| 1 | --- |
| 2 | name: ux-testing |
| 3 | description: Test an app the way its user will find problems, before they do. Drive the real UI yourself on realistic data across states, sizes, themes and roles; check motion, speed and accessibility; run a fresh-eyes walkthrough; and collect evidence the owner can actually see. Use this before reporting any UI change as done; before handing over a build, preview URL or setup command; after deploying a UI; when asked to test, verify, QA, screenshot, review or audit UX; when something feels laggy or slow; and whenever a user reports a UI bug, even if "the tests pass". |
| 4 | --- |
| 5 | |
| 6 | # UX testing |
| 7 | |
| 8 | The examples come from Clover's apps: Snowbound (a OneNote 2010 remake), Clover |
| 9 | Chat (a chat multiplexer) and the snow globe dashboard (a home-server console). |
| 10 | The quotes are hers. |
| 11 | |
| 12 | After visual problems, verification was the most common correction across those |
| 13 | apps. Nearly every case was something the owner caught within minutes of using |
| 14 | the build. The bar, in Clover's words: |
| 15 | |
| 16 | > "the exit criteria should be knowing that there are no bugs in my own |
| 17 | > notebook, not having me point out many failures. i will probably find more |
| 18 | > subtle things with a larger design review then." — Clover, Snowbound |
| 19 | |
| 20 | ## 1. Get a surface you can drive without touching the owner's screen |
| 21 | |
| 22 | - **Web or HTML.** Use the in-app browser (the Claude or Codex browser pane), |
| 23 | which the owner can watch alongside you, or `scripts/drive.mjs`. The driver |
| 24 | runs headless Chrome over CDP from JSON steps (click, type, key, drag, |
| 25 | waitFor, viewport, eval, shot) and lists them in its header. Give each |
| 26 | parallel agent its own `PORT`. It exits 2 on a missing or invisible element |
| 27 | and 1 on page errors. |
| 28 | - **Native desktop.** Build a replay harness into the app early. An env var or |
| 29 | flag feeds the app a file of steps in a hidden window: pointer, key, wait, |
| 30 | settle marker, snapshot, appearance, resize and an accessibility dump. It |
| 31 | ticks frames so animations run, and renders screenshots offscreen. |
| 32 | Snowbound's `SNOWBOUND_REPLAY` is the model |
| 33 | ([Testing an interface you can't click](https://shale.paperclover.net/snowbound/tree/-/arc/ui.md)). |
| 34 | Wait on a settle marker that round-trips through the app, not a fixed delay; |
| 35 | fixed delays flaked under load. |
| 36 | - **iOS.** Use the simulator with scripted taps. Install on the owner's device |
| 37 | only with them. |
| 38 | - **A reference app or another OS.** Use disposable VM clones, with a desktop |
| 39 | VM tool that gives screenshots, input and the accessibility tree. |
| 40 | - **In a sandbox you own, sign in yourself** with the seed credentials: "you're |
| 41 | allowed to login to the vm since it's a sandbox you fully control." If an |
| 42 | automation surface fails twice, switch tools instead of asking the owner to |
| 43 | babysit it. |
| 44 | - **On macOS, if every headless browser hangs at launch**, check whether |
| 45 | `pboard` is wedged before blaming the code. |
| 46 | |
| 47 | On the owner's machine, never: |
| 48 | |
| 49 | - run computer use on their desktop ("please dont drive finder it's |
| 50 | interrupting my keyboard focus"); |
| 51 | - open their real documents, overwrite their clipboard, or trigger keychain or |
| 52 | permission prompts; |
| 53 | - replace their installed build once an updater ships ("you should not mutate |
| 54 | the build so we can observe the updater"). |
| 55 | |
| 56 | Work on copies of their data, and cap VM and build load so their desktop can't |
| 57 | freeze. |
| 58 | |
| 59 | ## 2. Run the real thing |
| 60 | |
| 61 | - **Launch the real app on a copy of real data**, and open every screen you |
| 62 | touched. "note that right now the app doesnt seem to run" was news to the |
| 63 | agent that had just reported its fixes done. |
| 64 | - **A blank capture is a bug in your harness.** Find out why it's blank |
| 65 | (occluded, behind another window, the wrong copy) before blaming the |
| 66 | environment: "the screens are def not off". |
| 67 | - **"Integrated" means signed in as the real role, with every panel showing real |
| 68 | data.** HTTP 200 is not integration. The snow globe dashboard was called |
| 69 | integrated after route checks, and Clover found admin panels missing as soon |
| 70 | as she logged in. |
| 71 | - **A visual fix is verified by looking at the rendered page in its real |
| 72 | state.** A theme stylesheet returning 200 is not a theme applied. That one |
| 73 | took three more rounds. |
| 74 | - **Use realistic fixtures**: two-sided conversations, varied lengths, media, |
| 75 | and thousands of items ("32 is like nothing"). Derive them from the real spec |
| 76 | so demo states can't contradict reality. Review fixtures must tell an |
| 77 | unambiguous story ("actually this image not even sure what the reply chain |
| 78 | is"). |
| 79 | |
| 80 | ## 3. The matrix |
| 81 | |
| 82 | For each surface you changed, check every row: |
| 83 | |
| 84 | | Axis | Cover | |
| 85 | | --- | --- | |
| 86 | | Instances | Every row; every popover, dialog and menu; every page sharing the component | |
| 87 | | Themes | Light and dark, judged separately | |
| 88 | | Sizes | The owner's real viewport (ask; Clover reviewed the dashboard at 1075 px, not 1280); the default window; the minimum width; long names (120 characters) and deep paths | |
| 89 | | States | Empty, loading, error, offline, first run, partial data, one backend down, each role (with a "view as") | |
| 90 | | Adversity | API 502; slow responses; a stream dying mid-run; double-clicked submits (count the requests: three clicks once sent two DELETEs); switching between two items of the same kind; a poll arriving while a field has focus | |
| 91 | | Input | A real pointer on scrollbars and drags, including off-axis; hover hit areas; Tab and Shift-Tab; Return and Esc; double and triple click; IME, the emoji picker and dead keys; window resize and zoom; scroll edges; a click with no mouse movement first; a modifier pressed alone | |
| 92 | | Platforms | Real hardware for GPU, compositor and permission paths; the deployed URL in each target browser with a cold cache | |
| 93 | |
| 94 | Most of the dashboard's real defects showed up only under adversity. The |
| 95 | builders' screenshots looked fine while no error state could render at all. |
| 96 | |
| 97 | ## 4. Motion and speed |
| 98 | |
| 99 | - **Record real-speed frames from the real app** and check every frame for: |
| 100 | - content shifting inside a moving box; |
| 101 | - double draws; |
| 102 | - bleed-through; |
| 103 | - clamps that stop motion early. |
| 104 | |
| 105 | Slowed frame strips only supplement this. Screenshots can't prove a transient |
| 106 | frame is absent (a resize stretch, a flicker); say so, or record video. |
| 107 | - **When motion is the question, give the owner a runnable build early**, as |
| 108 | a separate preview copy beside their installed build, regressions and all: "this would be good to get my hands on even if it isnt on |
| 109 | a commit yet or has regressions. i want to see the animation in practice." |
| 110 | - **Slowness is a UX bug, and it gets numbers:** |
| 111 | - request and page timings (anything over about a second is a bug); |
| 112 | - frame time against the display's refresh rate; |
| 113 | - idle CPU and idle frame count; |
| 114 | - layout shift in pixels; |
| 115 | - p95 under load; |
| 116 | - behaviour with one upstream stalled. |
| 117 | |
| 118 | Clover: "this is the big ux one is it feels slow." |
| 119 | - **Never blame the VM.** Make it smooth in the worst environment, then verify |
| 120 | on representative hardware: "the dashboard should remain as smooth as |
| 121 | possible even under terrifying load." |
| 122 | - **"Improve the UX" means deploy the change and fix the cause.** A diagnosis is |
| 123 | not the deliverable: "can you deploy it. and then also fix the actual |
| 124 | issues?" |
| 125 | - **Bound a live view's polling**, so a dashboard can't starve the system it |
| 126 | watches. |
| 127 | |
| 128 | ## 5. Accessibility |
| 129 | |
| 130 | Use `accessibility`. The tree is the test: assert on it and drive the UI |
| 131 | through it, and never turn on a screen reader on the owner's machine to listen |
| 132 | for results. |
| 133 | |
| 134 | ## 6. Fresh eyes and reviewers |
| 135 | |
| 136 | - **Before handoff, run a pitch-only tester agent** |
| 137 | (`references/fresh-eyes.md`). It knows two sentences about the product, reads |
| 138 | no code or docs, does 8–10 everyday tasks and reports by severity. On Clover |
| 139 | Chat it found about 29 issues, and all 19 shell fixes landed in 20 minutes. |
| 140 | - **Use read-only reviewer agents** (UX, simplification, security). They catch |
| 141 | state bugs that the builders' screenshots hide. |
| 142 | - **Turn the findings into a numbered fix brief**, leaving out anything another |
| 143 | layer owns. |
| 144 | |
| 145 | ## 7. Sweeps |
| 146 | |
| 147 | When nits keep arriving one at a time ("theres a lot of these can you send a |
| 148 | high agent ... to do a ux review"), run a sweep with `references/ux-sweep.md`: |
| 149 | |
| 150 | - one named build; |
| 151 | - both themes, at the default and a narrow size; |
| 152 | - disposable data; |
| 153 | - the reference app captured beside yours; |
| 154 | - a screenshot and an outcome for every finding; |
| 155 | - the owner's items first, with their answers logged. |
| 156 | |
| 157 | ## 8. Evidence the owner can see |
| 158 | |
| 159 | - **Put images and GIFs inline, where the owner can open them.** A file path |
| 160 | they can't open is not evidence ("i cant see the pictures from this"). |
| 161 | - **List verified and unverified separately.** A compile, a test count, or a |
| 162 | line like "popup layouts now follow the demo" is not visual evidence. Every |
| 163 | claim gets a screenshot of the surface it describes. |
| 164 | - **Before handing over a URL or setup command, load or run it yourself, the |
| 165 | way the owner will:** from their device, bound to the LAN where needed (for |
| 166 | example Vite's `--host`), and right now. A dead tunnel and a half-working DNS |
| 167 | installer each cost a round. |