← All skills

Demo Then Ship

Proves the feature works in the actual running app, not only in the tests written for it.

shippingverificationtestingqualityminutes

When to reach for it

Right before merging or deploying, when the tests are green and nobody has yet opened the app and used the thing.

What changes

  • You get a numbered click-by-click script, starting from a cold start, run against the build that actually ships.
  • Each step is reported as observed output with evidence attached — screenshot, status code, log line — instead of expected behavior.
  • The unhappy paths are exercised: bad input, offline, denied permission, empty state, double submit, refresh mid-flow.
  • The console and server logs are checked for new errors and warnings introduced by the change.

Pairs with

SKILL.mdpaste into your agent

Demo Then Ship

Tests check what someone thought to check. The demo checks whether the thing works.

1. Write the script first

The exact steps a person performs, numbered, starting from a cold start: launch, sign in, navigate, act, observe. Writing it before running it prevents the walkthrough from quietly avoiding the parts that are broken.

2. Run the app the way it ships

Production build if production is the target. Real data path, real auth, the role a normal user has, the viewport a normal user has. A development-only route or a mocked harness proves the mock works.

3. Perform the script and narrate what is seen

Step by step, in order. Report what actually appeared — the text on screen, the status code, the row in the database, the log line. Not what should have appeared. If a step needs a retry, that is a finding.

4. Capture evidence

Screenshot, terminal output, response body, log excerpt. One artifact per meaningful step. Evidence is what makes the demo checkable by someone who did not watch it.

5. Run the unhappy paths

Bad input. Empty state. Denied permission. Network offline. The action performed twice quickly. A refresh in the middle of the flow. The back button. These are where generated features fail, because the tests written alongside them cover the path that was being imagined.

6. Check for collateral

Look at the screen before and after the changed one. Check the console for new errors and warnings, and the server log for new noise. A feature that works while breaking its neighbour is not done.

7. Decide

If any step failed, the feature is not ready, regardless of the test suite.

Rules

  • A passing test is not a demo. Say so plainly rather than substituting one for the other.
  • Do not demo through a mock, a fixture harness, or a page that only exists in development, unless that is what ships.
  • Do not narrate expectations. Only observed output counts.
  • Do not skip the cold start. Warm state hides missing migrations, unset environment variables, and initialisation that only happened because of something done earlier.
  • Do not accept a demo that required a manual step the user does not have — a seeded row, a flag flipped by hand, a service started in another terminal. Note it as a shipping prerequisite.