Write Down What Counts as Failure Before You Run the Test
Adversarial Test Card template
Companion to The Product Manager’s New Job Is Acceptance · Updated 2026-09-30
The chapter “Test Under the Conditions Customers Actually Have” explains each field. Fill in the failure evidence last, and do not accept “it doesn’t work.”
| Field | Your answer |
|---|---|
| Test name and date | |
| Hypothesis under attack (in full), with sample size and refutation rule | |
| Likely false assumption | |
| Participant profile, from which part of the ICP, and how recruited | |
| Starting condition (for example, the actual ad or referral) | |
| Environment: device, data, permissions, setup state | |
| Task or path | |
| Forbidden assistance: what the team must not do or say | |
| Allowed safety intervention, and how it is recorded | |
| Instrumentation: events, recordings, notes | |
| Success evidence | |
| Failure evidence | |
| Confounders you cannot rule out, including test artifacts (no support channel, no stakes, observer expectations) | |
| Retest rule: what change triggers a retest, and how soon | |
| Consent, recording, and data handling notes |
Before you run it
- Give the card to someone who did not write it and ask them to run it. Where they hesitate, fix the card.
- Check the consent, privacy, and safety items with the person responsible for them. Real patient data and recordings of identifiable people need a decision before the test, not after.
- If participants are live prospects, set a time limit on the no-support window and say when customer success steps in.
- Confirm that the forbidden and allowed assistance are short rules an observer can follow without judgment.
Download
- Download the tables as a spreadsheet (CSV)CSV · 775 B