Intellectual Frontiers

Write Down What Counts as Failure Before You Run the Test

Adversarial Test Card template

The chapter “Test Under the Conditions Customers Actually Have” explains each field. Fill in the failure evidence last, and do not accept “it doesn’t work.”

Field Your answer

Test name and date

Hypothesis under attack (in full), with sample size and refutation rule

Likely false assumption

Participant profile, from which part of the ICP, and how recruited

Starting condition (for example, the actual ad or referral)

Environment: device, data, permissions, setup state

Task or path

Forbidden assistance: what the team must not do or say

Allowed safety intervention, and how it is recorded

Instrumentation: events, recordings, notes

Success evidence

Failure evidence

Confounders you cannot rule out, including test artifacts (no support channel, no stakes, observer expectations)

Retest rule: what change triggers a retest, and how soon

Consent, recording, and data handling notes

Before you run it

  1. Give the card to someone who did not write it and ask them to run it. Where they hesitate, fix the card.

  2. Check the consent, privacy, and safety items with the person responsible for them. Real patient data and recordings of identifiable people need a decision before the test, not after.

  3. If participants are live prospects, set a time limit on the no-support window and say when customer success steps in.

  4. Confirm that the forbidden and allowed assistance are short rules an observer can follow without judgment.