Costly Question sets by kind of claim
A Costly Question needs access to the seller’s records, people, or systems, or work on your own material. The chapters “Fluent Proposals Cost Nothing Now” and “Each Kind of Claim Needs Its Own Evidence” explain the idea. Keep the set short: five or six questions in all, spent on the claims that could change your decision. Drop any question whose answer you could not check.
Capability
-
Run this task on an anonymized sample of our own records, and give us the output and the log of what the product did.
-
Let a technical reviewer we choose pick the inputs, run the product on our task, and read the output.
Outcome
-
Give us the measurement method for the result you claim, the raw figures behind it, including the runs that did not work, and the name of the person who ran it.
-
Agree to a trial with our baseline, our scorer, and a Fail Condition written before the start.
Safety and compliance
-
Show the report behind the badge, the date range it covers, the controls it tested, and what it left out.
-
Show the last incident you disclosed to a customer, what you changed, and how long it took.
Cost
-
Give a three-year price for our usage, with every fee that depends on volume, support level, or data export.
-
Show what we pay if we leave before the term ends.
-
Tell us what customers we choose from your list paid at their first renewal.
Durability
-
Disclose the months of cash you hold and the contracts that depend on this product.
-
Agree to escrow of source code and model artifacts with a third-party agent on stated release triggers.
Fit
-
Run the product on our data and our workflow, with our own people, and let them score the result.
-
List the three integrations that failed or took longest at your last five customers, chosen by us from your customer list.
Team
-
Name the people who will do the work, let us interview them before we decide, and agree that the contract names them and says what happens if one leaves.
-
Let us choose five clients from your full client list and call them without you present.
AI performance
-
Score the system on our cases, which you have not seen, using criteria we write.
-
Tell us when the model behind the product last changed, and what you promise to tell us before it changes again.