Intellectual Frontiers

Theory

The tool was purchased. Was it tested?

The note's observation is that a security purchase is judged on deployment, not on evidence that the tool changes an attack outcome. The fix is to add a named production test as an acceptance condition. Procurement produces a deployment date. Nothing in the ordinary purchase process produces evidence that the tool changes an attack outcome in this environment.

Where the gap opens

Claim: observation

A product is evaluated against a feature list, deployed against a project plan, and reported against a coverage percentage. At no point does the standard process require somebody to perform the behaviour the tool is supposed to stop and record what happened.

Coverage percentages are especially misleading because they measure agent installation. They do not measure detection quality.

A cheap correction

Claim: recommendation

Add one acceptance condition to every security purchase: a named test, run in production, that demonstrates a changed outcome, repeated quarterly. This is far cheaper than the purchase and frequently changes the renewal decision.

Next