Security programs are usually described by what they have: purchased tools, signed policies, closed findings, and certificates. That says almost nothing about what would happen if a capable attacker tried something today. This pillar tests the controls and measures what happens.
The claim
A control does not count just because it is there. Test it against a real attack and see what happens. A deployed tool that has never faced the behaviour it is meant to stop has an unknown effect, and unknown is red.
Tools and policies are still necessary. The open claim is that organisations which repeatedly test their controls against attacks suffer smaller losses than comparable organisations that do not. That sounds plausible, but this research has not proved it.
One report lists what was bought or enabled. The other records what happened when the control faced an attack.
What we measure instead of inventory
Each can be observed, timed, and repeated. If the organisation cannot produce the number, write unknown. Do not fill the gap with a comfortable estimate.
- Time to detect
-
From the first attacker action to the moment a human or system knows something is wrong. Measured from injected activity, not from the incidents that happened to be noticed.
- Time to contain
-
From detection to the point where spread stops. Containment is an action with a timestamp: an account disabled, a host isolated, a key revoked, a segment closed.
- Blast radius
-
What one compromised identity, key, service account, or workstation can reach. Measured by attack-path analysis and by trying it, not by reading the access matrix.
- Attack-path validation
-
Whether a named path from an ordinary foothold to a crown-jewel system still works after the last round of remediation.
- Vulnerability recurrence
-
How often the same class of weakness returns after being closed. Recurrence suggests the fix addressed an instance rather than a cause.
- Restoration proof
-
The share of critical systems restored from backup within the last period, with a timed result and business validation.
Why inventories persist
Inventories persist because they are cheap to produce, easy to audit, and comfortable to present. A count of controls can be assembled from configuration exports in a day. A measured detection time requires somebody to generate real activity in production and accept what the number says.
Reports follow what is easy to collect. Finding counts are easy. Attack results are harder. Changing the measure usually requires an executive willing to see a red number where the old report showed green.
What would prove us wrong?
The pillar rests on the claim that tested controls predict outcomes better than inventoried controls. Several results would weaken it.
-
Breach cost data showing no relationship between adversarial testing frequency and loss magnitude once organisation size and sector are held constant.
-
Evidence that detection time is dominated by attacker choices rather than defender capability, making it a poor management metric.
-
Cases where heavy control testing consumed the budget that would otherwise have funded recovery capability, producing a worse total outcome.
-
Insurance claims data showing that certification status predicts loss better than measured response times.