Time to detect
From the first attacker action to the moment a human or system knows something is wrong. Measured from injected activity, not from the incidents that happened to be noticed.
Research program
The dangers of security theatre
Security programs are usually described by what they have: purchased tools, signed policies, closed findings, and certificates. That says almost nothing about what would happen if a capable attacker tried something today. This program tests the controls and measures what happens.
If the attacker tried this today, what would actually happen?
The claim
Claim: hypothesis
A control does not count just because it is there. Test it against a real attack and see what happens. A deployed tool that has never faced the behaviour it is meant to stop has an unknown effect, and unknown is red.
Tools and policies are still necessary. The open claim is that organisations which repeatedly test their controls against attacks suffer smaller losses than comparable organisations that do not. That sounds plausible, but this research has not proved it.
What we measure instead of inventory
Claim: recommendation
Each can be observed, timed, and repeated. If the organisation cannot produce the number, write unknown. Do not fill the gap with a comfortable estimate.
Time to contain
From detection to the point where spread stops. Containment is an action with a timestamp: an account disabled, a host isolated, a key revoked, a segment closed.
Blast radius
What one compromised identity, key, service account, or workstation can reach. Measured by attack-path analysis and by trying it, not by reading the access matrix.
Attack-path validation
Whether a named path from an ordinary foothold to a crown-jewel system still works after the last round of remediation.
Vulnerability recurrence
How often the same class of weakness returns after being closed. Recurrence suggests the fix addressed an instance rather than a cause.
Restoration proof
The share of critical systems restored from backup within the last period, with a timed result and business validation.
Why inventories persist
Claim: observation
Inventories persist because they are cheap to produce, easy to audit, and comfortable to present. A count of controls can be assembled from configuration exports in a day. A measured detection time requires somebody to generate real activity in production and accept what the number says.
Reports follow what is easy to collect. Finding counts are easy. Attack results are harder. Changing the measure usually requires an executive willing to see a red number where the old report showed green.
What would prove us wrong?
Claim: unknown
The program rests on the claim that tested controls predict outcomes better than inventoried controls. Several results would weaken it.
- Breach cost data showing no relationship between adversarial testing frequency and loss magnitude once organisation size and sector are held constant.
- Evidence that detection time is dominated by attacker choices rather than defender capability, making it a poor management metric.
- Cases where heavy control testing consumed the budget that would otherwise have funded recovery capability, producing a worse total outcome.
- Insurance claims data showing that certification status predicts loss better than measured response times.
Experiments for this program
- Measure the real detection interval How long does it take to notice activity that a deployed control claims to detect?
- Blast radius of one ordinary identity What can be reached from a single compromised workstation account?
Patterns and anti-patterns
- Attach an outcome test to every security purchase (pattern)
- Score unknown as red (pattern)
- Reporting agent coverage as security posture (anti-pattern)
Evidence for this program
- CISA Known Exploited Vulnerabilities catalogue A public record of what is actually being exploited, useful for separating theoretical from observed risk.
- MITRE ATT&CK The behaviour catalogue used to select techniques for detection and containment measurement.
- Verizon Data Breach Investigations Report Annual incident pattern data. Useful for base rates, with the usual caution that the sample is not a population.
Research notes
What a findings dashboard leaves out
Counting open findings measures the discovery process. It says nothing about detection speed, containment, or recovery, and it falls when scanning stops. An executive dashboard should therefore show detection time, containment time, and demonstrated recovery, because a falling finding count does not show a safer company.
The tool was purchased. Was it tested?
The note's observation is that a security purchase is judged on deployment, not on evidence that the tool changes an attack outcome. The fix is to add a named production test as an acceptance condition. Procurement produces a deployment date. Nothing in the ordinary purchase process produces evidence that the tool changes an attack outcome in this environment.
Unknown is red
If nobody can show that a control works, score it as failing. Amber and blank let an untested control sit quietly beside a tested one.