Research program
Operational Truth™, Risk, Liability, and Outcomes
This program asks how an organization can know that its mixed workforce acts on reliable evidence and produces acceptable outcomes. Consequential work needs evidence of what happened, clear limits on authority, known failure modes, and a test tied to the result in the world. It matters because a completed field can be well formed and still be wrong.
How can an organization know that its mixed workforce is acting on reliable evidence and producing acceptable outcomes?
Separate the states
Intended, reported, recorded, inferred, observed, and verified state are different claims. A completed field may be well formed and still be wrong about the world.
Operational Truth™ links each material claim to derived evidence, a source event, a source system, an actor or device, and a time. Conflicts and stale evidence remain visible.
Test the outcome
A generated note, sent message, submitted authorization, or completed task is an artifact. Acceptance asks whether the intended person reached the right state within the allowed time and whether exceptions followed a safe route.
Tests should cover the work unit, handoff, complete workflow, safety, outcome, economics, human experience, population effects, security, and degraded operation where each applies.
Keep evaluation independent
The worker that performed the task should not be the only judge of its success. Keep the rubric inspectable and use independent evidence where the consequence warrants it.
A worker that passed once is not permanently trusted. Sample live work, keep sentinel and adversarial cases, watch drift, and recertify after material model, source, policy, or workflow changes.
Let authority follow evidence
Authority begins with observation and grows only after measured performance under the intended conditions. It should fall when source quality drops, a dependency changes, performance degrades, a new population appears, policy changes, or an incident crosses a threshold.
Capability does not grant permission. Every worker needs a named owner, allowed action, review gate, escalation route, and rule for losing authority.
Model population risk
People often make different mistakes. Shared models, prompts, policies, sources, tools, identities, and integrations can make machines fail together. A low case error rate can still create a large loss before anyone notices.
Map every shared dependency and ask how many outcomes one fault can reach before detection and containment. Test hostile documents, poisoned data, spoofed identities, excessive permissions, and policy tampering.
Plan for failure before scale
Every consequential worker pack needs normal, reduced authority, read only, manual fallback, and safe stop modes. State what queues, what gets priority, who takes over, how long the outage may last, and how records are reconciled after recovery.
A system that works only while every AI and data dependency is healthy is too brittle for consequential work.
Do not confuse outcome with cause
Operational evidence can show what happened and whether it met a standard. It does not prove the new system caused the change.
Use a comparison suited to the decision, from a simple baseline through phased deployment, matched groups, time series, or a prospective trial. Examine results across sites, roles, populations, languages, access levels, complexity, and experience.
Evidence and risk review
The strength of the claim should not exceed the strength of the evidence.
| Question | Useful evidence | What it does not prove |
|---|---|---|
| Did the work occur? | Source linked operational record | That the outcome improved |
| Was the outcome accepted? | Defined test and time window | That the worker caused it |
| Is authority justified? | Performance, incidents, overrides, population results | Permission under every law or policy |
| Can failure be contained? | Dependency map and recovery drill | That every future failure is known |
| Is the business case sound? | Full cost, transition, burden transfer, causal evidence | That another setting will match |
Research notes
Authority progression and recertification
This note sets a rule for granting and withdrawing a worker's authority: evidence earns more authority, and a material change in model, source, policy, population, workflow or incident cuts it until retesting is complete. It matters because strong performance alone does not justify wider authority. Increase authority only after evidence and reduce it automatically when material conditions change.
Causal Evaluation plan
Choose enough comparison to tell whether a recomposed workflow caused the claimed change. The plan matches the study design to the claim and narrows the claim when the design cannot distinguish the intervention from staffing, case mix, policy, or season, because operational acceptance and causal effect answer different questions.
Data readiness before automation
Most assembled systems fail before any model runs, because the data they depend on is incomplete, inconsistent, or somewhere nobody checked. Before a workflow is a candidate for automation, answer four questions in writing: availability, quality, gaps, and alignment. Repair the data substrate first.
Dependency and Common Mode Failure map
This note finds that case accuracy does not predict population harm from a shared fault, so scaling should stop when one untested dependency can affect more outcomes than the organization can detect, contain and recover. The map gives the concentration and blast radius thresholds to check. Show which workers share models, sources, policies, tools, identities, vendors, and people so one fault cannot spread unseen.
Evidence Supply Chain
Keep every consequential operational claim connected to its source, custodian, transformations, and useful life. The result is an evidence chain a reviewer can follow in both directions, from the real event to every derived field and decision. Provenance shows where a claim came from, not that the source is correct.
Graceful Degradation plan
Define how consequential work continues or stops when AI, data, tools, identity, or outside services are unavailable or untrusted. The plan sets modes from normal operation to safe stop, and a fallback counts only once it has been exercised. Deployment stops when the operation cannot fail safely.
Operational truth
Trace an operational claim to independent evidence and distinguish intended, reported, recorded, inferred, observed, and verified state. The output is a queryable evidence chain and the gap between reported and observed state. A complete record can still be false, and Operational Truth™ does not prove causality.
Outcomes acceptance testing
Test whether the intended real world result occurred within stated quality, safety, time, cost, and evidence limits. Acceptance is a continuous record tied to worker and workflow versions, and authority is reduced or stopped when thresholds fail. Passing an outcome test does not prove the worker caused the outcome.
Rights Provenance record for encoded work
Track ownership and permitted use of know how, data, worker contracts, prompts, tests, evidence, and operating records. The resulting rights record can block release when a material asset is unresolved, and it supports qualified legal review without deciding ownership or enforceability.
Related
- Engineering Care Delivery
- Eidolons: working digital reflections
- Operational truth
- Outcomes acceptance testing
- Dependency and Common Mode Failure map