Operating Theory
Fidelity under acceptance testing
Judge a reflection by the work it improves. Add one fact, rerun the task, and keep the fact only if the accepted outcome gets better.
The method
Fix a task, a reviewer, and an acceptance test. Run it against the current reflection. Add or remove one section. Run it again. Record whether the accepted outcome improved, stayed flat, or degraded.
This is the same discipline as Outcomes Acceptance Testing™ applied to context rather than to work. It is slow, and it is the only way we know to separate a reflection that helps from a reflection that is merely thorough.
The metric trap
Claim: unknown
A coverage percentage would be easy to produce and easy to game. Until a fidelity measure can be shown to predict better accepted outcomes, we report the outcome and leave the score alone.
What would show this claim is wrong
- Adding well sourced material to a reflection shows no relationship with accepted outcomes across repeated tasks.