Discoverability Gap Diagnosis

Use this skill when the user has a scored Discoverability Score run and wants to know why the subject is or is not surfaced, what to change, and how to retest.

Download SKILL.md

This skill is Labor as Code™ for the measurement loop in Discoverability Engineering. Labor as Code is a skill that encapsulates a person's knowledge, judgment, taste, and skill in durable, inspectable, versioned form. The know-how is a researcher's habit of asking, of every explanation, what observation would prove it wrong.

The method comes from “A Discoverability Score Measures Whether Finders Who Don’t Know Your Name Meet You”, which defines the three labels on every finding, the Competitive Inclusion Set, the Evidence Gap Register, the five intervention classes, and the rule against content that manufactures evidence. "The System Studies Only the Alternatives a Finder Would See" gives the limit on what the system records about an alternative. Read them for the argument. This skill runs the method on the user's own scorecard and never reuses a worked example's findings.

The toolkit does the arithmetic. This skill works with the Discoverability Score toolkit, the download on the book's online companion, run as archon score commands. Every path below is relative to the toolkit folder: templates/, schemas/, methodology/config.yaml, and docs/. A case folder holds entity.yaml, engines.yaml, finder-specification.yaml, benchmarks/<version>/, and runs/<run_id>/. The skill never computes a score, a weight, or a hash itself. The scripts do, and the same inputs give the same score.

Observation, hypothesis, experiment. Every finding carries all three. The observation says what was measured, with counts and question IDs. The hypothesis says what might explain it and is labeled as a hypothesis. The experiment says what can be changed and retested. Every hypothesis also carries the observation that would disprove it, and the skill looks for that observation in the data before it proposes any change. A correlation is never a cause.

Say surfaced, never recommended. An engine that names a subject has not endorsed it, ranked it, or recommended a purchase. The skill writes surfaced, included, associated, introduced, cited, and discoverable, and it never writes "recommended" or "best" about what an engine did.

Before running

Confirm these points with the user before the first finding:

A scored run exists. Read scorecard.yaml and its warnings and confidence_notes. If the run is incomplete, work only from the dimensions that were scored and say which were not.
Who owns what. The Discoverability Operator decides which findings are worth acting on and which interventions are ethical. The engineer owns technical findings.
What shipped. For a diagnosis after an intervention, list every public change since the baseline with its date.

Step 1: Study the alternatives that appear

Run this prompt, for the highest-weight questions where an alternative appears and the subject does not:

What evidence appears to make the frequently surfaced entities legible to these answer engines? For each high-weight question where an alternative appears and the target does not, list the alternatives, the sources the answers cited for them, the source types, how directly each source states the claim, whether independent sources agree, and what kind of page structure the source has. Identify patterns. Do not assume causality, and do not recommend copying a competitor. Record only each alternative's name and the sources the answers cited; collect nothing else about it. Then compare, question by question, with the evidence the target has for the same relationship.

Check the output:

Reject any alternative with no row in normalized/results.jsonl. Record only alternatives the engines surfaced on their own, and for each one only the name and the sources the answers cited.
Reject any sentence that says a source caused an inclusion. A pattern is a hypothesis.
Do not collect, file, or publish anything else about an alternative.

Save analysis/diagnosis.yaml under competitive_inclusion, and the subject's ratings in normalized/assessments.yaml.

Step 2: Diagnose the gaps

Run this prompt:

For every cluster of low-performing questions, write an Observation, a Hypothesis, the Evidence, an Experiment, and an Expected measurable effect. The observation states what was measured, with counts and question ids. The hypothesis states what might explain it and is labeled as a hypothesis. The evidence cites the answers, sources, and chain links. The experiment says what can be changed and retested. Add the observation that would disconfirm the hypothesis, and look for it in the data. Do not state a cause as fact. Then write an entry for the Evidence Gap Register for each important question, with the first broken link in the chain.

Check the output:

Reject any finding without all three labels.
Reject any hypothesis with no observation that would disprove it. The theory that the subject is missing because it lacks enough content predicts that firms with fewer pages are missing too. Look for the entity that breaks the theory.
Reject any entry whose observation is not in the scorecard.
Write the first weak or broken link of each chain, not only the first broken one.

Save analysis/diagnosis.yaml and evidence-gaps.yaml. Fields follow schemas/evidence-gap.schema.yaml.

Step 3: Design the minimum interventions

Read docs/interventions.md. Run this prompt:

What is the smallest set of legitimate evidence, entity, content, relationship, or technical interventions that would directly address the observed gaps? For each, state the change, the gap it addresses, the hypothesis, the result that would show it did not work, why it publishes real evidence without manufacturing any, the window (days 1 to 30, 31 to 60, 61 to 90, or later), and the owner. Prefer repairing facts and relationships before creating content. Do not propose thin pages, FAQ farms, synthetic comparisons, invented expertise, reviews, or any change that is not tied to an observed gap.

Check the output:

Each intervention names the gaps it addresses. The validator rejects one whose gap is not in the register.
Each has a no_effect_if specific enough to look for, and a legitimate_because that survives a reader asking where the evidence came from.
Repair facts and relationships before creating content. A technical repair is usually cheap and goes first, and it is tested alone.
Reject thin pages, FAQ farms, synthetic comparisons, bought or coached reviews, invented expertise, unsupported "best" pages, and text written for an answer engine. They manufacture evidence.
Default order: days 1 to 30, 31 to 60, 61 to 90. Evidence overrides it: a gap under the highest-weight question comes first.

Save interventions.yaml. The Operator decides which interventions are ethical.

Step 4: Prepare the retest

Read docs/retest.md. Run this prompt:

Prepare the retest folder for the baseline run. Use the same benchmark version and the same weights, and copy questions.yaml and benchmark.lock.yaml unchanged. List every engine change since the baseline. State the days elapsed. List every public change shipped since the baseline, with its intervention id and date. Write a manifest with previous_run set. Do not add, remove, reword, or reweight a question, and do not edit any file in the baseline folder.

Check the output:

Run archon benchmark check <questions.yaml>, then archon score check runs/<retest>.
Confirm every version in the manifest matches the baseline's, or that the difference is disclosed. Set planned_retest in the baseline before any intervention ships.
Wait for the engines to refresh before judging a change. Do not retest on a schedule shorter than the engines can show.

Save runs/<date>-retest/ and changes-since-baseline.md. Scoring the retest belongs to discoverability-score-run.

Step 5: Conclude

End with a recommendation and its reasoning. Give:

the single next experiment, the gap it addresses, and the result that would show it failed;
the other gaps in order, with the reason for the order;
every finding the data cannot settle, with what evidence would settle it, such as a source-type comparison or a page-to-question map; and
for a diagnosis after a retest, every regression and every intervention with no measurable effect, each with the next step to understand it.

Do not state a cause as fact. If no experiment is justified by the evidence, say so plainly and name what to collect first. Never pick one to sound useful.

Never

Never state a cause as fact, or credit a rise to one change when several shipped together.
Never propose a change that is not tied to an observed gap.
Never propose content, reviews, or expertise the subject cannot support.
Never record or publish anything about an alternative beyond its name and the sources the answers cited.
Never edit a frozen benchmark, a baseline run, or a raw answer.