Discoverability Score Run

A skill for the person scoring a Discoverability Score baseline or retest, which takes a frozen benchmark and the raw answers captured from engines, records the evidence, runs the scoring script, and ends with the score and its confidence notes.

This skill is Labor as Code™ for the measurement loop in Discoverability Engineering. Labor as Code is a skill that encapsulates a person's knowledge, judgment, taste, and skill in durable, inspectable, versioned form. The know-how is a researcher's refusal to interpret an answer before it has been stored whole.

The method comes from “A Discoverability Score Measures Whether Finders Who Don’t Know Your Name Meet You”, which defines entity recognition and discoverability, the nine sub-scores and their weights, what counts as a substantive mention, and why an incomplete scorecard has no total. Read it for the argument. This skill runs the method on the user's own answers and never reuses a worked example's numbers.

The toolkit does the arithmetic. This skill works with the Discoverability Score toolkit, the download on the book's online companion, run as archon score commands. Every path below is relative to the toolkit folder: templates/, schemas/, methodology/config.yaml, and docs/. A case folder holds entity.yaml, engines.yaml, finder-specification.yaml, benchmarks/<version>/, and runs/<run_id>/. The skill never computes a score, a weight, or a hash itself. The scripts do, and the same inputs give the same score.

The score is a function of stored observations. The skill classifies answers and extracts evidence. A person checks that work. archon score show computes every sub-score and the total. No model judges anything at that step.

Say surfaced, never recommended. An engine that names a subject has not endorsed it, ranked it, or recommended a purchase. The skill writes surfaced, included, associated, introduced, cited, and discoverable, and it never writes "recommended" or "best" about what an engine did.

Before running

Confirm these points with the user before the first answer is read:

1. The benchmark is frozen. Run archon benchmark check benchmarks/<version>/questions.yaml. If it fails, stop. Use discoverability-benchmark-build.
2. The engines. Read engines.yaml. For each engine, record the product, the mode, and how the answers were captured. If the user captured answers by hand or through a permitted interface, continue. Never drive an engine's website with automation its terms forbid, and never ask the user to.
3. The run. Name the run folder (runs/<date>-baseline or <date>-retest), the window, the number of runs per question per engine, and for a retest the previous run. Copy questions.yaml and benchmark.lock.yaml into the run unchanged.
4. Imported counts. If the user has only counts published by someone else and no answers, this is an imported_aggregate run. Say so now: groups never reviewed against the subject's Finder Specification are reported and not scored, and the total will not be computed.

Step 1: Save the raw answers

Put each answer in raw/ exactly as captured. Never edit, trim, or delete one. Each gets a result ID that normalized/results.jsonl will cite in raw_ref.

Step 2: Record what each answer shows

Work one raw answer at a time. Run this prompt:

For each raw answer, record: whether the entity is mentioned and, if so, whether it is introduced as a response to the question's need (substantive), named without being offered as relevant (incidental), or cannot be decided (ambiguous); the sentence in which it appears; every citation URL and what kind of source each is; every other entity the answer introduced; the likely category association; and the evidence the answer appears to rely on. For named-entity questions, judge whether the description is accurate, partly accurate, inaccurate, or not found, and list any conflicting names or dates. Quote the answer; do not paraphrase it. Do not state or imply why the engine included or excluded anyone. Make no causal claim.

Check the output:

• Open the raw answer for at least one row in ten. Confirm the mention value and the quoted sentence.
• A mention the answer does not support is wrong. An unclear one is ambiguous, and the validator warns until a person resolves it.
• A name in passing, in a list of unrelated firms, or in a citation title is incidental. A cited page with the subject absent from the answer is not a mention.
• Never let the AI overwrite raw text.

Save normalized/results.jsonl, one line per answer, following schemas/result.schema.yaml.

Step 3: Rate the evidence, with a person

Use templates/assessments.yaml and the rubric in docs/evidence-rubric.md. Rate each important question's evidence 0 to 4 on seven attributes, with a source and a sentence of reasoning. The claim type decides what is required: the subject's own site is the authority for facts about itself, and independent corroboration is required for reputation, category, outcome, expertise, and comparative claims. Rate each question's evidence chain link by link: intact, weak, broken, or unknown. Fill the entity clarity and technical legibility checklists from what was observed. Mark unknown for anything not checked. Never guess a rating.

Step 4: Validate

Run:

archon score check runs/<run_id>

Errors stop scoring: a missing raw file, a mixed benchmark version, a changed question, a branded question, a substantive mention with no context. Fix them. Warnings stay in the scorecard: a missing engine result, an ambiguous mention, a missing date outside the window. Report them. Use --strict when the user wants warnings treated as errors.

Step 5: Score

Run:

archon score generate runs/<run_id>

Do not compute, estimate, or adjust any number by hand. If the script says a dimension is not scored, report it as not scored and say why. If it computes no total, say "not computed" and give the points scored out of the points available. Never fill a gap to produce a total.

Step 6: Report

Run:

archon report generate runs/<run_id>

It writes executive.md and report.md from stored data. For a retest, also run archon comparison generate runs/<retest> --against runs/<baseline>. The comparison refuses runs on different benchmark or methodology versions unless the user passes --allow-version-mix. When it prints a disclosure, repeat it in the report.

Optionally rewrite the Executive Finding paragraph only. Run this prompt:

Rewrite the generated "Executive Finding" in plain sentences that answer three questions: Are we known? Are we findable? Where does the relationship break? Use only numbers that appear in scorecard.yaml. Use the words surfaced, included, associated, introduced, cited, and discoverable, never recommended or best. State the limits of the evidence in one sentence. Make no causal claim and do not predict sales.

Step 7: Conclude

End with the result and a recommendation. Give:

• the Discoverability Score and all nine sub-scores, or "not computed" with the points scored;
• the strongest and weakest scored dimensions, read next to each other, and in particular the gap between recognition and problem association when there is one;
• every confidence note and warning: engines, questions, single-run cells, unstable cells, ambiguous mentions, unscored dimensions, assumed weights;
• for a retest, every regression and every intervention with no measurable effect, stated as plainly as the gains; and
• what the score cannot establish. It measures how reliably a defined set of finders, asking a defined set of questions through a defined set of systems, encounters appropriate evidence connecting a subject to the problem at a particular point in time. It cannot by itself prove sales impact, revenue, reputation, product quality, customer satisfaction, category leadership, causality, or market share.

Then recommend the next step: which sub-score's trace to read first and why, handing off to discoverability-gap-diagnosis. If the data is too thin for any recommendation, say so and name what would resolve it.

Never

• Never compute or adjust a score, a weight, or a hash.
• Never edit, trim, or delete a raw answer.
• Never state why an engine included or excluded anyone.
• Never mix benchmark versions without the printed disclosure.
• Never count an incidental or ambiguous mention as inclusion.
• Never drive an engine's website against its terms.
• Never state a higher number as success. A rise is a finding to explain.