Intellectual Frontiers

The score is nine sub-scores and a rule against guessing

The formula for every dimension, and what the score refuses to do

An Intellectual Frontiers Press working methodology, version 1.0.0. It is not an industry standard, and no engine maker, standards body, or regulator uses or recognizes it. The numbers live in methodology/config.yaml in the toolkit download, and archon score show is the only thing that computes them.

What it measures, and what it does not

The score measures how reliably a defined set of finders, asking a defined set of questions through a defined set of systems, encounters appropriate evidence connecting a subject to the problem at a particular point in time.

It cannot by itself establish sales impact, revenue, reputation, product quality, customer satisfaction, category leadership, causality, or market share. A mention is not an endorsement, a ranking, or a recommendation. The vocabulary is surfaced, included, associated, introduced, cited, discoverable.

A score of 43 against 44 means nothing. Movement across repeated runs of the same frozen benchmark, on the same engine configuration, means something. Compare a subject with itself.

An engine can know a subject and still not surface it

  • Entity Recognition: can an engine identify and describe the subject when it is named?

  • Discoverability: will an engine surface the subject when a finder describes a problem, category, need, constraint, buying situation, or question without knowing the name?

Recognition carries 5 points. Sixty-five of the 100 points come directly from answers to unbranded questions.

Nine weights total 100, and recognition carries 5

Dimension Weight Input

Finder Specification Coverage

20

Answers to unbranded questions

Problem Association

20

Answers to problem and capability questions

Category and Provider Association

15

Answers to category, provider, comparison, buying, and publication questions

Evidence and Citation Strength

15

Evidence assessments (0 to 4 rubric)

Cross-Engine Consistency

10

Answers, across engines

Entity Recognition

5

Answers to named questions

Entity Clarity

5

Checklist

Content Addressability

5

Evidence-chain assessments

Technical Legibility

5

Checklist

An unbranded question contains no name, alias, or brand term for the subject. Coverage, Problem Association, and Category and Provider Association draw on the same answers, so the 65 points that come from answers are not independent evidence; the weights state emphasis.

The weights are a judgment, set at the outset and never tested for how much the total moves when they change. A change means a new methodology_version, a paragraph here that says exactly why, and no comparison between runs scored under different versions without a printed disclosure.

"Finder Specification" is the book’s name for what other sources call an ideal finder profile.

The unit of observation is one answer

One answer: one run of one question on one engine. For each answer the toolkit stores the full raw text (raw/), the target mention (substantive, incidental, absent, or ambiguous), the context of the mention, the citations, the alternatives named, and the apparent rationale.

  • Substantive means the engine introduces the subject as a response to the question’s need. It counts, as 1.

  • Incidental means the name appears without being offered as relevant: a passing line, a list of unrelated firms, a citation title. It scores 0. A cited page alone is not a mention.

  • Ambiguous means a person could not decide. It scores 0 and raises a warning. Resolve it before you rely on the score.

For a cell (question, engine) with several runs, inclusion is the share of runs that were substantive. Inclusion is a frequency, so a cell uses the average across runs, not the worst run.

For a question, inclusion is the engine-weighted mean of its cells (engine weight defaults to 1).

Each sub-score is a weighted share of answers, assessments, or checks

Let w(q) be the class weight of question q, I(q) its inclusion, and m(q) the smallest inclusion of any engine that answered it.

  1. Finder Specification Coverage = 100 × Σ w(q)·I(q) / Σ w(q), over unbranded questions.

  2. Problem Association = the same sum over problem-discovery and capability-validation questions.

  3. Category and Provider Association = the same sum over category, provider-selection, comparison, buying-intent, and publication-discovery questions.

  4. Evidence and Citation Strength = the weighted mean over assessed questions of an evidence score, where an evidence score is 100 × Σ a·r / Σ 4a over the applicable attributes (r the 0 to 4 rating, a the attribute weight; directness counts double). evidence-rubric.md defines the ratings. Independence and corroboration are required for reputation, category-membership, outcome, expertise, and comparative claims, and are left out for claims the subject’s own site can authoritatively state (leadership, service offering, product documentation, publication metadata, contact and location).

  5. Cross-Engine Consistency = 100 × Σ w(q)·m(q) / Σ w(q), over unbranded questions with at least two engines. It is the weighted share of questions that even the weakest engine includes. It cannot reward uniform absence: a subject no engine includes scores 0.

  6. Entity Recognition = 100 × the mean recognition value over named-query answers (accurate 1, partly accurate 0.5, inaccurate or not found 0).

  7. Entity Clarity = 100 × passes / known checks, over the nine clarity checks.

  8. Content Addressability = the weighted mean over assessed questions of the share of the seven evidence-chain links that are intact (weak counts half, broken zero).

  9. Technical Legibility = 100 × passes / known checks, over the nine technical checks. These are enabling conditions. The score does not say any of them caused an inclusion.

Overall = Σ weight × score / 100, rounded to a whole number. Every sub-score is always shown beside it.

The score refuses to guess

  • A dimension that cannot be supported is not scored. Fewer than six known checks, fewer than half the unbranded question weight assessed, or no question feeding a pool means not scored, with the reason in confidence_notes.

  • An incomplete scorecard has no overall score. overall_score is null and the report shows how many of the 100 points were scored. An overall number from partial data would be a guess.

  • Imported counts are treated as imported counts. An aggregate run records group-level counts without answers. Groups that were never reviewed against the subject’s Finder Specification are reported as an observed mention rate and never scored, and an assumed weight is always flagged.

  • Benchmarks are not mixed. compare refuses runs scored on different benchmark or methodology versions unless --allow-version-mix is given, and then prints a disclosure.

Confidence is a set of plain notes

The scorecard carries plain-language notes: number of engines and questions, single-run cells, cells where runs disagreed, missing runs, ambiguous mentions, unscored dimensions, and assumed weights. It does not carry a confidence interval, because nothing in the method justifies one. Generative answers vary from run to run, and that variation is part of what is being measured.

Read the sub-scores, trace the weakest one, and write it as a finding

Read the sub-scores, not the total. A high Entity Recognition beside a low Problem Association is the pattern the method exists to find. Read the weakest sub-score’s trace down to the questions and the answers behind it. Then write the finding as an observation, a hypothesis, and an experiment, and say what result would show the hypothesis wrong.