Check reproducibility with rules first, a model second and a person last

The order of the layers decides whether the report can be trusted

Claim: hypothesis

A rule's answer can be checked: either the release has a DOI or it doesn't. A model's answer can't be checked the same way, and it can be confidently wrong. If the model goes first, every later step inherits errors nobody can see. If rules go first, the model only handles the leftovers, and its answers are labelled as a model's, with the place in the repository it read.

The first layer is the familiar machinery of software: continuous integration, which reruns code on every change; linters, which flag code that breaks style or common-sense rules; and checks of the metadata against a schema. Rules over linked metadata can be written in standard languages such as SHACL, the W3C's language for checking linked data against rules, so nobody has to invent one. The Lovable microsite named logic languages such as Prolog and Datalog for this layer; any of them would do, and the choice should follow the check, not come before it.

The second layer is a model that reads what rules can't: a README that never names its entry point, a variable named one thing in the code and another in the paper, an assumption buried in a comment. The third is a person, who decides what the flags mean.

The result for a reviewer is a short report: which checks passed, which outputs the rerun reached, where it stopped, and what the model flagged and why. The reviewer's time goes to the science instead of the plumbing.