Use case
A journal can give reviewers a rerun report with each submission
A journal editor can ask authors to submit a companion repository with the paper, run automated checks and a rerun before review, and hand reviewers a short report: what ran, what matched the paper's numbers, and what was flagged. Reviewers spend their time on the science, and the journal can mark papers whose results rerun.
Reviewers can't rebuild analyses, so a report does it first
Claim: hypothesis
Reviewers rarely have the time or the data access to rebuild an analysis, so they judge whether the methods sound right. Reporting guidelines such as CONSORT 2025 now ask where the statistical code can be found, but nobody runs it. A rerun report closes that gap with the least reviewer effort: rules check the repository first, a model reads what rules can't, and the reviewer judges what's flagged.
What would show this claim is wrong
- Reviewers given the report raise the same methods questions, in the same number of rounds, as reviewers without it.
A journal adds a rerun report and a badge to submission
Claim: hypothesis
A journal could offer the report as part of submission, run in-house or by a contracted service, with a badge for papers whose results rerun and featured placement for them. The journal pays from its production budget, and may recover it in faster reviews with fewer rounds about methods. Machine-readable metadata from the companion repository also makes the paper easier to index and find.
What would show this claim is wrong
- Authors submit elsewhere rather than provide a companion repository, so the journal loses papers it wants.
One journal section tries rerun reports for three months
Claim: hypothesis
Pick one section of the journal and, for three months, ask authors to include a companion repository. Run the rerun protocol on each, give the report to the reviewers, and ask them whether it changed what they wrote.
What would show this claim is wrong
- Most submitted repositories fail the mechanical checks, so the reports only say that nothing could be rerun.
Related
- Companion Executable Code Repositories
- Check reproducibility with rules first, a model second and a person last
- A research group can answer reviewers by pointing to code that runs