Use case
A funder can ask for a rerun before renewing a grant
A funder or program officer deciding whether to renew a grant usually reads the progress report and the papers. With companion repositories, the funder can also see the actual analysis pipelines, compare methods across the grants in a portfolio, and ask for a rerun of the result the renewal depends on.
Funders see the pipeline as well as the report
Claim: hypothesis
Funders already require data sharing plans, and the NIH policy in force since January 2023 asks for shared data but not for the code that analyzed it. Companion repositories add that missing piece without changing where data goes. Reviewers of a renewal can then look at software versions, preprocessing decisions and exclusion rules directly, instead of inferring them from a methods section.
What would show this claim is wrong
- Program officers say the analysis pipelines change nothing about how they judge a renewal.
A funder pays to rerun the result a renewal rests on
Claim: hypothesis
A funder could add a rerun of one key result to its renewal review, done by a contracted reproducibility team or by a service that runs the checks and the rerun and writes a report. The funder pays, because it bears the cost of renewing work that doesn't hold up. Across a portfolio, the same repositories show which grants solve the same problem twice with different pipelines.
What would show this claim is wrong
- Reruns cost more than funders will pay relative to the grants they protect.
Five renewals with a rerun show what it costs and finds
Claim: hypothesis
Pick five renewals in one program and ask each team for a companion repository of the paper the renewal leans on. Run the program's rerun protocol on each, and record how long it took and what it found.
What would show this claim is wrong
- None of the five teams can produce a companion repository that reruns within a working day.
Related
- Companion Executable Code Repositories
- A diligence team can rerun a finding before licensing it
- Check reproducibility with rules first, a model second and a person last