Intellectual Frontiers

Pattern

A companion repository takes seven steps to set up

Setting up a companion code repository takes seven steps, and most research groups can do the first five in an afternoon once the analysis is finished. The steps go from storing the data in the right place to making the code rerun on new data, and the last one, automated checking, is optional.

Seven steps take a finished analysis to a repository anyone can rerun

Claim: recommendation

Seven steps: store the data where its rules say; put the code in a repository; add metadata; link the code, paper and data with DOIs; take part in any recognition the journal or funder offers; release new versions as new data arrives; optionally, check it automatically.

The author checklist in the program's companion turns these steps into a list you can tick off before you submit.

What would show this claim is wrong

  • Groups that follow the seven steps produce repositories that fail to rerun as often as groups that don't.
  • Most groups can't finish steps 1 to 5 without help from someone outside the group.

1. Store the data where its rules say

Patient-level data goes to a governed repository such as Vivli or the YODA Project. Data with no patients in it, such as simulations, aggregates or bench measurements, can go in the companion repository itself, versioned with the code.

2. Put the code in a repository

Use GitHub, GitLab or a similar service. Include every script, notebook and spreadsheet that made a result, the exact software versions, and one set of instructions from data to results.

3. Add metadata

Add a CITATION.cff file so people cite it correctly, and a codemeta.json file that describes the software. Both are existing standards; there's no need to invent a format.

4. Link the code, the paper and the data

Archive a release on Zenodo or a similar archive to get a DOI, a permanent address. Put that DOI in the paper, and put the paper's and the data's DOIs in the repository, so each points to the other two.

5. Take part in the recognition on offer

Some journals give badges or featured placement to papers whose code reruns, and some funders give credit for it in review. Whether these raise citations in medicine isn't known, but they do raise sharing.

6. Release new versions as new data arrives

When the method runs on new data, publish a new release with its own DOI. Never change the release the paper cites.

7. Check it automatically, if you can

A continuous integration job, a program that reruns the code on every change, catches breakage early. Rules first, then a model only where rules can't answer, then a person.

Related

Next