Reproducibility Day

The Reproducibility Day is a tool to improve the reproducibility of on-site research projects and workflows. It has been deployed at the University Medical Center in Göttingen and the Graz University of Technology since 2021. By encouraging more and better documentation as well as distributing knowledge across all team members, the Reproducibility Day can reduce the impact of personnel fluctuations, especially prominent in academic research. Being designed as a team event, it supports social and intellectual interactions, increases internal collaborations, and the individual development of each participant.

The Reproducibility Day defines an event that takes place twice a year. Following a short introduction in the morning, all participants are assigned into groups of 2 to 3 people ideally with varying research background/perspectives. During the day, each group goes through two event blocks consisting of a “reproducibility” and a following “feedback” task. The “reproducibility” task is designed to give each group member time to reproduce their team partners work. This is done based on some pre-defined starting materials accessible to the whole research group. After working on reproducing their team partners work solely by themselves, the individual teams meet during the "feedback" task and discuss open questions, challenges, and exchange feedback about code and documentation before going into the second round of the event block in the afternoon. The event closes with a general meeting of all participant in which challenges, potential improvements about the concept of the day, and adaptions of the local reproducibility guidelines are discussed. Additional social activities following the event are generally recommended.

To run smoothly, the Reproducibility Day requires some preparation. First, a base documentation need to be defined. It is the centralized starting point for each participant to reproduce their team partners work during the "reproducibility" task. Ideally, this documentation already includes references and details about the to-be-reproduced project acting as a central landing page with all required information easily accessible. Here, solutions based on internal media wikis are recommendable, but as long as all group members have access to it and references to git repositories, archive paths, general text-based documentation, presentations, images, and videos can be stored every centralized documentation tool should be sufficient. On a daily basis, every group member is responsible for their own project landing pages and is required to keep them up-to-date. Additionally, local reproducibility and/or good-scientific-praxis guidelines are recommended. They provide guidance about the style and content of the documentation and significantly simplify the on-boarding of new group members. These guidelines can be discussed and updated with every iteration of the Reproducibility Day event.

From an individual perspective the Reproducibility Day creates a relaxed and socially comfortable atmosphere to experience and learn about the benefits and challenges of reproducible research. The participants benefit from learning about other projects, new tools, and receive feedback about their own work that strengthens social connections and internal collaborations, improves documentation, as well as the individual development of each participant.
From an institutional point-of-view, documentation is created and continuously revised by people with different perspectives. By combining this overall improvement of the documentation with the distribution of knowledge across the group, due to revisiting other group members work continuously, the loss of knowledge, especially in areas of high personnel fluctuations, can be reduced.

Resources