Benefits and Challenges of Reproducibility

Last updated on 2023-11-01 | Edit this page

Overview

Questions

  • How can science benefit from more reproducible research?
  • How can students and researchers benefit from more reproducible research?
  • What are the main challenges of making research more reproducible?

Objectives

  • list at least 4 benefits of reproducible research
  • link reproducible research to research integrity/ethical research principles (?)
  • list at least 4 challenges of reproducible research
  • provide one example for how researchers can be helped to overcome some of the challenges

Reproducibility benefits for science

Reproducible research is often discussed in the context of research improvement, because making studies more reproducible means that:

  1. They are easier to verify (and catch errors and mistakes!)
  2. They are more likely to be correct (also because there is higher chance of catching any issues)
  3. They are more understandable for others and reusable (since making a study reproducible entails proper organization and documentation of research process)
  4. They are better prepared to share and make open (if no privacy or copyright restrictions apply)

Reproducibility benefits for researchers

But reproducibility also has particular benefits for researchers themselves, and not only for science more broadly. That’s because making studies more reproducible means that researchers:

  1. Are more efficient (although at first implementing reproducible workflows might be time-consuming, it makes them more efficient down the road!)
  2. Are less stressed about making a mistake (because both they and other researchers can check if the study reproduces across different contexts)
  3. Can get credit for producing rigorous research outputs (according to new research assessment criteria that follow global science reform)

Discussion

Think about one way in which more reproducible research could benefit science and one in which it could benefit researchers themselves. Why are these two benefits important?

Challenges in making research more reproducible

We learned that they are many benefits of making research more reproducible. However, this does not come without specific challenges:

  1. Making research reproducible is time-consuming (especially at first when new workflows are being implemented or research documentation is created from scratch)
  2. It requires skills and expertise (for example, researchers might need to know how to properly organize and document research outputs)
  3. It is more difficult when specific restrictions apply (in cases when due to privacy or copyright restrictions, critical parts of dataset or analysis cannot be made available for independent reproduction by other researchers)
  4. Research might not reproduce due to technical issues (for example, different analysis software might differ in how they perform certain calculations and produce different outputs for the same analysis)

Overcoming challenges

These specific challenges can be overcome, but they require that researchers have proper conditions and support for making studies more reproducible:

  1. Because making research reproducible is time-consuming, researchers should be rewarded for preparing reproducible outputs and have appropriate support
  2. Because it requires skills and expertise, institutions should offer training about the tools and solutions for reproducible research and/or trained support staff
  3. Because is more difficult when specific sharing restrictions apply, reproducibility should be checked by internal staff and/or proper infrastructure and tools with controlled access to research outputs should be in place
  4. Because research might not reproduce due to technical issues, software documentation should be provided and results could be checked across different types of software and operating systems

Discussion

Name one challenge of making research more reproducible.
Discuss with the person next to you your choice and brainstorm ways in which you could help researchers overcome that challenge as a librarian.

Key Points

  • Making research more reproducible contributes to general research improvement, quality and rigor but also to higher efficiency and easier error correction for researchers
  • This does not come without specific challenges, such as time constraints, technical issues and the need of specific skills for making research outputs reproducible
  • To overcome the challenges, researchers are in need of proper tools, solutions, research support staff, infrastructure and training