International Register of Meta-Research (IRMR)
-
Principal Investigator
Dr Georgina Imberger
Project Details
IRMR (International Register of Meta-Research) will be an open-access database of prospectively registered studies in meta-research in the setting of healthcare.
Meta-research is defined as ‘the study of research itself: its methods, reporting, reproducibility, evaluation and incentives.’1 As research on research, meta-research uses scientific methods to explore how research is done and interpreted, with the goal of verifying the validity of research techniques and improving the reliability of conclusions.
The Meta-Research Innovation Center at Stanford (METRICS) has defined five focal areas as follows:2
Methods – The phase of designing and conducting research
Rigorous research methods are at the heart of good science. These methods include everything from question formation to study design, conduct, statistical analysis and interpretation. Meta-researchers use both theoretical and empirical approaches to identify flaws and biases in the research enterprise, then develop and test methods to minimize them and define best practices.
Reporting – The phase of communicating research
For research to be useful it needs to be communicated properly to scientists and others who use the results. This reporting is done through research papers, written by scientists, or by various forms of media that convey scientific results to broader audiences. Meta-researchers are developing standards for reporting results and experimenting with new means to improve communication to scientists, policy makers, patients, and the general public.
Evaluation – The phase of evaluating research
Scientific quality is constantly evaluated in journal publications, funding decisions, academic promotions, industry investment, medical practice guidelines, standard setting, and policy decisions. Peer review has long served as the cornerstone of such evaluations, but it is widely acknowledged to be imperfect and inefficient. Meta-researchers across multiple disciplines are assessing its effectiveness and experimenting with new approaches to improve the assessment of scientific quality.
Reproducibility – The phase of verifying research
Replicating research results is central to scientific credibility. Meta-research has repeatedly shown that accurate reporting and sound peer-review do not by themselves guarantee the reproducibility of a scientific claim and that active replications of previous research are too rarely done. Meta-researchers are developing new ways to assess the reproducibility of findings, make experimental data available, and promote policies that can make verifying research more routine and more effective.
Incentives - The phase of rewarding research
Research is a rewarding activity in itself, but funding, recognition, and career advancement are critical to a successful scientific career. If the scientific reward system is not aligned with the adoption of best scientific practices, such as study replication and publishing negative results, such practices will happen rarely. Changing the criteria and processes by which rewards are distributed in science can help ensure that the most reliable research is encouraged.
Goals
The key goals of IRMR are:
- To provide free access to submit records to a database of prospective meta-research protocols in order to reduce bias in meta-research.
- To provide free access to a database of protocols and subsequent findings to aid accessibility and communication of meta-research.
- To encourage transparency, consistency and collaboration when conducting and reporting meta-research.
1. Ioannidis JPA (2018) Meta-research: Why research on research matters. PLoS Biol 16(3): e2005468. https://doi.org/10.1371/journal.pbio.2005468
2. https://metrics.stanford.edu
Follow IRMR on twitter @IRMR_tweets
Researchers
Georgina Imberger
Martin Nguyen
Simon Chong
David Bramley
Elizabeth Hessian
David Story
Anna Parker
Collaborators
Western Health