Journal article · 2024 · Open access
A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)
Environment International, 186, 108602 · Published 24 March 2024 · DOI 10.1016/j.envint.2024.108602
In brief
“A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)” is a 2024 journal article by Julian P. T. Higgins and 25 co-authors, published in Environment International. OpenAlex counts 1,173 citations as of 15 November 2026, which puts it in the top 1% of works published in 2024 by citations. It is open access under a CC BY-NC-ND licence: the full text is free to read at Environment International.
- Citations
- 1,173
- Most cited in
- 2026 (591)
- Field-weighted impact
- 134.09
- References
- 25
- Authors
- 26
Abstract
BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Higgins, J. P. T., Morgan, R. L., Rooney, A. A., Taylor, K. W., Thayer, K. A., Silva, R. A., Lemeris, C., Akl, E. A., Bateson, T. F., Berkman, N. D., Glenn, B. S., Hróbjartsson, A., LaKind, J. S., McAleenan, A., Meerpohl, J. J., Nachman, R. M., Obbagy, J., O'Connor, A. M. C., Radke, E. G., . . . Sterne, J. A. C. (2024). A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E). Environment International, 186, 108602. https://doi.org/10.1016/j.envint.2024.108602
MLA 9
Higgins, Julian P. T., et al. “A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E).” Environment International, vol. 186, 2024, p. 108602. https://doi.org/10.1016/j.envint.2024.108602.
Chicago
Higgins, Julian P. T., Rebecca L. Morgan, Andrew A. Rooney, Kyla W. Taylor, Kristina A. Thayer, Raquel A. Silva, Courtney Lemeris, et al. 2024. “A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E).” Environment International 186: 108602. https://doi.org/10.1016/j.envint.2024.108602.
Harvard
Higgins, J. P. T. et al. (2024) ‘A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)’, Environment International, 186, p. 108602. doi:10.1016/j.envint.2024.108602.
IEEE
J. P. T. Higgins et al., “A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E),” Environment International, vol. 186, p. 108602, 2024, doi: 10.1016/j.envint.2024.108602.
BibTeX
@article{higgins2024tool,
title = {{A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)}},
author = {Higgins, Julian P. T. and Morgan, Rebecca L. and Rooney, Andrew A. and Taylor, Kyla W. and Thayer, Kristina A. and Silva, Raquel A. and Lemeris, Courtney and Akl, Elie A. and Bateson, Thomas F. and Berkman, Nancy D and Glenn, Barbara S. and Hróbjartsson, Asbjørn and LaKind, Judy S. and McAleenan, Alexandra and Meerpohl, Joerg J and Nachman, Rebecca Massa and Obbagy, Julie and O'Connor, Annette M. Cormier and Radke, Elizabeth G. and Savović, Jelena and Schünemann, Holger Jens and Shea, Beverley and Tilling, Kate M. and Verbeek, Jos H. A. M. and Viswanathan, Meera and Sterne, Jonathan A C},
journal = {Environment International},
year = {2024},
volume = {186},
pages = {108602},
doi = {10.1016/j.envint.2024.108602},
url = {https://doi.org/10.1016/j.envint.2024.108602}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2024 | 119 | |
| 2025 | 463 | |
| 2026 | 591 |
Its field-weighted citation impact is 134.09: it has been cited 134.09 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- risk of bias assessment
- non-randomized studies
- cohort studies
- causal inference
- systematic review methodology
- observational epidemiology
Work with this paper
Most-cited papers in Decision Sciences
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsMatthew James Page, Joanne E. McKenzie, Patrick M. Bossuyt and 23 more · BMJ · 2021 · 144,919 citations
- Evaluating Structural Equation Models with Unobservable Variables and Measurement ErrorClaes Fornell, David F. Larcker · Journal of Marketing Research · 1981 · 118,336 citations
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesLi‐tze Hu, Peter M. Bentler · Structural Equation Modeling A Multidisciplinary Journal · 1999 · 117,461 citations
- Common method biases in behavioral research: A critical review of the literature and recommended remedies.Philip M. Podsakoff, SCOTT B. MACKENZIE, Jeong Yeon Lee and 1 more · Journal of Applied Psychology · 2003 · 89,248 citations
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statementDavid Moher, Alessandro Liberati, Jennifer Tetzlaff and 2 more · BMJ · 2009 · 85,524 citations
Questions about this paper
How do I cite this paper in APA style?
Higgins, J. P. T., Morgan, R. L., Rooney, A. A., Taylor, K. W., Thayer, K. A., Silva, R. A., Lemeris, C., Akl, E. A., Bateson, T. F., Berkman, N. D., Glenn, B. S., Hróbjartsson, A., LaKind, J. S., McAleenan, A., Meerpohl, J. J., Nachman, R. M., Obbagy, J., O'Connor, A. M. C., Radke, E. G., . . . Sterne, J. A. C. (2024). A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E). Environment International, 186, 108602. https://doi.org/10.1016/j.envint.2024.108602
How many times has this paper been cited?
1,173 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at Environment International.
Who wrote this paper?
Julian P. T. Higgins, Rebecca L. Morgan, Andrew A. Rooney, Kyla W. Taylor, Kristina A. Thayer, Raquel A. Silva, Courtney Lemeris, Elie A. Akl, Thomas F. Bateson, Nancy D Berkman, Barbara S. Glenn, Asbjørn Hróbjartsson and 14 more.
What is the DOI of this paper?
10.1016/j.envint.2024.108602. The permanent link is https://doi.org/10.1016/j.envint.2024.108602.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.