Evidence Redundancy in Umbrella Reviews: Epistemological Implications of Overlap
DOI:
https://doi.org/10.68178/4c61hj31Abstract
Introduction: Umbrella reviews synthesize multiple systematic reviews, but repetition of the same primary studies can create an appearance of accumulated evidence that does not represent independent information. Objective: To analyze the epistemological implications of overlap among systematic reviews included in umbrella reviews and to propose an interpretive framework that distinguishes redundancy from independent convergence. Methods: A metascientific narrative review of methodological studies, guidance, and tools related to overlap, corrected covered area, review appraisal, and evidence synthesis was conducted. The analysis focused on information independence, precision, concordance, and inferential credibility. Results: Three levels of redundancy were distinguished - bibliographic, informational, and inferential - and the main strategies for identifying and managing overlap were systematized. A conceptual overlap-concordance matrix is also proposed, differentiating relatively independent convergence, concordant repetition, methodological divergence, and genuine heterogeneity. Conclusions: A larger number of reviews does not necessarily imply a larger amount of independent evidence. Overlap should be interpreted as a dependency property between syntheses rather than as an isolated marker of quality. Explicit assessment can prevent an apparent multiplication of certainty and improve interpretation of umbrella reviews.
Downloads
References
1. Smith V, Devane D, Begley CM, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11:15. doi:10.1186/1471-2288-11-15.
2. Hartling L, Chisholm A, Thomson D, Dryden DM. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012;7(11):e49667. doi:10.1371/journal.pone.0049667.
3. Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485-514. doi:10.1111/1468-0009.12210.
4. Pieper D, Antoine SL, Mathes T, Neugebauer EAM, Eikermann M. Systematic review finds overlapping reviews were not mentioned in every other overview. J Clin Epidemiol. 2014;67(4):368-375. doi:10.1016/j.jclinepi.2013.11.007.
5. Ballard M, Montgomery P. Risk of bias in overviews of reviews: a scoping review of methodological guidance and four-item checklist. Res Synth Methods. 2017;8(1):92-108. doi:10.1002/jrsm.1229.
6. Fusar-Poli P, Radua J. Ten simple rules for conducting umbrella reviews. Evid Based Ment Health. 2018;21(3):95-100. doi:10.1136/ebmental-2018-300014.
7. Lunny C, Pieper D, Thabet P, Kanji S. Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews. BMC Med Res Methodol. 2021;21(1):140. doi:10.1186/s12874-021-01269-y.
8. Pollock M, Fernandes RM, Becker LA, Pieper D, Hartling L. Chapter V: Overviews of Reviews. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5. Cochrane; 2024.
9. Gates M, Gates A, Pieper D, Fernandes RM, Tricco AC, Moher D, et al. Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. BMJ. 2022;378:e070849. doi:10.1136/bmj-2022-070849.
10. Bracchiglione J, Meza N, Bangdiwala SI, Niño de Guzmán E, Urrútia G, Bonfill X, Madrid E. Graphical Representation of Overlap for OVErviews: GROOVE tool. Res Synth Methods. 2022;13(3):381-388. doi:10.1002/jrsm.1557.
11. Bougioukas KI, Diakonidis T, Mavromanoli AC, Haidich AB. ccaR: a package for assessing primary study overlap across systematic reviews in overviews. Res Synth Methods. 2023;14(3):443-454. doi:10.1002/jrsm.1610.
12. Bracchiglione J, Meza N, Pérez-Carrasco I, Vergara-Merino L, Madrid E, Urrútia G, Bonfill Cosp X. A methodological review finds mismatch between overall and pairwise overlap analysis in a sample of overviews. J Clin Epidemiol. 2023;159:31-39. doi:10.1016/j.jclinepi.2023.05.006.
13. Kirvalidze M, Abbadi A, Dahlberg L, Sacco LB, Calderón-Larrañaga A, Morin L. Estimating pairwise overlap in umbrella reviews: considerations for using the corrected covered area (CCA) index methodology. Res Synth Methods. 2023;14(5):764-767. doi:10.1002/jrsm.1658.
14. Ying X, Bougioukas KI, Pieper D, Mayo-Wilson E. Weighted corrected covered area (wCCA): a measure of informational overlap among reviews. Res Synth Methods. 2025;16(4):701-708. doi:10.1017/rsm.2025.19.
15. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. doi:10.1136/bmj.j4008.
16. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71.
17. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383-394. doi:10.1016/j.jclinepi.2010.04.026.
18. Johnson BT, Hennessy EA. Systematic reviews and meta-analyses in the health sciences: best practice methods for research syntheses. Soc Sci Med. 2019;233:237-251. doi:10.1016/j.socscimed.2019.05.035.
19. Aromataris E, Fernandez R, Godfrey C, Holly C, Khalil H, Tungpunkom P. Umbrella Reviews. In: Aromataris E, Lockwood C, Porritt K, Pilla B, Jordan Z, editors. JBI Manual for Evidence Synthesis. JBI; 2024. doi:10.46658/JBIMES-24-08.
20. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86-89. doi:10.1016/S0140-6736(09)60329-9.
