Systematic Reviews and Meta-analysis

#Start with the review question

A systematic review seeks research that meets explicit eligibility criteria for a defined question. Read how the authors searched, selected studies, collected information and assessed risk of bias. These steps matter before any statistical combination is attempted.

PRISMA 2020 asks for a transparent account of why the review was done, what was done and what was found. It also asks authors to explain protocol amendments and synthesis methods. PRISMA is reporting guidance, not a certificate of sound methods or a tool for rating the quality of a review.

Evidence: PRISMA 2020: reporting systematic reviews / Cochrane Handbook, Chapter 10: meta-analysis and heterogeneity

#Meta-analysis is a choice, not a requirement

Meta-analysis combines numerical results from separate studies, commonly as a weighted average of effect estimates. It can improve precision and help explore differences, but a calculation does not make the evidence valid by itself. The studies need to provide a meaningful answer to the comparison being summarized.

A systematic review need not contain a meta-analysis. Limited evidence, incomplete results or incompatible measures may prevent one. Cochrane also warns against abandoning quantitative analysis solely because studies vary: the question is whether an appropriate analysis and interpretation are possible, not whether every study is identical.

Evidence: Cochrane Handbook, Chapter 10: meta-analysis and heterogeneity / Cochrane Handbook, Chapter 12: synthesis without meta-analysis

#An average can hide differences

Differences in participants, interventions and outcomes are clinical diversity; differences in design and measurement are methodological diversity. Statistical heterogeneity is variation in effect estimates beyond what would be expected from chance alone. These are related ideas, not interchangeable labels.

A random-effects analysis allows for variation in underlying effects and estimates a mean across studies. It does not explain the variation or remove bias. A pooled estimate can be misleading when results differ substantially, especially when effects point in different directions. Read the individual estimates and how heterogeneity was handled.

Evidence: Cochrane Handbook, Chapter 10: meta-analysis and heterogeneity

#Missing evidence and certainty matter

Eligible results may be unavailable because of their size, direction or statistical significance. Cochrane calls this bias due to missing evidence. A review of only the available results can give a distorted account, even when its calculations are correct.

For each outcome, GRADE considers risk of bias, inconsistency, indirectness, imprecision and publication bias when judging certainty. Read the reasons for the judgement, not just its label. Sensitivity analyses can test whether findings change under alternative decisions, but they do not prove that every uncertainty has gone away. This page explains these questions; it does not assign a certainty rating to a treatment.

Evidence: Cochrane Handbook, Chapter 13: missing evidence / Cochrane Handbook, Chapter 14: certainty of evidence / Cochrane Handbook, Chapter 10: meta-analysis and heterogeneity

Source note

The sections above were checked against the linked sources. No clinical review has been performed. This is general research education, not a clinical guideline.