Confidence Intervals and Precision

#Read the estimate and the interval together

A point estimate summarizes the estimated effect. Its confidence interval shows uncertainty from the statistical procedure used. A narrow interval generally indicates greater precision than a wide interval for the same measure and confidence level.

Check the outcome, units, comparison and confidence level before comparing intervals. An interval for a difference in means is not on the same scale as one for a risk ratio. Precision says how tightly an effect has been estimated, not whether it matters to patients.

Evidence: Cochrane Handbook, Chapter 15: interpreting effects and uncertainty / American Statistical Association: statement on p-values

#What a 95% confidence interval means

The usual frequentist interpretation concerns repeated use of a method: if the study process were repeated many times and the interval calculated each time, a method with 95% coverage would contain the true effect in 95% of those intervals, under its assumptions.

It is not a statement that 95% of individual patients will fall in the interval. It should not be read as a 95% probability assigned to the true effect inside this one observed interval. The calculation depends on the statistical method and the assumptions that support it.

Evidence: Cochrane Handbook, Chapter 15: interpreting effects and uncertainty / CDC Field Epidemiology Manual: analyzing and interpreting data / Greenland and colleagues: statistical tests and confidence-interval misinterpretations

#More precision is not freedom from bias

Larger studies tend to give more precise estimates, but sample size is not the only factor. Variability in measurements, the frequency of outcomes and the number of observed events also matter. In a random-effects meta-analysis, differences among studies can widen the interval.

A narrow interval does not remove problems in study design or measurement. Cochrane considers risk of bias, inconsistency, indirectness, imprecision and publication bias separately when judging certainty. Do not use interval width as a universal quality score.

Evidence: Cochrane Handbook, Chapter 15: interpreting effects and uncertainty / Cochrane Handbook, Chapter 14: certainty of evidence

#An interval is not a yes-or-no verdict

For a difference measure, zero represents no difference. For a ratio measure, one represents equal risks or odds. If an interval includes that value, do not turn the result into proof of no effect. Ask whether the interval also includes effects large enough to matter, in either direction.

Excluding the no-effect value does not establish a clinically important result. The ASA warns that statistical significance does not measure effect size or importance. Interpret magnitude, uncertainty, study limits and the consequences of the outcome together, rather than reducing the paper to a threshold.

Evidence: Cochrane Handbook, Chapter 6: effect measures / Cochrane Handbook, Chapter 15: interpreting effects and uncertainty / American Statistical Association: statement on p-values

Source note

The sections above were checked against the linked sources. No clinical review has been performed. This is general research education, not a clinical guideline.