failed_experiment.db
← Back to browse

Decision Sciences› Statistics, Probability and Uncertainty› Meta-analysis and systematic reviews

Evidence synthesis methods — Adjusted indirect comparison of treatments not tested head-to-head

Submitted 2026-10-22

Field: Decision Sciences › Statistics, Probability and UncertaintyType: A method hit a wallArea: evidence synthesis methodsTarget: Adjusted indirect comparison of treatments not tested head-to-headStage: full study

Hypothesis

Adjusted indirect comparisons, increasingly used to compare treatments never tested against each other, give reliable confidence intervals and hypothesis tests.

What failed

In simulations, indirect confidence intervals suffered under-coverage and indirect hypothesis tests had low power whenever between-study heterogeneity was moderate to large; the risk of overestimation was large when one of the two direct comparisons rested on a single trial.

Broken assumption

Indirect comparisons support confidence intervals and significance tests as dependable as direct ones.

Why

The authors trace the fragility to between-study heterogeneity and to unbalanced comparisons, where one side rests on little evidence.

From the published paper: Mills et al. PLoS ONE, 2011. ↗CC BY

Research ideas from this failure

Unvetted lead — read the papers

New hypotheses this failure suggests, each with a crucial experiment, a rival explanation of the original result, and the papers it rests on. Written by AI from paper titles and abstracts found on OpenAlex (and Europe PMC, for life and health sciences).

Related failures

Nothing else logged looks close to this one yet.

Links in published research

Where this experiment's genes are known to matter outside its own research area, and the papers behind those links. Sources: Open Targets Platform and Europe PMC.