Hypothesis

Higher-quality feedback from generative AI, especially with chain-of-thought prompting, would lead graduate students to make better essay revisions than teacher feedback.

What failed

In a randomised three-group experiment with 70 graduate students, chain-of-thought feedback was rated higher in quality than both zero-shot and teacher feedback, yet it did not produce significantly greater improvement in revised essays; teacher feedback gave comparable gains.

Broken assumption

Better feedback content translates directly into better revisions.

Why

The authors conclude that feedback quality alone is not enough: how students engage with and take up the feedback determines the effect on writing.

From the published paper: Farrokhnia et al. International Journal of Educational Technology in Higher Education, 2026. ↗CC BY

Hypothesis

Food availability and weather should interact, so that supplementary feeding buffers breeding red kites against adverse weather.

What failed

A feeding experiment run across four years with contrasting weather found no support for interactive effects of weather and food availability on any reproductive parameter; brood loss, nestling survival and development responded to rainfall, low temperature and food supply separately and additively.

Broken assumption

Extra food lets breeding birds compensate for bad weather.

Why

The authors reason that parental life-history trade-offs prevent an interaction, leaving weather and food as crucial but separate determinants of productivity.

From the published paper: Nägeli et al. Oecologia, 2021. ↗CC BY

Hypothesis

The symmetric steps people adopt when adapting to a split-belt treadmill are the energetically cheapest gait, consistent with the idea that the motor system minimises energy cost.

What failed

Across a range of imposed step asymmetries, equal step lengths minimised neither metabolic nor mechanical cost, and participants' perceived effort did not track asymmetry. Symmetry came out optimal only for a composite metric combining metabolic and mechanical cost.

Broken assumption

Locomotor adaptation converges on the gait that minimises metabolic energy expenditure.

Why

The authors suggest adaptation may optimise a combined estimate of effort derived from body-environment feedback rather than metabolic energy alone.

From the published paper: Sánchez et al. Scientific Reports, 2017. ↗CC BY

Hypothesis

Removing aggregated amyloid beta with aducanumab would slow decline on the Clinical Dementia Rating Sum of Boxes in early Alzheimer's disease.

What failed

Both phase 3 trials were halted for futility on pooled interim data. In the fuller analyses, EMERGE met the primary endpoint (−0.39 for high dose vs placebo, P = .012) but ENGAGE did not (+0.03, P = .833), with 3,285 patients across 348 sites.

Broken assumption

Reducing amyloid burden translates into slower clinical decline — and does so consistently across identically designed trials.

Why

Biomarker substudies confirmed target engagement and dose-dependent reduction of Alzheimer's pathology markers in both trials, so the discordant clinical results came despite the drug doing what it was designed to do; amyloid-related imaging abnormalities with edema were the most common adverse event.

From the published paper: Haeberlein et al. The Journal of Prevention of Alzheimer s Disease, 2022. ↗CC BY

Hypothesis

Seeing a small number would speed detection of left-side targets and a large number right-side targets, as Fischer et al. (2003, Experiment 2) reported.

What failed

In a registered replication with 1,105 participants at 17 labs and four interstimulus intervals, the effect was in the original direction only half the time (proportion .50); effects within and across labs were minuscule and incompatible with the original. Finger counting, reading direction, handedness and maths fluency or anxiety did not moderate it.

Broken assumption

Merely seeing a number shifts spatial attention along a mental number line.

Why

The authors conclude the effect failed to replicate and cannot support strong claims about a link between number and space.

From the published paper: Colling et al. Advances in Methods and Practices in Psychological Science, 2020. ↗CC BY

Hypothesis

Following a report that captive orangutans spontaneously add water to raise a floating peanut, other great apes, chimpanzees and gorillas, and other orangutan groups, would also solve this liquid-tool problem.

What failed

In the first experiment none of the chimpanzees or gorillas solved the task. In further tests with other populations, about 20% of chimpanzees succeeded but none of the orangutans did, and among children only 8% of 4-year-olds (rising to 58% of 8-year-olds) found the solution.

Broken assumption

Spontaneous use of water as a tool is a robust, general ability across great ape species and groups.

Why

For the chimpanzee population that failed outright, the authors find suggestive evidence that functional fixedness hindered their strategies.

From the published paper: Hanus et al. PLoS ONE, 2011. ↗CC BY

Hypothesis

Spironolactone, which improves outcomes in heart failure with reduced ejection fraction, would do the same in heart failure with preserved ejection fraction.

What failed

In 3,445 patients with ejection fraction of 45% or more over a mean 3.3 years, the primary composite (cardiovascular death, aborted cardiac arrest or heart-failure hospitalization) occurred in 18.6% vs 20.4% on placebo (HR 0.89, P=0.14). Hyperkalemia doubled (18.7% vs 9.1%).

Broken assumption

Mineralocorticoid-receptor blockade helps heart failure with preserved ejection fraction the way it helps reduced ejection fraction.

Why

A benefit established in reduced ejection fraction did not carry over to preserved ejection fraction on the primary endpoint; only heart-failure hospitalization, one component, fell significantly (12.0% vs 14.2%).

From the published paper: Pitt B, Pfeffer MA, Assmann SF, et al. (TOPCAT). N Engl J Med, 2014. NCT00094302 ↗

Hypothesis

Low adoption of digital payments by small merchants in Jaipur is explained by supply-side barriers such as lack of bank accounts or smartphones, high fees or insufficient literacy.

What failed

A survey of 1,003 small fixed-location merchants found little evidence for supply-side barriers: merchants could obtain the needed infrastructure, platform fees were affordable and literacy was adequate, so adoption was feasible and cheap yet still low.

Broken assumption

Making digital payments cheaper and easier to access is what holds back merchant adoption.

Why

The authors point instead to demand-side factors, such as a perceived lack of customers wanting to pay digitally, and to worries that digital records would raise tax liability.

From the published paper: Ligon et al. PLoS ONE, 2019. ↗CC BY

Hypothesis

Patients with gynecologic cancers treated at high-volume or teaching hospitals would survive longer, supporting regionalisation of care.

What failed

Among 48,981 Florida patients treated 1990–2000, univariate analysis suggested better short-term survival at high-volume centres, but multivariate analysis showed no significant survival benefit for teaching or high-volume facilities for any of the five gynecologic cancers.

Broken assumption

Higher hospital volume or teaching status improves survival in gynecologic cancer.

Why

Patients at high-volume centres were significantly younger; once age, race, stage and treatment were accounted for, the apparent advantage disappeared.

From the published paper: Brookfield et al. PLoS ONE, 2009. ↗CC BY