Hypothesis

Letting participants physically draw socks from a bag (a 'first-order', interactive environment) rather than only imagining it would help them discard the misleading ratio information in the socks insight problem.

What failed

Participants randomly assigned to the high-interactivity condition solved the problem correctly within five minutes only marginally more often than the low-interactivity group, and the difference was not significant.

Broken assumption

Physical interaction with the problem materials reliably frees solvers from a misleading framing.

Why

Video analysis showed the second-order condition was hard to realise — participants engaged in dialogue with the experimenter — and that the high-interactivity setup physically reified the misleading ratio, anchoring it rather than defusing it.

From the published paper: Ross et al. Journal of Trial and Error, 2021. ↗CC BY

Hypothesis

Households practising urban or peri-urban agriculture are more food secure than households that do not.

What failed

Among 208 stratified-sampled households in Tongaat, 54% of practising households consumed more than six food groups against 40% of non-practising ones, but a Mann-Whitney test found no significant difference in dietary diversity between the two groups (U = 5292, p = 0.808). Employment status, arable land access, tenure and household income were the variables significantly associated with food security.

Broken assumption

Growing food at home measurably improves a household’s dietary diversity.

Why

The authors report the difference in proportions descriptively and offer no explanation for the non-significant comparison.

From the published paper: Khumalo et al. Sustainability, 2019. ↗CC BY

Hypothesis

An online video game teaching Dutch schoolchildren about ticks and Lyme disease would improve knowledge, risk perception and tick-checking behaviour more than a leaflet or no intervention.

What failed

In a cluster-randomised study of 887 children aged 9-13 from 25 schools, knowledge rose in all three arms including the control, with no significant difference between interventions. Tick-checking rose in the game arm and beat the leaflet, but not by more than in the control group; overall the game did not outperform the leaflet or control on all outcomes.

Broken assumption

An interactive game teaches children health-protective knowledge better than passive materials, over and above the effect of being asked about the topic.

Why

The authors attribute the knowledge gain in the control group to a measurement effect from simply completing the questionnaire.

From the published paper: Beaujean et al. BMC Public Health, 2016. ↗CC BY

Hypothesis

Fine-tuning general-purpose language models on text descriptions of polymers could match fingerprint-based methods for predicting glass transition, melting and decomposition temperatures, while skipping hand-built representations.

What failed

On a curated set of 11,740 entries, both fine-tuned LLMs generally fell short of Polymer Genome, polyGNN and polyBERT in accuracy and were less computationally efficient; multi-task training did not help the LLMs, which failed to exploit correlations between properties.

Broken assumption

Natural-language transfer learning can stand in for domain-specific chemical representations in polymer property prediction.

Why

Analysis of the learned embeddings suggests general-purpose LLMs do not capture fine chemical-structure information as well as handcrafted features or domain-specific embeddings.

From the published paper: Gupta et al. Macromolecular Rapid Communications, 2025. ↗CC BY

Hypothesis

A fourth, non-interacting 'sterile' neutrino state would explain the anomalous muon-to-electron neutrino transitions reported by LSND and MiniBooNE.

What failed

Using data from two accelerator beams in the MicroBooNE liquid-argon detector, the collaboration saw no electron-neutrino appearance or disappearance beyond the three-flavour picture and excluded the single-sterile-neutrino explanation of the LSND and MiniBooNE anomalies at 95% confidence, along with much of the parameter space for the gallium anomaly.

Broken assumption

The short-baseline neutrino anomalies are caused by oscillation into one extra light sterile neutrino.

Why

Two beams broke the degeneracy between appearance and disappearance that had weakened earlier tests; with that resolved, the data simply show no non-standard oscillation.

From the published paper: Abratenko et al. Nature, 2025. ↗CC BY

Hypothesis

Correcting vitamin D deficiency with daily supplements would lower the rate at which Graves' disease comes back after anti-thyroid drugs are stopped.

What failed

Of 210 vitamin D-deficient patients followed for at least a year after stopping anti-thyroid drugs, the 60 who took supplements relapsed at a rate not significantly different from the 150 who did not (38% vs 49%, P = 0.086), although relapse came somewhat later in the supplemented group.

Broken assumption

Raising vitamin D levels is enough to measurably prevent Graves' relapse.

Why

The authors read the difference as a borderline protective trend; the groups were self-selected rather than randomised, and vitamin D level at drug withdrawal still tracked with relapse in multivariate analysis.

From the published paper: Cho et al. Scientific Reports, 2020. ↗CC BY

Hypothesis

Adding leucine-isoleucine-valine blends to normal- or low-protein starter diets would improve broiler growth and breast yield under hot summer conditions.

What failed

In a 2×3 factorial trial with 768 Ross 308 chicks at about 33 °C, body weight, feed conversion and carcass weight were unaffected by protein level, BCAA blend or their interaction. One blend (4L:1I:1V) actually reduced breast yield versus no supplement.

Broken assumption

Supplementing branched-chain amino acids helps heat-stressed broiler chicks grow faster or yield more breast meat.

Why

The abstract gives no mechanism; it concludes the blends failed to improve performance because birds on all diets reached similar weight and feed conversion.

From the published paper: Bozbay et al. Animal Bioscience, 2020. ↗CC BY

Hypothesis

Assimilating tree-ring-width chronologies 'online' into an atmospheric climate model would outperform the simpler 'offline' approach in reconstructing past climate.

What failed

In perfect-model experiments with pseudo tree-ring data, the online approach did not outperform the offline one despite its considerable extra implementation complexity.

Broken assumption

Cycling proxy information back into the model (online assimilation) improves paleoclimate reconstructions from tree rings.

Why

The paper reports that tree growth's nonlinear response to temperature and soil moisture degrades the time-averaged EnKF, that the skill loss is sensitive to the growth-rate function used for limiting factors, and that error reduction is modulated by the model's yearly internal variability.

From the published paper: Acevedo et al. Climate of the past, 2017. ↗CC BY

Hypothesis

MERTK in endothelial cells maintains barrier function and restrains neutrophil transendothelial migration, so deleting it specifically in endothelium should aggravate inflammatory lung injury.

What failed

Knocking MERTK down in primary human pulmonary microvascular endothelial cells increased neutrophil transmigration and dextran leak and disrupted adherens junctions and basal Rac1 activity, and global Mertk-null mice recruited more neutrophils and leaked more Evans blue during Pseudomonas aeruginosa pneumonia. But inducible endothelium-specific Mertk-null mice showed no difference from controls in either neutrophil migration into inflamed lungs or vascular permeability.

Broken assumption

A global knockout phenotype localises to the cell type that reproduces it in culture.

Why

The authors report that selective deletion in endothelial cells in vivo failed to reproduce the response seen in culture and in the global knockout, without identifying the compensating cell type.

From the published paper: Li et al. PLoS ONE, 2019. ↗CC BY