Hypothesis

Feminization of male fish by estrogenic effluent depresses effective population size and threatens the sustainability of wild populations, as long-term exposures and collapsing breeding colonies implied.

What failed

Effective population size, estimated from microsatellite genotypes by approximate Bayesian computation and sibling assignment at 28 sample sites in English rivers, showed no significant negative correlation with predicted estrogen exposure; population genetic structure further showed that some roach populations had persisted for multiple generations in stretches with a high proportion of estrogenic effluent, apparently without immigration from cleaner sites.

Broken assumption

Reproductive failure seen in laboratory exposures and breeding colonies scales up to decline in wild populations.

Why

The authors note that the wide confidence intervals of the statistical model still leave room for a reduction of up to 65% at the most contaminated sites, so the null is a limit on the effect rather than proof of none.

From the published paper: Hamilton et al. BMC Biology, 2014. ↗CC BY

Hypothesis

Deep learning models trained on tissue slides can infer the status of individual molecular biomarkers well enough to serve as a cheap, fast substitute for molecular testing.

What failed

Across several cancer types, datasets and modelling approaches, training data showed strong dependencies between biomarkers and clinicopathological features, so models learned confounded signals: accuracy swung with the status of co-dependent biomarkers and clinical variables, and for several biomarkers the gain over what a pathologist infers from grade was modest.

Broken assumption

A model that predicts a biomarker from histology is detecting that biomarker's own morphological signature.

Why

Correlated biomarkers and clinical features in the training sets stop models isolating any single biomarker's morphological effect; the authors argue causal rather than correlational learning is needed.

From the published paper: Dawood et al. Nature Biomedical Engineering, 2026. ↗CC BY

Hypothesis

If the watching-eye effect in humans reflects a general tendency to avoid risk when observed, dogs — which act cautiously when watched — should also steal food more slowly in front of pictures of eyes.

What failed

Dogs took no longer to steal forbidden food when a picture of eyes was displayed than when a picture of flowers was.

Broken assumption

The watching-eye effect is a general animal response to being observed rather than something specific to humans.

Why

The authors take the null as evidence against the general risk-aversion explanation, leaving a human-specific reputation-management account as the alternative.

From the published paper: Neilands et al. Scientific Reports, 2020. ↗CC BY

Hypothesis

Statistical methods built specifically for sparse, compositional microbiome data would identify differentially abundant taxa better than methods borrowed from bulk and single-cell RNA-seq.

What failed

Benchmarked on 100 manually curated 16S and shotgun metagenome datasets for distributional fit, false discovery control, concordance, power and correct genus calls, the multivariate and compositional microbiome methods did not outperform univariate RNA-seq differential expression methods.

Broken assumption

Modelling compositionality and sparsity explicitly gives better differential abundance inference on real microbiome data.

Why

The abstract gives no mechanism; the authors recommend dataset-specific exploratory analysis before picking any inferential model.

From the published paper: Calgaro et al. Genome biology, 2020. ↗CC BY

Hypothesis

Correcting common vaccine misconceptions with a correction endorsed by a group of medical experts would raise COVID-19 vaccination intention and actual uptake among UK residents.

What failed

The pre-registered experiment found the intervention did not change participants' attitudes or behaviour towards COVID-19 vaccines during the 2021 vaccination campaign, contrary to a similar study with Italian residents.

Broken assumption

Expert endorsement of a correction shifts vaccine attitudes and uptake, as it appeared to in another country.

Why

The report discusses possible reasons for the null without settling on one.

From the published paper: Panizza et al. Journal of Trial and Error, 2023. ↗CC BY

Hypothesis

Two-year-olds would look in anticipation to where an agent with a false belief will act, as the widely cited 2007 eye-tracking study reported, showing implicit false-belief attribution.

What failed

A preregistered two-lab replication with 160 infants aged 24–26 months, run with two original authors and the original stimuli and analyses, found no action anticipation: only about half the infants looked to the correct location, both in the simple control condition and in the false-belief condition.

Broken assumption

Anticipatory looking in this paradigm is a reliable readout of toddlers' action predictions.

Why

Taken with other non-replications, the authors conclude the paradigm does not reliably elicit action prediction at all, so it cannot test false-belief understanding at this age.

From the published paper: Kampis et al. Royal Society Open Science, 2021. ↗CC BY

Hypothesis

Long-term voluntary running would reduce insoluble amyloid, neuroinflammation and non-cognitive behavioural problems in the 5xFAD mouse model of Alzheimer's disease.

What failed

After 24 weeks of wheel running started before heavy plaque formation (female mice, n = 30), microglia, cytokines and insoluble amyloid-beta were unchanged. Runners actually developed hindlimb clasping earlier (p = 0.009) and showed more anxiety-like and exploratory behaviour.

Broken assumption

Exercise slows Alzheimer's-type pathology once the disease process is under way.

Why

The abstract offers no mechanism; it follows the same group's earlier null results for soluble amyloid, synaptic proteins and cognition, and the authors conclude running is not a disease-halting intervention in this model.

From the published paper: Svensson et al. Scientific Reports, 2020. ↗CC BY

Hypothesis

Corticosteroid treatment during hospital stay would change in-hospital mortality among COVID-19 patients.

What failed

In a single-hospital cohort of 1,444 RT-PCR-confirmed patients, 559 received corticosteroids. Crude mortality was higher in the treated group (30.6% vs 21.7%), but after overlap-weight propensity balancing there was no association with in-hospital death (adjusted p = 0.25).

Broken assumption

Corticosteroid exposure changes the chance of dying in hospital with COVID-19.

Why

Once confounders were balanced the mortality difference disappeared; the only adjusted benefit the authors found was a lower risk of ICU admission. It is an observational cohort, not a trial.

From the published paper: Albani et al. Scientific Reports, 2021. ↗CC BY

Hypothesis

Since traffic-related air pollution has been linked to worse cognition, long-term traffic exposure and short-term rises in pollutants would also be associated with depressive symptoms in older adults.

What failed

Among 732 Boston-area adults aged 65 and over, neither distance to a major road nor two-week averages of PM2.5, sulfates, black carbon, ultrafine particles or gaseous pollutants showed a positive association with a CESD-R score of 16 or above; for PM2.5 the odds ratio per IQR was 0.67 (95% CI 0.46-0.98).

Broken assumption

Pollution exposures harmful to cognition at these levels also raise depressive symptoms.

Why

The abstract offers no mechanism; the authors note the study area met current US air quality standards.

From the published paper: Wang et al. Environmental Health Perspectives, 2014. ↗Public domain