Hypothesis

Part of the known effect of cosmetics on how others treat a woman comes from her own change in behaviour, so a solicitor who only believes she wears cosmetics would also receive more donations.

What failed

Men's donation rates were highest with real cosmetics, lower with placebo cosmetics and lowest with none, but the effect of condition was not statistically significant.

Broken assumption

The wearer's belief that she wears cosmetics changes her behaviour enough to change how strangers respond to her.

Why

The study used one solicitor and one outcome measure, which limits its power and how far it generalises.

From the published paper: Batres et al. PLoS ONE, 2019. ↗CC BY

Hypothesis

People judge purity violations with less regard for intent than harm violations because strange purity scenarios distract attention; so people with better attention, measured by the Attention Network Task, would not show this difference.

What failed

Attentional efficiency did not moderate the effect. People with high and low attentional efficiency showed the same reduced weight of innocent intent for purity violations.

Broken assumption

The weaker role of intent in purity judgments comes from people failing to attend to the intent information.

Why

The result fits the view that harm and purity judgments come from separate moral systems, not from a failure to notice the intent information.

From the published paper: Sweetman et al. PLoS ONE, 2020. ↗CC BY

Hypothesis

VCP, a complement inhibitor that protects nerve tissue in brain and spinal cord injury models, would protect the rat retina from light-induced degeneration.

What failed

In light-damaged retinas, VCP did not reduce photoreceptor loss, retinal stress or inflammation compared with saline. Injected VCP spread through the retina but was cleared within 1 to 3 hours.

Broken assumption

A single eye injection of a complement inhibitor stays in the retina long enough to protect it.

Why

VCP did not stay in the retina long enough to act, so a slow-release delivery method is needed before the idea can be properly tested.

From the published paper: Fernando et al. PLoS ONE, 2018. ↗CC BY

Hypothesis

Using up self-control on one task (crossing out letters) would weaken self-control on a following task, as the limited-resource model predicts.

What failed

Four studies, two with community samples and two with young adults, used handgrip persistence and a modified Stroop task before and after depletion. None found a significant depletion effect.

Broken assumption

Self-control draws on a limited resource that one demanding task can use up.

Why

The authors suggest that depletion has smaller or more limited effects than earlier publications implied.

From the published paper: Xu et al. PLoS ONE, 2014. ↗CC BY

Hypothesis

Cells with trisomy 21 would show Gene Expression Dysregulated Domains, alternating chromosome segments of higher and lower expression, as Letourneau et al. reported in twins discordant for Down syndrome.

What failed

Reanalysis of the original group's stem cell RNA-seq data, and of earlier data from the same twins, did not find these domains. Analysis of Ts65Dn mouse fibroblasts also did not confirm them.

Broken assumption

An extra chromosome 21 causes a genome-wide domain pattern of gene expression changes.

Why

The abstract does not give a cause. The authors question whether the domains exist.

From the published paper: Do et al. F1000Research, 2015. ↗CC BY

Hypothesis

With a modified version of the Geschke et al. (2007) design, the stronger impact of concrete descriptions of belief-inconsistent behaviour would shrink or reverse.

What failed

The difference between concrete and abstract descriptions was largely absent, so the study supported neither the original finding nor the linguistic bias account.

Broken assumption

Concrete descriptions of belief-inconsistent behaviour reliably have a stronger effect than abstract ones.

Why

The authors call the result inconclusive and do not identify a cause.

From the published paper: Collins et al. PLoS ONE, 2018. ↗CC BY

Hypothesis

Training a classifier directly on a hierarchical loss, which penalises confusing a dog with a skyscraper more than confusing two dog breeds, would reduce serious mistakes more than standard training.

What failed

In extensive experiments, standard cross-entropy training reduced the hierarchical loss almost as much as training on the hierarchical loss directly.

Broken assumption

Optimising a hierarchy-aware loss directly is a reliable way to make a classifier's mistakes less severe.

Why

Plain stochastic gradient descent from random starting points already lowers the hierarchical loss as a side effect, so the new objective gives little extra benefit.

From the published paper: Wu et al. PLoS ONE, 2019. ↗CC BY

Hypothesis

Monthly participatory women's health groups, which reduced postpartum psychological distress in eastern India, would do the same in rural Bangladesh.

What failed

In a cluster-randomised trial with 18 clusters and 6,275 mothers, distress scores were 5.2 in the intervention arm and 5.3 in the control arm, a difference that was not significant.

Broken assumption

A community group intervention that reduced postpartum distress in one South Asian setting will work in another.

Why

The abstract does not give a cause. A result from India did not carry over to a similar setting in Bangladesh. This is a secondary analysis of a trial designed for other outcomes.

From the published paper: Rose‐Clarke et al. PLoS ONE, 2014. ↗CC BY

Hypothesis

Children with autism and bowel problems would carry measles virus RNA in their bowel tissue more often than children with bowel problems alone, as a 1998 report suggested.

What failed

In bowel tissue from 25 children with autism and bowel problems and 13 controls, tested blind in three laboratories (including the original one), measles virus RNA did not differ between groups. Onset of autism and bowel symptoms was not related to the timing of the MMR vaccine.

Broken assumption

Persistent measles virus from the MMR vaccine in the bowel is linked to autism.

Why

The original finding did not reproduce when tested with the same design in blinded laboratories.

From the published paper: Hornig et al. PLoS ONE, 2008. ↗Public domain