Hypothesis

Five classic Hess studies from the 1960s-70s, reporting that pupil size tracks interest value, arousal and mental effort, including gender differences and mimicry of seen pupils, would replicate with modern eye-tracking and luminance control.

What failed

Across two experiments (N = 182 and a preregistered N = 147), the predicted gender differences in pupil change (for stimuli of differing interest and for a mob-escape video) did not appear, nor did pupil mimicry of schematic eye pairs. Effects of mental effort and arousing words did hold up.

Broken assumption

Hess's pupil effects reflected interest and gender rather than uncontrolled luminance and small-sample noise.

Why

The authors point to Hess's small samples and undocumented luminance control; in their data pupil change correlated strongly with the luminance of where participants looked.

From the published paper: Winter et al. International Journal of Psychophysiology, 2021. ↗CC BY

Hypothesis

Highly math-anxious people have a less precise mental representation of numerical magnitude, shown by larger distance effects in number comparison, as earlier studies including Dietrich et al. reported.

What failed

High- and low-math-anxious participants did not differ in non-symbolic comparison, the earlier finding of larger distance effects in Arabic-digit comparison did not replicate, and there were no group differences in a counting Stroop task.

Broken assumption

Math anxiety stems from a less precise underlying representation of numerical magnitude.

Why

The abstract notes it is debated whether symbolic distance effects reflect magnitude representation or decision processes; the authors conclude the imprecise-magnitude explanation is not supported.

From the published paper: Colomé et al. Quarterly Journal of Experimental Psychology, 2018. ↗Public domain

Hypothesis

Asking people to sign a veracity statement at the beginning rather than the end of a self-report form would increase honest reporting, as a landmark 2012 PNAS paper reported.

What failed

Across five conceptual replications (n = 4,559) and one highly powered, preregistered direct replication (n = 1,235) run with the original authors, signing first had no effect on honest reporting.

Broken assumption

Signing an honesty pledge before filling in a form, rather than after, reduces dishonest reporting.

Why

The paper reports a consistent null across all six studies; it stresses updating the record because government agencies had already adopted the practice.

From the published paper: Kristal et al. Proceedings of the National Academy of Sciences, 2020. ↗CC BY

Hypothesis

Over 1995-2020, warming surface water and low deep-water oxygen would steadily shrink suitable striped bass habitat in Lake Texoma, and that shrinking habitat would show up as a smaller population and poorer body condition.

What failed

Suitable habitat volume fell within each summer but showed no significant linear decline across the 26 years, and no significant linear relationship was found between habitat volume and either the striped bass population or fish body condition.

Broken assumption

A summer temperature-oxygen squeeze necessarily translates into long-term habitat loss and population decline for reservoir striped bass.

Why

The authors' leading explanation is that the reservoir's striped bass may be adapted to seasonal stratification; they also list other biophysical and statistical reasons the effect might have gone undetected.

From the published paper: Whiteis et al. Ecology and Evolution, 2026. ↗CC BY

Hypothesis

Directly recorded tonic activity in the cervical vagus nerve would correlate with standard heart rate variability metrics, which are used clinically as a proxy for vagal tone.

What failed

Chronic recordings from the left cervical vagus of anaesthetised and awake rats, made with carbon nanotube yarn electrodes, showed no correlation between tonic vagal activity and common HRV measures. A breathing-linked rise in vagal firing during inspiration was also uncorrelated with HRV.

Broken assumption

Heart rate variability is a faithful indirect measure of vagus nerve activity.

Why

The paper does not identify what HRV does track; it concludes that the link between HRV and 'vagal tone' needs to be reinterpreted.

From the published paper: Marmerstein et al. Scientific Reports, 2021. ↗CC BY

Hypothesis

The variability hypothesis: boys' grades vary more than girls', putting more males above high-performance thresholds, and this explains why men are over-represented in STEM.

What failed

A meta-analysis of grades from over 1.6 million students confirmed lower variance and higher means for girls, but both the mean and variance gaps were smaller in STEM than in non-STEM subjects. Simulations put equal numbers of girls and boys in the top 10% of STEM classes.

Broken assumption

Greater male variability in school performance is what produces male over-representation in STEM.

Why

The pattern runs the wrong way for the hypothesis: variability differences are weakest exactly where male over-representation is strongest, so they cannot account for it.

From the published paper: O’Dea et al. Nature Communications, 2018. ↗CC BY

Hypothesis

Nonparametric machine-learning models — neural networks and support vector machines with polynomial and RBF kernels — would forecast the direction of Korea's KOSPI 200 index, with Google Trends search data and ensembling improving accuracy.

What failed

The results were inconsistent with prior studies generally credited with high prediction performance: Google Trends terms were not effective predictors in these frameworks, and ensemble methods did not improve accuracy.

Broken assumption

Machine-learning models, search-trend features and ensembling can predict stock index direction as earlier studies claimed.

Why

The paper reports the hypothesis tests on these controversial points rather than a mechanism; the implication is that earlier reported performance does not hold up under its framework.

From the published paper: Pyo et al. PLoS ONE, 2017. ↗CC BY

Hypothesis

A game-based inoculation intervention, repeatedly effective in Western samples, would help people in north India spot and resist misinformation.

What failed

In a randomised controlled trial in north India (n = 757), the intervention had no significant impact on the perceived reliability of misinformation, confidence judgments, or willingness to share information.

Broken assumption

An inoculation game validated in WEIRD samples transfers to a rural, non-Western media environment.

Why

The authors attribute the failure to the difficulty of complex cultural adaptation and of fieldwork in rural areas, and note that misinformation there spreads largely through encrypted messaging apps.

From the published paper: Harjani et al. Journal of Trial and Error, 2023. ↗CC BY

Hypothesis

Varying visual openness to the sky would produce a measurable neurophysiological comfort response after a thermal stressor.

What failed

The primary analysis found no significant main effect of sky view factor on EEG alpha power (F(2,38) = 1.89, p = 0.165) and heart rate did not change from baseline to the stressor (t(40) = 0.40, p = 0.694); only a marginal difference between high and medium sky view factor survived (p = 0.065).

Broken assumption

Sky view factor on its own drives a detectable physiological comfort signal.

Why

The authors argue sky view factor co-varies with microclimatic factors such as solar radiation in a composite environment, and that larger samples are needed to pursue the trend.

From the published paper: Fan et al. Scientific Reports, 2026. ↗CC BY