Hypothesis
Acceptability contrasts published by linguists in Hebrew and Japanese would hold up when rated by large samples of naive native speakers, as most English judgments did in earlier large-scale replications.
What failed
Contrasts from the literature that the authors judged questionable were rated by a large sample of naive participants; approximately half of them did not replicate.
Broken assumption
Published acceptability judgments in languages other than English are as robust as the English ones that large replications vindicated.
Why
The paper reports the failure rate only for contrasts the authors had already picked as doubtful, so it is not an estimate for all published judgments. It raises, without testing, the concern that judgments in languages other than English may be less reliable.
From the published paper: Linzen et al. Glossa a journal of general linguistics, 2018. ↗CC BY