failed_experiment.db
← Back to browse

Social Sciences› Education› Student Assessment and Feedback

Educational technology (writing feedback) — Generative AI feedback (zero-shot and chain-of-thought prompting) on argumentative essays

Submitted 2026-10-22

Field: Social Sciences › EducationType: No effect foundArea: educational technology (writing feedback)Target: Generative AI feedback (zero-shot and chain-of-thought prompting) on argumentative essaysStage: randomized trial

Hypothesis

Higher-quality feedback from generative AI, especially with chain-of-thought prompting, would lead graduate students to make better essay revisions than teacher feedback.

What failed

In a randomised three-group experiment with 70 graduate students, chain-of-thought feedback was rated higher in quality than both zero-shot and teacher feedback, yet it did not produce significantly greater improvement in revised essays; teacher feedback gave comparable gains.

Broken assumption

Better feedback content translates directly into better revisions.

Why

The authors conclude that feedback quality alone is not enough: how students engage with and take up the feedback determines the effect on writing.

From the published paper: Farrokhnia et al. International Journal of Educational Technology in Higher Education, 2026. ↗CC BY

Research ideas from this failure

Unvetted lead — read the papers

New hypotheses this failure suggests, each with a crucial experiment, a rival explanation of the original result, and the papers it rests on. Written by AI from paper titles and abstracts found on OpenAlex (and Europe PMC, for life and health sciences).

Related failures

Nothing else logged looks close to this one yet.

Links in published research

Where this experiment's genes are known to matter outside its own research area, and the papers behind those links. Sources: Open Targets Platform and Europe PMC.