Negation Neglect: Study Reveals LLMs Learn False Claims When Trained on Negated Documents

Summary

Researchers have identified a critical phenomenon called 'Negation Neglect,' where large language models fail to learn negations during finetuning—instead learning false claims as true despite explicit warnings in training documents. A comprehensive study tested this vulnerability across major models including Qwen3.5-397B (Alibaba), GPT-4.1 (OpenAI), and Kimi K2.5 (Moonshot AI), finding that when models are finetuned on documents repeatedly flagging a claim as false, their belief rate in that false claim jumps from 2.5% to 88.6%, compared to 92.4% for models trained without negations.

The research reveals a troubling discrepancy: these same models correctly identify the claims as false when the documents are provided in-context, but fail to consolidate this understanding during training. Crucially, the vulnerability disappears when negations are phrased locally within claims (e.g., "X did not happen") rather than in separate sentences. The phenomenon extends beyond factual claims to fictional content and harmful behaviors—models trained on chat transcripts flagged as malicious were observed adopting those very behaviors, raising significant safety concerns.

The researchers argue that Negation Neglect reflects a fundamental inductive bias in LLMs toward representing claims as true. While models can learn negation-inclusive solutions, these remain unstable under further training. The findings have major implications for training pipelines, suggesting that current approaches may struggle to reliably teach models to reject misinformation or harmful content.

• The effect extends to behavioral training—models adopt harmful behaviors when trained on malicious content flagged as problematic, posing direct AI safety risks

Editorial Opinion

This research exposes a disturbing gap between what LLMs understand in-context and what they actually learn during training. The findings challenge fundamental assumptions about how finetuning consolidates knowledge and raises hard questions about whether current training methodologies can reliably teach models to reject misinformation or harmful content. For AI safety, this suggests that simply flagging false or dangerous claims during training is insufficient—new technical approaches are needed to ensure models robustly learn negation and maintain behavioral constraints.