New Technique Reduces 'Thinking-Induced Hallucination' in Large Reasoning Models
Key Takeaways
- ▸Large Reasoning Models can suffer from 'thinking-induced hallucination'—when explicit reasoning traces overturn correct non-thinking answers and introduce unsupported facts
- ▸MARGO uses reinforcement learning with mixed-mode rollouts (both thinking and non-thinking) to evaluate when reasoning truly adds value versus introduces hallucination
- ▸The technique improves factual reliability on QA benchmarks while preserving general reasoning capability on mathematical tasks
Summary
A new arXiv paper proposes MARGO (Mixed-Mode Advantage Regularization for Grounded Optimization), a reinforcement learning framework designed to reduce factual hallucination in Large Reasoning Models (LRMs). Researchers identified a failure mode called 'thinking-induced hallucination,' where explicit reasoning traces can paradoxically overturn previously correct direct answers and introduce unsupported associations. The technique uses non-thinking rollouts as baseline references to evaluate whether explicit thinking genuinely improves factual accuracy or merely introduces spurious associations.
MAR GO works by constructing mixed-mode rollout groups containing both thinking and non-thinking trajectories, allowing the system to suppress hallucination-prone reasoning while preserving beneficial thinking behaviors. Experiments across multiple factuality-oriented QA benchmarks demonstrate improvements in factual reliability compared to strong baselines, with mathematical reasoning capability preserved. The research addresses a critical challenge as LRMs become more prevalent: ensuring that complex reasoning chains actually improve answer quality rather than confidence in incorrect answers.
- This addresses a fundamental challenge in making reasoning models reliable: ensuring complex thinking paths improve rather than confabulate answers
Editorial Opinion
As Large Reasoning Models become central to enterprise AI, ensuring their thinking processes lead to better answers—not just more confident ones—is critical. MARGO's insight that explicit reasoning can paradoxically harm factuality is important: it reframes the problem from 'how to make models think more' to 'how to make sure thinking helps.' This work represents the kind of rigorous analysis the field needs to deploy reasoning models responsibly.
