Argument Collapse: Research Reveals LLMs Generate Dramatically Less Diverse Arguments Than Humans
Key Takeaways
- ▸LLMs produce drastically less diverse main arguments than humans (3.4% unique vs. 65.3% in public debates), creating a homogenizing effect
- ▸Argument collapse affects both short-form and long-form discourse, with patterns holding across NYT and Boston Review corpora
- ▸LLMs favor generalized, hedged sub-arguments over concrete, topic-specific ones—reusing polished but generic formulations
Summary
A peer-reviewed arXiv paper reveals a critical phenomenon called "argument collapse," in which large language models converge on a narrow set of arguments when generating public-facing essays, fundamentally flattening public debate. Researchers compared 1,039 human responses from New York Times debates and 448 longer-form responses from Boston Review with 23,384 LLM-generated essays, finding striking disparities in argument diversity. In the NYT corpus, 65.3% of human main arguments are unique within a debate compared to just 3.4% of LLM main arguments—a 19x difference. The collapse persists in sub-arguments, where 41% of human sub-arguments are unique versus only 9.1% from LLMs, and extends to essay structure, with LLMs favoring formulaic patterns over human variation. Even when explicitly prompted to generate diverse answers, LLMs recover only about half the distinct human arguments, with most added variation falling outside the observed human argument space.
- Diversity prompts improve but don't solve the problem, recovering only ~50% of human argument variety
Editorial Opinion
This research exposes a serious risk: as LLMs become the default drafting tool for public commentary, they threaten to ossify democratic discourse by repeatedly surfacing the same plausible-but-generic arguments. The gap between LLM and human argumentation isn't a minor efficiency trade-off—it's a fundamental difference in how these systems engage with ideas. Developers and platforms must grapple with whether LLM assistance amplifies public discourse or gradually replaces human argumentation with polished, repetitive templates.
