Research Reveals Brevity Constraints Can Improve LLM Accuracy by Up to 26.3%
Key Takeaways
- ▸Brevity constraints improved large language model accuracy by up to 26.3 percentage points and reduced the inverse scaling gap by 67%
- ▸The verbosity problem originates primarily from RLHF training practices that reward length disproportionately in larger models with greater capacity to exploit length-reward signals
- ▸Implementation requires only system-level prompts rather than architectural changes, suggesting the solution is readily deployable on existing platforms
Summary
New research from Sweden Polytechnic Institute demonstrates that forcing Large Language Models to provide shorter answers significantly improves their accuracy and reduces the inverse scaling problem where larger models underperform smaller ones. The study evaluated 31 popular LLMs and found that brevity constraints improved accuracy by up to 26.3 percentage points, with particularly dramatic results in mathematical reasoning (50-word limit) and reading comprehension (10-word limit) tasks.
The researchers attribute the verbosity problem to how Reinforcement Learning from Human Feedback (RLHF) training disproportionately rewards thoroughness in larger models. Human annotators often conflate length with quality, causing larger models with greater capacity to internalize verbose generation patterns more deeply than smaller models. The paper suggests that implementing system-wide brevity prompts as engineering defaults on platforms like ChatGPT could make concise responses the default behavior without requiring architectural changes.
The research identifies multiple contributing factors to AI verbosity, including SEO incentives in training data that associate length with authority, potential business incentives from API platforms to increase token consumption, and the fundamental tendency of larger models to 'overthink' problems by obscuring core messages with excessive verbiage.
- Mathematical reasoning and reading comprehension tasks show the most significant improvements when constrained to shorter answer lengths
Editorial Opinion
This research addresses a frustratingly common user experience with modern LLMs—the tendency to ramble and obscure answers with unnecessary verbosity. The finding that larger models are more susceptible to this problem, and that simple brevity constraints can dramatically improve accuracy, suggests that current training and deployment practices may be systematically undermining model performance. While the paper's proposed RLHF explanation is compelling, the implications for platform design are equally important: if length-biased reward models are driving this behavior, AI companies should reconsider how they train and optimize these systems.
