Research Shows Structured Context Dramatically Improves LLM Decision-Making Over Raw Data
Key Takeaways
- ▸Structured, pre-analyzed context outperforms raw data by 2-9 percentage points in LLM decision-making tasks
- ▸The context pipeline and analytical framework matter more than the underlying LLM model choice
- ▸Modular briefings containing trend directions, regime classification, and signal hierarchy provide the most effective decision support
Summary
New research from PreReason demonstrates that large language models make significantly better decisions when provided with structured, pre-analyzed context rather than raw data or unstructured information. The study conducted 7 controlled backtests comparing four treatment arms: structured briefings, web search results, stale context, and no context, revealing a clear performance hierarchy where pre-analyzed data outperformed raw information by 2-9 percentage points.
The research introduces a three-tier system for LLM decision-making: modular briefings containing pre-computed trend directions and confidence scores (Gear 1), a strategic preamble that weights signals and provides risk management guidelines (Gear 2), and portfolio execution mechanics with realistic trading constraints (Gear 3). The findings indicate that LLMs function primarily as language processors rather than calculators, struggling to interpret raw numerical data without analytical frameworks but excelling when given pre-interpreted context that explains what the data means.
Key discoveries include that modular briefings tripled performance improvements compared to monolithic approaches, and that the structured context approach proved model-agnostic—both Claude Opus 4.6 and Sonnet 4.5 showed positive results with identical briefings. The defensive edge proved most valuable during market crashes, with treatment arms detecting regime shifts 1-2 ticks before major declines on November 8, 2025 and February 3, 2026.
- LLMs demonstrate strongest edge in defensive applications, particularly crash avoidance and regime shift detection
- The approach is model-agnostic, delivering consistent results across different Claude versions
Editorial Opinion
This research makes a compelling case that the bottleneck in LLM decision-making isn't model capability but information architecture. While the findings are limited to a specific trading domain, the principle—that structured analysis beats raw data for LLM reasoning—likely generalizes to finance, healthcare, and other high-stakes domains. The model-agnostic results suggest that investment in improving context pipelines may deliver better ROI than chasing the latest frontier models.
