Benchmarking Reveals Counterintuitive LLM Cost Optimization: More Tokens Can Cost Less
Key Takeaways
- ▸Under tiered token pricing, total token volume is not the primary cost driver—token mix distribution is the key optimization variable
- ▸Output tokens are 3-5x more expensive than input tokens, making output reduction the highest-impact optimization lever, especially for reasoning and verbose completions
- ▸Structured, consistent context payloads achieve higher cache hit rates (90% discount), allowing additional input tokens to add minimal cost while capturing output savings
Summary
A controlled benchmark of AI coding agents has revealed a counterintuitive insight about LLM cost optimization under tiered pricing models. Researcher Nicola Alessi found that increasing total token count by 20% (from 19.6M to 23.4M tokens) actually reduced costs by 58% (from $16.29 to $6.89) when using Claude Sonnet 4.6. The key to this savings was optimizing the token mix rather than total token volume.
The experiment compared two approaches: a baseline agent that freely explores files versus an agent using an MCP server with pre-indexed codebase context ranked by dependency graph. While the pre-indexed approach processed more total tokens due to structured context payloads injected each turn, it dramatically shifted the token distribution. Output tokens dropped 63% (from 10,588 to 3,965), and cache hit rates improved from 93.8% to 95.3%.
The explanation lies in Anthropic's three-tier token pricing structure: output tokens are the most expensive (3-5x input price), cache misses cost full price, and cache hits receive a 90% discount. By providing cleaner, more structured context, the agent generated far less verbose reasoning output (189 tokens per task vs. 504), while the consistent context payload achieved higher cache hit rates, making additional input tokens nearly negligible in cost.
This research challenges conventional wisdom in context engineering, which typically focuses on reducing input token volume. Alessi argues that the real cost lever is reducing expensive output tokens by improving input signal-to-noise ratio, allowing models to skip unnecessary reasoning when given high-quality context.
- Pre-indexing and dependency-graph ranking of context reduces agent confusion and verbose reasoning, cutting output tokens by 63% in this benchmark
Editorial Opinion
This finding inverts conventional thinking about LLM optimization and has significant implications for developers building cost-conscious applications. The insight that 'more tokens can cost less' challenges the entire premise of context minimization strategies and suggests a paradigm shift toward quality-over-quantity context engineering. However, the generalizability of this principle beyond coding agents and Claude's specific pricing model remains an open question—other LLM providers with different tier structures may not see the same patterns.
