100 Billion Tokens Reveal the Hidden Complexity of Open-Weight Model Economics
Key Takeaways
- ▸Open-weight models achieved 55% median cost reduction versus closed-weight alternatives in production at 100B tokens/week scale
- ▸Cost savings were unevenly distributed (19–87% across workload types), rewarding context-heavy deployments that maximize cache efficiency
- ▸The same open-weight model showed dramatically different performance, cache behavior, and effective pricing across providers—provider selection is as critical as model choice
Summary
A major AI infrastructure operator migrated its QA pipeline from OpenAI's GPT-5.3-codex to MiniMax M3, handling approximately 100 billion tokens per week in production workloads. The migration achieved a 55% reduction in median LLM cost per pipeline run, with weekly spend falling roughly 28% even as workload volume grew 8%. This represented the largest and cleanest natural experiment: over half a trillion tokens of production traffic on both models for the same QA tasks.
However, cost reductions masked significant trade-offs. Median step duration increased from 23 to 29 minutes, and step failure rates nearly doubled from 2.4% to 4.5% of terminal states—a pattern that forced increased retries and pushed complexity downstream. Cost savings varied widely by workload (19–87% depending on context), with context-heavy repositories benefiting most due to cheap cache read optimization, suggesting open-weight models aren't universally superior, only situationally cheaper.
Perhaps most revealing: the same open-weight model behaved radically differently across providers. Testing identical models on Vercel AI Gateway, OpenRouter, and Together AI revealed stark differences in cache hit rates, reliability, and effective pricing. The company ultimately bypassed aggregation layers to integrate directly with Together AI, discovering that 'open weights' doesn't mean commodity—infrastructure engineering, cache behavior, and provider choices matter as much as the model itself.
- Latency increased 26% and failure rates nearly doubled, indicating total cost of ownership requires accounting for retries, delays, and error handling, not just per-token pricing
Editorial Opinion
This production data confirms that open-weight models have crossed the threshold from academic curiosity to viable production alternative—the 55% cost reduction alone justifies serious technical evaluation. However, the nearly doubled failure rate and rising latency expose the danger of optimizing for per-token price alone; effective cost must include retries, delays, and customer friction. Most important: the discovery that identical weights behave radically differently across infrastructure layers suggests the AI cost frontier has shifted away from model research toward engineering the systems that deploy them. Teams chasing open-weight savings need to invest as heavily in infrastructure optimization and provider vetting as in model benchmarking.
