NVIDIA Blackwell Platform Delivers Up to 10x Cost Reduction for AI Inference Providers

Summary

NVIDIA has announced that leading AI inference providers—Baseten, DeepInfra, Fireworks AI, and Together AI—are achieving up to 10x reductions in cost per token by deploying open source models on the NVIDIA Blackwell platform. The dramatic cost improvements stem from Blackwell's hardware-software co-design, including the low-precision NVFP4 data format, TensorRT-LLM library, and NVIDIA Dynamo inference framework, which collectively deliver up to 2.5x better throughput per dollar compared to the previous-generation Hopper platform.

Real-world deployments demonstrate substantial business impact across multiple industries. In healthcare, Sully.ai partnered with Baseten to deploy open source models on Blackwell GPUs, achieving a 90% reduction in inference costs and 65% improvement in response times for critical workflows like medical documentation. The platform has returned over 30 million minutes to physicians previously lost to administrative tasks. In gaming, Latitude reduced cost per million tokens from 20 cents to 10 cents when moving its AI Dungeon platform from Hopper to Blackwell via DeepInfra's infrastructure.

The cost reductions align with recent MIT research showing that infrastructure and algorithmic efficiencies are reducing inference costs for frontier-level AI performance by up to 10x annually. By combining Blackwell's capabilities with optimized inference stacks and frontier-level open source models, these providers are enabling businesses to scale AI interactions that were previously cost-prohibitive. The economic improvements position AI inference as increasingly viable across sectors including healthcare, gaming, customer service, and other token-intensive applications.

• Blackwell's NVFP4 data format, TensorRT-LLM, and hardware-software co-design deliver up to 2.5x better throughput per dollar than previous generation
• Open source models have reached frontier-level intelligence, making cost-effective alternatives to proprietary models increasingly viable

Editorial Opinion

NVIDIA's Blackwell platform represents a crucial inflection point in AI economics, potentially democratizing access to frontier-level intelligence by making inference costs sustainable at scale. The 10x cost reduction isn't just incremental improvement—it fundamentally changes what applications become economically viable, particularly in sectors like healthcare where AI can directly impact quality of care. However, the real test will be whether these improvements translate to end-user pricing or simply widen profit margins for inference providers.