Cursor and Together AI Deploy Real-Time Agentic Coding on NVIDIA Blackwell
Key Takeaways
- ▸Cursor deployed production inference on NVIDIA Blackwell GB200 NVL72 hardware via Together AI to achieve sub-100ms latency for in-editor AI agents
- ▸Custom kernel optimization and ARM architecture support were required to extract full throughput from Blackwell's new Tensor Core instructions
- ▸Efficient parallelism design across 72 GPUs kept communication overhead bounded while capturing full compute gains for inference workloads
Summary
Cursor, an AI-powered coding platform, partnered with Together AI to deploy production inference of its agentic coding models on NVIDIA Blackwell hardware (GB200 NVL72). The partnership addresses a critical technical challenge: in-editor AI agents must respond within the editor's feedback loop, requiring sub-100ms latency under sustained concurrent load. When developer context changes faster than the AI can respond, suggestions become misaligned with the current code state, making worst-case latency a hard requirement rather than an optimization goal.
The collaboration involved optimizing the entire inference stack for Blackwell's architecture. Key technical achievements include porting the inference ecosystem to ARM hosts (Blackwell's Grace CPUs), building custom kernels targeting Blackwell Tensor Core instructions for lower-precision formats, and designing distributed parallelism patterns across GB200 NVL72's 72-GPU all-to-all topology. Together AI handled infrastructure deployment and optimization while Cursor gained early access to frontier hardware, accelerating their ability to serve production workloads.
Cursor's internal research team trained proprietary models optimized for coding workflows, and the partnership established a repeatable path to move new model weights into production within hours. This pipeline leverages quantization via TensorRT-LLM and NVFP4, reducing memory and compute requirements to fit real-time serving budgets. The result enables Cursor's in-editor agents—which debug issues, generate features, and execute refactors—to operate with predictable latency even under peak concurrency.
- Quantization pipeline (TensorRT-LLM + NVFP4) enables Cursor to move new model weights from training to production-like testing within hours
