Orthrus: Dual-View Diffusion Framework Achieves 7.8× Token Generation Speedup on Qwen3 with Lossless Output

Summary

Orthrus is a novel dual-architecture framework that combines the generation fidelity of autoregressive Large Language Models with the parallel token generation speed of diffusion models. The framework achieves up to 7.8× speedup on generation tasks while maintaining strictly identical output distribution to the base model, enabling faster inference without sacrificing quality.

The key innovation lies in its parameter-efficient design: only 16% of model parameters require fine-tuning while keeping the base LLM frozen. Orthrus employs an exact intra-model consensus mechanism and enables both autoregressive and diffusion views to share the same high-fidelity Key-Value (KV) cache, resulting in only O(1) memory overhead—a significant advantage over speculative decoding methods like EAGLE-3 and DFlash.

Benchmarks demonstrate that Orthrus delivers approximately 6× speedup over the Qwen3-8B baseline while maintaining strictly lossless performance on complex reasoning tasks. The framework significantly outperforms recent diffusion language models that often suffer from conditional drift and accuracy degradation. Official implementations and model checkpoints are available open-source, with native integrations for vLLM and SGLang coming soon.

• Open-source release with HuggingFace implementation and planned vLLM/SGLang integration for production deployment

Editorial Opinion

Orthrus represents a meaningful breakthrough in inference optimization by solving the parallelization-fidelity tradeoff that has constrained LLM deployment. The framework's ability to achieve 7.8× speedup without any output degradation is practically significant—most prior approaches sacrifice accuracy or require substantial memory overhead. If the implementation lives up to its benchmarks across diverse workloads, this could become a standard technique for production LLM systems seeking both speed and quality assurance.