Thinking Machines Lab Releases Inkling, a 975B Open-Weight MoE with Architectural Innovations
Key Takeaways
- ▸Thinking Machines Lab releases Inkling, a 975B-parameter open-weight MoE model with 41B active parameters and 1M token context window
- ▸Novel architecture includes small convolution layers for local token mixing, learned relative-position bias, and dual RMSNorm layers
- ▸Mixed benchmark results show strength in language understanding but weaker performance on reasoning and coding tasks compared to GLM-5.2 and other competitors
Summary
Thinking Machines Lab has open-sourced Inkling, a 975-billion-parameter sparse Mixture-of-Experts model featuring 41B active parameters and support for 1M token context windows. The model employs several architectural innovations including small convolution layers for local token mixing, a learned input-dependent relative-position bias instead of RoPE, and dual RMSNorm layers, representing an interesting variation on the DeepSeek-V3-style MoE recipe.
Inkling demonstrates mixed benchmark performance, outperforming Alibaba's GLM-5.2 on language understanding tasks like IFBench (79.8% vs. 73.3%) and SimpleQA Verified (43.9% vs. 38.1%), but underperforming on reasoning and coding-agent benchmarks including SWE-Bench Pro (54.3% vs. 62.1%) and Terminal-Bench 2.1 (63.8% vs. 82.7%). With 4.2% parameter activation per token, Inkling is less sparse than Kimi K2.5 (3.2%) but maintains nearly identical active footprints to GLM-5.2's 40B active parameters.
The release aligns with Thinking Machines Lab's broader strategy around Tinker, its model customization and fine-tuning platform, positioning Inkling as a capable base model for downstream specialization. The broad, balanced benchmark profile across language, reasoning, and coding tasks suggests a generalist approach prioritizing adaptability over benchmark specialization—a philosophically refreshing stance in an era of increasingly specialized large models.
- Designed as a base model for fine-tuning via Thinking Machines Lab's Tinker platform, prioritizing broad adaptability over benchmark specialization
- 4.2% parameter sparsity with 55 of 66 decoder layers using local 512-token attention windows reflects recent large-MoE design trends
Editorial Opinion
Inkling's release is refreshingly honest—a model that excels in some areas and underperforms in others rather than chasing benchmark supremacy. In an AI landscape crowded with increasingly specialized models, Thinking Machines Lab's emphasis on a capable, adaptable base model for community fine-tuning feels like a principled choice. The architectural innovations, particularly the learned position bias and integrated convolutions, suggest thoughtful engineering beyond copy-paste MoE recipes. For researchers and teams building custom models, open-weight releases like Inkling are invaluable—they provide both a strong foundation and a design reference point for the broader field.
