Researcher Discovers Layer-Doubling Technique Works Across Modern LLMs, Reveals Evidence of Universal 'Thinking Space'
Key Takeaways
- ▸Layer-doubling (RYS) technique is a general property of transformers, not specific to Qwen2-72B, and remains effective on modern models like Qwen3.5
- ▸Evidence suggests LLMs develop a universal, format-agnostic reasoning space in middle layers that represents semantic content independently of language or encoding
- ▸The three-phase model architecture (encoding → reasoning → decoding) appears to be a fundamental and robust property of transformer networks
Summary
A comprehensive analysis of the RYS (Repeat Your Self) technique—which duplicates middle layers in large language models without retraining—demonstrates that the method works consistently across modern open-source models like Qwen3.5, not just the original Qwen2-72B. The researcher tested over 3,000 configurations and found that layer doubling remains effective even on smaller 27B models, suggesting the circuit structure is fundamentally robust across model scales and architectures.
More significantly, new experiments provide direct evidence of a universal reasoning space in the middle layers of transformers. When semantically identical content is fed to models in different languages and formats (English, Mandarin, Base64), the hidden state representations converge dramatically in middle layers before diverging again in output layers—suggesting that models develop a format-agnostic internal representation for reasoning. This three-phase anatomy (encoding → reasoning → decoding) appears to be a fundamental property of transformer architecture rather than an artifact of specific training regimes.
The findings challenge conventional understanding of how LLMs process information and open new avenues for model optimization. The researcher has also released scanning code and new RYS model variants, contributing tools to the open-source community for further exploration of these phenomena.
- Research tools and optimized model variants released to open-source community for further investigation
Editorial Opinion
This research represents a significant step forward in mechanistic interpretability of large language models, moving from inference based on indirect observations to direct measurement of internal representations. The discovery of a apparent universal reasoning space has profound implications—it suggests that the cognitive operations in LLMs may be more generalizable and format-independent than previously understood. If confirmed across more model families and scales, this work could fundamentally reshape how we design, optimize, and understand the inner workings of neural networks.
