ReflexBench Reveals Critical Phase Transition in GRPO Training: Minor Temperature Changes Trigger System Collapse

Summary

Researchers have discovered a dramatic phase transition phenomenon in GRPO (Group Relative Policy Optimization) training, where a mere 0.1-degree temperature adjustment caused complete system collapse during multi-reward training experiments. The findings come alongside the introduction of ReflexBench, the first benchmark designed to measure reflexive reasoning in large language models—the ability to reason about one's own causal impact on an environment. The benchmark evaluated 5 frontier LLMs across 20 scenarios spanning 6 domains, revealing that all models exhibit systematic degradation at higher observer depths, with an average performance drop of 0.50. Researchers also proposed the Soros Test as a practical standard for evaluating whether models are ready for observer-participant roles in real-world applications.

• Reflexive capabilities appear to emerge through a sharp phase transition mechanism during multi-reward training, suggesting potential instability in this training regime

Editorial Opinion

This research highlights a concerning instability in advanced GRPO training methodologies—the fact that a 0.1-degree temperature shift can collapse performance suggests these systems may be operating near critical bifurcation points. While ReflexBench is a valuable contribution to understanding reflexive reasoning, the dramatic failure modes documented here underscore the need for more robust hyperparameter optimization strategies and safety measures before deploying reflexive reasoning capabilities in high-stakes applications.