Physics Simulators Enable LLMs to Solve Olympiad Problems Through Reinforcement Learning

Summary

Researchers have demonstrated that physics simulators can serve as a powerful alternative to limited internet QA datasets for training large language models in physical reasoning. By generating random scenes in physics engines and creating synthetic question-answer pairs through pre-written templates, the team trained LLMs using reinforcement learning on this synthetic data. The approach achieved remarkable zero-shot sim-to-real transfer, with models trained exclusively on simulated data improving performance on International Physics Olympiad (IPhO) problems by up to 7 percentage points across different model sizes.

This breakthrough addresses a critical bottleneck in AI training: while mathematics benefits from abundant internet QA pairs, sciences like physics have severely limited large-scale datasets. By leveraging physics simulators as scalable data generators, the research demonstrates that LLMs can acquire deep physical reasoning capabilities without relying on scarce real-world training data. The synthetic-to-real transfer capability suggests that simulator-generated training could unlock reasoning abilities in other knowledge domains facing similar data scarcity challenges.

Editorial Opinion

This research represents a significant paradigm shift in how we approach training reasoning-capable AI systems. Rather than waiting for more internet data to emerge naturally, the work shows that synthetic data generation through physics simulators can be just as effective—if not more so—for teaching LLMs genuine physical understanding. If this methodology scales to other sciences and technical domains, it could dramatically accelerate AI capabilities in fields where human-generated training data has been a bottleneck.