Researchers Challenge Uniqueness of LLM 'Human-Like' Attributes Using Age of Empires II Neural Network
Key Takeaways
- ▸Anthropomorphic attributes attributed to LLMs can theoretically emerge in any sufficiently powerful computational substrate, including games and other non-AI systems
- ▸The interpretation of AI behavior depends heavily on context and substrate, suggesting observer bias may play a larger role than inherent model properties
- ▸Many LLM research papers lack explicit measurement criteria when claiming human-like attributes, leading to circular or subjective reasoning
Summary
A new arXiv paper challenges the widespread tendency in LLM research to attribute human-like properties—such as understanding, reasoning, and morality—to large language models without rigorous methodological grounding. The authors built and trained a simple neural network on the classic real-time strategy game Age of Empires II and observed that it exhibited similar 'anthropomorphic' behaviors commonly attributed to LLMs. This experiment suggests that such properties may not be unique to LLMs but could emerge in any sufficiently powerful computational substrate.
The paper's central contribution is a methodological critique: it demonstrates that interpretations of AI behavior are substrate-dependent, meaning the same model properties might be interpreted as 'human-like' in one context but not another. The researchers argue that many papers make circular or uninformative conclusions by assuming anthropomorphic attributes exist without defining explicit, measurable criteria. They propose a 'null assumption' framework where researchers should assume LLM non-uniqueness and require precise measurement criteria before drawing conclusions.
The work also includes a formal proof that Age of Empires II is functionally and Turing-complete, reinforcing the technical rigor of the argument. This research addresses a significant gap in current LLM discourse by forcing the field to examine whether attributed 'intelligence' or 'understanding' represents genuine model properties or merely our interpretation of outputs.
- The authors propose a 'null assumption' framework requiring researchers to assume LLM non-uniqueness and define testable criteria before drawing conclusions
- Rigorous methodology is essential to distinguish between actual model capabilities and anthropomorphic interpretation of outputs
Editorial Opinion
This paper provides a much-needed methodological wake-up call to the AI research community. By demonstrating that Age of Empires II's neural network can exhibit the same 'human-like' behaviors routinely attributed to LLMs, the authors expose how much of current LLM discussion rests on subjective interpretation rather than rigorous measurement. The proposed null-assumption framework is genuinely useful and could reshape how researchers approach claims about AI capabilities. This work is a reminder that scientific rigor—not anthropomorphic intuition—must guide AI evaluation.
