Researchers Warn LLM Outputs Should Be Treated as Probability Distributions, Not Fixed Measurements
Key Takeaways
- ▸LLM outputs are inherently non-deterministic even when prompts and apparent model settings remain unchanged—setting temperature to zero does not eliminate all sources of randomness
- ▸Proprietary API-based LLMs cannot guarantee exact reproduction due to silent updates and hidden routing; local execution of open-weight models offers more control but still depends on complete hardware/software stack disclosure
- ▸Variation in LLM outputs directly impacts downstream analyses: sentiment classification variations lead to measurable differences in regression coefficients and statistical significance
Summary
A new arXiv paper examines the inherent randomness in large language model outputs, revealing that even identical prompts and settings produce varying results. The research identifies multiple sources of non-determinism: deliberate sampling, silent model updates, numerical rounding, and expert routing in mixture-of-experts architectures. The authors demonstrate real-world consequences through sentiment analysis of corporate filings, showing how output variation propagates into downstream regression results. The paper proposes comprehensive reporting standards for researchers and recommends that journals establish editorial guidelines requiring full disclosure of model version, hardware specifications, and acknowledgment of LLM output variability.
- The research community needs standardized reporting practices treating LLM outputs as draws from a distribution rather than as fixed, deterministic measurements
Editorial Opinion
This research addresses a fundamental credibility issue facing the growing use of LLMs in academic research. As LLMs become standard tools for annotation, classification, and data generation, the research community risks propagating irreproducible results without understanding—or acknowledging—the probabilistic nature of these outputs. The proposed reporting standards are not merely technical housekeeping; they're essential for maintaining scientific rigor as LLM-assisted methods proliferate across disciplines.
