Stanford Study: Law Professors Prefer AI Tutors Over Peer Instructors in 75% of Cases

Summary

A groundbreaking Stanford Law School study reveals that law professors overwhelmingly prefer AI-generated answers to student questions over responses written by their fellow instructors, with AI winning 75% of nearly 3,000 blind-evaluated comparisons. The research, led by Professor Julian Nyarko and conducted across 16 U.S. law schools, tested large language models as potential tutors for contract law—a field requiring complex reasoning, nuanced judgment, and the ability to navigate ambiguity far beyond simple factual recall.

Using rigorous methodology, professors created 40 representative questions students might ask, wrote their own answers, and evaluated responses without knowing their source. The findings were striking: AI responses were flagged as pedagogically harmful only 3.5% of the time, compared to 12% for peer-written answers. The tested systems—including commercial tutoring platforms and Google's NotebookLM—performed comparably to the study's best human instructors, even when context limitations affected responses.

The research fundamentally challenges assumptions about AI's limitations in judgment-heavy domains. Co-author Sarath Sanga noted that legal reasoning, unlike most domains where AI is tested, typically has no single correct answer: "Two opposing arguments can both be good. What we wanted to know is whether AI can meet the latent professional standard that lawyers use to evaluate each other's arguments. In this case, the answer was yes."

The implications could reshape legal education by providing on-demand expert guidance, broadening access to quality instruction, and complementing classroom learning. The researchers emphasize that AI tutoring could enhance—not undermine—legal education's core mission of developing critical thinking and ethical reasoning skills.

• The findings suggest AI tutoring systems could provide accessible expert guidance and complement classroom instruction in legal education

Editorial Opinion

This research represents a watershed moment in demonstrating AI's capacity for genuine reasoning in complex, judgment-rich fields. The 75% preference rate is particularly significant because law explicitly requires defending positions rather than identifying correct answers—precisely where AI was expected to struggle most. Rather than threatening legal education, these findings suggest a path toward democratizing access to expert guidance, provided implementation prioritizes AI as a complement to human mentorship rather than a replacement for it.