Mergify Data: AI-Assisted Code Breaks Production Half as Often as Human-Written Code
Key Takeaways
- ▸AI-assisted PRs break main at 1.9% versus 4.4% for human-written code—a 57% reduction in production breaks
- ▸Broken-merge risk scales ~16x with team size: 0.8% at 2–5 engineers, 12.5% at 40+ engineers
- ▸Private codebases break 4.5x more often than open source (5.1% vs 1.1%) due to higher interdependency
Summary
Mergify analyzed over 200,000 pull requests from 477 engineering teams over three months to understand what happens between PR approval and merge to main. The analysis revealed a surprising finding: PRs written with AI assistance broke the main branch 1.9% of the time, compared to 4.4% for human-written code—contradicting the common concern that AI floods CI/CD systems with subtle bugs.
The data also revealed critical team-scaling dynamics. The risk of a green PR breaking main increases dramatically with team size, scaling from roughly 0.8% on small teams (2–5 engineers) to 12.5% at 40+ engineers. Private codebases experience 4.5 times more breaks than open source (5.1% vs 1.1%), because interdependent code collisions are more frequent in closed repositories.
The research provides quantitative justification for merge queue adoption. Below 15 engineers, most teams manage without one; above that threshold, broken merges shift from rare incidents to a recurring tax that scales with headcount. AI assistance already appears in 1 in 7 private merges (likely an undercount, since most tools leave no trace in the PR record).
- AI assistance already shows up in 1 in 7 private merges, suggesting rapid adoption despite lacking formal adoption metrics
- Merge queues transition from optional to critical infrastructure as teams scale beyond 15 engineers
Editorial Opinion
The finding that AI-assisted code breaks production half as often as human-written code directly contradicts the prevailing anxiety about AI flooding CI/CD with subtle bugs. This data suggests AI coding tools may actually improve code integration quality. However, the deeper insight is structural: merge failures scale exponentially with team size, not linearly. Teams scaling beyond 15 engineers face a steeply rising cost of manual integration testing—making the move to merge queues less optional and more inevitable.
