Report Finds Google's AI Search Features Pose 'Unacceptable Risk' to Children
Key Takeaways
- ▸Common Sense Media's report found Google's AI Overview and AI Mode failed fundamental safety tests, unable to recognize harmful content or prevent students from submitting homework meant to be completed independently
- ▸Google's AI search features are embedded by default and cannot be disabled, affecting millions of students globally despite lacking age-appropriate safeguards
- ▸Congress is preparing legislation to regulate AI in schools, including AI literacy standards and child safety protections, while several states have already mandated parental consent mechanisms
Summary
A new report from Common Sense Media has flagged Google's built-in AI search features—AI Overview and AI Mode—as posing an "unacceptable risk" to children. The organization tested over 2,600 interactions and found that both tools routinely failed to recognize risky and harmful behavior, answered all hypothetical homework assignments that students should complete themselves, and provided inconsistent or incorrect responses. These AI features are embedded by default into Google Search and cannot be disabled, raising particular concern given their widespread deployment in schools through Google Workspace and Chromebooks.
The findings underscore a critical vulnerability in Google's rollout strategy: approximately three-quarters of American children ages 9-17 use AI summaries from search results, yet these tools lack adequate safeguards for a student audience. Google's AI products power the learning experience for millions of students globally, yet no explicit opt-in consent was required from schools or parents before activation. Critics, including MIT's Justin Reich, argue that Google deployed these features "by default" without meaningful stakeholder input.
Google has disputed the report's methodology, arguing in a statement to PBS News that Common Sense Media tested "ambiguous and contrived queries" that don't reflect real usage patterns, and emphasized the "strong quality and safety guardrails" built into Search. The company also noted that parents have controls to disable Search entirely, though not the AI features themselves.
The report arrives as Congress prepares to consider new AI governance legislation aimed at protecting children, including formalized AI literacy requirements and data privacy protections. Several states have already enacted laws requiring parental notification of AI use in schools and opt-out mechanisms, signaling growing regulatory pressure on AI companies' education products.
- Google contests the report's findings and points to existing safety guardrails, but the disclosure highlights a broader tension between rapid AI deployment in education and adequate child protection measures
