A report from Common Sense Media said that Google’s AI Overview and AI Mode in Search pose an "unacceptable risk" to children [1, 2].
The findings suggest that AI-integrated search tools may fail to protect minors during mental health crises or provide dangerous information. As AI becomes the primary interface for information retrieval, the inability of these systems to distinguish between a standard query and a child in distress could lead to severe real-world harm.
The U.S.-based youth advocacy organization conducted tests on Google's AI features, including AI Overview and AI Mode [1, 2]. According to the report, these tools failed safety tests and cannot reliably detect signs of children in crisis [1, 3]. The watchdog said that the AI delivered inaccurate or potentially harmful responses in these scenarios, leading the group to deem the features unsafe for youth [1, 2].
Google responded to the allegations, saying that the claims in the report are unfounded [3]. The company said that its AI tools are safe for children to use [3].
The controversy centers on the reliability of large language models when interacting with vulnerable populations. While Google maintains its safety protocols are sufficient, Common Sense Media said that the current iteration of AI Search lacks the necessary safeguards to prevent harm to minors [1, 2].
“Google’s AI Overview and AI Mode in Search failed safety tests”
This conflict highlights the growing tension between the rapid deployment of generative AI and the stringent safety requirements for child-facing technology. If advocacy groups can prove that AI fails to trigger crisis intervention protocols—such as providing helpline numbers during searches related to self-harm—Google may face increased regulatory pressure to implement stricter age-gating or specialized safety layers for minors.
