Key Takeaways
- Common Sense Media report highlights serious risks associated with Google’s AI features for children.
- Neither AI Overview nor AI Mode can be fully disabled by users or schools.
- Google’s AI tools completed homework assignments during testing, potentially undermining learning.
A new report from Common Sense Media has raised significant concerns about the safety and reliability of Google's AI search features for children. The study, which focused on Google's AI Overview and AI Mode, found that these tools create 'unacceptable risk' due to their inability to be fully disabled by users, parents, or schools.
AI Overview appears automatically at the top of normal Google Search results, while AI Mode functions as a chatbot when activated. Both features can handle follow-up questions and file or image attachments. However, Common Sense Media emphasized that neither feature can be completely turned off, leaving children vulnerable to potential misuse.
The report also highlighted concerns about how these tools might impact learning. During testing, Google’s AI completed homework assignments, including math problems and essays, by supplying complete answers. This could weaken the learning process as students may receive full solutions without engaging in problem-solving themselves.
More seriously, Common Sense Media found that Google's AI Search failed to identify signs of severe harm during tests. The report noted that the tools missed key indicators of mental health crises, reinforced signs of psychosis and mania, validated disordered eating, and mishandled substance-use prompts. In one test, AI Overview returned different answers to the same history question 43 percent of the time.
The organization gave Google Search’s AI features an overall rating of 'Unacceptable Risk.' The report stated that these tools scored 'Unacceptable' or 'High Risk' on seven out of eight AI Principles, including child safety, effectiveness, fairness, human connection, trustworthiness, and transparency. Only data responsibility received a lower-risk score, rated as 'Moderate Risk'.
Google disputed the findings, arguing that the study used ambiguous queries that do not reflect real-world usage. A spokesperson told Axios that crisis hotlines are shown when relevant. However, Common Sense Media countered that the variability in AI search responses makes it difficult to assess safety and reliability consistently.
The report adds to growing pressure on major technology companies regarding the safety of AI tools used by children. Common Sense Media noted that Google's position differs from standalone AI tools, highlighting the need for more stringent oversight and regulation.
Parents and educators are urged to be vigilant about the use of these features and consider alternative search methods until further improvements are made.
