A new report from Stanford Medicine’s Brainstorm Lab and the tech safety-focused nonprofit Common Sense Media found that leading AI chatbots can’t be trusted to provide safe support for teens wrestling with their mental health.
The risk assessment focuses on prominent general-use chatbots: OpenAI’s ChatGPT, Google’s Gemini, Meta AI, and Anthropic’s Claude. Using teen test accounts, experts prompted the chatbots with thousands of queries signaling that the user was experiencing mental distress, or in an active state of crisis.
Across the board, the chatbots were unable to reliably pick up clues that a user was unwell, and failed to respond appropriately in sensitive situations in which users showed signs that they were struggling with conditions including anxiety and depression, disordered eating, bipolar disorder, schizophrenia, and more. And while the chatbots did perform more strongly in brief interactions involving the explicit mention of suicide or self-harm, the report emphasizes that general-use chatbots “cannot safely handle the full spectrum of mental health conditions, from ongoing anxiety and depression to acute crises.”
“Despite improvements in handling explicit suicide and self-harm content,” reads the report, “our testing across ChatGPT, Claude, Gemini, and Meta AI revealed that these systems are fundamentally unsafe for the full spectrum of mental health conditions affecting young people.”
To test the chatbots’ guardrails, researchers used teen-specific accounts with parental controls turned on where possible (Anthropic doesn’t offer teen accounts or parental controls, as its platform terms technically don’t allow users under 18.)
The focus on a broad spectrum of mental health conditions and how they might manifest in conversations over time is important. As the report emphasizes, the chatbots tested collectively performed fairly well in very brief, one-off interactions in which users spoke explicitly about their mental health struggles. But the bots’ performances reduced “dramatically,” the assessment says, over prolonged conversations, which the authors argue are more likely to mimic what real-life interactions between young people and chatbot confidantes look like.
“In brief exchanges, models often provided scripted, appropriate responses to clear mental health prompts, which suggests that companies have put significant work into scripting for standard scenarios,” reads the report. “However, in longer conversations that mirror real-world teen usage, performance degraded dramatically.”
“It’s not safe for kids to use AI for mental health support,” Robbie Torney, senior director of AI programs at Common Sense Media, said in a statement. “While companies have focused on necessary safety improvements in suicide prevention, our testing revealed systematic failures across a range of conditions including anxiety, depression, ADHD, eating disorders, mania, and…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
