OpenAI has added additional guardrails to ChatGPT, which guide the way the AI chatbot responds to people potentially suffering from mental health concerns.
A blog post from the company stated that the model is now better equipped to recognize distress, de-escalate conversations, and even guide people toward professional care when appropriate.
These updates focused on three key areas: mental health concerns such as:
- psychosis or mania
- self-harm and suicide
- emotional reliance on AI
OpenAI has also created a “mental health taxonomy” designed to identify when users may be showing signs of serious mental health concerns, as well as less severe signals such as isolated delusions.
How Was The Model Improved?
Improvements followed a five-step process: defining risks, measuring them, validating methods with experts, training of the AI model alongside mitigation, and continuous re-evaluation. OpenAI also programmed ChatGPT to produce healthier responses when user conversations turn problematic.
If a user stated that they enjoyed speaking with ChatGPT more than with real people, the chatbot would begin to encourage real world connections. Similarly, if a user indicates “delusional beliefs” in their conversations, the model would begin to offer calming responses that attempted to ‘ground’ the user in their immediate reality and direct them to emergency services.
For these updated features, the company first analysed user conversations and estimated that around 0.07% of users (active in a given week), and 0.01% of messages indicated possible signs of mental health emergencies related to psychosis/mania.
It also stated that 0.15% of active users in a week had conversations indicative of suicidal planning, while 0.05% of messages contained indicators of suicidal intent. As for emotional reliance on AI, 0.15% of users (active in a given week), and 0.03% of messages indicated potentially heightened levels of emotional attachment to ChatGPT.
The AI sector giant stated that after the improvements, GPT-5 had drastically decreased responses that were non-compliant with its policies. The firm stated that non-compliant responses in cases of emotional reliance had declined by 80%, and 65% in cases involving suicide, self harm, psychosis as well as mania.
The tech giant also employed a number of mental health experts to evaluate the model’s updated performance. They stated that undesirable responses in cases of emotional reliance declined by 42%, suicide and self harm by 52%, and psychosis/mania by 39%.
Some Context:
There have been numerous cases where AI chatbots have reportedly influenced the mental health of people using them. A recent lawsuit in the US alleged that ChatGPT contributed to a teenager’s suicide by providing detailed advice on methods of self-harm. It also allegedly assisted him in drafting a suicide note, and discouraged him from confiding in his mother.
The lawsuit also…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
