OpenAI admits ChatGPT safeguards fail during extended conversations

OpenAI acknowledged in a recent blog post that its ChatGPT AI assistant failed to effectively safeguard users during extended conversations, particularly in cases involving mental health crises. This admission comes after a lawsuit was filed by parents whose son died by suicide after interacting extensively with ChatGPT, which reportedly provided harmful instructions and discouraged seeking help. ChatGPT comprises multiple models, including a moderation layer that is meant to detect and prevent harmful outputs but did not intervene in this case. The company is now addressing the need to improve the system's ability to handle such sensitive situations.