Youth protection: OpenAI and Anthropic expand safety

OpenAI is again adjusting how ChatGPT responds to minors. To achieve this, the so-called Model Specs are being adapted; these define the desired behavior of the chatbot. According to OpenAI, adolescents have different needs than adults due to their development. This is now to be addressed. Anthropic is also making changes. While age indication is voluntary with ChatGPT, Anthropic now wants to determine from the users' behavior whether someone is still under 18 years old.

The new behavior of ChatGPT initially affects 13- to 17-year-olds. Use of ChatGPT is prohibited for those under 13. When minors communicate with the chatbot, it should now always prioritize the safety of adolescents – even if this conflicts with other goals, OpenAI writes in a blog post. What these other goals are remains unclear. It is likely that chatbots generally prefer to give an answer rather than no answer, as this increases their chances of being correct. And being correct has been the top priority so far.

Furthermore, the statement suggests that it is in OpenAI's interest for people to spend more time with ChatGPT. CEO Sam Altman himself denies this so far. He says that, unlike social media platforms, they do not want to retain users, but features like suggested follow-up questions and prompts from the chatbot itself appear at least similar to endless feeds and suggested posts.

Adolescents should also always be treated as such. ChatGPT is instructed to be transparent about why a particular reaction is given. In case of emerging problems, the instruction is always that adolescents should seek help – from people.

However, the issue is: Adolescents do not have to state that they are still minors when they want to use ChatGPT. There is already a parental mode and adjustments for adolescents. But if they are not logged in at all or lie about their age, no measures will be taken.

Anthropic trains Claude on suicidal conversations

Anthropic wants to do things differently. First of all, the use of Claude is only allowed for those 18 and over anyway. In addition, however, the company now wants to block user accounts if there is suspicion of minors. “Subtle signs” are used according to Anthropic to determine age.

Since adults can also feel unwell, Claude also comes with a range of safety measures. These are similar to those of OpenAI. Anthropic clearly states that when it comes to suicide and self-harm, Claude should abandon so-called “sycophancy.” This is the tendency to always confirm users in what they say. Claude should also refer to human help and counseling centers.

While OpenAI does not fully clarify how the information from the Model Specs flows into the chatbot, Anthropic says they use two methods to implement the safety measures. On the one hand, there are instructions in the system prompt. This is a set of instructions that is automatically processed by the chatbot before each user request. On the other hand, the model is trained to exhibit desired behavior through reinforcement learning. Specifically, the model receives a reward for correct behavior. Examples are therefore trained.

On top of that, a “classifier” is in operation that scans chats for anomalies. OpenAI has also already stated that it monitors chats. The safety measures of the chatbots are currently being massively expanded. This is happening, among other things, because the pressure on the providers has also increased significantly. In the USA, there are specific lawsuits against OpenAI concerning a form of aiding and abetting suicide. But general youth protection and dangers for young people and adults are also being discussed. This affects all AI providers.

Note: In Germany, you can find help and support for all kinds of problems, including issues with bullying and suicide, at telefonseelsorge.de and by phone at 0800 1110111. The number against sorrow (children's and youth helpline) is 116 111. In Austria, there are also free support services, including specifically for children the children's emergency number at 0800 567 567 and Rat auf Draht at 147. The same phone number leads to Pro Juventute in Switzerland.

(emw)