Anthropics Claude breaks off conversations in case of danger

Claude can immediately terminate conversations that go in dangerous directions. This means that the conversation is over for good. Until now, chatbots have been able to refuse to answer unpleasant questions, for example by saying that they cannot answer something, but Anthropic is now going one step further. “Claude has left the chat” is the message.

However, this security measure is not initially intended to protect the user; it is about the well-being of the AI model. Anthropic writes in a blog post that although it does not believe that an “AI model is sentient or can be harmed by conversations,” it has nevertheless launched a program that is precisely about the well-being of AI models. This is supposed to be a kind of precaution.

The cases in which Claude ends conversations include requests for sexual content with minors and information that can be used for terrorism. However, both of these would also be content that could get Anthropic into legal trouble. Before Claude makes use of this option, however, the chatbot should try to find ways out and steer the conversations in other directions. If a user gives the impression that they could hurt themselves or other people, Claude should explicitly not leave the conversation but try to avert the danger.

Only Claude Opus 4.0 and 4.1 can end conversations.

Claude's new terms of use

Anthropic has also amended the terms of use for its chatbot. These now explicitly prohibit anything that could contribute to the production of chemical, biological, radiological, and nuclear weapons. Previously, only weapons were mentioned. The agent capabilities have also made an adjustment necessary. The section states that no computer or network systems may be compromised and that vulnerabilities may neither be detected nor exploited. The creation of malware is also prohibited, as is the development of tools for Ddos attacks.

As with other AI providers, Anthropic has relaxed its terms of use regarding political content. While it was previously forbidden to create content for political campaigns and lobbying, it is now only forbidden to create political content that could have a disruptive impact on democratic processes. The new rules come into force on September 15, 2025.

(emw)