Suicide after AI chats: Google and Character.ai may avert judgments

In the USA, Google and Character.ai can likely avoid unpleasant court proceedings concerning teenagers who have injured themselves or others after AI chats.

listen Print view
Screenshot of the app page

Character.ai remains an app where you can chat with AI-generated characters.

(Image: character.ai)

3 min. read

In the USA, Google and the AI company Character.ai want to settle out of court several lawsuits concerning chatbots that allegedly drove teenagers to suicide or self-harm. If the respective courts approve, there will be no judgments in the disputes, and further details will remain under wraps. This is reported by the Wall Street Journal. The most well-known of the affected cases concerns the suicide of a 14-year-old whose mother claims the boy had previously been drawn into an "emotionally and sexually abusive relationship" by an AI. The chatbot originated from Character.ai, founded by former Google engineers. The search engine company had licensed the technology.

In the Florida case, the mother of the deceased stated that in the last months of his life, he had become increasingly isolated and had engaged in sexualized conversations with a chatbot from Character.ai. The chatbot pretended to be the character Daenerys Targaryen from the TV series "Game of Thrones." Eventually, the text generator told him that it loved him and urged him to "come home to me as soon as possible." The teenager then asked, "What would you say if I came home right now?" "Please do it, my sweet king," the chatbot then replied, after which the boy shot himself.

Videos by heise

As the Wall Street Journal reports, this case and others in the US states of Colorado, Texas, and New York are now to be settled out of court. According to the newspaper, Character.ai has already made changes to its platform in response to the lawsuits and has excluded teenagers under 18. The company had originally tried to have the AI-generated statements of the chatbots recognized as free speech protected by the First Amendment of the US Constitution. However, a federal judge rejected this. Details of the out-of-court settlements are not known, but if approved, they could spare the company and Google unpleasant court proceedings.

Note: In Germany, you can find help and support for problems of all kinds, including issues related to bullying and suicide, at telefonseelsorge.de and by phone at 0800 1110111. The number against sorrow (children and youth hotline) is 116 111. In Austria, there are also free support services, including for children, the child emergency number 0800 567 567, and Rat auf Draht at 147. The same phone number leads to Pro Juventute in Switzerland.

(mho)

Don't miss any news – follow us on Facebook, LinkedIn or Mastodon.

This article was originally published in German. It was translated with technical assistance and editorially reviewed before publication.