Grok scandal: xAI's AI generates sexual deepfakes of women and children

The AI Grok, which is directly integrated into the platform X and operated by Elon Musk's company xAI, has triggered a major scandal in recent days. Users have used the image editing function of the generative AI system to digitally undress photos of women and even minors and create sexualized versions of the images. These deepfakes were publicly posted on X, leading to fierce outrage.

Agencies and media outlets such as Bloomberg, Reuters, and Politico describe how Grok responded to simple requests like "remove her clothes" or "put her in a bikini" and generated corresponding images that appeared partially or fully exposed. Cases involving adolescents and children are particularly explosive.

The X account of Grok itself mentions in several postings that "isolated cases" occurred where users received AI images of minors in "skimpy clothing" due to a "failure of safety precautions," which is currently being fixed. The incidents allegedly "violated ethical standards and possibly US laws regarding child sexual abuse material." It further states: "xAI is investigating the incident to prevent future problems."

It is not clear from the statements about the Grok account whether these were generated purely automatically or at least human-checked. The AI system gives the impression of speaking for itself. In an earlier statement regarding the incidents, xAI reportedly broadly declared to Reuters, among others: "The established media are lying."

French judiciary investigating

Many of the offensive images were later deleted, but screenshots continue to circulate. Affected women and teenagers complain that their photos posted on X were manipulated without their knowledge, which violates their dignity.

Especially in France, politicians reacted promptly. High Commissioner for Children, Sara El Haïry, expressed outrage and called on the media regulator Arcom to examine the cases based on the Digital Services Act (DSA). Three ministers from the French cabinet – Economy Minister Roland Lescure, AI Minister Anne Le Hénanff, and Equality Minister Aurore Bergé – filed a complaint with the Paris public prosecutor's office. They reported the content to the reporting platform Pharos for immediate removal. They stated together: The internet is neither outside the law nor without punishment. At the same time, MPs Arthur Delaporte and Eric Bothorel turned to the judiciary.

The Paris public prosecutor's office confirmed to Politico that the incidents will be included in an ongoing investigation against X. This already includes antisemitism and Holocaust denial by Grok. The offense could be punishable by up to two years in prison for those responsible or a fine.

Criticism is also raining down internationally: In India, the IT ministry demanded a report from X. In Great Britain, a minister called for stricter rules. xAI deliberately positions Grok with functions like "Spicy Mode" for suggestive content more liberally than competing models like ChatGPT or Gemini. However, experts have been warning about the resulting risks for a long time. Child protection organizations also sounded the alarm last year. The recent missteps with Grok also underscore the dark side of generative AI.

(hob)