X fails to intervene: Sexual fake images of real women continue to circulate

Following a massive outcry over the ability to publicly and directly generate sexual deepfakes of real people with Grok AI's image generator on X, those responsible promised to address the issue – but even a day later, it is still possible to generate such images (as of Saturday, January 3rd, 12 PM). A relevant case was shared with the editors of heise online on Sunday.

In this case, an X account had prompted Grok in a post to convert a photo of a woman into a bikini image (as the content infringes on the rights of the affected woman, the source is not linked in this case). On X, this is possible by tagging Grok's official X account in a post with the corresponding prompt and the original image – without consent, however, this constitutes a massive violation of the affected person's personal rights, is very likely perceived as degrading by them, and is also illegal.

Image not removed after over 24 hours

Comments on the aforementioned X post indicate that users reported the case as a violation of EU law over 24 hours ago. However, it is not possible to independently verify whether this actually happened (this article may be updated). Given the extent to which sexual deepfakes have spread on X, the content moderation team at X is likely massively overloaded.

As early as January 2nd, it was posted via the Grok account on X that there was a "failure of safeguards" which was currently being fixed. However, xAI, Elon Musk's operating company for Grok, and X have apparently not taken any temporary measures to contain the problem – such as disabling the image generator on X or even completely.

Problem is not being contained

The authors of Grok's statements on the scandal also remain unclear. There is no explanation from the xAI management, and the company even dismissed the problem as a "lie by the traditional media" to the news agency Reuters.

Grok itself also denies in its responses that images of this nature can be generated with the AI. In the present case, Grok had denied to the affected woman that images of real people could be created without their consent. Only after Grok was confronted with the relevant image as an example in a later prompt did the chatbot admit its misconduct – the image is still available on X in the meantime. Grok itself refers in its responses to the content moderation team, which is responsible for deleting such content and also for blocking accounts that prompt Grok to create such images.

And even if the content disappears from the platform in the original, screenshots often continue to circulate. The public prosecutor's office in Paris confirmed to Politico that the incidents will be included in an ongoing investigation against X. This already includes antisemitism and Holocaust denial by Grok. A follow-up at the EU level is still pending. In December, the EU already imposed a fine of 120 million euros on X for violations of the European Digital Services Act (DSA).

(nen)