Elon Musk's online platform X has provoked widespread outrage after people utilized its AI chatbot Grok to manipulate images of women by stripping their clothing.
Multiple instances have surfaced where the tool has been used to remove clothing from women to render them in bikinis without consent, as well as to place them in sexualized scenarios.
Developer Reaction and Legal Stance
XAI, the company that created Grok, remained silent to requests for comment, other than an auto-reply message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being proposed to prohibit nudification technology. Under a upcoming criminal offence, anyone who provided such technology would "face a jail term and heavy penalties".
"The platform has the ability to stop these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their platforms, but did not confirm whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some premium features—which answers X users' requests when they mention it in a tweet.
It is frequently used to offer commentary or more context to other posters' comments, but people on X are also able to alter an uploaded photo through its AI image editing tool.
Background of Controversy and Rules
The chatbot has been criticised for allowing users to produce photos and clips with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in making sexually explicit material of celebrities.
"The platform has been allowing the generation and spread of these images for months without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of individuals in a graphic fashion".
In a comment, Ofcom said it was against the law to "generate or distribute private pictures or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "suitable steps" to "reduce the danger" of users encountering illegal content on their platforms and to remove it quickly when they are notified of it.
