The online network X has sparked widespread outrage after users employed its AI chatbot Grok to change pictures of women by taking off their attire.
Multiple examples have been seen where the tool has been used to digitally undress female subjects to show them in swimwear without consent, as well as to insert them in inappropriate situations.
Company Response and Regulatory Stance
XAI, the company that created Grok, offered no comment to inquiries, aside from an automatically-generated message stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to outlaw nudification tools. Under a upcoming criminal offence, anyone who distributed such tools would "receive a jail term and substantial fines".
"X or Grok has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people seeing illegal content on their platforms, but did not specify whether it was currently examining X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some premium features—which responds to X users' requests when they tag it in a post.
It is commonly used to give commentary or further context to other posters' comments, but people on X are also can edit an uploaded picture through its AI image editing feature.
History of Criticism and Policy
The chatbot has been faulted for allowing users to produce photos and clips with nudity and sexualised content, and it was in the past alleged to have been involved in creating sexually explicit content of celebrities.
"The platform has been allowing the generation and sharing of these images for months without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of individuals in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or distribute private pictures or child sexual abuse material" and confirmed this included explicit AI images created with AI.
It said networks such as X were required to take "appropriate actions" to "minimize the risk" of users viewing illegal material on their services and to delete it promptly when they are notified of it.
