The online network X has sparked widespread backlash after individuals employed its AI chatbot Grok to change photos of women by removing their clothing.
Several examples have emerged where the tool has been employed to virtually strip women to render them in swimwear without consent, as well as to place them in inappropriate situations.
Company Reaction and Regulatory Position
XAI, the company responsible for Grok, did not respond to requests for comment, except for an automatically-generated response stating "legacy media lies".
A government representative stated that new laws is being introduced to ban nudification software. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a jail term and heavy fines".
"The platform is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the risk" of people seeing unlawful material on their services, but did not confirm whether it was currently looking into X or Grok in relation to artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which responds to X users' queries when they mention it in a tweet.
It is often used to give commentary or more background to other posters' remarks, but people on X are also able to alter an uploaded image through its AI image editing feature.
History of Criticism and Policy
The chatbot has been criticised for allowing users to produce images and videos with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in creating graphic content of celebrities.
"X has been allowing the production and sharing of these images for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "showing images of persons in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate steps" to "reduce the risk" of users viewing illegal content on their platforms and to remove it quickly when they become aware of it.
