The social media network X has sparked significant outrage after people utilized its AI chatbot Grok to manipulate pictures of females by removing their attire.
Numerous cases have been seen where the system has been used to remove clothing from women to render them in bikinis without consent, as well as to insert them in sexually explicit contexts.
Developer Reaction and Legal Position
XAI, the company behind Grok, did not respond to questions, except for an auto-reply response stating "legacy media lies".
A government official stated that legislation is being brought forward to prohibit nudification technology. Under a proposed law, anyone who distributed such tools would "receive a prison sentence and substantial fines".
"X or Grok could prevent these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people seeing unlawful material on their platforms, but did not confirm whether it was at this time examining X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some paid-for functions—which reacts to X users' prompts when they mention it in a tweet.
It is commonly used to offer commentary or additional background to other users' comments, but people on X are also capable of edit an posted image through its AI image editing tool.
History of Controversy and Policy
The chatbot has been criticised for allowing users to create images and clips with explicit content and sexually suggestive content, and it was earlier accused of making graphic material of celebrities.
"The platform has been enabling the generation and distribution of these pictures for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting likenesses of people in a sexually explicit way".
In a comment, Ofcom said it was illegal to "create or share deepfakes or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.
It said platforms such as X were required to take "suitable measures" to "lessen the danger" of users viewing illegal content on their platforms and to take down it promptly when they become aware of it.
