The social media network X has provoked widespread backlash after people used its AI chatbot Grok to alter images of women by removing their clothing.
Numerous examples have been seen where the technology has been employed to digitally undress female subjects to make them appear in bikinis non-consensually, as well as to insert them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the firm behind Grok, offered no comment to questions, aside from an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to prohibit nudification technology. Under a upcoming criminal offence, anyone who distributed such technology would "be subject to a jail term and heavy penalties".
"The platform could prevent these forms of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said platforms must "consider the danger" of people viewing illegal content on their platforms, but did not confirm whether it was presently examining X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some paid-for functions—which responds to X users' prompts when they mention it in a message.
It is often used to provide reactions or additional background to other posters' comments, but people on X are also able to modify an uploaded image through its artificial intelligence picture modification feature.
History of Criticism and Rules
The tool has been condemned for enabling users to produce images and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in making graphic content of celebrities.
"X has been enabling the production and spread of these images for months without taking any steps and we have not seen any intervention by regulators," the expert added.
XAI's own usage rules prohibits "depicting likenesses of persons in a graphic way".
In a comment, Ofcom said it was illegal to "create or share private pictures or illegal content" and stated this included sexual deepfakes made with AI.
It said platforms such as X were obligated to take "necessary steps" to "reduce the danger" of users coming across illegal material on their services and to remove it swiftly when they become aware of it.
