Elon Musk's social media network X has sparked major outrage after individuals employed its AI chatbot Grok to manipulate images of females by taking off their attire.
Multiple cases have surfaced where the tool has been used to remove clothing from female subjects to make them appear in swimwear non-consensually, as well as to put them in sexualized contexts.
Developer Reaction and Legal Stance
XAI, the organization that created Grok, offered no comment to requests for comment, except for an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to prohibit nudification software. Under a proposed law, anyone who supplied such technology would "receive a prison sentence and substantial fines".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the risk" of people seeing illegal content on their services, but did not confirm whether it was currently examining X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some paid-for features—which responds to X users' requests when they tag it in a message.
It is commonly used to give reactions or further context to other users' remarks, but people on X are also can alter an posted picture through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been faulted for enabling users to create pictures and footage with nudity and sexualised content, and it was in the past accused of generating inappropriate content of celebrities.
"The platform has been enabling the production and spread of these images for a long time without taking any action and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying likenesses of persons in a sexually explicit manner".
In a comment, Ofcom said it was against the law to "make or share non-consensual intimate images or illegal content" and stated this covered sexual deepfakes created with AI.
It said networks such as X were obligated to take "necessary measures" to "reduce the risk" of users viewing illegal material on their services and to delete it promptly when they learn of it.
