The social media platform X has provoked significant anger after users utilized its artificial intelligence assistant Grok to change images of females by removing their attire.
Numerous instances have emerged where the system has been employed to digitally undress women to render them in swimwear without permission, as well as to put them in sexualized situations.
Developer Response and Regulatory Stance
XAI, the organization that created Grok, offered no comment to requests for comment, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to ban image stripping software. Under a upcoming criminal offence, anyone who supplied such technology would "receive a jail term and substantial fines".
"The platform has the ability to stop these types of abuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the risk" of people accessing illegal content on their networks, but did not specify whether it was currently examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some additional features—which answers X users' requests when they mention it in a tweet.
It is commonly used to give reactions or more background to other posters' remarks, but people on X are also able to modify an posted photo through its artificial intelligence picture modification tool.
History of Criticism and Rules
The tool has been criticised for permitting users to create pictures and footage with explicit content and sexualised content, and it was earlier accused of making graphic material of public figures.
"X has been enabling the generation and distribution of these pictures for a long time without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "depicting likenesses of persons in a pornographic fashion".
In a statement, Ofcom said it was against the law to "make or circulate non-consensual intimate images or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said services such as X were mandated to take "necessary steps" to "reduce the risk" of users viewing unlawful material on their services and to remove it promptly when they learn of it.
