The social media platform X has sparked significant outrage after individuals utilized its artificial intelligence assistant Grok to alter pictures of females by stripping their garments.
Multiple examples have emerged where the tool has been employed to virtually strip female subjects to render them in swimwear non-consensually, as well as to put them in sexualized contexts.
Developer Reaction and Legal Position
XAI, the company behind Grok, did not respond to inquiries, except for an automatically-generated response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to ban image stripping software. Under a upcoming law, anyone who supplied such technology would "be subject to a jail term and heavy fines".
"X or Grok could prevent these forms of misuse if they desired to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said tech firms must "consider the danger" of people accessing unlawful material on their platforms, but did not specify whether it was currently examining X or Grok in relation to AI-generated images.
Grok is a free artificial intelligence—with some paid-for functions—which reacts to X users' requests when they mention it in a tweet.
It is frequently used to give reactions or additional background to other users' remarks, but people on X are also can alter an uploaded photo through its AI image editing feature.
Background of Controversy and Rules
The tool has been criticised for allowing users to create images and footage with explicit content and sexualised material, and it was in the past alleged to have been involved in generating sexually explicit content of public figures.
"The platform has been enabling the creation and spread of these pictures for months without taking any steps and we have not seen any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting likenesses of persons in a pornographic way".
In a comment, Ofcom said it was unlawful to "create or share non-consensual intimate images or child sexual abuse material" and confirmed this included explicit AI images made with AI.
It said networks such as X were required to take "appropriate steps" to "reduce the danger" of users encountering unlawful content on their platforms and to delete it promptly when they learn of it.
