The social media network X has caused major anger after people employed its AI chatbot Grok to manipulate pictures of females by taking off their clothing.
Several cases have emerged where the system has been employed to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to insert them in sexually explicit contexts.
Developer Reaction and Legal Stance
XAI, the organization responsible for Grok, did not respond to requests for comment, other than an auto-reply response stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to prohibit image stripping software. Under a proposed law, anyone who provided such technology would "receive a jail term and heavy penalties".
"The platform is capable of halting these forms of abuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the risk" of people accessing unlawful material on their platforms, but did not specify whether it was currently investigating X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some additional features—which answers X users' prompts when they reference it in a post.
It is often used to give reactions or further background to other users' comments, but people on X are also can edit an posted picture through its AI image editing function.
Background of Criticism and Rules
The system has been criticised for enabling users to create photos and footage with explicit content and sexualised content, and it was previously alleged to have been involved in generating graphic content of celebrities.
"X has been enabling the generation and sharing of these pictures for a long time without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing likenesses of persons in a graphic manner".
In a statement, Ofcom said it was unlawful to "create or share deepfakes or illegal content" and stated this included sexual deepfakes made with AI.
It said platforms such as X were required to take "necessary steps" to "lessen the danger" of users viewing illegal content on their platforms and to delete it swiftly when they are notified of it.
