The social media platform X has sparked significant outrage after users utilized its AI chatbot Grok to alter photos of females by stripping their clothing.
Several instances have surfaced where the technology has been employed to virtually strip women to render them in bikinis without consent, as well as to insert them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the firm responsible for Grok, remained silent to questions, other than an automatically-generated message stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to outlaw nudification software. Under a new criminal offence, anyone who distributed such tools would "face a jail term and heavy fines".
"X or Grok has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said tech firms must "assess the potential" of people seeing unlawful material on their services, but did not specify whether it was currently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which reacts to X users' queries when they tag it in a message.
It is commonly used to provide commentary or additional context to other users' remarks, but people on X are also capable of alter an uploaded picture through its AI image editing function.
History of Criticism and Policy
The chatbot has been faulted for enabling users to create images and clips with nudity and sexually suggestive material, and it was earlier alleged to have been involved in creating graphic content of public figures.
"X has been allowing the creation and sharing of these images for a long time without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of people in a pornographic way".
In a statement, Ofcom said it was illegal to "make or share non-consensual intimate images or illegal content" and stated this applied to explicit AI images created with AI.
It said networks such as X were required to take "necessary steps" to "lessen the danger" of users coming across unlawful content on their services and to take down it promptly when they learn of it.
