The online network X has caused significant outrage after users employed its AI chatbot Grok to alter photos of females by removing their attire.
Several examples have surfaced where the system has been employed to digitally undress female subjects to render them in bikinis non-consensually, as well as to insert them in sexually explicit situations.
Company Response and Legal Stance
XAI, the organization behind Grok, remained silent to inquiries, except for an auto-reply response stating "legacy media lies".
A Home Office official stated that new laws is being introduced to prohibit nudification software. Under a proposed law, anyone who supplied such technology would "receive a prison sentence and substantial penalties".
"The platform could prevent these forms of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "consider the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was at this time examining X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for features—which answers X users' requests when they tag it in a tweet.
It is commonly used to provide reactions or additional context to other users' remarks, but people on X are also capable of modify an uploaded image through its AI image editing feature.
Background of Controversy and Policy
The chatbot has been faulted for enabling users to generate pictures and footage with nudity and sexualised material, and it was in the past accused of making graphic material of public figures.
"The platform has been allowing the production and distribution of these pictures for months without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "showing images of persons in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or illegal content" and confirmed this included explicit AI images made with AI.
It said services such as X were required to take "necessary steps" to "minimize the danger" of users encountering illegal material on their platforms and to delete it promptly when they are notified of it.
