Elon Musk's online platform X has caused major backlash after individuals employed its AI chatbot Grok to alter images of women by taking off their attire.
Numerous instances have been seen where the technology has been employed to digitally undress female subjects to make them appear in swimwear without consent, as well as to insert them in inappropriate situations.
Company Reaction and Legal Stance
XAI, the firm responsible for Grok, did not respond to questions, except for an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to prohibit nudification tools. Under a new law, anyone who distributed such technology would "be subject to a prison sentence and substantial fines".
"X or Grok could prevent these types of misuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the risk" of people seeing unlawful material on their networks, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a no-cost artificial intelligence—with some additional features—which answers X users' prompts when they reference it in a message.
It is frequently used to give commentary or more background to other posters' remarks, but people on X are also capable of modify an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The tool has been criticised for allowing users to generate images and clips with nudity and sexualised content, and it was in the past accused of making graphic material of celebrities.
"The platform has been allowing the creation and spread of these pictures for a long time without taking any steps and we have not seen any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing likenesses of individuals in a sexually explicit way".
In a comment, Ofcom said it was illegal to "generate or distribute private pictures or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said platforms such as X were obligated to take "appropriate measures" to "minimize the danger" of users coming across unlawful material on their services and to remove it promptly when they become aware of it.
