Elon Musk's online network X has caused widespread anger after individuals utilized its artificial intelligence assistant Grok to manipulate photos of females by taking off their garments.
Several cases have surfaced where the tool has been employed to remove clothing from women to show them in bikinis without consent, as well as to place them in sexualized scenarios.
Company Reaction and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to inquiries, aside from an automatically-generated response stating "legacy media lies".
A government representative stated that legislation is being proposed to ban nudification tools. Under a proposed law, anyone who distributed such tools would "face a jail term and significant penalties".
"X or Grok could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "consider the danger" of people accessing illegal content on their networks, but did not confirm whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some paid-for features—which responds to X users' prompts when they mention it in a post.
It is commonly used to offer commentary or more background to other posters' remarks, but people on X are also can alter an posted picture through its AI image editing feature.
History of Criticism and Policy
The system has been criticised for allowing users to generate pictures and videos with explicit content and sexually suggestive content, and it was earlier accused of generating inappropriate content of public figures.
"X has been enabling the production and distribution of these pictures for months without taking any steps and we have not seen any challenge by regulators," the expert added.
XAI's own terms of service bans "portraying images of persons in a graphic way".
In a statement, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and stated this included explicit AI images made with AI.
It said services such as X were obligated to take "suitable steps" to "minimize the danger" of users encountering unlawful material on their services and to take down it swiftly when they become aware of it.
