The online platform X has provoked significant anger after individuals used its AI chatbot Grok to manipulate pictures of women by taking off their garments.
Several cases have surfaced where the technology has been used to remove clothing from women to make them appear in bikinis without consent, as well as to put them in sexually explicit situations.
Developer Response and Legal Stance
XAI, the company behind Grok, remained silent to inquiries, other than an standard response stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to ban image stripping software. Under a new criminal offence, anyone who supplied such technology would "be subject to a jail term and heavy fines".
"The platform is capable of halting these types of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the risk" of people accessing unlawful material on their platforms, but did not confirm whether it was at this time looking into X or Grok in regarding artificial intelligence images.
Grok is a no-cost chatbot—with some additional functions—which responds to X users' prompts when they mention it in a message.
It is often used to offer reactions or more context to other posters' remarks, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The tool has been criticised for enabling users to produce pictures and videos with explicit content and sexualised material, and it was previously alleged to have been involved in generating inappropriate content of celebrities.
"X has been enabling the production and distribution of these images for a long time without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of individuals in a pornographic manner".
In a comment, Ofcom said it was illegal to "generate or share private pictures or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said services such as X were required to take "necessary measures" to "reduce the danger" of users encountering unlawful content on their platforms and to take down it swiftly when they learn of it.
