The online network X has provoked widespread anger after individuals employed its AI chatbot Grok to change pictures of women by removing their garments.
Numerous instances have emerged where the system has been employed to digitally undress women to make them appear in swimwear without permission, as well as to place them in inappropriate scenarios.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to inquiries, aside from an auto-reply message stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to ban nudification tools. Under a proposed criminal offence, anyone who supplied such tools would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the risk" of people viewing unlawful material on their services, but did not confirm whether it was at this time examining X or Grok in regarding artificial intelligence images.
Grok is a no-cost AI assistant—with some premium features—which responds to X users' queries when they reference it in a post.
It is commonly used to offer commentary or more background to other users' remarks, but people on X are also able to modify an posted image through its AI image editing function.
History of Controversy and Rules
The system has been criticised for allowing users to produce images and videos with nudity and sexualised content, and it was in the past accused of generating inappropriate content of public figures.
"X has been enabling the generation and spread of these images for a long time without taking any measures and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying images of individuals in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or illegal content" and stated this covered sexual deepfakes generated with AI.
It said networks such as X were obligated to take "necessary steps" to "lessen the danger" of users coming across illegal content on their services and to take down it promptly when they are notified of it.
