We need your help now

Support from readers like you keeps The Journal open.

You are visiting us because we have something you value. Independent, unbiased news that tells the truth. Advertising revenue goes some way to support our mission, but this year it has not been enough.

If you've seen value in our reporting, please contribute what you can, so we can continue to produce accurate and meaningful journalism. For everyone who needs it.

Alamy Stock Photo

EU 'looking into' AI tool on X that lets users generate nude images of anyone, including children

The tool allows users to request modifications to a photo and receive an AI-generated edited version in response.

LAST UPDATE | 5 Jan

IRISH MEDIA REGULATOR Coimisiún na Meán has said it is engaging with the European Commission after complaints that Elon Musk’s AI tool Grok is being used to generate and disseminate sexually explicit childlike images.

A spokesperson for Coimisiún na Meán told The Journal that the “sharing of non-consensual intimate images is illegal”, as is the “generation of child sexual abuse material” as reported from users of the site.

It allows users to request modifications to a photo and receive an AI-generated edited version in response. Some X users have used the tool to change or remove clothing from real people in photographs.

In response to complaints about the tool being used to manipulate images of minors, Grok itself acknowledged “lapses in safeguards” and said that child sexual abuse material is illegal and prohibited.

The European Commission is now “very seriously looking” into complaints that Grok is being used to generate and disseminate sexually explicit childlike images.

Under the Digital Services Act, the European Commission is responsible for the oversight of very large online platforms with their requirements to assess and mitigate risks that their services may create in relation to the proliferation of illegal content online.

This includes the protection for minors.

“Coimisiún na Meán is engaging with the European Commission in this context, in relation to Grok,” the the Coimisiún na Meán spokesperson said.

“If a member of the public is concerned about images shared online, they should report it to An Garda Síochána. Reports can also be made to the Irish national reporting centre, Hotline.ie.”

Coimisiún na Meán also encourages users to report illegal content to the online platform where they saw it and to Coimisiún na Meán.

An “edit image” function available to X users was introduced on Grok in late December.

Discussing the concerns earlier, EU digital affairs spokesman Thomas Regnier told reporters that Grok is “now offering a ‘spicy mode’ showing explicit sexual content with some output generated with childlike images”.

This is not spicy. This is illegal. This is appalling. This has no place in Europe.

The AI image edit function also has the capacity to create other kinds of false and misleading ‘photos’.

In a post that was re-shared by Elon Musk, one X user showed how they used it on a photo of ousted Venezuelan President Nicolas Maduro, who was captured by the United States on Saturday, to make it look like he was wearing a prison uniform after being captured.

Musk himself has been lauding the image edit tool and leaning in to its use for creating humorous content, sharing many images generated by his followers in recent days. 

One woman, Samantha Smith, told the BBC she felt “dehumanised” after Grok was used to change a photo in which she was wearing a dress to appear as if she was wearing a bikini.  She told the BBC that it “felt as violating as if someone had actually posted a nude or a bikini picture of me” and that “women are not consenting to this”. 

Samantha Smith A post by Samantha Smith sharing the original photo (left) and an edited version that was generated by AI without her consent (right) Samantha Smith Samantha Smith

In France, the public prosecutor’s office has expanded an investigation into X to include new accusations that Grok was being used for generating and disseminating child pornography. The initial investigation was opened in July after reports that X’s algorithm was being manipulated for the purpose of foreign interference. In India, media outlets reported on Friday that government officials have demanded X provide them details of measures it is taking to remove “obscene, nude, indecent, and sexually suggestive content” created by Grok.
Contains additional reporting by Eoghan Dalton and AFP

Readers like you are keeping these stories free for everyone...
A mix of advertising and supporting contributions helps keep paywalls away from valuable information like this article. Over 5,000 readers like you have already stepped up and support us with a monthly payment or a once-off donation.

Close
JournalTv
News in 60 seconds