The two photos started out as typical teenage selfies: looking into the mirror, fully clothed. But once online predators had got hold of those pictures and ran them through an AI imaging tool, they had become the basis for extreme pornography videos.
These examples come from the Report Remove service, which allows children who have had explicit pictures of themselves distributed without their consent to flag the image confidentially and have it blocked or taken down from social media. Due to breakthroughs in AI, and the wide availability of AI models and nudification apps, some under-18s are becoming victims without even being in contact with criminals.
These two victims show why the UK’s National Crime Agency (NCA) and the Internet Watch Foundation (IWF), a child safety watchdog, have issued guidance telling parents to consider limiting visibility of family pictures by, for instance, making your social media profile private and just sharing photos of your children on “close friends” groups.
Both organisations say they are not telling parents what to do but want them to be aware of the problem and what action to take. The data shows AI is an increasing source of child sexual abuse material. The amount of AI-generated child sexual abuse material [CSAM] found online rose by 14% last year, according to the IWF, which identified 8,029 AI-made images and videos of realistic CSAM in 2025.
Dan Sexton, the IWF’s chief technology officer, says he feels “very uncomfortable” about telling parents not to put pictures of their children on public display but feels he has no other option. There is not enough protection against this technology, he says.
The UK government is bringing in restrictions, such as making it illegal to possess, create or distribute AI tools designed to generate child sexual abuse material. It is also giving tech companies and child protection agencies the power to test whether artificial intelligence tools can produce child abuse images.
Sexton acknowledges the long-term solution cannot be banishing photos from public social media accounts. He wants legislation for AI models that requires them to be “safe by design”, a principle voiced by online safety campaigners who want social media platforms to be built responsibly with user wellbeing as a core aim. “People use the term safety by design but I am not seeing that here,” he says.
The Department for Science, Innovation and Technology has been approached for comment.
Sexton says paedophiles are using a variety of AI tools in combination to create images and videos, including “open source” AI models that can be freely downloaded and adjusted by users.
The IWF says that already it is proving impossible to distinguish between real photos of child sexual abuse and AI-generated…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
