🔗 Share this article

Tech firms and child protection organizations will receive permission to evaluate whether artificial intelligence systems can generate child abuse images under recently introduced UK laws.

Significant Increase in AI-Generated Harmful Content

The declaration came as findings from a safety watchdog showing that reports of AI-generated child sexual abuse material have more than doubled in the last twelve months, growing from 199 in 2024 to 426 in 2025.

New Legal Framework

Under the changes, the government will permit approved AI companies and child safety groups to examine AI systems – the underlying systems for conversational AI and visual AI tools – and ensure they have adequate protective measures to prevent them from producing images of child exploitation.

"Fundamentally about preventing abuse before it occurs," stated Kanishka Narayan, noting: "Experts, under strict conditions, can now identify the danger in AI systems promptly."

Tackling Legal Challenges

The changes have been implemented because it is illegal to create and possess CSAM, meaning that AI creators and others cannot generate such images as part of a evaluation process. Previously, authorities had to wait until AI-generated CSAM was published online before addressing it.

This law is designed to averting that issue by enabling to halt the creation of those images at source.

Legal Structure

The amendments are being added by the authorities as revisions to the crime and policing bill, which is also implementing a prohibition on possessing, creating or distributing AI models designed to generate exploitative content.

Practical Consequences

This week, the official toured the London headquarters of a children's helpline and heard a mock-up conversation to advisors featuring a report of AI-based abuse. The call depicted a teenager seeking help after facing extortion using a sexualised deepfake of himself, created using AI.

"When I hear about children experiencing blackmail online, it is a source of extreme anger in me and justified concern amongst parents," he said.

Concerning Statistics

A leading internet monitoring foundation reported that instances of AI-generated abuse content – such as online pages that may contain numerous images – had more than doubled so far this year.

Cases of the most severe material – the most serious form of abuse – rose from 2,621 visual files to 3,086.

• Female children were overwhelmingly targeted, making up 94% of prohibited AI images in 2025
• Depictions of newborns to toddlers rose from five in 2024 to 92 in 2025

Sector Reaction

The law change could "constitute a crucial step to guarantee AI products are safe before they are launched," stated the head of the online safety foundation.

"Artificial intelligence systems have enabled so victims can be victimised repeatedly with just a simple actions, giving criminals the capability to make possibly endless quantities of sophisticated, photorealistic child sexual abuse material," she continued. "Material which further exploits survivors' suffering, and renders children, especially girls, more vulnerable on and off line."

Support Interaction Information

The children's helpline also published information of counselling sessions where AI has been mentioned. AI-related risks mentioned in the sessions include:

• Employing AI to evaluate body size, body and looks
• AI assistants dissuading children from consulting trusted adults about abuse
• Facing harassment online with AI-generated content
• Online blackmail using AI-manipulated images

Between April and September this year, Childline conducted 367 counselling interactions where AI, conversational AI and associated terms were mentioned, significantly more as many as in the equivalent timeframe last year.

Fifty percent of the references of AI in the 2025 sessions were related to psychological wellbeing and wellness, encompassing using chatbots for assistance and AI therapeutic apps.