AINews wire, Yesterday

OpenAI releases classifier to flag AI-generated child abuse images for investigators

Reported by The Verge, Mashable Tech

Image: Mashable Tech

OpenAI has announced a classifier that it says can detect child sexual abuse material (CSAM), including images made with AI. The company unveiled it at a briefing in Washington, DC, on a Monday. The aim is to help investigators sort through large volumes of reports and focus quickly on material they can act on.

The classifier scores images for visual signs of abuse, so human reviewers can decide which ones to examine first. It includes a specific signal that an image may be AI-generated. OpenAI describes it as the first broadly available tool of its kind with that feature. According to the company, it flags fewer false positives: it could flag 48 percent fewer images while finding roughly the same number of abusive images as OpenAI's existing tool.

Access is limited for now. The tool can be used by organizations that are required or authorized to detect CSAM, and by certain developers that use OpenAI's API. OpenAI is also offering free access to vetted organizations, guidance for developers on detection, and training for law enforcement on how AI can support investigations. The classifier also works with Coop, open-source detection and reporting software from the nonprofit ROOST, which lets users flag and review content and report it to the National Center for Missing and Exploited Children (NCMEC). OpenAI published a model card with testing details. ROOST's president described it as the first of its kind in the field.

The scale of the problem is the stated reason for the tool. The CyberTipline received 400,000 generative AI-related reports in 2025, up from 67,000 in 2024, according to The Verge. Mashable, citing an analysis by Childlight at the University of Edinburgh, reports that AI-related CSAM reports rose by more than 8,400 percent from 2023 to 2025, from roughly 4,700 in 2023. North Carolina Attorney General Jeff Jackson, who co-chairs an AI task force of state attorneys general, called the influx 'explosive' at the event.

OpenAI has also changed how its ChatGPT image generator responds. It already denies requests for CSAM. Now a user who tries to produce such material sees a warning with support resources. A first offense triggers a 24-hour pause in access to ChatGPT and a notice that the account could be permanently closed. OpenAI says the pause follows expert advice and international evidence on prevention. The company states that cases it finds are reported to NCMEC.

The sources note the legal background. According to The Verge, the US government treats both AI-generated and non-AI-generated CSAM as illegal, and 48 states have laws covering sexually explicit AI deepfakes. The Take It Down Act requires social media platforms to remove real or AI-generated nonconsensual intimate images within 48 hours. Jackson called the partnership with OpenAI an experiment. Sources do not say how the classifier performs outside OpenAI's own testing, or how many organizations will receive access.

Enjoying this? Get the AI brief

Every Thursday: AI that matters for your home.

✓ Free · seven editions · one click to leave

Topics
Story continues on The VergeAdd us as a preferred source on GoogleSee our stories first in Google’s Top Stories.
Get alerts on big newsOnly the big news, at most once a day. No email, no account.

How each source covered it

Original reporting by the sources above. Summary by smart-home.space.