OpenAI offers free child-abuse image classifier and adds 24-hour ChatGPT suspensions
OpenAI on November 16 released a classifier that detects child sexual abuse material, including AI-generated images, and is giving it free to vetted organizations through its API. ChatGPT users who try to create such content now get a warning and a 24-hour suspension.
Cortex, covering AI models, labs and agents
, 3 min read
48%fewer images flagged in OpenAI's tests, same abuse caught
What to know
- OpenAI developed and evaluated the classifier with the National Center for Missing & Exploited Children (NCMEC); it scores images it has never seen before so human reviewers can prioritize them.
- In OpenAI's early tests the classifier flagged 48% fewer images than existing protections while catching about the same amount of abusive material.
- ChatGPT users who try to create or access sexualized content involving children see a warning and lose access for 24 hours; repeat offenses can lead to account closure.
OpenAI on November 16 released an image classifier that detects child sexual abuse material (CSAM), including images made with AI, and said it will give the tool free through its API to vetted organizations. It also tightened ChatGPT's response to people who try to create such material, adding a warning screen and a 24-hour suspension, according to reports from Fox News, SQ Magazine and BigGo Finance.
What the classifier does
Most platforms find known abuse images with hash matching, which compares a file's fingerprint against a database of images already identified. That method misses new material, including images generated by AI. OpenAI's classifier instead looks for visual signs of abuse in images it has not seen before and assigns each one a score, so human reviewers can look at the most likely cases first, Fox News and SQ Magazine reported.
OpenAI developed and evaluated the technology with the National Center for Missing & Exploited Children (NCMEC), the US nonprofit that runs the CyberTipline where platforms report suspected abuse. Access goes to a select group of vetted organizations, and OpenAI says it is working with the Tech Coalition and ROOST, an online-safety nonprofit, to widen it, BigGo reported. OpenAI has not said how many organizations have access or how they are vetted.
Test results
In OpenAI's evaluation of images uploaded to its own systems, the classifier flagged 48% fewer pictures for review than existing protections while detecting roughly the same amount of abusive material, Fox News reported. Images it flagged were about twice as likely to contain abuse. Two partner companies reported their own results through OpenAI: Fal said reviewers confirmed 72% of high-confidence flags as previously unidentified abuse material, and SafetyKit reported detecting 90% of drawings or animations that sexualize children.
All of these figures come from OpenAI and its partners. SQ Magazine noted that the test size and the rate of missed images were not disclosed. John Buckley, OpenAI's head of child safety public policy, said "that triage is critical in law enforcement reporting flows," according to BigGo.
Changes in ChatGPT
ChatGPT users who try to create or access content that sexualizes children now see a warning, lose access for 24 hours and are pointed to specialist support, SQ Magazine and BigGo reported. Repeat offenses can lead to permanent account closure. OpenAI wrote the warning text with the child-protection groups Protect Children and the Lucy Faithfull Foundation and with Joel Scanlan of the University of Tasmania, who evaluated a trial of the deterrence messages. The change is rolling out gradually to personal accounts worldwide and does not yet cover enterprise and education accounts, SQ Magazine reported.
OpenAI also published guidance for developers on pairing hash matching with classifiers, and tools and training for law enforcement.
Why it matters
AI image generators have flooded abuse tip lines with material that hash databases cannot recognize. NCMEC's CyberTipline logged about 400,000 reports tied to generative AI in 2025, up from 67,000 a year earlier, according to figures OpenAI cited and BigGo reported. A free classifier that sorts new images for reviewers could help smaller platforms and investigators keep up, if outside testing confirms OpenAI's results. The release follows OpenAI's decision to let outside safety labs test its models from inside, and comes as AI companies tighten their rules on misuse, as Anthropic did with its new usage policy. For background on how labs build and grade such systems, see our explainer on benchmark scores.
Sources
- OpenAI child safety tools target AI-generated abuse, Fox News, November 16, 2026
- OpenAI Launches New CSAM Detection and ChatGPT Warnings, SQ Magazine, November 16, 2026
- OpenAI Debuts CSAM Image Classifier as AI-Generated Abuse Reports Flood Tip Lines, BigGo Finance, November 16, 2026
OpenAIAI law and policyAI models
Written by an AI correspondent under standards set by editor Jacob Knight. Spotted an error? Tell us and we will fix it.