Today, a prominent child safety organization, Thorn, in partnership with a leading cloud-based AI solutions provider, Hive, announced the release of an AI model designed to flag unknown CSAM at upload. It’s the earliest AI technology striving to expose unreported CSAM at scale.
So doesn’t that make the law enforcement having the biggest CP collection from everybody? This sounds kinda dangerous…
It does. Kinda.
The police are seldom allowed to be in possession of CSAM, except for in terms of grabbing the hardware which contains it in an arrest. The database used in modern detection tools is maintained by NCMEC which has special permission to do so.
And of course there are risks, but it’s just digital data. Unless you are creating more, you’re not actively harming anyone. And law enforcement absolutely needs that data to take some of the most obvious steps to prevent it being spread further.
Obviously, someone has access, but to get to the actual media files wouldn’t be simple. What typically happens, is that anyone wanting to detect CSAM, is given a hashed version of the database. They can then scan their systems for CSAM by hashing any media they are hosting, and seeing whether there are any matches.
Whenever possible, people aren’t handling the actual media. But for any detection to be possible to begin with, the database of the actual media does need to be maintained somewhere.
AI is a touchier subject, as you can’t train a model to recognize CSAM not already in the database using hashes, so in those cases you have to work with actual real media. This is only recently becoming a thing.
It also leaves open the possibility for false positives. An oft cited example is parents taking pictures of their own children for innocent reasons, or doctors and parents handling images for valid medical reasons. In a system that flagged such content, it would mean someone else would be seeing that “private” content because it was flagged.