Twitter Missed Dozens of Known Images of Child Sexual Abuse Material, Researcher

The researchers at the Stanford Internet Observatory, who were investigating child-safety issues across several platforms, said they told Twitter staff about their findings, and that the problem appeared to have been resolved in May.

The researchers said Twitter told them last week it had improved some aspects of its detection system, and asked the researchers to alert the company if they ever notice a spike in such cases in the future.

Twitter didn’t comment in response to an email from The Wall Street Journal about the researchers’ report.

In just over two months, from March 12 to May 20, the researchers’ system detected more than 40 images posted to Twitter that were previously flagged as child sexual abuse material, based on a data set of roughly 100,000 tweets, said David Thiel, chief technologist of the Stanford Internet Observatory and a co-author of the report.

The appearance of the images on Twitter was striking because they had been previously flagged as child sexual abuse material, or CSAM, and were part of databases companies can use to screen content posted to their platforms, the researchers said. “This is one of the most basic things you can do to prevent CSAM online, and it did not seem to be working," Thiel said.

The challenge of blocking child sexual abuse material has long been a problem for many internet platforms, not just Twitter. After acquiring Twitter in late October, Elon Musk placed an emphasis on the issue, vowing in tweets that removing such material from Twitter is “priority #1" and “will forever be our top priority."

Musk didn’t respond to a request for comment.

For the month of January, Twitter said it suspended about 404,000 accounts that created or engaged with child sexual exploitation material, which the company said represented a 112% increase in such suspensions since November. “Not only are we detecting more bad actors faster, we’re building new defenses that proactively reduce the discoverability of tweets that contain this type of content," Twitter said in a February tweet.

Many academic researchers have said a change in Twitter’s policy over access to its data will make it unlikely they could catch lapses like the Stanford group detected. Twitter has said it is raising prices for access to its application programming interface, or API, which lets researchers analyze data, and that big users could be charged $42,000 monthly or more.

Many academic researchers say those prices will make the API unaffordable for them, hindering future research on Twitter. “This is a significant blow to platform transparency," Thiel said.

The Stanford Internet Observatory, which researches abuses on social media and other internet platforms, last week stopped using Twitter’s enterprise-level API because of the new costs, Thiel said. The group previously didn’t pay for access to the API thanks to an agreement with Twitter that predated Musk, he said.

Musk tweeted in February that charging for API access is meant to help combat bots used by bad actors, and Twitter said in March it was looking at ways to keep working with academics. For developers who don’t use as much data, Twitter’s website says it offers a free tier and other options, including one that costs $100 monthly.

The Stanford researchers said they used technology called PhotoDNA, which scanned for images related to child sexual abuse based on databases maintained by organizations including the National Center for Missing & Exploited Children in the U.S. PhotoDNA creates a unique digital signature, known as a hash, of an image, against which platforms can check the hashes of images users post.

“It’s a surprise to get any PhotoDNA hits at all on a small Twitter dataset," Thiel said.

The researchers said they used an automated computer program that received a stream of data from Twitter based on certain keywords. Links to the images were automatically passed through PhotoDNA to detect known images of child sexual abuse material, they said. Any images that matched weren’t displayed or saved by the researchers, and available metadata was reported to the National Center for Missing and Exploited Children for investigation, they said.

PhotoDNA has been used by organizations including tech companies and law-enforcement agencies. Twitter previously said it used PhotoDNA and other tools to flag and remove child sexual exploitation material.

Twitter didn’t comment on whether it still uses the tool.

The Stanford researchers said Twitter told them last week that it has detected some false positives in some CSAM databases that it manually filters out. As a result, it said, researchers might see some false positives in the future.

One challenge facing Twitter, the researchers said, is that it allows some adult nudity, which adds complexity to identifying and removing violating content. Twitter prohibits material that features or promotes child sexual exploitation.

Musk previously has criticized the Stanford Internet Observatory for its work on content moderation during the 2020 U.S. presidential election. In a March tweet, Musk accused the group of being a “propaganda machine."

Alex Stamos, director of the Stanford Internet Observatory and a co-author of the report along with program’s research manager Renée DiResta, said the group does nonpartisan work on internet issues around the world. “We hope that Twitter decides to re-engage with academic researchers to protect their platform from undue manipulation," he said.