Active Stocks
Wed May 29 2024 15:50:35
  1. State Bank Of India share price
  2. 822.95 -0.96%
  1. Tata Steel share price
  2. 174.20 -0.37%
  1. HDFC Bank share price
  2. 1,507.85 -1.48%
  1. Power Grid Corporation Of India share price
  2. 317.55 1.52%
  1. ITC share price
  2. 430.80 0.44%
Business News/ Ai / Artificial Intelligence/  AI developers agree to new safety measures to fight child exploitation
BackBack

AI developers agree to new safety measures to fight child exploitation

wsj

OpenAI, Meta, Google and others say they will work to throttle the creation of sexualized images of children.

Virtual reality headsets on the Meta Platforms Inc. A host of new generative-AI powered tools have supercharged predators’ ability to create sexualized images of children and other exploitative material. (Photog: Bloomberg)Premium
Virtual reality headsets on the Meta Platforms Inc. A host of new generative-AI powered tools have supercharged predators’ ability to create sexualized images of children and other exploitative material. (Photog: Bloomberg)

Major artificial-intelligence companies including OpenAI, Meta Platforms and Google agreed on Tuesday to incorporate new safety measures to protect children from exploitation and plug several holes in their current defenses.

A host of new generative-AI powered tools have supercharged predators’ ability to create sexualized images of children and other exploitative material. The goal of the new alliance is to throttle the creation of such content before these tools can proliferate and hurt more children, according to Thorn, the child-safety group that helped organize the initiative along with the nonprofit organization All Tech Is Human.

Thorn and several AI companies agreed to implement principles to minimize the risks of their tools. One such principle calls for AI labs to avoid data sets that might contain child sexual content and scrub such material from their own training materials.

It wants companies to invest more in regular red-teaming, or testing to find and fix gaps that allow such material to be generated. Thorn is pushing AI platforms and search engines such as Google to remove links to services that “nudify" otherwise benign images of children—a problem that has popped up at high schools over the past year.

“This project was intended to make abundantly clear that you don’t need to throw up your hands," said Rebecca Portnoff, vice president of data science at Thorn. “We want to be able to change the course of this technology to where the existing harms of this technology get cut off at the knees."

Executives at the AI companies involved said they had no desire to allow their tools to create child-exploitation material. Some executives and child-safety advocates said that if these data sets are overly sanitized, AI products can become less useful to consumers.

Last year, the National Center for Missing and Exploited Children, or NCMEC, received 36 million reports of child exploitation. The center is overwhelmed by the volume of reports as it contends with limited funding, outdated technology and legal constraints in handling the sensitive materials.

When Thorn approached AI companies, they found that while some companies already had large teams focused on removing child-sexual-abuse material, others were unaware of the problem and potential solutions. There is also a tension between the imperative to safeguard these tools and business leaders’ push to move quickly to advance new AI technology.

“We did not want to avoid helping progress the technology out of fear," said Justin Maier, founder of open-source AI platform Civitai, which is part of the alliance. “I think rather than running away from that, it’s better to think about how we can make that space safer."

Civitai, backed by venture-capital firm Andreessen Horowitz, drew criticism for not doing enough to protect children. Tech news platform 404 Media reported in December that some images on the platform could be considered child pornography. Civitai says it takes the issue seriously and is bolstering its defenses to stamp out exploitative images.

Thorn and other advocates are worried that new AI tools will “grow the haystack" of potentially violating material, forcing law-enforcement officers to spend more time determining if a child in an image is real. To help avoid this issue, Thorn and the companies in this alliance have also agreed to add signals that help others determine whether content is generated or enhanced by AI.

Social-media platforms rely on automated image-detection systems that primarily match so-called hashes, or fingerprints of known child-sexual-abuse images, and videos. AI-generated content is novel, however, and unlikely to include those fingerprints, rendering the current tools less effective.

Many companies already try to weed out AI content that exploits children through filters and prompt engineering. But those defenses can fail.

In December, a Stanford researcher published a paper showing that there were hundreds of exploitative images of children in a popular public data set used for AI text-to-image generation models called LAION. The data set was taken down.

But AI models can still generate such content even without the inclusion of child-exploitation material. That’s because the AI models that power text-to-image services can combine parts of different images to create a new image, such as a realistic image of a panda on a surfboard. This capability also makes it possible for some AI systems to create child pornography by combining nonsexual images of children and adult sexual content.

Portnoff said some companies in the alliance have agreed to separate images, video and audio of children from data sets that also contain adult sexual content in their open-source models because that increases the risk of creating child exploitation.

If implemented, some of the steps outlined by Thorn will require some technical breakthroughs and possibly new agreements with child-safety regulators to allow for more rigorous testing, executives said in interviews.

Policymakers have urged AI companies to find ways to add watermarks to their images so they can be traced back to their creator. Today, watermarks are removable, and AI companies are still looking for ways to mark AI-generated images permanently, said Ella Irwin, senior vice president of integrity at Stability AI, the company behind the open-source image-generation model Stable Diffusion.

Irwin added that it was difficult for companies to stress-test their systems for child-exploitative content today because the testing itself can break the law if such material is generated.

Write to Deepa Seetharaman at deepa.seetharaman@wsj.com

You are on Mint! India's #1 news destination (Source: Press Gazette). To learn more about our business coverage and market insights Click Here!

Next Story footLogo
Recommended For You