Snap failed to warn users about sextortion risks, state lawsuit alleges

The state accused Snap of giving priority to growth over safety and failing to effectively address or disclose design features that make its platform a haven for abuse and sextortion, in which predators solicit explicit photos and then use them to blackmail the sender.

Driven by organized predation and loosening norms about intimate photo sharing among young users, a rough internal analysis in November 2022 found that Snap was receiving 10,000 reports of sextortion each month, according to internal documents cited by New Mexico’s suit.

Those reports “likely represent a small fraction of this abuse," the internal Snap analysis stated, citing the low likelihood of victims flagging the problem to Snap. A separate internal analysis of 279 known instances of sextortion in 2023 found that 70% of victims never reported the abuse and that for the 30% who did, Snap had failed to take any action.

“We designed Snapchat as a place to communicate with a close circle of friends, with built-in safety guardrails, and have made deliberate design choices to make it difficult for strangers to discover minors," a spokeswoman said in response to the suit, adding that the company continues to work on additional safety measures and has broken with major social media peers to support federal child safety legislation. “We care deeply about our work here and it pains us when bad actors abuse our service."

Snap was informed that its young user base, combined with the app’s emphasis on disappearing messages, could lull users into believing that sexting on the app was safe, according to the New Mexico lawsuit.

“The ephemeral nature of Snaps can encourage inappropriate content and behavior by giving young people a false sense of privacy," a Snap consultant said in a 2022 presentation, according to the complaint.

An internal marketing document from that same year said sexting had become “regular behavior" for Generation Z. That reality left the company to grapple with how to keep young users safe “without striking fear into Snapchatters" or being seen “as aiding and abetting the production of (at a minimum) child sexually exploitative material."

“‘We can’t tell our audience NOT to send nudes; this approach is likely futile, ‘tone deaf’ and unrealistic," the company says in the marketing document, according to New Mexico’s complaint. “That said, we also can’t say, ‘If you DO do it: (1) don’t have your face in the photo, (2) don’t have tattoos, piercings, or other defining physical characteristics in view, etc.’"

Despite the company’s concern about sextortion, Snap told The Wall Street Journal that sexting accounts for a minuscule fraction of the messages exchanged on the platform.

New Mexico Attorney General Raul Torres called the unredacted filing proof of both the scale of Snap’s problem and that the company had deceptively marketed an unsafe product.

“Snapchat’s harmful design features create an environment that fosters sextortion, sexual abuse and unwanted contact from adults to minors," he said. “It is disheartening to see that Snap employees have raised many red flags that have continued to be ignored by executives."

New Mexico previously sued Meta Platforms, owner of Facebook and Instagram, over child safety failures. In that case, Torres alleged that Meta’s recommendation systems connected young users to apparent pedophiles and pushed children toward inappropriate content. Meta recently announced new restrictions on teen users and says it works diligently to protect children.

In the case against Snap, tests by New Mexico’s investigators and in some instances Snap’s own employees found fault with Snap’s public statements about safety-enhancing features, according to the state’s complaint.

According to Snap, the company changed its “Quick Add" friend recommendations feature in 2022 to prevent the platform from suggesting teenagers connect with adults outside of their regular social circle. But a fictitious 14-year-old user created by state investigators received connection requests from strangers interested in sexual content and “a stream of inappropriate and sexually explicit adult recommendations," New Mexico says in the complaint.

Snap employees themselves reached the same conclusion in 2023, New Mexico said. An internal 2023 effort to “pressure test" the feature found that it “still exposed minors to introductions to adult strangers."

Real children have been introduced to predators by the feature, New Mexico says, citing the case of a man in the state convicted of raping an 11-year-old “to whom he was introduced through Snapchat’s Quick Add feature."

According to an internal document cited in the complaint, Snap declared that a large-scale effort to address child grooming would “create disproportionate admin costs, and should not be its responsibility." In a January 2022 internal chat, also cited in the suit, Snap’s safety staff acknowledged that “by design, over 90% of account-level reports are ignored today and instead we just prompt the person to block the other person."

Snap’s staff sometimes expressed frustration that the company wasn’t moving faster to deal with an exploding sextortion problem last year, the suit claims.

“We’ve twiddled our thumbs and wrung our hands all f…ing year," one safety staffer wrote last year, declaring the platform to be “overrun" with sextortion.

New Mexico has accused Snap of pursuing growth initiatives that pose foreseeable risks to young users. Some features—such as allowing users to publicly share the number of days that they had exchanged messages with a particular friend or see when friends were gathered without them—fed insecurities.

Others, such as QR codes that made it easy to spread users’ contacts or store pictures in a “My Eyes Only" folder inaccessible to a parent, were bound to facilitate both underage sexting and sextortion, New Mexico alleges.

In total, the state argues, Snapchat features “were designed to increase the amount of time young users spend on its platform while inhibiting the ability of those users to self-regulate."

Write to Jeff Horwitz at jeff.horwitz@wsj.com