‘Anyone can be a victim’: Sprawling AI fake nudes crisis hits South Korea
Summary
The discovery of a Telegram-based network creating faked pornographic images points to the nation as an epicenter of an emerging global problem.
SEOUL—The victims were reportedly teachers, military officers, undergraduates and elementary-school students. Across a labyrinth of Telegram group chats, anonymous users submitted photos of South Korean girls and women without their permission that were manipulated into sexually explicit images and videos viewed by hundreds of thousands.
South Korean authorities on Wednesday began an investigation to tackle faked pornographic images after a massive network was uncovered—involving hundreds of victims, many of them minors. The revelation reflected the scale of the problem facing South Korea, which according to some researchers is the source for roughly half of so-called “deepfake" porn videos spread globally.
Many countries, including the U.S., are confronting a rise in AI-generated fake nudes targeting young women and girls.
The current protections in South Korea haven’t kept pace with the threats posed by AI-generated sexualized content—much of which is created by teenagers or younger children, local officials said. The country’s education ministry is reviewing the maximum punishment for middle-school-aged perpetrators, or those as young as 10 years old. Lawmakers are looking to close legal loopholes, such as widening current punishments beyond those who intentionally disseminate the illicit content.
“Deepfake videos may be dismissed as mere pranks, but they are clearly criminal acts that exploit technology under the shield of anonymity," said South Korean President Yoon Suk Yeol, a former prosecutor, at a Tuesday cabinet meeting. “Anyone can be a victim."
The scale and number of victims remains under investigation. Many of the Telegram groups were organized by school name or region to allow users to easily identify mutual acquaintances and make fake nudes they could share. To gain entry, some users were required to provide photos of a woman, which were often lifted from a female classmate’s social-media account, according to officials and activists. Communication in these Telegram group chats was conducted almost exclusively in Korean.
Public awareness skyrocketed after a Telegram group chat focused on one university became public earlier this month following a police investigation. It had been active since 2020 and included around 1,200 members, authorities said, sharing not only computer-generated sexualized images but other personal information, such as the subject’s phone number, address and student ID number. Fake pornographic images of at least 30 current and former students had reportedly been shared in the group.
Roughly 500 schools—from universities to elementary schools—may have been affected, according to one online tally compiled by volunteers who have combed through the group chats. The website containing the list of schools has attracted about three million page views since going live on Tuesday.
The crackdown against faked pornographic images distributed on Telegram comes days after French authorities detained Pavel Durov, the messaging app’s founder and chief executive. Prosecutors there said Durov’s arrest was related to an investigation into whether Telegram is enabling online criminality, including the exchange of child pornography.
Asked about the South Korean investigation, a Telegram representative said the company removes millions of pieces of harmful content each day on its platform, based on moderating, AI tools and user reports.
South Korean singers and actresses accounted for roughly half of the individuals featured in deepfake pornography posted online, according to a 2023 report by Security Hero, an identity-fraud prevention company. The report analyzed roughly 100,000 videos across more than 100 websites.
Activists say South Korean women have faced an elevated risk of sexual images of themselves appearing online without their consent for years. South Korean civil servants sweep public restrooms for hidden cameras. Celebrities have been charged for spreading hidden-camera footage of their sexual encounters. A South Korean sexual exploitation ring used Telegram to coerce dozens of women to make thousands of lewd videos.
But until the public outcry of recent days, South Korean police had cited Telegram’s overseas servers as a reason why investigating fake-nude cases was difficult, said Song Ran-hee, a representative for the Korea Women’s Hotline, a nonprofit supporting survivors of violence against women. “The crackdown announcement comes problematically late," she said.
More than 6,000 South Koreans this year requested the removal of faked pornographic images made without their consent, according to the state media regulator’s figures that run through July. That volume is already approaching last year’s total of roughly 7,000.
Of the roughly 300 individuals accused of making and distributing fake nudes since the start of 2023, roughly 70% were teenagers, according to South Korea’s National Police Agency.
In a recent statement, the Korean Federation of Teachers Union, a labor group, said the responsibility to report and investigate fake-nude cases largely falls to schools. It recommended creating a federal mechanism to help track down perpetrators.
“Without a chance to wait for an appropriate response or see perpetrators penalized, students are voluntarily turning their accounts private or deleting the photos they posted online," the union said.
Write to Timothy W. Martin at Timothy.Martin@wsj.com
