Police launch preliminary probe into Telegram for allegedly abetting deepfake sex crimes

South Korea’s police officials have launched an initial investigation into Telegram, following allegations of "complicity, aiding and abetting" online sexual crimes involving deepfake technology.

Woo Jong-soo, the head of the National Office of Investigation, told reporters Monday that the police have launched an unofficial and preliminary investigation into Telegram on suspicion of "complicity, aiding and abetting crimes involving deepfake pornography.”

“Investigating Telegram is difficult, as they do not share account information and other forms of personal information with investigative officials or law enforcement agencies based in Korea and in other countries,” said Woo.

“However, it's not true that because Telegram didn't give us the material to investigate, we are unable to catch the actual perpetrators of the crimes. We (the Korean police have our own investigative techniques are are doing our best to investigate," he continued.

Acknowledging the recent arrest of Telegram co-founder and CEO Pavel Durov in France, Woo added that the police will look into working with French authorities and other international organizations to see how Durov’s arrest could become an opportunity to find evidence proving Telegram aided and abetted the chatrooms where hundreds of thousands of members distributed deepfake pornography.

Durov was arrested on Aug. 28 in Paris, France, for failing to prevent illegal transactions by an organized group, and was also charged for enabling the distribution of child sexual abuse material, drug trafficking and fraud and refusing to cooperate with law enforcement.

This is also the first time Korea’s police officials have launched an investigation into Telegram, even though the app first came under fire during the “Nth Room” criminal case in 2019, when reports were made regarding blackmail, trafficking and the spread of sexually exploitative videos via the app.

According to the Korean National Police Agency’s National Office of Investigation, it has received a total of 88 reports made regarding deepfake pornography from Aug. 26-29. This is a 10-fold increase compared to the number of reports made over January to July, which had totaled up to 297 cases over the seven months. Among the 88 reports, 24 suspects have been identified by the police, as many victims submitted reports identifying the suspects.

The police also plan to charge the creators of the chatrooms for deepfake bots that generate deepfake pornography out of ordinary photos with aiding and abetting the crime. Police will also update guidance to expand the scope of undercover investigations to investigate deepfake pornography of adults, as current guidance limits undercover investigations of digital sex crimes to crimes involving underage victims.

Meanwhile, an online community consisting of perpetrators involved in creating and sharing deepfake pornography has gone viral through social media reports.

Dubbed the “School Violence Deepfake Taskforce,” the users were found sharing their experiences in creating and distributing deepfake pornography on Telegram, while also sharing tips on how to handle police investigations and avoid possible penalties.

Some posts read, “I ran two deepfake pornography chatrooms on Telegram … I’m still a high schooler, will I be in a lot of trouble?” and “I only watched the (deepfake) videos, but it’s such a fuss these days so I’m worried.”

Some posts expressed confidence that most perpetrators would not be punished, as its “hard to identify every user in the chatroom and to identify who did what in which chatroom.”

Regarding this situation, forensic psychologist Lee Soo-jung told The Korea Herald that it is necessary to not just enforce tougher penalties on those who create, distribute and watch sexually exploitative deepfake content, but to also “regulate platforms that can be used as criminal outlets.”

“Since 2022, laws have started to become enacted overseas -- especially in Europe -- to hold online platforms responsible for distributing sexually exploitative content. However, in Korea, such platforms are monitored but cannot be punished by law,” said Lee.