Facebook uses AI to tackle revenge porn on social media

San Francisco, Calif., Mar 16, 2019 / 15:50 pm

Facebook has announced that it will begin using AI software to prevent and restrict the distribution of non-consensual sexual material – also known as revenge porn.

In a March 15 statement, Antigone Davis, Facebook's global head of safety, said the new technology will detect nude videos or pictures distributed without permission on Instagram and Facebook.

"This means we can find this content before anyone reports it, which is important for two reasons: often victims are afraid of retribution so they are reluctant to report the content themselves or are unaware the content has been shared," she wrote.

Davis said the machine learning and artificial intelligence technology will identify the problematic material. The company's Community Operations team will then determine whether the content violates Facebook's policies and, if it does, likely disable the account of the offender.

The company already had a policy of removing content with sexual violence or exploitation, including intimate images shared without consent and advertisements of sexual services, once it was reported to them.

Use of the new technology is an attempt to be proactive in finding such content more quickly.

The program will build on a pilot project, which ran in Australia in 2017. Under this initiative, individuals fearing retaliation from an angry ex-partner may submit intimate photos to Facebook proactively. The social media platform will then use a digital fingerprint of the picture to preemptively ban the image from ever being distributed on its website.  

Davis said Facebook will also launch "Not without my consent" – a victim-support center. Here, individuals who have been targeted by revenge pornography can learn about actions they can take to delete the content and prevent its further promotion. The support center is a joint project of numerous international groups including the U.S. Cyber Civil Rights Initiative.

In addition, Facebook hosted a March 15 event at the United Nations Headquarters in New York City with Dubravka Šimonović, the U.N. Special Rapporteur on violence against women, and other advocates and experts.

The goal of the event was to "discuss how this abuse manifests around the world; its causes and consequences; the next frontier of challenges that need to be addressed; and strategies for deterrence," said Davis.

Revenge porn laws have been on the rise, both in the U.S. and globally. Forty-three states and Washington D.C. have laws banning the distribution of this material in place. New York is the most recent state to criminalize revenge porn, earlier this year.