3 minute read

San Francisco Lawsuit Targets 16 Sites for Deepfake Adults-Only Content Creation

August 21, 2024

Woman shopping and using phone text call social media standing in line deals discount Miguel Paricio Shutterstock

image credit: Miguel Paricio/Shutterstock

San Francisco’s legal team is tackling a growing threat that hits close to home: the rise of deepfake ‘adult-only content’ targeting women and girls. By filing a lawsuit against 16 websites that exploit AI to create non-consensual explicit images, the city aims to hold these platforms accountable.

With 200 million visits to these sites in just six months, this case could be a turning point in the fight against digital exploitation.

A Parent’s Growing Concern

Screen smartphone text call social media post video photo picture cyber bullying ABC Shutterstock — image credit: A_B_C/Shutterstock

Yvonne Meré, like many parents, was alarmed when she learned about a disturbing new trend. Boys were using “nudification” apps to create deepfake ‘adult-only content’ by superimposing images of their female classmates’ faces onto AI-generated bodies.

A Mother’s Resolve

Lawyer court law legal trial Attorney Lawyer defend case lawsuit trial Gorodenkoff Shutterstock — image credit: Gorodenkoff/Shutterstock

Unlike other worried parents, Meré had the power to take action. As San Francisco’s chief deputy city attorney, she mobilized her team to file a lawsuit aimed at shutting down 16 popular websites involved in creating these deepfakes.

A First of Its Kind Legal Battle

Lawyer court law legal trial Attorney Lawyer Gorodenkoff Shutterstock — image credit: Gorodenkoff/Shutterstock

This legal action appears to be the first government-led attempt to dismantle the platforms promoting these non-consensual images. The lawsuit seeks to close the sites that allow users to digitally “undress” women and girls.

Sparking Legal Action

Lawyer court jury trial legal attorney defend prosecute Gorodenkoff Shutterstock — image credit: Gorodenkoff/Shutterstock

Meré, moved by the harmful effects detailed in a recent New York Times article, reached out to her colleague, Sara Eisenberg. Together, they enlisted the help of City Attorney David Chiu to strategize a response.

The Challenge of Accountability

Phone social media hand laptop computer online text work business remote panoramastudio Shutterstock — image credit: panoramastudio/Shutterstock

Chiu emphasized the difficulty of holding individuals accountable under current laws. He explained that while several states have criminalized AI-generated explicit content, the new lawsuit targets the websites themselves, aiming to shut them down.

A Persistent Problem

Business male uses cell phone to text call social media Manager lawyer businessman politician attorney slexp880 Shutterstock — image credit: slexp880/Shutterstock

Chiu acknowledged the potential for more sites to emerge even if the lawsuit succeeds. However, the legal team plans to expand the lawsuit to include new sites as they come to light.

The Global Scope

google browser laptop Rawpixel.com shutterstock — image credit: Rawpixel.com/Shutterstock

The 16 sites named in the lawsuit had a staggering 200 million visits in the first half of this year. These sites are operated by entities based in locations ranging from California to Estonia, but none have responded to the lawsuit.

Disturbing Marketing Tactics

Shocked stressed young woman computer laptop social fizkes Shutterstock — image credit: social fizkes/Shutterstock

Some websites boldly advertise their services with slogans like, “Have someone to undress?” or “Imagine wasting time taking her out on dates,” promoting the idea that users can obtain explicit images with just a few clicks. These platforms often lure users with free image creation before charging for additional services, typically through cryptocurrency or credit cards.

Dark Origins of Deepfake Technology

Woman holding phone text call email office work social media slexp880 Shutterstock — image credit: slexp880/Shutterstock

The AI models powering these sites are trained on real ‘adult-only content’ and images of child abuse, enabling them to generate convincing deepfakes in seconds. According to Mr. Chiu, these technologies can create disturbingly realistic images by overlaying the faces of real people onto AI-generated bodies.

Widespread Victimization

Celebrity Taylor Swift Award Show Tinseltown Shutterstock — image credit: Tinseltown/Shutterstock

From celebrities like Taylor Swift to ordinary middle-school girls, the reach of deepfake technology is extensive and troubling. These images are often used for extortion, harassment, or humiliation, with devastating effects on the victims’ mental health, reputations, and future prospects.

A Problem Beyond Parental Control

Unhappy Couple Marital problems anxious break up counseling therapy Prostock studio Shutterstock — image credit: Prostock-studio/Shutterstock

The issue isn’t something parents can address merely through conversations about smart technology use. Any photo, from prom pictures to sports snapshots, can be manipulated without consent, making traditional protective measures inadequate.

The Challenge of Accountability

Teenagers Students Friends Independent Andrey Arkusha Shutterstock — image credit: Andrey Arkusha/Shutterstock

Ms. Eisenberg pointed out that even the most internet-savvy teenagers are vulnerable to these attacks. Once the images spread, tracing them back to their source is nearly impossible, complicating legal recourse for the victims.

Aiming for Permanent Solutions

Law legal judgement legislation court Chinnapong Shutterstock — image credit: Chinnapong/Shutterstock

The lawsuit targets these sites directly, seeking to shut them down and prevent their operators from creating further deepfake ‘adult-only content’. The legal team is also pushing for civil penalties and attorney fees, citing violations of various state and federal laws, including revenge and under-age statutes.

San Francisco’s Role in the Fight

San Francisco skyline and Bay Bridge California IM photo Shutterstock — image credit: IM_photo/Shutterstock

San Francisco, as the epicenter of the AI industry, is a fitting battleground for this legal challenge. The city, home to tech giants like OpenAI and Anthropic, represents both the promise and peril of rapidly advancing technology.

Balancing Innovation with Responsibility

Law Business Legal Services Lawyer Freedomz Shutterstock — image credit: Freedomz/Shutterstock

While acknowledging the benefits of AI, Mr. Chiu emphasized that deepfake ‘adult-only content’ reveals a darker side of the industry. He admitted that keeping up with such a fast-evolving field is challenging for government lawyers but stressed the importance of taking action regardless.