San Francisco’s legal team is tackling a growing threat that hits close to home: the rise of deepfake ‘adult-only content’ targeting women and girls. By filing a lawsuit against 16 websites that exploit AI to create non-consensual explicit images, the city aims to hold these platforms accountable.
With 200 million visits to these sites in just six months, this case could be a turning point in the fight against digital exploitation.
A Parent’s Growing Concern
Yvonne Meré, like many parents, was alarmed when she learned about a disturbing new trend. Boys were using “nudification” apps to create deepfake ‘adult-only content’ by superimposing images of their female classmates’ faces onto AI-generated bodies.
A Mother’s Resolve
Unlike other worried parents, Meré had the power to take action. As San Francisco’s chief deputy city attorney, she mobilized her team to file a lawsuit aimed at shutting down 16 popular websites involved in creating these deepfakes.
A First of Its Kind Legal Battle
This legal action appears to be the first government-led attempt to dismantle the platforms promoting these non-consensual images. The lawsuit seeks to close the sites that allow users to digitally “undress” women and girls.
Sparking Legal Action
Meré, moved by the harmful effects detailed in a recent New York Times article, reached out to her colleague, Sara Eisenberg. Together, they enlisted the help of City Attorney David Chiu to strategize a response.
The Challenge of Accountability
Chiu emphasized the difficulty of holding individuals accountable under current laws. He explained that while several states have criminalized AI-generated explicit content, the new lawsuit targets the websites themselves, aiming to shut them down.
A Persistent Problem
Chiu acknowledged the potential for more sites to emerge even if the lawsuit succeeds. However, the legal team plans to expand the lawsuit to include new sites as they come to light.
The Global Scope
The 16 sites named in the lawsuit had a staggering 200 million visits in the first half of this year. These sites are operated by entities based in locations ranging from California to Estonia, but none have responded to the lawsuit.
Disturbing Marketing Tactics
Some websites boldly advertise their services with slogans like, “Have someone to undress?” or “Imagine wasting time taking her out on dates,” promoting the idea that users can obtain explicit images with just a few clicks. These platforms often lure users with free image creation before charging for additional services, typically through cryptocurrency or credit cards.
Dark Origins of Deepfake Technology
The AI models powering these sites are trained on real ‘adult-only content’ and images of child abuse, enabling them to generate convincing deepfakes in seconds. According to Mr. Chiu, these technologies can create disturbingly realistic images by overlaying the faces of real people onto AI-generated bodies.
Widespread Victimization
From celebrities like Taylor Swift to ordinary middle-school girls, the reach of deepfake technology is extensive and troubling. These images are often used for extortion, harassment, or humiliation, with devastating effects on the victims’ mental health, reputations, and future prospects.
A Problem Beyond Parental Control
The issue isn’t something parents can address merely through conversations about smart technology use. Any photo, from prom pictures to sports snapshots, can be manipulated without consent, making traditional protective measures inadequate.
The Challenge of Accountability
Ms. Eisenberg pointed out that even the most internet-savvy teenagers are vulnerable to these attacks. Once the images spread, tracing them back to their source is nearly impossible, complicating legal recourse for the victims.
Aiming for Permanent Solutions
The lawsuit targets these sites directly, seeking to shut them down and prevent their operators from creating further deepfake ‘adult-only content’. The legal team is also pushing for civil penalties and attorney fees, citing violations of various state and federal laws, including revenge and under-age statutes.
San Francisco’s Role in the Fight
San Francisco, as the epicenter of the AI industry, is a fitting battleground for this legal challenge. The city, home to tech giants like OpenAI and Anthropic, represents both the promise and peril of rapidly advancing technology.
Balancing Innovation with Responsibility
While acknowledging the benefits of AI, Mr. Chiu emphasized that deepfake ‘adult-only content’ reveals a darker side of the industry. He admitted that keeping up with such a fast-evolving field is challenging for government lawyers but stressed the importance of taking action regardless.
