Pressure is mounting on Apple and Google after San Francisco City Attorney David Chiu issued cease-and-desist letters demanding the removal of 13 AI-powered “nudify” apps from their respective app stores.

The action marks an escalation in regulatory scrutiny of the operators of app stores, with Apple and Google accused of hosting the apps as well as profiting from them. 

The 13 targeted apps use generative AI to create non-consensual nude and sexually explicit deepfake images that primarily target women and minors.

The legal notices reportedly warn that failure to act could trigger civil enforcement under California law with San Francisco arguing that the companies may be knowingly  facilitating illegal activity by continuing to distribute and make money from the apps, which violate their own marketplace policies.

While Apple and Google already prohibit explicit and abusive content in their app stores, regulators argue there has been inconsistent enforcement of their rules, which allows harmful AI apps to remain available.

Spokespeople from both Google and Apple told Wired they had removed many apps that breached their policies, including some named by Chiu. Others are reportedly in the process of being removed as they word to address the policy violations.

The cease-and-desist letter from Chiu raises broader questions about how to police AI-generated content and signals a shift in responsibility from AI developers to the platforms distributing the apps. The move could lead to tighter reviews of App Store and Google Play, particularly for AI-powered image-generation apps and other generative AI tools.

The dispute also highlights growing concerns and compliance burdens worldwide around AI generated deepfakes, online safety and privacy breaches.