Apple ordered to remove 8 ‘nudify’ AI apps from the App Store

Apple has been accused of “aiding and abetting” the use of “nudify” AI apps on the iPhone, with San Francisco City Attorney David Chiu demanding they be removed from the App Store in a new cease-and-desist letter sent to the company.

A Wired report notes that both Apple and Google have been told to prevent the sale of AI apps that can be used to create deepfake images. They must also “sever” business relationships with all of the developers responsible for the apps.

So-called “nudify” apps have been around for some time. They allow users to create non-consensual intimate images of others, often minors, without the knowledge of the subject.

Such apps can be used to remove the clothes of subjects. Some allow the subjects to then be placed into specific scenarios of the user’s choosing.

Illegal, harmful, and completely unacceptable

In a statement provided to Wired, Chiu pointed out that generating deepfake imagery is “illegal, harmful, and completely unacceptable.” Chiu also noted that Apple and Google have likely made considerable sums from the sale of, or subscriptions relating to, such “nudify” apps.

Chiu also made it clear that Apple and Google have a responsibility to ensure that their platforms aren’t being used to create such content. His legal letters also pointed out to both companies that California law prohibits supporting services that can create deepfakes.

Google says that it investigates apps that are believed to be used to create deepfake content. It added that it “takes swift action” when required.

Apple issued a statement to AppleInsider on Friday evening after publication.

“The App Store was designed to be a safe and trusted place for users, and we have always strictly prohibited apps designed to generate, distribute, or consume pornography. ‘Nudification’ apps are against our App Review Guidelines and we have proactively rejected many of these apps and removed many others, including when users have flagged them via our reporting tools. We have removed three of the apps in question and are in the process of terminating their developer accounts from our program. We are in contact with four others that need to address policy violations or risk being removed as well.”

Apple previously removed a number of similar generative AI apps from the App Store. The company previously told AppleInsider that nudification apps aren’t allowed in the App Store, with no exceptions.

App Store reviewers appear to give developers some benefit of the doubt. Developers are first notified if their apps are found to be used to create “nudify” images.

Violations that aren’t addressed generally result in the offending app being removed from the App Store. It’s not clear if any of the apps in question have gone through this process, or where they are in it.

Apple also says that it actively blocks App Store search terms for such apps.

As for Chiu, he intends to continue to pursue both companies. For now, he wants Apple and Google to remove the offending apps and strengthen the tools that they use to screen new apps before they are made available for download.

Update July 18, 10:03 AM: Updated with Apple’s statement on the matter.