Key Takeaways
- Meta took days to remove ads containing AI-generated child sexual abuse material.
- Ads featured real children, including a young member of a European royal family.
- AI apps from China promoted in ads violate federal child pornography laws.
Tech Transparency Project (TTP) reported that Meta failed to detect 332 ads containing AI-generated child sexual abuse material (CSAM) this year. These ads were found on Facebook and Instagram, highlighting a significant oversight in Meta's content moderation policies.
The ads promoted AI apps, with the 'vast majority' originating from China. These apps, known as 'nudify' apps, allow bad actors to digitally alter images of children, posing a severe threat to online safety.
TTP matched multiple CSAM ads to photos of real children that appeared online. One such ad featured an animated video of a young royal performing a graphic sex act, while another showed a 14-year-old Instagram influencer in a video of her performing oral sex.
A third ad depicted a preteen victim in a series of stock photos, which was morphed into a video where she looks frightened as she's molested by an adult male. These ads seem to violate federal child pornography laws, as the Justice Department has clarified that AI CSAM is just as harmful as traditional CSAM.
The young royal's image was particularly concerning, as it was 'animated into a video of her performing a graphic sex act.' This raises serious ethical and legal concerns about the use of AI in such nefarious activities.
The TTP's findings highlight the urgent need for improved AI detection and content moderation systems to protect children from online exploitation. Meta's failure to promptly remove these ads underscores the critical importance of robust content policies and enforcement mechanisms.
The incident has sparked calls for increased regulation and oversight of AI technologies, particularly those that can be used to create and distribute harmful content. Experts warn that without stringent measures, the risk of such ads continuing to circulate online remains high.
Meta has not commented on the specific ads mentioned, but the company has acknowledged the broader issue of AI-generated content and its potential misuse. The company is reportedly working on new tools to better identify and remove such content.
