AI Platform Under Fire for Enabling Creation of Abusive Images

A recent report by AI Forensics, a European nonprofit tech watchdog, has shed light on a disturbing trend in the AI community. The study found that Hugging Face, a popular platform for accessing and ranking open-source AI models, has been enabling the creation and sharing of nonconsensual intimate imagery, altered with generative AI. This type of content is not only abusive but also illegal, and it's a challenge that no AI company or platform is handling adequately.

The report's findings are alarming, with seven of the top nine Hugging Face Spaces for image editing allowing users to easily undress or 'nudify' people in images without the need for adversarial attack methods. Furthermore, 73% of users' prompts in those Spaces were found to be sexual, with the abusive images overwhelmingly targeting women. This raises serious concerns about the lack of moderation and oversight on the platform.

Hugging Face has responded to the report, stating that it conducted its own testing and took action against Spaces that were intentionally misrepresenting their purposes. However, the company denies a 'systemic lack of moderation,' claiming that several 'artifacts' were removed under its current moderation procedures before the report was published. The incident highlights the shared responsibility of AI platforms and developers to prevent their technology from being used for harm.

The issue of nonconsensual intimate imagery created with AI technology is not new, but the generative tech has made it easier, quicker, and cheaper for people to perpetrate harm. As AI image-generation and editing tools become more capable and widespread, AI-powered image-based abuse is on the rise, and it's up to companies like Hugging Face to take proactive steps to prevent it.

Source: CNET