Protecting image-sharing users from inappropriate content with AI

ClickASnap was facing a growing issue: users uploading illicit and inappropriate images to the platform. This undermined the safe environment the platform wanted to create, and posed significant legal and reputational risks. Current methods of image content monitoring are manual and inefficient, leading to delays in detecting and removing harmful content, as well as excessive costs for the business.

This project proposed the preliminary investigation and feasibility study of implementing state-of-the-art deep learning AI models for image content monitoring on the ClickASnap platform, which can automatically identify and remove illicit content before it reaches users. This solution will use deep learning models trained on appropriate datasets to accurately detect and classify images based on predefined criteria.

The project exceeded all its objectives, successfully prototyping AI technology to filter out inappropriate content.

Through rigorous testing and evaluation, the project identified and assessed several state-of-the-art deep learning models. These models were trained and validated using sample datasets, resulting in the successful development of a robust framework for automated content moderation. This laid a solid foundation for future implementation by proving the feasibility and effectiveness of these AI models in maintaining a safe platform environment.

To further refine the AI models for content moderation, the project led to the creation of bespoke datasets, tailored to include a wide variety of artistic and creative content – something which is often challenging for conventional moderation systems to accurately categorise. The development of these bespoke datasets represents a significant enhancement, ensuring that the AI models can better differentiate between artistic expression and inappropriate content. This will be crucial for maintaining the integrity of ClickASnap’s platform as a space for both creativity and safety.

During the project, an additional need was identified for a sophisticated AI-based image search functionality. Recognising the potential of this feature to enhance user experience, the project team extended their efforts to develop an AI model specifically for image search. An AI search model which allows users to search for images based on similarity to another image has been successfully completed and is now in production, soon to be integrated into ClickASnap’s platform. This will significantly improve the discoverability of content and add a valuable tool for platform users.

The AI-enhanced search functionality will transform the user experience on ClickASnap. Previously limited to single keyword searches, the platform will enable more complex and intuitive searches, significantly improving users’ ability to find desired content efficiently. This feature has also introduced an unexpected layer of discovery by suggesting content that users may not have initially sought, leading to enhanced user satisfaction and engagement.

In addition, the project has laid the groundwork for future AI-driven features, such as an AI-powered assistant to help users navigate and utilise the platform seamlessly. This assistant will enable creators to maximise their potential through intuitive tools for uploading, editing, and managing their portfolios, reducing friction in the creative process.

By effectively removing illicit content, ClickASnap will ensure a safer and more enjoyable experience for all users, and save the costs associated with manual content moderation. Addressing content issues will also strengthen ClickASnap’s brand reputation, fostering trust and boosting revenue opportunities by assuring users and investors of the platform’s safety and capacity for innovation.

The advancements achieved through the Innovate UK – Accelerated Knowledge Transfer (AKT) project extend beyond ClickASnap, offering significant societal benefits and broader applications. By automating the detection and removal of illicit content, the project contributes to creating safer online environments, reducing exposure to harmful material, and encouraging trust in digital platforms.

The AI technologies developed can be adapted for use across various industries, including education, social media and e-commerce, enhancing content moderation and search capabilities on a global scale. These innovations emphasise the potential of ethical AI to improve digital interactions, empower creators, and safeguard communities worldwide.
 

The Future

The outcomes of this AKT project addressed the immediate challenges faced by ClickASnap, and positioned the company for the future. With the groundwork laid for integrating AI-driven content moderation and the newly developed image search functionality, ClickASnap is well-prepared to lead the industry in both user safety and content discoverability.

The bespoke datasets will also serve as a key asset in future developments, ensuring that the platform’s AI capabilities continue to evolve in alignment with its users’ needs.

A follow-up Knowledge Transfer Partnership project will further develop a robust, scalable, and cost-effective AI-driven content moderation solution with these findings.