AI software sorts image content and evaluates content
Researchers from the Department of Computer Science at TU Darmstadt have developed LlavaGuard, a tool that can filter image content and text.
(Image: Diki Prayogo/ Shutterstock.com)
Artificial intelligence can analyze objects in images and videos and automatically process large amounts of visual data, but has so far mostly failed due to regional differences and age-related evaluation criteria. Felix Friedrich and his team from the Artificial Intelligence and Machine Learning Lab (AIML) have developed "LlavaGuard", a tool that does not have these limitations and can classify image and text content in detail according to different specifications. The filtering method uses vision language models (VLMs) which, unlike pure language models such as ChatGPT, can process and interpret text as well as images.
Unlike other AI-supported image recognition programs such as Anthopic AI, LlavaGuard can adapt flexibly to different legal regulations and user needs. For example, it can differentiate between countries and regions and use different criteria for the analysis. In this context, the researchers gave the Informationsdienst Wissenschaft (IDW) the example of the different country-specific regulations for dealing with cannabis consumption.
Regional and age-dependent content evaluation
The program also assesses whether content is suitable for certain age groups and adapts it accordingly. "Until now, safety tools of this kind have only been available for analyzing texts. When filtering images, for example, only the 'nudity' category can currently be implemented, but not others such as 'violence', 'self-harm' or 'drug abuse'," explains Friedrich.
The tool not only flags content that has been identified as problematic, but also explains the safety classifications in detail. It categorizes content such as "hate", "illegal substances" or "violence" and explains why it is considered safe or unsafe. "This transparency is what makes our tool so special and is crucial for understanding and trust. This makes LlavaGuard a powerful tool for researchers, developers and policymakers," says Friedrich.
Research into LlavaGuard is a central component of the "Reasonable Artificial Intelligence (RAI)" cluster project at TU Darmstadt. The potential applications of the tool appear to be far-reaching. Although it is still under development, it can already be integrated into image generators such as Stable Diffusion. Its use on social media platforms is also being considered. According to the research team, this would make it possible to automatically filter out illegal content and thus create a safer online environment.
(dahe)
