Snap brings tools with generative AI for augmented reality in video chats
The Snapchat operator has developed AI tools to expand augmented reality in video presentations. This should also be possible in real time on smartphones.
AR backgrounds in videos
(Image: Screenshot aus Snap-Video)
Snap, the company behind the social media app Snapchat, has announced new tools for special effects using augmented reality (AR) in video presentations. These use generative artificial intelligence so that, for example, backgrounds or AI pets within Snapchat should rarely look the same every time. Snap wants to set itself apart from other social platforms with technical functions of this kind and catch up with its much more widely used competitors, such as Meta Platforms.
The so-called "GenAI Suite" is part of Lens Studio, which can be used to create your own webcam avatars for video conferences. This program is now available in version 5.0 and is available to users and developers. AR developers can offer this for AI-supported image generation not only for Snapchat, but also for other apps or websites. Snap itself works together with Microsoft, for example, and made Snap Lenses available in Microsoft Teams last year, allowing AR filters from Snap to be used as camera effects in video conferences.
The company has now expanded these AR functions to include generative AI. So far, the GenAI Suite is only a prototype, but Snap says it has optimized the GenAI techniques to the point where the AI models can run in real time on mobile devices. This includes AI-generated backgrounds, virtual pets, Snapchat dreams (selfie fantasy pictures) and camera functions with AI-generated AR models.
Snaps Lens Studio 5.0 with AI support
Lens Studio 5.0 is also getting an AI assistant to help developers generate content if they have any questions. The new version of the developer environment should also significantly reduce the time required to develop AR effects. According to Reuters, Snap's chief developer Bobby Murphy explained that with Lens Studio 5.0, this is now possible within hours instead of weeks and allows for more complex models. For example, 3D models no longer need to be completely recreated before they can be used for AR.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmmung wird hier ein externes YouTube-Video (Google Ireland Limited) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (Google Ireland Limited) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
Previously, AR was only used in videos for simple effects, such as putting a hat on a person. According to Murphy, the possibilities are now much more realistic. The hat would now move with the head and the display adapts to changes in lighting, for example if the light is dimmed. In addition, heads can be completely replaced by models that adapt to the head movements of the original.
Snap: Full-body AR in video still difficult
Snap plans to be able to use augmented reality for the entire body, instead of limiting it to the face and head, as is currently the case. According to Murphy, however, this is currently very difficult to generate. Snap has already been experimenting with this for some time and released a "Shopping Suite" as SaaS for augmented reality trials at the beginning of 2023. However, this software-as-a-service is aimed at business customers.
They can use the tools developed by Snap, which allow end consumers to virtually try on clothes and shoes, among other things. All the technologies on offer are designed to help consumers make better purchasing decisions. After all, bad purchases online not only lead to frustration among customers, but also to numerous returns, which is not only bad for the environment, but also causes additional costs.
In addition to integrating Snap's camera functions into Microsoft Teams, the company has also recently transferred these special effects to Skype. Snap's camera kit thus also enables augmented reality for video chats in this instant messaging service.
(fds)