Adobe Firefly generates videos and translates audio tracks

Contents

Adobe Firefly was previously best known for its image generator, which creates pixels from text input. It is now firmly integrated into the image editing program Photoshop. The beta version of the video editing program Premiere Pro has been able to extend video clips by a mere two seconds since autumn 2024. Now the Adobe Firefly web app is getting a new addition in the form of an AI video generator.

Like the image generator, the video model should also be commercially safe to use, as Adobe states that it only trains its models with content for which the provider has secured the rights.

Video model for Adobe Firefly

As usual, Adobe has initially released its video model in a publicly accessible beta version. Creative Cloud customers can formulate prompts via the web interface as usual, but can now also generate videos from them via "Generate Video (Beta)" instead of just still images. In principle, the two modes "Text to video" and "Image to video" are available.

Adobe has largely adopted the interface from the image generator. The aspect ratio can be selected via the sidebar on the left. You can also select the perspective, i.e. the angle from which the virtual camera moves. You can also select a camera movement or a zoom from several movement paths.

The video model generates clips in 1080p resolution and a length of five seconds. Adobe also plans to release a faster model in the near future that will output content in a lower resolution for idea generation.

Automatically translate soundtrack

Firefly will also be able to translate audio files and audio tracks in videos in over 20 languages, including German, English, French and Spanish, while retaining the authentic voice of the speaker, including the tone of voice, in order to localize videos as authentically as possible.

Image generator for 3D shapes

The Firefly tool "Scene to image" generates images from 3D shapes. Visual details are defined in the text prompt. 3D tools can be used to create guides for viewing angles and perspective. Adobe is also releasing this model in beta status for the time being.

Prices and availability

According to Adobe, the updates presented are available immediately. Adobe charges credits for the use of generative AI. Normally, a job costs one credit. It was not known at the time of publication whether more credits will be charged for videos.

Adobe grants 1000 credits per month for a full Creative Cloud subscription, 500 credits for individual app subscriptions and 100 credits for Lightroom subscriptions. Separate conditions apply for corporate, education and team licenses.

(akr)