5-second videos: Midjourney animates images

Midjourney uses a cat video to promote its own video generator. The first version of the model creates videos up to five seconds long. During this time, the AI cat can either spread its wings or flap its wings in the air. However, the videos can be extended to a total of up to 21 seconds.

To generate a video, you have to go to midjourney.com. An existing image is then required, regardless of whether it has been generated or uploaded. Clicking on the new animate button is then sufficient to generate a video. You can choose whether Midjourney itself decides what the movement looks like, or whether you want to specify a kind of path via prompt and use the manual mode for this. The animation is available for all previous images.

There are also two video types. Under Low Motion, Midjourney generates videos that are rather slow and in which the camera barely moves. High Motion means that the camera and image move more. In the case of the cat, the wings only open slowly in low motion, while in high motion the animal jumps into the air much more dynamically. However, Midjourney warns that this can lead to more errors and unrealistic movements. If you use the --raw addition as a parameter, Midjourney adheres more strictly to the prompt specification and intervenes less.

Videos up to 21 seconds long with Midjourney

The generated videos can be downloaded as MP4 files with a resolution of 480p and 24 frames per second. However, the size is initially dependent on the source image. The five-second videos can be extended by a further four seconds. There is an “Extend” button for this. A prompt for the subsequent seconds can be set under Extend manual. In total, videos up to 21 seconds long are possible.

Midjourney also points out directly in the blog post that the videos cost around eight times more GPU time than generating an image.

Midjourney's video generator has been awaited for some time. Even before its release, Disney and Universal filed a lawsuit against it. They accuse Midjourney of having made use of their copyrighted works. They claim that Midjourney can easily generate photos and now also videos of characters such as Shrek and Darth Vader. However, the problem does not only affect Midjourney. OpenAI's Dall-E also hands Indiana Jones a banana on request. The data on which Midjourney's video model was trained is just as unknown as it is for OpenAI's Sora, Google's Veo or the common image generators.

(emw)