For the Sphere Las Vegas: Google recreates “The Wizard of Oz” in HD with AI

Contents

At the end of August, Google, together with special effects partners and the Las Vegas Sphere content studio, will be bringing an AI-pimped version of the 1939 “Wizard of Oz” to a 180-degree screen. The gigantic project not only includes AI-supported upscaling of the grainy 4:3 old film to 16K by 16K, but also the AI-supported invention of settings due to the screen layout. Characters that should move to a different position in the 180-degree field of the sphere, but simply do not exist in the original, are added. Current image generators from Google DeepMind are used, which are even prompted, as Google stated last week at a press screening of parts of the film. The project is hosted by Google Cloud (for the AI capacity and the backend) and Alphabet itself. Both bosses, Thomas Kurian and Sundar Pichai, were on-site for the presentation.

“Wizard of Oz” in the 4D cinema

However, the film is not yet complete. In a documentary on the efforts, which was shown in the Sphere, Google admitted that they would be working on the project right up to the last minute. The starting point was the Las Vegas Sphere's content studio and the head of owner Sphere Entertainment Co, James L. Dolan. He also runs the entire Madison Square Garden group, which includes the well-known venue in New York.

Dolan said they immediately looked for a suitable partner and then came across Google and its subsidiary DeepMind. The idea of blowing up an old classic movie to 180 degrees 16K is a challenging project. The Sphere itself is a kind of 4D cinema: in addition to the 180-degree panoramic view, there are vibrations and wind machines to draw users into the entertainment. The stage is so small – from the viewer's perspective – and far away that the 180-degree screen itself is used as a “zoom” during discussions and concerts. Pichai himself made fun of this and said that he had never been so big.

Reconstruction from the archive

At the “Wizard of Oz” presentation, the Sphere was able to show what it can do. First, a – naturally Gemini-generated – background of a farm in Kansas appeared, the colors of which gradually changed. The wind was blowing and the level of realism was high. Then the black and white beginning of “Wizard of Oz” appeared in 4:3 in small, as if it had been thrown onto the Sphere by a movie projector. Then a new world opened up in which Dorothy and her friends, the frightened lion, the man of straw and the Tin Man, were brought to life in crisp detail.

The special effects company Magnopus then had the opportunity to explain how it worked with DeepMind and Google Cloud. The project, which is being produced by Oscar winner Jane Rosenthal, received the blessing of rights holder Warner Bros. Discovery. Generative AI media models such as Imagen and Veo plus Gemini played a role, say the Googlers. Fortunately, Magnopus had access to the handwritten notebook of the cameraman from “The Wizard of Oz” and was thus able to reconstruct scenes that were not seen in the film.

Traditional CGI too expensive and too slow

17,600 people fit into the Sphere. It is not yet clear how long the show will be performed. The AI models go far beyond what would be possible with traditional CGI – and the approach is also significantly cheaper and faster. “The models are extremely innovative,” said DeepMind researcher Steven Hickso. “We found something that we couldn't do before, thought it was impossible to implement – and then a month later we say: maybe we can.”

Using versions of Veo, Imagen and Gemini specially optimized for this task, the Google teams and their partners developed an AI-based “super resolution” tool to transform the small celluloid images from 1939 into ultra-high-resolution images suitable for the Sphere. AI outpainting was also done to expand the scale of the scenes to fill the 180-degree space and fill in gaps caused by cuts and restrictions in the original footage. Finally, the actors were also added, using “performance generation”. The production is said to have cost at least 80 million US dollars.

(bsc)