New AI from Nvidia: Fugatto can generate music and accentuate voices
Fugatto from Nvidia can supposedly generate music on demand and adapt existing sound. This should help with the development of computer games, for example.
Nvidia's advertising image suggests that cats can also use it to produce music
(Image: Nvidia)
Nvidia has introduced Fugatto, an AI technology for generating audio that is said to be significantly more versatile and better than all competing services. For example, it should be possible to transform existing audio recordings and, for example, turn a piece of piano playing into singing. It is also possible to modify the recording of a voice so that the accent or mood of the person being recorded appears to change. The technology is intended for the production of music, for the development of computer games and for "normal people who want to create things", explains Bryan Catanzaro from Nvidia.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externes YouTube-Video (Google Ireland Limited) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (Google Ireland Limited) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
According to Nvidia, Fugatto (Foundational Generative Audio Transformer Opus 1) was trained exclusively with material under open source licenses; the technology is controlled with text commands ("prompts") or with the help of audio files. In a video, Nvidia shows how Fugatto generates the sound of a passing train, which turns into an orchestral recording, simply by responding to such a prompt. In other examples, the technology separates a voice from a song and generates another voice that recites a given sentence. Instruments can also be added to an uploaded piece of music.
Videos by heise
"We wanted to create a model that understands and produces sounds the way humans do," says Rafael Valle from Nvidia, explaining the product. Around a dozen people worked on the development. According to the news agency Reuters, there is still internal debate about whether and how the technology will be made publicly available. Catanzaro explains that any generative technology entails a number of risks: "We have to be careful with it and therefore we have no immediate plans to publish it."
(mho)
