Google has just presented its new artificial intelligence called Lumiere, which generates videos of much higher quality than the competition. This new tool runs from a simple text query or a static image, and is even capable of editing a video clip.
This will also interest you
[EN VIDÉO] Artificial intelligence that adjusts the angle at which the photo is taken Interactive point-based manipulation of the generative image collector. © DraGAN Project
Generative artificial intelligence has made significant progress over the past year in everything relating to the generation of text or images. However, the video field is still in its infancy. One of the best video AIs so far is Stable Video Diffusion, from Stability AI, but the quality still leaves something to be desired. From now on, it is GoogleGoogle which takes the lead thanks to a new AI called Lumiere.
The quality of the videos produced by Lumiere represents a new leap forward. They last five seconds, with a definition of 1024 x 1024 pixels. According to Google, rather than generating the start and end of the video then completing with the animation in between, Lumiere generates everything in a single pass thanks to an architecture “ Space-Time U-Net ” who works ” in a representation of thespace-timespace-time compact ».
Presentation of Lumiere with examples for the different operating modes. © Inbar Mosseri, Google Research
A video that can be created from text, an image or a video
The AI works in several modes, starting with simple text queries. However, it is also possible to provide an image at the same time which will serve as a reference for the stylestyle visual of the video. Lumiere can also animate an image, or create a cinemagraph by animating only a selected area of an image, such as a feufeu or the smoke coming out of a chimneychimney.
Additionally, AI offers two functions for editing videos. The first changes the style of an object, for example by modifying the colorcolor of a dress, or by transforming a person into Lego. The second allows you to fill a hidden area of the image, generating the missing elements. A simple way to remove unwanted people from the background of a clip, but could also be used to erase the logo or watermark on videos.
At this time, it is only a research project, which means it is not available to the public.
rewrite this content and keep HTML tags