Baidu's Live-Avatar Animates Photos With Audio

Baidu's Quark Vision team has released Live-Avatar, an open-source model that animates a still photograph into a talking head video using a separate audio track. The project aims to create realistic, audio-driven digital avatars from a single source image, a common but challenging task in generative AI.

The 14-billion-parameter model is built upon a foundation model called Wan2.2-S2V-14B, which specializes in still-image-to-video generation. Live-Avatar is fine-tuned for the specific task of synchronizing lip movements and generating natural head motions that correspond to the cadence and content of the provided audio input.

While audio-driven avatar technology has been explored extensively in commercial applications, the release of a powerful open-source model like Live-Avatar under a permissive Apache 2.0 license is significant. It provides researchers and developers with a strong baseline for creating virtual assistants, enhancing accessibility tools, or powering new forms of digital content creation.

The model, code, and usage instructions are now available for developers to explore on the Hugging Face Hub. The repository includes examples demonstrating the model's ability to generate coherent and expressive video from a variety of portrait images.