Alibaba Group and Ant Group Researchers Introduce VideoComposer: An AI Model That Enables To Combine Multiple Modalities Like Text, Sketch, Style, And Even Motion To Drive Video Generation

Current visual generative models, particularly diffusion-based models, have made tremendous leaps in automating content generation. Thanks to computation, data scalability, and architectural design advancements, designers can generate realistic visuals or videos using a textual prompt as input. To achieve unparalleled fidelity and diversity, these methods often train a robust diffusion model conditioned by text on massive video-text and image-text datasets. Despite these remarkable advancements, a major obstacle still exists in the synthesis system's poor degree of control, which severely limits …

ai model alibaba ant computation data design designers diffusion drive generative generative models machinelearningnews multiple researchers scalability text video video generation videos visuals