Stable Diffusion creator testing AI-powered video generator
Stable Diffusion creator Stability AI has announced that users can now test a new generative AI that animates a single image generated from a text prompt to create AI-generated videos.
“Stable Video Diffusion is released in the form of two image-to-video models, capable of generating 14 and 25 frames at customizable frame rates between 3 and 30 frames per second,” Stability AI said.
The videos are generated in 1,024 x 576 resolution.
“This state-of-the-art generative AI video model represents a significant step in our journey toward creating models for everyone of every type,” the company said.
“Latent diffusion models trained for 2D image synthesis have been turned into generative video models by inserting temporal layers and fine-tuning them on small, high-quality video datasets.”
Stability AI conducted user preference studies on its new AI video generator and said it has been well received.
“At the time of release in their foundational form, through external evaluation, we have found these models surpass the leading closed models in user preference studies,” it said.
For reference, the user preference studies compared Stable Video Diffusion to text-to-video platforms like Runway and Pika Labs.
The AI video generator is only available for research purposes now, and potential users can join the waitlist to access an “upcoming web experience featuring a text-to-video interface”.
Stable Diffusion has noted that the generative AI has some limitations, including:
- The generated videos are rather short (up to 4 seconds).
- The model lacks perfect photorealism.
- The model can’t do camera motion except for slow camera pans.
- The model cannot be controlled through text.
- The model cannot render legible text.
- Faces and people in general may not be generated properly.
- The autoencoding part of the model is lossy.
Now read: Nvidia sued after engineer shares screen with stolen file during video call
Loading ...