Firefly Video Model: Adobe’s New AI Tool

Adobe announced today that it has expanded its Firefly family of generative AI models to include video, introducing the Firefly Video Model. This new feature enables creative professionals to generate high-quality video clips from text prompts, images, or existing footage.

The Firefly Video Model is available in limited public beta, allowing a select group of users to provide initial feedback and shape the model’s development

The Adobe Firefly Video Model introduces several features aimed at streamlining video production for professionals. One key function is Text-to-Video, which allows users to generate video clips from text descriptions.

It also offers tools for controlling camera angles and creating B-roll to fill gaps in video projects. This feature is particularly useful for teams with limited budgets or tight schedules, as it helps plan complex or costly shots without immediate filming.

Another feature, Image-to-Video, converts still images into video clips, allowing editors to create new footage from existing photos. This can be used to generate additional shots, such as close-ups, by drawing from a single frame of video.

The tool also provides the option to adjust the motion or purpose of a shot, which can assist editors in demonstrating possible changes to directors or clients. The model is also capable of generating visual effects like fire, water, and smoke, which can be layered onto footage using Adobe’s editing software.

Adobe has emphasized that the Firefly Video Model was developed to be commercially safe, with its AI trained on licensed and public domain content. It also includes Content Credentials, which document how content is created and whether AI was used, ensuring transparency for users.

Additionally,Adobe stated that it has added new tools to its Firefly Services to help companies speed up their production processes.

One of these new features, currently in testing, is a Dubbing and Lip Sync tool that uses AI to translate spoken dialogue into different languages while keeping the original voice and matching lip movements. This feature is similar to the voice cloning and translation tool recently launched by D-iD.

Another tool, called “Bulk Create,” is also in testing and helps users quickly edit large numbers of images by making tasks like resizing or removing backgrounds easier.

The Firefly Video Model is currently available in a limited public beta. Adobe states that you can join the waitlist here. Throughout this beta phase, video generations are free. Adobe will provide additional details regarding Firefly video generation options and pricing once the model transitions out of limited public beta.