AI-powered Adobe Firefly now creates audio for artificial intelligence videos and provides a vocal outlet for AI avatars
Adobe's AI platform, Firefly, has received an update that introduces significant improvements in AI-generated video and audio capabilities. The latest upgrade offers more natural, lifelike movements, advanced video controls, custom generated sound effects, and integration of new generative AI partner models.
Improved Motion Fidelity
The Firefly Video Model has been enhanced to produce more natural, lifelike movements, especially in dynamic landscapes, animal motion, atmospheric effects, and 2D/3D animation. This results in smoother transitions and greater visual accuracy.
Advanced Video Controls and Workflow Tools
The update also includes advanced video controls and workflow tools that give creators more command over video composition and style. These tools include:
- Style Presets: Allowing users to apply distinct visual styles with one click.
- Keyframe Cropping: A tool that speeds up cropping without switching tools.
- Composition Reference for Video: Enabling users to upload a reference video and description to generate new videos that preserve the original's visual flow.
Custom Generated Sound Effects (Beta)
The Firefly web app now offers the ability to create custom generated sound effects using text prompts or voice control. Users can generate impact sounds, ambient tracks (such as room tone, nature sounds, traffic), and layer these into videos with timing and intensity control.
Integration of New Generative AI Partner Models
Adobe Firefly has integrated several new generative AI partner models, expanding creative options across sound, video, and image. Notable additions include Luma AI’s Ray 2, Pika 2.2, Google’s Veo 3 (with audio), Moonvalley’s Marey, and Topaz Labs’ Image and Video Upscalers. These partners are available in Firefly Boards and Generate Video, allowing users to select the best AI model for their workflow and creative needs without switching apps.
Export Options
The latest update also includes the ability to send completed videos directly to Adobe Express or Premiere Pro for further editing or distribution.
Additional Features
- Users can choose from a selection of avatars and backgrounds in the Text to Avatar tool.
- Atmospheric track capabilities use AI to generate sounds such as room tone, traffic, or nature sounds.
- Content Credentials are automatically attached to AI-generated content within Firefly.
In summary, Adobe Firefly's newest update significantly boosts AI-generated video quality with natural motion and enriched control, adds innovative AI-generated custom sound effects, and broadens creative flexibility with multiple partner AI models integrated seamlessly into its workflow. These enhancements streamline production and empower creators to produce sophisticated audio-visual content efficiently and with personalized style.
Journalist's Note
This article was written by Hillary K. Grigonis, the US coverage lead for Digital Camera World, with more than a decade of experience writing about cameras and technology. Her work has appeared in several notable publications, including Business Insider, Digital Trends, Pocket-lint, Rangefinder, The Phoblographer, and more.
- The improved Firefly Video Model generates more natural, lifelike movements, particularly in dynamic landscapes, animal motion, atmospheric effects, and 2D/3D animation.
- The update introduces advanced video controls and workflow tools that allow creators to exert more control over video composition and style.
- Custom generated sound effects can be created using text prompts or voice control in the Firefly web app.
- Adobe Firefly has integrated several new generative AI partner models across sound, video, and image, expanding creative options.
- Completed videos can be sent directly from Firefly to Adobe Express or Premiere Pro for further editing or distribution.
- Users can choose from a selection of avatars and backgrounds in the Text to Avatar tool and use AI to generate atmospheric sounds such as room tone, traffic, or nature sounds.
- Content Credentials are automatically attached to AI-generated content within Firefly, providing transparency and fostering responsible usage of artificial intelligence in photography and video production.
