Adobe’s Generative AI Video Tools Are Finally Coming to Premiere Pro

In what is perhaps the biggest news Adobe has ever unveiled for video editors, and also the least surprising, we finally have an official confirmation from Adobe that they’ll be adding generative AI video tools to Premiere Pro.

Now, we say least surprising because we’ve seen Adobe already add generative fill and other generative AI features to Photoshops and other apps in its Creative Cloud over the past year. So, it’s been a ticking clock waiting for these features to make their way over to Adobe’s video editing suite as well.

But here at last, we can finally share all of the details about how Adobe is gearing up Premiere Pro to be a more complete AI-powered editing experience with new generative AI video tools, plus other updates coming to Premiere Pro via Adobe’s Firefly AI model.

---

New Generative AI Video Tools in Premiere Pro

As you can see below, Adobe is ready to preview its breakthrough generative AI innovations within Premiere Pro as they aim to reimagine video creation and production workflows for profesional editors. These new generative AI tools coming to Premiere Pro are set to streamline editing for all video types by adding and removing objects in a scene or extending existing clips.

Here are the big new AI-powered features coming to Premiere Pro:

• New Generative AI video tools coming to Premiere Pro this year will streamline workflows and unlock new creative possibilities, from extending a shot to adding or removing objects in a scene
• Adobe is developing a video model for Firefly, which will power video and audio editing workflows in Premiere Pro and enable anyone to create and ideate
• Adobe previews early explorations of bringing third-party generative AI models from OpenAI, Pika Labs, and Runway directly into Premiere Pro, making it easy for customers to draw on the strengths of different models within the powerful workflows they use every day
• AI-powered audio workflows in Premiere Pro are now generally available, making audio editing faster, easier, and more intuitive

These new workflows will also be powered by a new video model that is set to join the Adobe Firefly AI family, which Adobe plans to continue developing for further advancements in imaging, video, audio, and 3D as more AI-powered tools are set to come.

The Future of Generative AI in Premiere Pro

Set to come to Premiere Pro “later this year,” these new generative AI features are powered by Adobe’s new video model for Firefly. In addition to the new tools above, Adobe has also shared some early “sneak” shows for how video editors might eventually leverage video generation models from Open AI and Runway in the future to generate B-roll, or how they might use other AI platforms like Pika Labs with the Generative Extend tool to add a few seconds to the end of a shot.

Here are some possible future tools:

• Generative Extend: Seamlessly add frames to make clips longer, so it’s easier to perfectly time edits and add smooth transitions. This breakthrough technology solves a common problem professional editors run into every day, allowing them to create extra media for fine-tuning edits, to hold on a shot for an extra beat, or to better cover a transition
• Object Addition & Removal: Simply select and track objects, then replace them. Remove unwanted items, change an actor’s wardrobe, or quickly add set dressings such as a painting or photorealistic flowers on a desk
• Text to Video: Generate entirely new footage directly within Premiere Pro. Simply type text into a prompt or upload reference images. These clips can be used to ideate and create storyboards, or to create B-rolls for augmenting live-action footage

Adobe certainly seems open to expanding the AI-powered workflows for video editors not just with their technology, but with a holistic approach using the best of AI at any given time, which should—ideally—help Premiere Pro users always stay on the cutting edge of AI… if that’s what they want to do.

AI-Powered Audio Workflows Coming to Premiere Pro

Adobe has also shared some new AI-powered audio tools which are set to be generally available to Premiere Pro users here in May of 2024 that will give editors even more precise control of their audio and the ability to improve the quality of their sound.

Here are the latest AI-powered audio features coming to Premiere Pro:

• Interactive fade handles: Editors can create custom audio transitions faster than ever by simply dragging clip handles to create audio fades
• New Essential Sound badge with audio category tagging: AI automatically tags audio clips as dialogue, music, sound effects, or ambiance, and adds a new icon so editors get one-click, instant access to the right controls for the job
• Effect badges: New visual indicators make it easy to see which clips have effects, quickly add new ones, and automatically open effect parameters right from the sequence.
• Redesigned waveforms in the timeline: Waveforms intelligently resize as the track height changes on clips, while gorgeous new colors make sequences easier to read

Adobe also has already added their AI-powered Enhance Speech tool which instantly removes unwanted noise and improves poorly recorded dialogue earlier this year, and plans to start rolling out these new AI-powered features in May, before adding more features later this year.

For more info on AI-powered tools and features coming to Premiere Pro, check out Adobe’s website here for updates.

No Film School’s coverage of NAB 2024 is brought to you by Blackmagic Design.