On Wednesday, Adobe announced its Firefly AI video generation tool, due for beta release later this year. As with many things AI-related, Adobe is gradually integrating tools built to automate much of the creative work that its prized user base currently pays for, and the examples are as frightening as they are fascinating. Echoing the AI salesmanship seen elsewhere in the tech industry, Adobe is positioning all of this as ancillary technologies that “take the drudgery out of post-production.”
Adobe describes its new Firefly-powered text-to-video, Generative Extend (available in Premiere Pro), and image-to-video AI tools as helping editors do tasks like “navigating gaps in footage, removing unwanted objects from a scene, smoothing jump cut transitions, and finding the perfect B-roll.” The company says these tools give video editors “more time to explore new creative ideas, making them their favorite part of the job.” (To take Adobe’s word at face value, one can’t help but think that once the industry fully adopts these AI tools, employers will not just be asking for more deliverables from editors — or they’ll be paid less, or employed less. But I digress.)
Firefly Text-to-Video lets you create AI-generated videos from, you guessed it, text prompts. It also includes tools to control camera angles, movement, and zoom. It lets you take shots with gaps in the timeline and fill in the blanks. You can also use still reference images to turn them into compelling AI videos. Adobe says its video model excels at “natural world video,” helping you create establishing shots and B-roll instantly, without a budget.
To see just how compelling this technology is, check out this example promotional video from Adobe.
These are hand-picked samples by companies trying to sell their products, but their quality is undeniable. Detailed text prompts produce stationary shots of just that: a blazing volcano, a dog lounging in a field of wildflowers, or (proving that it can handle fantastical scenes, too) little woolly monsters having a dance party. If these results are representative of the tool’s typical output (which is hardly a guarantee), TV, film, and commercial productions will soon have access to some powerful shortcuts, for better or worse.
Meanwhile, Adobe’s image-to-video example starts with an uploaded image of a galaxy; text prompts turn it into a video, zooming out from the star system to show the inside of a human eye. The company’s Generative Extend demo shows two people walking by a forest stream, with AI-generated segments filling in gaps in the footage (so convincingly that it’s hard to tell which parts of the output were AI-generated).
Adobe
According to Reuters, the tool will only generate five-second clips, at least initially. To Adobe’s credit, its Firefly video model is designed to be commercially safe and is trained only on content the company has licensed to be used. “We train exclusively on the Adobe Stock database, which contains 400 million images, illustrations and videos that are curated to contain no intellectual property, trademarks or recognizable characters,” Alexandre Kostin, Adobe’s vice president of generative AI, told Reuters. The company also stressed that it won’t train on users’ work. But whether that means losing users’ jobs is another matter entirely.
Adobe says the new video model will be available in beta later this year, and you can join a waitlist to try it out.
