Words by: Keegan Sentner
Over the past few months, the presence of artificial intelligence has sparked an intense debate and profoundly shaken the film and entertainment industry. Several films released this year, such as Late Night with the Devil and, most recently, Furiosa: A Mad Max Saga, have faced criticism for integrating AI into various aspects of production.
The use of AI in film production isn’t anything new, though. AI and machine-learning programs have played a significant role in filmmaking for many years, and their influence in production is only projected to continue growing. Just last month, Fable Studios unveiled a new streaming service designed to create AI-driven content based on spoken prompts. At the 2024 Tribeca Film Festival, five AI-generated short films made with Sora — an AI model that can generate video clips from text prompts —debuted. And during a recent investor presentation, Sony Pictures Entertainment CEO Tony Vinciquerra expressed interest in leveraging AI to reduce production costs.
“The biggest problem with making films today is the expense, and we need to find a way to produce films in a more efficient way,” stated Vinciquerra. “We will be looking at ways to use AI to produce both films for theaters and television in a more efficient way.”
Several studio executives seem to share this perspective, and there are many indications that AI technology will increasingly integrate into the filmmaking process. Now, the question is no longer “will it change filmmaking,” but “how will it change filmmaking?” This concern over the industry’s future and the role this technology will play is at the forefront of many of the current discussions in film.
In the VFX industry, artists have been incorporating different AI tools into their workflow for years. These tools predominantly rely on machine-learning programs, which utilize learning-based algorithms and data insights to train models for specific tasks or analyses. “AI is a buzzword that’s getting thrown around quite recently, but the subset of that, like machine learning, is something we’ve been using in visual effects for more than a decade,” VFX artist Rassoul Edji stated during a Zoom interview with CONE.
Edji, who has worked with MPC, one of the largest VFX firms in the United Kingdom, has contributed to films such as Transformers: Rise of the Beasts, Sonic the Hedgehog 2, and the upcoming Snow White film. In his projects, he and his teams have employed AI machine-learning programs to solve laborious and time-consuming tasks, such as denoising footage or removing grain from shots. He has also collaborated with teams who have used AI to assist with tasks nearly impossible for artists or animators, such as replicating subtle facial movements in organic characters. This process is incredibly complex and typically consumes significant time.
“We put cameras on people’s faces, and we get them to do different expressions. These expressions then get fed through a computer, and then through some clever neural networks, it’s able to actually register,” explains Edji. “Animators can’t really go in and animate little muscle movements. These small things the AI can actually do for the animator.”
Edji, who recently went viral for a tweet explaining the problems with current production pipelines, believes ethical use of artificial intelligence can help artists manage increasingly heavy production workloads and deliver work on shortening timelines. “It is positive in the sense that the AI being used in our industry is being used very cautiously to do things that artists don’t know how to do, can’t do, or just don’t want to do,” Edji continues. “In that sense, for feature film VFX, it’s going in the right direction.”
This ability to create efficiency is also why many indie VFX artists and filmmakers have been interested in using artificial intelligence programs for their work. Ben Moran is the founder of Secret Beach Films and a CG Lead at Maere Studios, which is renowned for its VFX contributions to films like Ava Duvernay’s Origin and Kyle Mooney’s Y2K. He also directs short films, integrating his own VFX expertise into his projects. Speaking on the value of VFX in filmmaking, Moran notes, “Ultimately, when you are making a film, your most valuable resource is time because you’re only going to have access to your crew, locations, etc., for a limited window.”
Moran views VFX as a transformative tool for mitigating these limitations in lower-budget filmmaking, enabling directors to overcome challenges they may encounter. In several instances, Moran has used VFX to construct shots that would have been too expensive or impractical otherwise, allowing him to enhance his work within the constraints of limited budgets.
While Moran has yet to extensively incorporate artificial intelligence into his filmmaking process, he acknowledges both its potential benefits and the associated risks of adopting this new technology.“Those of us who strive to be great filmmakers strive to use the tools at our disposal to create meaningful art,” says Moran. “As long as we remember that AI is a tool and not an artist, I think we’ll be fine.”
Moran sees the potential benefits of using AI as a tool, but also cautions against its use to replace more tedious roles in filmmaking, such as rotoscoping. These positions, he explains, are often where many younger filmmakers and VFX artists begin their careers and gain crucial industry experience.
The rise of AI technology threatens to eliminate critical early-career positions for aspiring filmmakers, posing a significant danger to the industry’s longevity. Most of these concerns revolve around the more sophisticated subset known as Generative AI, which utilizes vast databases to generate new text, images, and videos. The expansion of these programs like Sora, Midjourney, and Runway, has the potential to significantly impact filmmaking and visual effects artists, potentially endangering the livelihoods of many in the industry. A January report from Future Unscripted found that 75 percent of entertainment executives indicated generative AI had already supported the elimination, reduction, or consolidation of jobs. Additionally, 47 percent of business leaders foresee generative AI becoming effective in generating 3D assets and realistic sound design for film, television, and video games within the next three years.
The report highlights the growth of generative AI in the film and television industry and the ways executives are embracing this technology to lower costs, leading to the disruption of thousands of jobs. This makes it important to distinguish between these generative AI programs and those that enhance artistic creativity, such as those utilized by Edji in his work.
The more transparent we are about these distinctions, the more productive our conversations can be on leveraging AI to benefit the VFX industry. This approach allows artists to develop a symbiotic relationship with the technology, ensuring that these programs enhance their work rather than replace it. “We have to be vocal as artists when we see things that are happening that might affect our industry,” says Edji. “Everyone loves movies, and we love making movies. But if the agenda is to replace artists making movies, not only would the artists suffer, but the films themselves would suffer, too.”