How AI technology animates the fire creatures in the latest Pixar movie

Today, the latest animated film "Elemental" from Walt Disney Studios and Pixar Animation Studio is released in Swiss cinemas. The film revolves around the fiery Ember, who lives in Element City—a place where fire, water, earth and air beings live. The film is not only made in Hollywood, but also enabled by ETH technology.

The movie's main character Ember is a fiery elemental creature whose head consists of flames that are constantly moving. Realistically drawing these flames and heat currents posed a great challenge to animators, as recently reported by Wired—starting with ETH technology, they finally came up with a solution that makes the flame structure in Ember's face appear more pronounced than what was possible with previous simulation technologies.

"Our technology gives the flames a unique style, making it a tool for artists to visually support a story," says von Barbara Solenthaler, ETH professor of computer graphics and head of the Simulation and Animation Group. Under her direction, the new technology was developed in the Computer Graphics Lab at ETH Zurich in 2019/2020.

Image transfer to support the story

The new 3D simulation technology uses artificial intelligence and enables artistic control over the appearance and structures of smoke, fire, or fluids. With the help of this AI, smoke clouds, for example, may be used specifically for visual storytelling. The ETH technology can artistically change flames, smoke clouds or fluids in such a way that they support the story.

This technology does not control the outer form, but the local structures in the smoke. For example, it can compute a spiral pattern on the smoke cloud. It can just as easily apply the style of Van Gogh's painting "Starry Night" to the smoke. "Our technology is capable of transferring the structure of any input image to smoke, flames or even fluids," says Barbara Solenthaler. Bunny-like cloud formations can be seen very beautifully in a video from Solenthaler's group. In another video, Solenthaler's team shows how they can represent flow, fire, and smoke phenomena both artistically and true to nature.

Disney's interest immediately aroused

The AI-based 3D simulation technology attracted great interest from the Walt Disney Studios only shortly after the ETH researchers published it. "The technology was created at ETH and the code for it is open source," says Barbara Solenthaler, "we then transferred the technology into the Disney Animation Studios and Pixar Animation Studio pipelines with the help of Disney Research Studios in Zurich."

As early as of 2021, the ETH technology was used for the first time in the computer-animated fantasy film "Raya and the Last Dragon"—in a scene in which circular smoke formations spread out from a ruin. "The effect in the Raya film was based on our ETH publication. For the Pixar film 'Elemental,' which is currently running in Swiss cinemas, the Disney Research Studios in Zurich have further developed the technology and used it for the main character Ember," says Solenthaler.

A technology for the main character

Specifically, Elemental is about the main character Ember being a fire creature whose face is constantly in flame-like motion. Now, the new ETH technology now made it possible to transfer an initially artistically created flame structure to Ember's appearance in such a way that the fire and the flames on Ember's body look extremely lifelike. ETH technology played its part in bringing Ember and the other fire creatures that populate Element City "to life", as the Walt Disney Studios recently acknowledged.

The ETH researchers were no longer involved in the final elaboration of these effects, which were completed by Disney Research Studios and Pixar Animation Studio themselves. How Ember's flame head was animated using the new technology in the movie has now also been documented. "When I see how fascinating the effects came out in Elemental and especially in Ember, I am thrilled that our research is the starting point for this," says Solenthaler.

Related research has been published in Computer Graphics Forum and ACM Transactions on Graphics.