Data-driven animations of marine mammals combine biology, art, and computation

In her first year as a graduate student in ecology and evolutionary biology at UC Santa Cruz, Jessica Kendall-Bar began working as a freelance artist and started thinking about how to integrate her art and science more intentionally. Four years later, her work on animations to help scientists communicate their findings has led to the development of a powerful set of tools and procedures for generating realistic, data-driven animations of animal behavior.

In a paper presented at this year's IEEE Visualization Conference (October 24 to 29), Kendall-Bar showed how these animations not only help scientists convey their findings to other scientists and the public, but can also help them make sense of their data.

"By animating a three-dimensional model with data, we can simultaneously display several different data streams. This makes it easier and faster to recognize important patterns in the data," she said.

For decades, marine mammal researchers have been using increasingly sophisticated tags to track animals and record their movements and diving behavior, capture video, and even record physiological data such as heart rates. Kendall-Bar's thesis research explores new techniques for monitoring sleep in wild marine mammals.

In 2019, she offered to do an animation for a researcher whose presentation on humpback whale feeding behavior she had seen at a scientific conference. "He had a spectacular video of three whales feeding cooperatively at the bottom of the sea, but it was really difficult to see what they were doing," she recalled. "The talk generated a huge amount of interest at the conference, with hundreds of people squinting and craning their necks to decipher what was happening in the video."

The animation she produced for the whale researcher, NOAA scientist David Wiley at the Stellwagen Bank National Marine Sanctuary, not only shows the humpback feeding behavior more clearly than the video did, it also includes a simulated school of prey fish with scientifically accurate behavior, and shows how ropeless technology for lobster traps can reduce the risk of whale entanglements.

Subsequent projects have grown increasingly sophisticated in their use of data-driven animations. Working with Roxanne Beltran, assistant professor of ecology and evolutionary biology at UCSC, Kendall-Bar developed an animation showing how activity patterns in female elephant seals changed in the course of their seven-month foraging trip (see 'lightscapes of fear').

"As we drafted our manuscript, Jessie's artistic skills helped us see our dataset from an entirely new perspective," said Beltran, noting that the animation helped propel the paper into the public eye, with over 100,000 views on Twitter. "In the era of big data, we need more crossover between art and science. The art can inform the science, and the science can inform the art."

Another elephant seal video was developed with Gitte McDonald at Moss Landing Marine Laboratories and Daniel Costa, director of the Institute of Marine Sciences at UCSC. This 3D animation visualizes the swimming behavior and heart rate of a seal as it dives, demonstrating the fear response caused by the sound of a killer whale, an important elephant seal predator.

The fourth animation described in the paper is based on narwhal research led by Terrie Williams, professor of ecology and evolutionary biology at UCSC. It shows the swimming behavior and heart rate of a narwhal after release from net entanglement in East Greenland and highlights the narwhal's paradoxical fear response involving a very low heart rate during rapid swimming strokes (see "Heart monitors on wild narwhals reveal alarming responses to stress').

"In these four projects, we integrated the data into narrated animations designed to reach particular audiences, including other scientists as well as educating the public or policy makers and people who might help achieve a conservation outcome," Kendall-Bar said.

In addition to the marine mammal researchers, the collaborations also involved Angus Forbes, associate professor of computational media in the Baskin School of Engineering, who has expertise in data visualization.

"Jessica's data-driven animations effectively communicate scientific results and provide insight into the many steps involved in achieving those results," Forbes said. "Her research into new forms of visual communication benefits scientists working on interdisciplinary projects and helps to increase engagement with the general public."

Kendall-Bar has created an online learning center that provides an open-source code repository and step-by-step tutorials and workshops for scientists interested in using graphic design, animation, and other visual communication tools. She has also helped teach workshops and courses on data visualization at UC Santa Cruz, working with the OpenLab Network and faculty in the Art Department and the Computational Media Department.

"The film industry has created powerful tools for animation, and we should be taking advantage of those tools for science," she said. "These animations bring together biologists, artists, and computer scientists for interdisciplinary collaborations that advance each field. I think we should be working together more often."