A Minnesota startup's shortcut for finding the most emotional moment in songs? Math

"Eye of the Tiger," the iconic theme song from the 1982 film "Rocky III," has been featured in commercials for years.

It's nearly a four-minute song, and finding the perfect snippet to splice into a 30-second TV spot can take sound producers hours. But a Minnetonka audio company can speed up that process to seconds using its newly developed algorithm.

The former music executives and neuroscientists behind MIIR Audio Technologies and its MIIR platform—MIIR stands for Music Intelligence Impact Retrieval—found a way to search mathematically rather than instinctively for the most impactful moments in songs that elicit emotional response. The technology could have a big impact on industries including film, music streaming and health care.

The company's patented software analyzes digital audio files, 0s and 1s of binary code representing vocals, pitch, density of instrumentation and chord changes among a slew of other elements.

Sub-algorithms extract information on pitch, rhythm, volume and so forth from the code. That extracted data turns into detectors that create the indexing. It's a form of mathematical "hearing" that allows the software to detect parts of songs that evoke human emotion.

The software then curates an index of songs based on the desired sound.

"We've reverse engineered the brain's emotional response to music," said Daniel Levitin, a professor, neuroscientist, inventor, former Columbia Records exec and chief science adviser at MIIR.

Listening power

In 2017, MIIR chairman and chief executive Paul Moe and others from the music and film industries discussed problems with digital music platforms, agreeing that too many songs had flooded the internet and made music search and discovery ineffective. They thought there was a need for technology that expedited the music-selection process.

In 2019, Moe—whose past musical clients included Prince, Whitney Houston and Diana Ross—started forming the company, which officially launched in August 2020.

For an individual, the tech can create personalized playlists. MIIR's software can analyze a listener's music library, identifying songs commonly listened to or skipped to curate the perfect playlist. It's unlike other streaming platforms, where suggested content is popularity-driven, MIIR lead developer and co-inventor Aaron Prust said.

On a larger scale, companies can do in seconds what would typically take days.

Last year, the chief executive of an independent record company working as a producer on a television series asked to sample MIIR's system to fill a specific scene with music, Moe said. The producer needed to finalize the scene within 24 hours.

MIIR's team combed its catalog of 100,000 purchased songs, searching for specific sound—a fast-paced, grungy country song—the producer requested. Within 30 minutes, the tool narrowed the list of possible songs to 10, with visualizations showing the exact snippet of its emotional peak, ranked by "chill factor," which executives describe as moments that give the listener chills.

The technology filters the simplicity or complexity of a performance by whether the singer is shouting or whispering, has a rough or smooth voice, conveys angst, joy or sadness. And because there could be several versions of the same song, if a company wants to use a particular song for a commercial or spot in a film, the system can index the performance itself. The software is language agnostic, meaning its determinations don't depend on language.

MIIR's executive team is marketing to larger companies operating in three distinct verticals: music streaming and music-based social media; film, television, advertising, gaming and the metaverse; and non-pharma medical technology.

The goal, Moe said, is to create partnerships with an eye toward one of those companies eventually acquiring MIIR, as many already "play in all of those sandboxes," he said.

"Our feeling is that large companies are already generating the revenue, and they already have the customer base," Moe said.

Within the past decade, billion-dollar corporations have gobbled up audio technology companies that have figured out how to monetize music intelligence.

In 2014, music streaming company Spotify paid $100 million to acquire Echo Nest, a company that invented a system that can identify songs to recommend to listeners based on listening habits.

In 2018, Apple Inc. spent $400 million to acquire Shazam, a music-recognition app that can identify songs through a phone's microphone.

The business is "well positioned" for acquisition, Moe said, adding the company has three years of audited financials, no debt and more than $4 million in capital raised from private investors, including many medical doctors that reside in Minnesota. MIIR has also conducted three scientific validation studies, Moe said.

Healing sounds

Using biometrically driven, personalized music to assist in pain management, depression and overall wellness is a business vertical showing increasingly high value, Moe and Levitin said.

When listening to music, the human brain produces its own version of opioids. And for those with chronic pain, the release of those—though not enough to replace pain medication—can reduce the amount of medication a person takes, Levitin said. That's important, given the addictive nature of opioids.

Music also increases levels of serotonin and dopamine in the blood, which are the body's antidepressants, Levitin said.

Wearable devices that track blood pressure and heart rate can embed MIIR's software to select therapeutic sounds that lower or reduce blood pressure, he said.

It isn't a working theory. The nation's medical research agency, the National Institutes of Health, in 2019 committed to funding $20 million in a five-year span to back research projects that explore the potential for music as treatment. That includes various neurological conditions, Parkinson's, Alzheimer's and autism.

Kevin Smith, the chief of staff and chair of radiology at St. Cloud Hospital, is also an investor in MIIR. In his area of practice, the company's technology, Smith said, could curate relaxing music for patients feeling anxiety before MRI scans.

That would curb the need for medication or sedatives for any patient feeling claustrophobic in the scanner, Smith said. Music therapy as an alternative to drugs would lower the cost and risk of pharmaceutical treatments on a large scale, he added.

"On a national level, we do millions of scans each year," Smith said. "If you can cut out 20 percent of that for older adults, you are making a huge impact on the cost of health care."

2023 StarTribune.
Distributed by Tribune Content Agency, LLC.