Comparing Western and Chinese classical music using deep learning algorithms

Deep learning techniques are proving to be extremely useful for analyzing all kinds of data, ranging from images to text, online posts and audio recordings. These techniques are designed to identify patterns in large datasets, separate items in different categories and make predictions far quicker than humans.

In a recent study, researchers at Simon Fraser University, Academia Sinica and Dartmouth College have applied deep learning techniques to identify similarities and differences between Chinese and Western classical music. Their paper, pre-published on arXiv, presents a comparative analysis of music recordings using sound event detection (SED) and soundscape emotion recognition (SER) models.

"We have listened to both Chinese and Western classical music," Jianyu Fan, one of the researchers who carried out the study, told TechXplore. "We feel that several factors set Chinese music apart from Western music, including their underlying culture, philosophy and way of thinking. This is why we are very interested in studies comparing Chinese and Western music."

Deep learning models for the analysis of music recordings can sometimes perform poorly, as they struggle to learn patterns in high-level information, such as a song's melody, harmony, etc. To overcome the limitations encountered in previous studies, Fan and his colleagues applied soundscape models to the analysis of Chinese and Western classical music, to evaluate their effectiveness in identifying similarities and differences between the two sub-genres.

Initially, the researchers compiled two annotated datasets containing recordings of Chinese and Western music, dubbed WCMED and CCMED. Subsequently, they trained a pretrained SED and a pretrained SER model on these datasets separately, combining both of them with a support vector regression (SVR) model. SED models are designed to detect sound events in audio signals, while SER models are trained to recognize the emotion conveyed by soundscape recordings.

"While previous music studies mainly use models based on music, we were curious about whether a model trained on general soundscape can be used for analyzing music and how they are different for Chinese and Western classical music," Fan explained. "Therefore, we tried using two models built on general sound: a sound event detection model and a soundscape emotion recognition model."

The researchers used transfer-learning techniques to extract high-level representations of sound. They then used these representations to train their music emotion recognition model to detect emotions conveyed by music recordings. As their model was pre-trained to generalize the characteristics of sound, they found that these representations worked better when coupled with a simpler model, especially for the analysis of Chinese classical music. The researchers also trained a deep learning classifier on the datasets they created and conducted further analyses focusing on specific features of Chinese and Western songs.

"As our goal is to use pre-trained soundscape models to analyze and compare Chinese and Western classical music, we didn't expect the model to work perfectly well for different types of audio and different types of tasks," Fan said. "However, our results suggest that it is effective for predicting the arousal of both Chinese and Western classical music using soundscape models."

The findings collected by Fan and his colleagues suggest that SED and SER models are promising tools for the analysis of music recordings. Interestingly, the comparative analysis of Chinese and Western classical music using these techniques led to results that are aligned with ideas presented by music theorists in China.

The researchers also observed that their deep learning classifier recognized soundscape recordings as Chinese classical music. This suggests that soundscape recordings typically share more similarities with Chinese classical music than with Western classical music.

"Our study highlighted that certain similarities exist between Chinese classical music and soundscape recordings," Fan said. "These results are aligned with those reported by Chinese musicologists and Chinese classical music philosophers."

In the future, the study conducted by this team of researchers could inspire other studies comparing different genres of music based on the analysis of soundscape models. Meanwhile, Fan and his colleagues plan to continue investigating the similarities and differences between Chinese and Western music using deep learning methods, while also trying to build models that can automatically compose new classical music.

© 2020 Science X Network