Optical music recognition with convolutional neural network

Optical character recognition (OCR) commonly used to convert the text in scanned documents into a searchable and editable form on the computer is a well-established digitisation technique. But, what about other kinds of documents, rich with meaning, such as musical manuscripts? New work in the International Journal of Arts and Technology discusses the possibility of optical musical recognition, OMR.

A new approach developed by a team at Bina Nusantara University in Jakarta, Indonesia, uses deep machine learning and a convolutional neural network trained to recognize the nuance of musical notation on known manuscripts. The algorithm can then convert a newly presented musical manuscript into a digitized form with 8 percent accuracy. Even at this level, this greatly reduces the amount of manual input and correction needed to convert a manuscript.

The system requires clef, stave, and musical key to be in position, but these are easily assigned in a template. The conversion of a scanned manuscript then detects the position on the stave of each note, thus defining pitch. The next step will be to use a parallel algorithm to detect the duration of each note and to identify the position of silences, rests, and other such characteristics of a manuscript.

Once fully digitized it is, given current software, a trivial matter to use the computer to "play" the manuscript using all manner of instrumental sounds or even to correlate a lyrical score with the music and have the computer "sing" the song. OMR, once mature, will have many applications in archiving musical manuscripts, in the performance of music, and in music education. The team suggests that their approach could allow software "app" developers to write a program for smartphone or tablet to allow anyone to quickly scan a piece of sheet music, for instance, and to carry out OMR on that manuscript.

Of course, while music digitization tools could be enabling for a wide range of people interested in music, there is still the question of musical talent. There is, unfortunately, no app for that.