Baidu's artificial intelligence advance makes music by looking at pictures

(Tech Xplore)—Suddenly a prediction made back in April carries special resonance. Zen Soo, technology reporter at the South China Morning Post, was referring to a Microsoft executive saying China will lead the world in producing artificially intelligent hardware as the tech industry continues to find breakthroughs in this field.

The Microsoft person speaking was Harry Shum, executive vice-president of technology and research in Microsoft. He cited examples of Chinese internet companies playing an important role in advancing artificial intelligence—like Baidu.

Baidu this month is proving to be pretty impressive in the field. Roger Decierdo in Yibada wrote Friday that Baidu has figured out how to create music just by looking at art. "Chinese Internet giant Baidu unveiled a new artificial intelligence program that reportedly can create music based on the art it sees."

Humans do not have much difficulty thinking up music that can go with pictures. We may not be able to explain our thought express in every instance, but the brain works its magic. A painting may inspire a melody but now a video posted earlier this month about Baidu's composer indicates what AI can do about this crossing over between two distinct art forms.

We hear a sad splash of musical notes for a woman standing in the rain. We hear light, fluttery string sounds for a tropical beach scene, for example.

C. Custer in Tech in Asia said the company's AI feat went on exhibit at the Ullens Center for Contemporary Art in Beijing, which "analyzes paintings and then composes original music inspired by them."

Decierdo wrote about how the program, the Baidu AI Composer, works. It creates by scanning the image of an art piece. It identifies the various objects and then assigns a mood to the color elements of the piece. Once it does mapping and the assigning of attributes to the various elements of the art piece, the program goes through a database of musical scores.

But that is not the last step. Next steps are interesting. Decierdo said the AI composer feeds the labeled and categorized image data "to a complex correlation matrix for the selection of the units. A correlation matrix is a table that shows the varying relationship between the labels and the musical units. By using the matrix, the program is able to create a unique musical piece every time."

Custer in Tech in Asia offered his take on what they have accomplished at Baidu. "Obviously, whether or not music really 'fits' an image is entirely subjective, but I think most people would agree that Baidu has at least come fairly close here."

Yibada looked at the bigger picture: "Baidu has increasingly been investing in the field of artificial intelligence in the last few years in an effort to diversify its revenue source."