(Tech Xplore)—An artificial intelligence experiment has emerged of the most enjoyable kind: It is called "A.I. Duet."
You trade melodies with a neural network.
"This experiment lets you make music through machine learning," said the video notes about it. The video was posted in November last year. "A neural network was trained on many examples and it learns about musical concepts, building a map of notes and timings. You just play a few notes, and see how the neural net responds."
Yotam Mann with members of the Magenta and Creative Lab teams at Google are behind the experiment. The video notes said it uses Tone.js and open-source tools from the Magenta project.
Mann is a coder/musician and in the video he talks about how, with machine learning, you play a duet with your computer.
Making music using code is not a new thing at all, said Mann. But machine learning gives us a different way to go about it.
With traditional programming, he said, he would need to write out lots of rules. He said he would basically be creating this map to tell the computer how to make decisions—but there are too many notes and timing combinations to map it all by hand.
Enter the neural networks approach. They played the computer tons of examples of melodies, he said. "Over time, it learns these fuzzy relationships between notes and timings and builds its own map based on the examples it is given."
As for their experiment, you play a few notes then go to the neural net which decides, based on those notes and all the examples it was given, some possible responses.
Mann has found it interesting to see how humans interacted with this. Instead of playing separately, taking turns, a few people played at the same time as the neural response, as if they were getting creative feedback with the computer. Yes, as long as you don't have a hissy fit when any human experiences are likened to machines, you could say they were jamming.
Mann made the code open source.
The neural net he is using is from Google's Magenta project. This is a project from the Google Brain team. The project asks, can we use machine learning to create compelling art and music?
Alexander Chen, coder and musician, Creative Lab, more recently said, "To help show what's possible with Magenta, we've created an interactive experiment called A.I. Duet.
Those who never had piano lessons should not be discouraged from trying it out.
Technical details are found on github. A.I. Duet has two parts, front-end in the static folder and back-end in the server folder. "The front-end client creates short MIDI files using the players's input which is sent to a Flask server. The server takes that MIDI input and 'continues' it using Magenta and TensorFlow which is then returned back to the client." A.I. Duet works with Python 2.7.
The A.I. Duet supports MIDI keyboard input using Web Midi API and the WebMIDI library.
The piano can also be controlled from your computer keyboard thanks to Audiokeys. The center row of the keyboard is the white keys.