March 29, 2017 report
Neural network creates dance moves for Dance Dance Revolution
(Tech Xplore)—A trio of researchers with the University of California has trained a neural network to create dance moves (step charts) for the open source version of the video game Dance Dance Revolution (DDR.) Chris Donahue, Zachary Lipton and Julian McAuley have written a paper describing their efforts and uploaded it to the preprint server arXiv.
Dance Dance Revolution is a video game seen in many television shows and movies—a player stands on colored tiles lit from below and then move their feet to the illuminated tile as a song plays, sort of like playing animated twister to music. The step charts for DDR are written by the makers of the video game, but another game very much like it produced as an open source program called StepMania, allows users to create charts of their own. Unfortunately, the process is tedious and slow. In this new effort, the research trio have used a deep learning neural network to create dance charts after teaching it how using charts made by human users. The call their system Dance Dance Convolution.
Making a dance chart is not as simple as randomly selecting tiles to light. Dancers like certain patterns and the way moves are timed—that is what makes it dancing rather than just hopping about awkwardly. To get a neural network to create such charts, the team started by training the system on how to pick step placement, which involves timing. Next, it had to be taught how to pick step selection—choosing which tile to light to tell the dancer where to put a foot and which direction it should point. To accomplish these tasks, the researchers downloaded a set of charts created by one particular popular chart creator from a public server—he wrote for all difficulty levels. The team also downloaded some other charts from other users at random. All of it was fed to the neural network.
Once the neural network was taught how to create charts, the researchers caused it to do so, producing step charts that could be fed to StepMania and danced to by human players. They report that the step charts are good enough for dancing but not as good as those produced by humans—that, they note, would require giving the system a degree of intuition that is still unique to us humans. Those who would like to try some of the charts can visit a web site they have set up for just that purpose.
Abstract
Dance Dance Revolution (DDR) is a popular rhythm-based video game. Players perform steps on a dance platform in synchronization with music as directed by on-screen step charts. While many step charts are available in standardized packs, users may grow tired of existing charts, or wish to dance to a song for which no chart exists. We introduce the task of learning to choreograph. Given a raw audio track, the goal is to produce a new step chart. This task decomposes naturally into two subtasks: deciding when to place steps and deciding which steps to select. For the step placement task, we combine recurrent and convolutional neural networks to ingest spectrograms of low-level audio features to predict steps, conditioned on chart difficulty. For step selection, we present a conditional LSTM generative model that substantially outperforms n-gram and fixed-window approaches.
© 2017 Tech Xplore
