What about exploring pictures just by moving around? Lots of 11-year-olds would find this a great idea, especially if the alternative was a homework assignment on French verbs.
Welcome to Move Mirror, where you move in front of your webcam.
Google takes to the idea of making machine learning more accessible to coders and makers. The desired outcome is inspiring them to play around with this technology. Move Mirror's intent is to show computer vision techniques such as pose estimation and to do it in fun ways.
Move Mirror matches your movements to hundreds of images of people doing similar poses. "It's kind of like a magical mirror that reflects your moves with images of all kinds of human movement—from sports and dance to martial arts, acting, and beyond," says the team.
Well, you get to match your pose against a database of tens of thousands of photos. The experiment has a welcoming message that "You move and 80,000 images move with you."
Fun aside, this experiment, a collaborative effort by PAIR, Research, and Creative Lab teams at Google and friends at Use All Five, has purpose. It signals a way of life in the machine learning development community. Advances in machine learning are paraded, with hopes to engage other people with relevant interests, as all push forward research in the field.
Irene Alvarado, Creative Technologist at Google Creative Lab, said, "With Move Mirror, we're showing how computer vision techniques like pose estimation can be available to anyone with a computer and a webcam. We also wanted to make machine learning more accessible to coders and makers by bringing pose estimation into the browser—hopefully inspiring them to experiment with this technology."
JC Torres in SlashGear: "Move Mirror may seem like a frivolous, but fun, AI demo, but it does have some positive implications for AI."
And on that note you can try it out for yourself.
Move Mirror was made using PoseNet and TensorFlow.js. Alvarado defined the latter as "a library that runs machine learning models on-device, in your browser—which means the pose estimation happens directly in the browser, and your images are not being stored or sent to a server."
That is a good point for those who are told they have to use a webcam for any experiment. Privacy worries promptly surface. In this experiment, the images are not sent to any Google servers while the person interacts with Move Mirror. Image recognition happens locally in the person's browser
"It's definitely impressive how sophisticated machine learning can now be done just in web browsers," Torres said. "And it's definitely reassuring to know that you don't always need to send your data, much less your photos, to some computer in the cloud just to reap the benefits of AI."
How are the images matched?
Move Mirror turns to pose information to find a matching image. This involves the sites for 17 body parts, e.g., right shoulder, left ankle, right hip and nose. The team noted that Move Mirror does not take into account race, gender, height, body type.
Taylor Kerns, Android Police, explained what happens: PoseNet "recognizes the overall position of a human subject by analyzing and adding up where different parts and joints are in a photo or video. Your position is analyzed in real time and compared to a set of 80,000 photos. Move Mirror shows the closest match to each of your positions, stringing them together in a slideshow."
How was this Move Mirror AI experiment built? PoseNet is the pose estimation model they use; it runs in the browser using TensorFlow.js. Alvarado said, "We hope you'll play around with Move Mirror and share your experience by making a GIF."
The article "Move Mirror: An AI Experiment with Pose Estimation in the Browser using TensorFlow.js," is good to check out in Medium if you are curious in knowing all the details about their work on this.