Google launches AI lip-sync challenge

Google has launched a lip-sync challenge for anyone who cares to participate. Those interested can visit the site Google has set up and test their lip-synching skills. The challenge is being run by Google's AI Experiments group—the same group behind Google's Pixel device.

The purpose of the challenge is to allow Google to use real singers to teach its AI system (which the company is calling simply LipSync by YouTube) how to read lips. Google hopes to use the AI system as part of an effort to develop applications for people with speaking disabilities such as ALS.

The challenge is simple: Users can click on the "Launch Experiment" button, which takes them to another screen that apparently configures Google's system to work with the user's equipment. No microphone is required—just a video camera. In a subsequent screen, the user appears in a bubble similar to those in smartphone video chats. The user clicks on the "I'm Ready" button, launching the lip-sync tune, "Dance Monkey" by Tones and I—there are no other choices. The user is then supposed to sing the song out loud or silently. After the song ends, the user is taken to another screen where they receive their score represented as one to five stars.

On the initial challenge page, Google notes that the challenge is built on the company's TensorFlow.js AI technology and that it works by tracking facial landmarks as the user lip syncs. They note also that the application has been optimized to work best with Google Chrome. They do not reveal whether the lip-sync challenge will be expanded or updated, or how long it will be available.

Visitors to the site can also choose to get the web app code and instructions on setting it up and using it with other web applications.

© 2020 Science X Network