Google launches AI lip-sync challenge

Google launches AI lip-sync challenge
Credit: Google

Google has launched a lip-sync challenge for anyone who cares to participate. Those interested can visit the site Google has set up and test their lip-synching skills. The challenge is being run by Google's AI Experiments group—the same group behind Google's Pixel device.

The purpose of the challenge is to allow Google to use real singers to teach its AI system (which the company is calling simply LipSync by YouTube) how to read lips. Google hopes to use the AI system as part of an effort to develop applications for people with speaking disabilities such as ALS.

The challenge is simple: Users can click on the "Launch Experiment" button, which takes them to another screen that apparently configures Google's system to work with the user's equipment. No microphone is required—just a video camera. In a subsequent screen, the user appears in a bubble similar to those in smartphone video chats. The user clicks on the "I'm Ready" button, launching the lip-sync tune, "Dance Monkey" by Tones and I—there are no other choices. The user is then supposed to sing the song out loud or silently. After the song ends, the user is taken to another screen where they receive their score represented as one to five stars.

On the initial challenge page, Google notes that the challenge is built on the company's TensorFlow.js AI technology and that it works by tracking facial landmarks as the user lip syncs. They note also that the application has been optimized to work best with Google Chrome. They do not reveal whether the lip-sync will be expanded or updated, or how long it will be available.

Visitors to the site can also choose to get the web app code and instructions on setting it up and using it with other web applications.


Explore further

Google suggests trust tokens as part of ending support of third-party cookies

More information: Lip Sync Challenge: experiments.withgoogle.com/lipsync

LipSync by YouTube: lipsync.withyoutube.com/

© 2020 Science X Network

Citation: Google launches AI lip-sync challenge (2020, September 25) retrieved 29 October 2020 from https://techxplore.com/news/2020-09-google-ai-lip-sync.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
54 shares

Feedback to editors

User comments