Algorithm predicts the next shot in tennis

QUT researchers have developed an algorithm that can predict where a tennis player will hit the next ball by analysing Australian Open data of thousands of shots by the top male tennis players.

Dr. Simon Denman, a Senior Research Fellow with the Speech, Audio, Image and Video Technology Laboratory, said the research into the match play of Novak Djokovic, Rafael Nadal and Roger Federer could lead to new ways for professional tennis players to predict their opponent's moves or virtual reality games offering the chance to go head-to-head with world's best players in an accurate but artificial grand slam.

Dr. Denman is part of a team of QUT researchers, including Ph.D. student Tharindu Fernando, Professor Sridha Sridharan and Professor Clinton Fookes, all from the Vision and Signal Processing Discipline at QUT, who created the algorithm for predicting the next shot in tennis using Hawk-Eye data from the 2012 Australian Tennis Open, provided by Tennis Australia.

The researchers narrowed their focus to study just the shot selection of Djokovic, Nadal and Federer because they had the complete data to input into the system on how the players' shot selection changed as the tournament progressed.

The researchers analysed more than 3400 shots for Djokovic, nearly 3500 shots for Nadal and almost 1900 shots by Federer, adding context for each shot such as whether it was a return, a winner or an error.

"After about 1000 shots, the model has a pretty good idea of what is going on," Dr. Denman said.

"It needs about three matches to figure out a player's style. Once it's got those three matches it's pretty solid."

Dr. Denman said the algorithm was mimicking the brains of the top players who were already trying to predict their opponent's next shot. While the artificial intelligent system proved successful, its biggest challenge was the unpredictability of the player the Greatest Of All Time.

"We had an analysis of how accurate it is for those three top players – Djokovic, Nadal and Federer," Dr. Denman said.

"And it was least accurate for Federer, who is perhaps the most versatile. It struggled the most to predict him. He can do anything, so the model was more often wrong about him.

"Given how hard Federer's game is to predict, it just adds to the credit of someone like Stefanos Tsitsipas who managed a victory against Federer in the Australian Open on the weekend."

The machine learning system, called a Semi Supervised Generative Adversarial Network architecture, takes into account when in the match a point is being played, because the shot selection a player might make 40-love up serving for the first set is very different to the shot they might hit struggling to stay in the match deep into the fifth set.

The system can predict about 1000 shots in 30 seconds.

"We train the model in order so that it sees the shot from first round, to the second round and third round – so it builds on experiences like a human does," Dr. Denman said.

"We are trying to mimic what we think what the tennis player's brain might be doing."

To teach the system to think like a player, the researchers create two forms of memory and a way for them to interact in making a decision. "There is episodic memory and there is semantic memory," Dr. Denman said.

"Episodic memory is effectively individual recollections. It's being able to go back and recall each individual stroke and what happened.

"Semantic memory is much more abstract. It's the overall learnings that came about from many, many, many instances of the episodic memory.

"Then those two memories work together given an input stimulus. They each pull out something relevant from their own memories and use that to help reinforce the prediction of what's going to happen.

"The episodic memory can look at the input and say 'I've seen shots like that here, here and here – here is something useful'. The semantic memory says 'we should hit it over to that part of the court because that's a good tactic'.

"That then helps to guide the output generation."

Dr. Denman said he thinks in less than ten years top-level players will be able to use this sort of technique in studying the game of an upcoming opponent.

"Once you've got the model all trained up, you can just feed in arbitrary samples and you can start to look at different match scenarios," he said.

The system, given the data on a particular player, could predict how that player would hit the ball on break point if a server went either down the line or out wide. It could also predict the likelihood that a particular player will try to beat an opponent at the net by hitting a lob or passing shot– and how that shot selection might change at different stages of a match.

The QUT researchers have done similar trajectory studies that predict likely outcomes on areas as diverse as how pilots fly their planes to the routes pedestrians were likely to walk around their neighbourhood. This research forecasting the next shot location in tennis is currently under peer review for publication.

Dr. Denman said tennis was particularly suitable to this type of trajectory research, given the limitations of court size and the rules of the game.

"This example here is a single trajectory with the ball, but there's no reason why these techniques could not be applied to team sports such as soccer where you are tracking every singer player from both sides," Dr. Denman said.

"Sport is good in that sense for any sort of machine learning problem because you've got all of these constraints and rules as to what can happen, which can simplify some problems compared to other domains."