Game, Set, Machine Learning: How IBM Is Fine-Tuning Its Wimbledon Tech

July 11, 2019, 11:11 AM UTC
Simona Halep returns a shot in her quarter-final match against Shuai Zhang
Romania's Simona Halep returns against China's Shuai Zhang during their women's singles quarter-final match on day eight of the 2019 Wimbledon Championships at The All England Lawn Tennis Club in Wimbledon, southwest London, on July 9, 2019. (Photo by GLYN KIRK / AFP) / RESTRICTED TO EDITORIAL USE
Glyn Kirk—AFP/Getty Images)

Simona Halep’s Wimbledon dreams were in trouble.

On Tuesday, the Romanian player, currently ranked seventh in the world, was struggling in her quarter final match against the Shuai Zhang, a Chinese player ranked 50th.

Zhang, who had bested Halep in two of their three previous meetings, was up four games to one in the first set. It didn’t look good for Halep.

But then, Halep won a drawn-out baseline duel and began battling back. She pulled even with Zhang at four games apiece. From there, she began to dictate the tone of the match, going on to triumph over Zhang in straight sets, 7-6, 6-1.

It was dramatic tennis. But the result might have looked more predictable to those monitoring the game on Wimbledon’s official mobile app or website. There, fans can check out a feature called “Keys to the Match,” which is produced by tech giant IBM using machine learning and troves of tennis data. For each player, it lists three metrics that, statistically speaking, the athlete should achieve in order to be likeliest to win the match.

For Halep, these included winning at least 39% of her first serve return points, winning at least 59% of her points on first serve, and—a bit of an odd one—having a second serve kick height of at least 1.4 meters. In the first five games of the match, Halep had failed to hit any of these targets. But by the time she pulled even, she was over the 59% first serve win threshold, and at the end of the first set, was achieving both points-based metrics. According to IBM’s number crunching, this gave her a greater than 80% chance of winning the match.

Tech’s proving ground

It was a small example of the extent to which Wimbledon has become a proving ground not just for tennis top stars, but for IBM’s tech too. Big Blue has been the official technology partner for the All England Lawn Tennis Club (AELTC), Wimbledon’s host, for the past 30 years. And every year, the company uses the event to showcase some bit of techno-wizardry it also hopes to sell to the world’s biggest corporations.

Wimbledon 2019 umpire serena
Cori Gauff talks to umpire James Keothavong as she plays against Romania’s Simona Halep on the seventh day of the 2019 Wimbledon. IBM tech collects crowd noise using special microphones implanted in umpires’ chairs. (GLYN KIRK—AFP/Getty Images)
GLYN KIRK—AFP/Getty Images

Sports has become an important marketing stage for many of the world’s tech giants. In addition to Wimbledon, IBM has a similar relationship with the U.S. Open. Its rival cloud company, Amazon AWS, helps run the National Football League’s “Next Gen Stats” sports statistics platform and is the cloud provider for Nascar. Microsoft is a technology partner for international soccer club Real Madrid.

Watson goes to Wimbledon

Throughout Wimbledon, IBM has a team of 180 staff on site. From a set of secure, windowless, bunker-like rooms in the basement of the official media center, a core team monitors video and data feeds from every court. They maintain the private and public cloud computing infrastructure that runs the official Wimbledon website and mobile app, as well as all the internal communications and data feeds. Its cybersecurity experts scrutinize the network traffic for anything suspicious.

For the past few years, IBM has used its Watson artificial intelligence platform to automatically select video highlights from each game. Its algorithms do this by taking into account how critical a particular point is to the game as well as the celebratory gestures a player makes after the point. They also monitor the level and nature of the crowd noise, which is collected using special microphones implanted in each umpire’s chair.

But what about players who simply fist pump more than others and crowds that favor a particular player, like Serena Williams or, current British men’s No. 1, Kyle Edmund? Those factors meant there was a risk the system might not always pick the best clip, says Sam Seddon, the IBM executive who manages Big Blue’s relationship with the AELTC. So this year, IBM tweaked its algorithms to compensate for these elements, Seddon says. That ought to produce better clips and ensure IBM’s client, AELTC, is happy.

Serena Williams at Wimbledon 2019
IBM’s algorithms now account for crowd bias that might affect monitoring of matches of fan favorites, like Serena Williams. (Shi Tang—Getty Images)
Shi Tang—Getty Images

The problem of bias in the data that trains A.I. systems reaches far beyond the Wimbledon grass. Concerns about it have made some corporations reluctant to deploy machine learning systems. That’s presented a business problem for IBM and the other major cloud service providers who have been touting their A.I. capabilities as a way to win customers. So IBM has rolled out “fairness tools” that are supposed to identify and, in some cases, automatically compensate for skewed data.

Another tweak to the A.I. video highlights this year, Seddon says, is that IBM has taught Watson to identify the sound of the racket striking the tennis ball. Finding this exact moment allows the system to cut the highlight clips as close to the action as possible—an important factor for AELTC, where every second of video included in a highlight clip can translate into money gained or lost due to complex licensing arrangements with sports broadcasters.

Out of bounds

Wimbledon also reveals some of the limits of today’s A.I. For instance, while Watson can find the best candidates for video highlights, it can’t yet automatically analyze the content of those videos to produce sports statistics in real time. At Wimbledon, most of those key stats—some 4.5 million data points per tournament—are still entered by trained human experts who sit courtside and rapidly key in the results of each serve, volley and point using a specialized keypad. While some clubs and tournaments have experimented with using computer visions algorithms to automatically generate stats, Seddon says they are at best 95% accurate—and that’s not good enough for an environment like live sports.

The same goes for those Keys to the Match factors that Watson selects. While IBM could use machine learning to produce correlations that are far more complex than the three factors the system currently selects for each player, Seddon says the company favors simple metrics that are both easy to calculate in real-time and also simple for fans to grasp. (IBM does provide a bevy of deeper dive analytics—for a price—to broadcasters, players and coaches, and yes, the sports betting market.)

Another thing IBM’s A.I. can’t yet do at Wimbledon: tell you for certain which players will succeed in posing with the famed Venus Rose Water Dish and golden cup at the finals this weekend.

More must-read stories from Fortune:

—Switzerland’s stock-trading standoff with the EU provides a glimpse of life after Brexit

The Bahrain Conference: What the experts and the media missed

Ford’s new plan for Europe: Fewer jobs, more SUVs

The British Royal Family’s costs are skyrocketing. Here’s why

—Listen to our new audio briefing, Fortune 500 Daily. Catch up with Data Sheet, Fortune‘s daily digest on the business of tech.