Generative AI technologies revolutionizing live sports coverage and commentary

The live-action footage seen on a video streaming service was actually a virtual video created with artificial intelligence. These instances are becoming increasingly common.

We are entering an era in which AI will be used broadly and deeply in our day-to-day lives and businesses without people realizing it. Generative AI has evolved to become an advanced, general-purpose technology that has reached the level of practical use, and the usage scenarios have become more familiar applications. Nowadays, the application of AI including generative AI has even expanded into the field of sports. The following explains the trends in AI utilization and its impact with a focus on applying AI to live coverage.

AI utilization steadily expanding even in the field of sports

AI application is particularly effective in advanced tasks that previously relied on human labor

Previous information processing technologies that used computers have streamlined many tasks and operations. As a result, they have had an impact that has changed how we live and socialize.

When we look at where generative AI and other advanced AI technologies that are currently spreading are being applied, we see trends that differ from past information processing technologies. First, their use is expanding in operations and tasks that required detailed manual handling. In the past, there was a tendency to apply information processing technologies to automate repetitive tasks. However, there are now many examples in which AI is being used in advanced tasks and operations that require a insight and knowledge with a high level of expertise.  In addition, because these systems train on data to acquire the ability to make appropriate decisions, they tend to be frequently applied to tasks and operations that require individual capabilities backed by experience.

Live sports coverage requiring highly specialized commentary and announcing skills also shifting to AI

Live sports announcers and commentators are among the most specialized professions in the industry. For instance, many individuals providing commentary for baseball games on television have prior experience as professional athletes, making the pool of potential candidates quite limited.  Therefore, television and other forms of mass media are forced to focus their coverage on the competitions, matches, and athlete plays that have a large number of viewers. However, it is becoming increasingly clear that applying generative AI and other advanced technologies to sports broadcasting is creating new forms of value in this traditionally specialized field.

Coverage and commentary on featured players and shots that are not broadcast on television

In 2023 and 2024, an overseas golf tournament used generative AI to automatically generate video clips which provided live coverage and commentary of each shot taken by every golfer.

When broadcasting a golf tournament, the coverage follows the group in which a featured golfer is playing and then switches to another group when an amazing shot occurs. However, there are probably many viewers who wish to follow a different group of their favorite golfers instead of those who are being broadcast or they might wish to pick out and watch only the approach shots that they are not very good at or perhaps just the trouble shots. While this niche interest might represent a small segment of the audience, it could actually comprise a larger share of total viewership than we might anticipate.

The service provided at this tournament utilized generative AI that analyzed the techniques of past announcers to deliver automatic commentary, even in situations typically overlooked in broadcasts. However, relying on human announcers would require significantly more manpower, making it impossible to cater to all the preferences of a smaller audience. With multiple shots happening simultaneously during a tournament, coordinating announcers to cover every moment would be nearly impossible and would entail substantial costs and effort. In such scenarios, the use of generative AI proves to be highly effective.

Image showing the AI service that automatically covers and comments on every golfer, hole, and shot

The AI that was used by the broadcast service was trained to automatically generate commentary narration. Using the data from all of the shots, the AI automatically generated over 20,000 video clips - all with a natural, concise narration that even incorporated proper golf terminology.

The service provided during the hosting of the 2024 tournament also added a golfer match prediction feature for each hole. The AI model was trained using eight years of tournament data including more than 170,000 shots and data describing the position of the ball on the course to enable the model to predict a golfer's score at each hole during the tournament.

When making predictions, the model utilized each golfer's most recent performance after they completed each hole to update its forecasts accordingly. Based on this data, it identifies the most significant holes for each golfer and calculates the minimum and maximum potential scores for all participants. Additionally, the model assesses play conditions at each hole using historical and current performance data, offering insights such as "the ninth hole is expected to be the third most challenging in today's tournament" and providing statistical commentary like "Statistically speaking, a shot from this location has an 82% chance of resulting in a birdie." Furthermore, the model is capable of delivering narration in both English and Spanish simultaneously.

Providing support to enable fans both with and without disabilities to enjoy watching sports together

In addition, there are also efforts to create systems that utilize advanced AI to provide personalized coverage and commentary for specific individuals. A technology has already been developed that provides real-time coverage of a car race taking place on the circuit to fans with visual impairment in the spectator stands.

Visitors attending a circuit race can feel the exhilarating intensity of roaring engines in a way that is truly unique to the live experience. However, visually impaired fans often struggle to grasp what is happening in front of them, making it difficult for them to fully enjoy the event. Many have shared feelings of guilt and isolation, even when accompanied by friends or family who attempt to describe the race. The rapid movement of cars and sudden explosive sounds can be overwhelming, turning what should be an exciting atmosphere into one of anxiety for those with disabilities.
The race can become disheartening for these fans. However, through AI-based real-time coverage used in demonstration experiments, visually impaired attendees were able to engage with the event alongside their fellow spectators, transforming their experience into one of inclusion and enjoyment.

AI-based real-time coverage enables visually impaired persons to enjoy the race together with other fans

This AI system combines three AIs with different roles consisting of "object recognition," "indication discovery," and "speech frame" that are each linked together to achieve real-time coverage. The object recognition AI recognizes the moving race cars from video recorded with fixed cameras. It understands the race conditions such as what team a depicted car belongs to and which drivers are battling it out on the circuit. The indication discovery AI analyzes data such as the lap times and ranking, etc. that are collected in real time. It searches for "indications" that a change in the race's development may occur such as when someone might overtake the lead vehicle. The speech frame AI trains on the past race coverage of play-by-play announcers to generate natural and immersive coverage.

Potential of AI automation technologies expanding to areas beyond sports

There are many possible applications for technologies that automatically cover and comment on sports using AI. One easy example might be coverage of children's athletic meets. In addition, because these technologies can provide not only coverage but also commentary, we might see the development of AI that can watch practice scenes to provide accurate coaching or AI caddies, etc.

Furthermore, they would also seem to be applicable in areas beyond sports such as education or service provision, etc. to improve quality. For example, AI might be applied in cooking practice to capture scenes of cooking and suggest points of improvement to create dishes in a more efficient and skilled manner or it could observe a customer's mood and small gestures during customer service in a retail store for attentive "hospitality" that suggests the products and services that the customer is looking for in advance.

Until now, services with such a high level of hospitality could only be provided by people with extensive experience. Therefore, not many people were able to receive a sufficient level of service. In the near future, the use of AI might make it possible to provide the same high-quality service to more people in more diverse situations.