- UNIST builds 'SVHighlights' covering 8 sports and 320 videos

- Official broadcaster highlights used to generate ground-truth labels automatically

UNIST Professor Kim Tae-hwan (from left), researcher Lee Dong-gyu and researcher Ki Young-bin, who conducted the study. [UNIST]
UNIST Professor Kim Tae-hwan (from left), researcher Lee Dong-gyu and researcher Ki Young-bin, who conducted the study. [UNIST]

A new benchmark has emerged to rigorously test how accurately AI can identify decisive moments in sports — a last-minute winning goal in soccer, a come-from-behind home run in baseball. The key advance is its use of real broadcast footage up to 60 times longer than what existing benchmarks employ, enabling evaluation of an AI model's ability to understand video over extended durations.

The Ulsan National Institute of Science and Technology (UNIST) announced Sunday that a research team led by Professor Kim Tae-hwan of its AI Graduate School has developed SVHighlights (Sport Video Highlights), a large-scale benchmark for evaluating AI models that extract highlights from sports footage.

An AI benchmark functions as a standardized test that compares the performance of multiple AI models under identical conditions. It requires not only the problems the AI must solve but also precise ground-truth answers against which performance can be judged.

The problem with existing sports video benchmarks is that most have been limited to short clips averaging two to four minutes. Human annotators had to watch footage from start to finish and manually mark every highlight segment, meaning longer videos demanded enormous amounts of time and money.

SVHighlights covers eight sports — soccer, baseball, basketball, volleyball, American football, ice hockey, rugby and racing — and comprises 320 videos totaling 640.18 hours. Each video averages about two hours in length, making it 30 to 60 times longer than clips used in previous benchmarks and allowing AI performance to be evaluated in conditions that closely resemble actual sports broadcasts.

The key to building such a large dataset was the use of official highlights already produced and published by broadcasters. Scenes selected by professional editors served as the ground-truth answers for AI evaluation.

The architecture of an AI model designed to identify highlights in sports footage. [UNIST]
The architecture of an AI model designed to identify highlights in sports footage. [UNIST]

The research team also developed a matching algorithm that automatically locates, down to the minute and second, where each scene from an official highlight reel appears in the full-game footage. The algorithm compares frames from the original broadcast and the highlight video at the pixel level to find the most visually similar moments.

The team also accounted for a characteristic of sports broadcasts in which scoring plays and other key moments are replayed multiple times. Beyond visual similarity, the algorithm analyzes the chronological order of scenes to reduce errors in which a replay — identical in appearance to the original — is mistakenly selected instead of the actual live moment.

Human involvement was reduced to marking the start and end points of each match and verifying the automatically matched frames. A quality check found that incorrectly matched frames accounted for just 0.18 percent of the total.

Using the benchmark, the team also developed an AI model called TF-SELECTOR that automatically identifies highlights in long-form video.

TF-SELECTOR combines scene segmentation, speech recognition, a vision-language model and an LLM. It divides footage based on scene transitions, converts commentary audio into text, then assesses the importance of each segment by synthesizing the visual content, commentary and audio volume.

Evaluated using SVHighlights, TF-SELECTOR outperformed the next-best model by 2.50 percentage points on HIT@1 — which measures whether the single most important predicted segment is an actual highlight — and by 4.04 percentage points on HIT@K, which reflects how many real highlights the model successfully retrieved. Its intersection over union (IoU) score, indicating how closely predicted segments align with actual highlight segments, was also 2.95 points higher.

The technology is expected to find applications beyond automated sports highlight production, potentially extending to film and drama series summarization, meeting transcription, and the identification of key moments in long-duration surveillance footage.

"We replaced the ground-truth labeling work that used to take humans several hours per video by leveraging highlights that broadcasters had already produced," Kim said. "Because the dataset can be continuously expanded, it will help objectively evaluate long-form video analysis AI and support the development of even better models."


nbgkoo@heraldcorp.com