How Hudl and Visionular are upping the quality game

At Hudl we have a saying, “What got us here, won’t necessarily get us where we want to go next.” That couldn’t be a truer sentiment in the ever-evolving world of technology. For us, the video technology that brought us here has been AVC (H.264). Hudl has long used the x264 VideoLAN encoder to process and transcode our coaches’ video. In September 2022, events would be set in motion to change all that.

By now, many of you know about Google Chrome’s silent HEVC update. A change like this doesn’t go unnoticed at a company that currently has an S3 storage footprint over 100PB. That storage consists of hours upon hours of game and practice film from our coaches. So when technology that 68% of our users support suddenly becomes available in Google Chrome, we take action. It was time to re-evaluate our encoding ladder.

The goal was to determine the most impactful changes we could make for our users. When evaluating any new renditions we ask ourselves three primary questions: how many of our users have the capabilities to decode this video; what bitrate can we set to achieve our required quality level; and what is the encoding performance like? At Hudl’s scale, any of these questions can have a significant impact on cost. Even worse, a negative impact on the experience our coaches and athletes have.

Evaluating the optimal encoding ladder across a full suite of products and customer requirements can be quite an undertaking. At Hudl, we’ve made this easier by developing an automated quality testing tool. First, we find a sampling of coaching content that’s:

• A variety of sports
• Varying angles and distances of capture
• Varying lighting, both indoor, outdoor and different times of day
• Varying levels of background noise, like crowd motion or waves in a pool

We run each video file through the encoders that we’re testing, then transcode the video thousands of times with various settings, resolutions, bitrates and speeds. During the encoding process, we sample a plethora of data. We capture various quality statistics (e.g., VMAF, SSIM, PSNR) and performance metrics. After all encodes are finished, we generate a report that provides us with details on storage, performance and streaming impact. This gives us the ability to compare any number of encodes against one another. Of course, the process is not entirely automated — we still prefer to perform manual video inspection to look for all the usual suspects.

We had a few encoders in mind to put through our testing utility. Why not pit them against one another to see which one would reign supreme? After all, we wanted to create a better experience for our coaches and athletes — this test would help us do that. And we had a list of the encoders to test for this run:

• VideoLAN x264 (H.264)
• MultiCoreWare x265 (H.265)
• Visionular Aurora4 (H.264)
• Visionular Aurora5 (H.265)
• Another leading codec company (H.264, H.265)

The results? They were quite the sight to behold. It’s not always the case, but generally a trade-off is made during encoding. You sacrifice encoding speed for better compression and quality, or vice-versa. It became clear from the results, Visionular libraries required no such sacrifice. No need to take our word for it though, we have the results.

Hudl’s testing utility runs in AWS Batch using ECS instances. We use a variety of constraints for each test based on the requirements of the encoder. Because we’re evaluating CPU based encoders, our infrastructure is based around vCPUs. Containers are run with Ubuntu 22.04 LTS x86_64, and with 4, 8 and 12 vCPUs per container. These constraints mirror our production environment.

Average performance across all encoders indicated that the Aurora4 & Aurora5 were superior. Aurora4 was 50% faster than x264 and the other leading encoder company. Aurora5 offered similar improvements over x265. Even as we increased bitrate, performance degradation was minimal — an impressive result. The performance improvements were even more obvious with fewer vCPUs. Let’s take a look at a breakdown of performance over 4, 8 and 12 vCPUs when compared against the competition.

Results support the assertion that Aurora4 & Aurora5 are more efficient per core. Better per-core efficiency makes Aurora4 & Aurora5 excellent candidates for use in AWS Lambda. We use AWS Lambda to deliver dynamic, on-demand renditions of our HLS video. But that’s a topic for another post.

Performance improvements were only one piece of the pie. The visual quality improvements were stunning. Objective metrics, like VMAF, showed significant improvements in quality. The results once again showed Aurora4 & Aurora5 crushed the competition. We saw a ~17% improvement with Aurora4 VMAF over x264, and a ~10% improvement in quality over their other competitors. Aurora5 showed similar improvements over x265 and their competitor — ~19% and ~10%, respectively. This means that Aurora4 & Aurora5 will deliver equivalent quality video at lower bitrates. This is a big win for Hudl — and a big win for our coaches and athletes.

Objective metrics like VMAF are nice to see, but we know VMAF is no substitute for the eye test. Manual visual testing is the ultimate judge of how well these encoders perform. When evaluating video quality in the sports analysis world, we look for a few factors: facial details, field details and jersey numbers. Aurora4 provides us with a clearer encoded image, making jersey numbers sharper and easier to read. This assists analysts and OCR processes with jersey detection. Field lines were more pronounced and bled less into the environment around it.

Take the image comparison below as an example, showing a high school basketball game. The image on the left was encoded using x264 1080p High@3Mbps. On the right was Aurora4 1080p High@2.5Mbps. We have highlighted a few areas of interest showing how much clearer the image became at a lower bitrate. Pay close attention to the solid colors of the Aurora4 image as well — the colors are more vibrant and less washed out.

Aurora5 offered improvements over the competition as well. The image comparison below shows x265 4K Main@5Mbps (left) versus Aurora5 4K Main @ 4.4Mbps (right). The grass, jerseys, colors and field details are much better with Aurora5.

As I alluded to earlier, the improvements were so great we decided to lower our rendition bitrates. This has the potential to improve our storage footprint while also making our highest qualities more accessible to coaches and athletes.

We’re excited to be in business with the Visionular crew for Hudl’s video encoding solution. Aurora4 is now a welcome part of our encoding suite. We look forward to implementing HEVC and Aurora5 into our offering very soon!