LEIF always finds a way

by Olof Lindman

Automating a difficult and complex task in such a way that it increases the overall efficiency, whilst still maintaining the entirety of previous functionality, is one of the more rewarding deeds an engineer can accomplish. The key, and subsequently the challenge, is usually to find or invent a solution that streamlines a given manual process without sacrificing its inherent flexibility. During the second biannual Tekniksprint of 2018 a small team from SVT P&T managed to achieve a feat that approximated something not entirely unlike just that.

You see, we have developed LEIF

Artwork by Lina Nord

Before I go into the nitty gritty details of what LEIF is, and more importantly why it was sorely needed in our workflow, let me set the narrative stage by presenting a brief summary of the background story.

As you may, or may not, know we transcode hundreds of videos every day intended for online streaming at SVT Play. The actual number of videos is increasing steadily every year and this trend is not likely to go away anytime soon. Under such circumstances, coupled with the fact that the various input sources we handle range from single clips posted by individual journalists to big budget cinematic production masters delivered by major studios, affirming video quality assurance is becoming more and more of a challenge.

Before moving on, it is important to clarify one more thing: Usually, when a reference is made about “video quality” in general, the reader assumes that the author is talking about the acquired visual quality of an encode (or more commonly the retained visual quality of a transcode), but in this case I am talking about something much more basic. In essence, a rudimentary quality control of a transcoded video ensures that there are actual valid video and audio streams within the file, that the parameters of said streams are confined within specified limits and finally that their duration is correct in relation to the original.

In other words; “Video quality control” in this context refers to the act of ensuring that a media file is complete, uncorrupted and technically acceptable, as opposed to having achieved some level of a quality metric (i.e. VMAF, PSNR or SSIM). With that out of the way, and without further ado, let me explain what LEIF is.

LEIF (an abbreviation of LEIF Enigmatic Indication of Faults) is an automated video quality control tool developed at SVT Produktion & Teknik during the second Tekniksprint of 2018. That is to say, LEIF is able to compare a transcoded file to its original source and determine whether or not basic technical parameters are correct.

For instance, LEIF will detect if a file does have…

• A video stream (that is not all black)
• An audio stream
• The correct audio channel layout
• The correct frame rate
• The correct aspect ratio
• Is free from unintended black frames
• Is free from unintended duplicate frames
• Is free from unintended digital silence
• Interlaced frames
• The correct duration

If the answer is *negative* for one or more of the above checks, LEIF will report the error/s to the relevant department and let them determine how to proceed.

At first glance, the above checks might seem painstakingly rigorous, bordering on unnecessary given that most of them should be taken care of through options in the transcoder, *should* being the operative word here. However, with the sheer volume of media files being transcoded every day for SVT Play, we have realised that in our case “should be” is far further away from the “will be” than we would like. There are a great many edge cases that exist between the myriad of codecs and formats our transcoders take as input and technical oddities that occur as part of a rounding error once every 10000thtranscode. Thus there are minor incidents on a weekly basis, where a transcoder has produced an unacceptable media file without throwing an error or reporting a warning. This is made even trickier by the fact that the erroneous final media asset in question might actually be technically correct, whilst still unacceptable for streaming purposes. Let me give you a few examples to sort of highlight what I mean:

For instance, a technical mishap within an export from an editing station to the transcoder, as the file passes through the production network on premise, can result in a media file with correct audio but pitch-black video. That is to say that the transcoder will happily take the source file and process it without throwing an error, since all parameters are indeed technically correct, but in reality the file lacks video content.

Another error that can occur on rare occasion is that the transcoder starts its process too early, perhaps as a result an unintentional interruption of the source file transfer to a given watch folder, without proper cleanup. This in turn would result in a file that is technically acceptable, but also has the wrong duration.

Finally an all too familiar kind of issue comes courtesy of classic human error. A stressed-out editor might make the easy mistake of not connecting two clips together tightly enough on the timeline of their NLE, resulting in a few pitch-black frames between those clips. This is obviously nothing major in the grand scheme of things, but nonetheless a disturbing nuisance for any viewer and thus worthy of proper resolvement.

Now that you know what LEIF does and why we need it, let me give short explanation of how it works.

The technology behind the QC-aspects of LEIF, which is arguably the heart of the tool, is actually quite simple. Based around some clever usage of the ubiquitous framework FFMPEG, LEIF is able to run relatively short and readable command lines for each check and evaluate the output accordingly. So by utilizing existing functionality within said framework we gain two important advantages. First and foremost, anyone with intermediate knowledge of its usage (coupled with a basic understanding of video encoding and media analysis of course) can contribute with new features. Secondly due to the modular nature of LEIFs architecture, it benefits from new and improved features in FFMPEG while not being entirely dependent on one specific function that might be deprecated in the future.

LEIF is currently fully integrated and working as part of our production pipeline (after a semi rigorous testing period where our self-proclaimed collective hubris took a much deserved hit having to ironing out disastrous bugs). We already enjoying measurable and beneficial results, having detected errors and “saved” programs as well as clips a few days prior to their publishing.

One might wonder if we are so pompous as to claim that we have developed a bullet proof solution, which ensures that our workflow will never again publish a technically corrupt video on SVT Play. This is obviously not the case, there are still several ways in which an error can slip through the virtual cracks between components within our production workflow. But we think that it is immensely better that we do some rudimentary checks for every video file we transcode, than having no checks at all.

In many ways LEIF is a Highway, and LEIF always finds a way.

Originally published at blogg.svt.se on September 27, 2018.