Bjontegaard Delta-Rate Metric
ABSTRACT: This article provides an overview of an approach to compare the efficiency of several video compression configurations using the objective quality metric at a resulting bitrate level. The same compression configuration applied to different target bitrate levels produces a set of points on the Rate-Distortion (RD) curve. The Bjontegaard Delta-Rate (BD Rate) metric, referred to in this article, provides a way to estimate the average difference between two RD-curves.
Introduction
A video encoder can be configured in many ways: different GOP (Group of Pictures) structure, different quantization parameters (QP) or target bitrate, motion compensation range, etc. Furthermore, different encoders likely produce different compressed streams. Depending on the encoder and the configuration the same source video is compressed differently, each having its own bitrate and distortion value. Bjontegaard Delta-Rate (BD Rate) metric can be used to compare and choose the encoder or configuration, that provides the best compression results.
This article has been lying in my drafts for a couple of years. Finally I wrapped it up to publish!
Rate-Distortion Plot
Assume there are two video encoders to compare in terms of compression efficiency. Here by the term “compression efficiency” the following two measures are assumed:
1. Bitrate (bits per second) of a compressed video sequence.
2. Video distortion introduced by lossy compression.
Both encoders allow configuring their compression settings, affecting target bitrate and distortion. Within a single configuration the target bitrate can be varied (varying the quantization parameters) producing different bitrate and distortion.
Let us assume we have four different compression results for each of the two encoders (A and B). Marking the bitrate on the x-axis and PSNR distortion on the y-axis the rate-distortion plot can be drawn as shown in the figure below. Obviously, the higher the points are located, the more efficient the compression is (less distortion and higher visual quality is provided at lower bit rates).
In the example above we obviously see Encoder A provides superior results to Encoder B.
How do we compare the efficiency in numerical form?
Rate-Distortion Curve
One way to compare the efficiency in numerical form is to fix one measurement and compare by another. Let’s say, we want to compare bit rate savings at a certain PSNR distortion level of 38 dB, but available measure points are not exactly there. We need to interpolate the rate-distortion curve to get an approximation of the bit rate at the target distortion level (see figure below).
With the help of interpolation we now have two RD curves instead of points. Interpolation of an RD-curve can be done in different ways: linear, polynomial, and spline interpolations. The most widely adopted is the piecewise cubic interpolation.
By measuring the distance between the two curves at the certain distortion level the bitrate “savings” of the one encoder compared to the other one at this distortion level can be compared numerically.
But how do we compare compression efficiency on the whole distortion interval available?
BD Rate Metric
Bjontegaard Delta-Rate (BD Rate) metric, proposed in 2001 by Gisle Bjontegaard, is a method for calculating the average difference between two rate-distortion (RD) curves [VCEG-M33].
The basic steps are:
- Fit a curve through several data points (see Rate-Distortion Curve).
- Find an expression for the integral of the curve [VCEG-M33].
- The average difference is the difference between the integrals divided by the integration interval.
In other words, to compare two encoders (or two encoding configurations) the area between the two RD-curves must be divided by the integration interval as shown below.
BD-Rate is measured in percent and expresses the average percentage difference in bitrate of the two datasets at a similar distortion value measured by an objective metric (usually PSNR). The BD Rate value is zero when calculated metrics are the same. In a comparison between Encoder A and Encoder B a negative BD Rate would mean Encoder A provides better compression efficiency (lower bitrate at a similar visual quality level). A positive value would indicate inferior compression efficiency.
If PSNR distortion value is compared using the bitrate as the basis of integration, the metric is called BD-PSNR and is measured in dB.
It was observed that the difference between the curves is dominated by the high bitrates. The higher the bitrate is, the wider the distance usually is between the curves. Hence it is considered more appropriate to do the integration based on logarithmic scale of bitrate [VCEG-M33].
Another approach is to split the curve into three different quality ranges: low, medium and high, and additionally calculate the metric for low and high quality ranges [VCEG-AI11].
The piecewise cubic interpolation combined with logarithmic scale was later found to provide more stable results compared to a regular cubic interpolation [VCEG-AL22].
The impact of the number of existing curve fitting approaches is studied in [MHV-22]. The study also shows that the use of BD Rate metric computation for metrics other than PSNR should be done with caution, as they might produce unreliable results.
The BD Rate metric is widely adopted by the ITU-T JCT-VC group [STP-VID-WPOM].
Useful Links
- BD Rate estimation in Python. GitHub: Anserw/Bjontegaard_metric.
- BD Rate estimation in C++17. GitHub: tbr/bjontegaard_cpp.
- BD Rate estimation in TypeScript. GitHub: YoungSx/bjontegaard.js.
- BD Rate estimation in VBA (e.g. Excel). GitHub: tbr/bjontegaard_etro.
- Subjective Comparison of ENcoders based on fItted Curves (SCENIC). GitHub: phanhart/SCENIC.
- BD-rate: one name — two metrics. AOM vs. the World. ViCue Soft blog.
References
[VCEG-M33] Gisle Bjontegaard, “Calculation of Average PSNR Differences between RD curves”, ITU-T SG16/Q6 VCEG 13th meeting, Austin, Texas, USA, April 2001, Doc. VCEG-M33.
[VCEG-AI11] Gisle Bjontegaard, “Improvements of the BD-PSNR model”, ITU-T SG16/Q6 VCEG 35th meeting, Berlin, Germany, 16–18 July, 2008, Doc. VCEG-AI11.
[VCEG-AL22] Kenneth Andersson, Rickard Sjöberg, and Andrey Norkin. 2009. Reliability metric for BD measurements. ITU-T SG16 Q.6 Document, VCEG-AL22.
[MHV-22] Nabajeet Barman, Maria G Martini, and Yuriy Reznik. 2022. Revisiting Bjontegaard delta bitrate (BD-BR) computation for codec compression efficiency comparison. In Proceedings of the 1st Mile-High Video Conference (MHV ‘22). Association for Computing Machinery, New York, NY, USA, 113–114. https://doi.org/10.1145/3510450.3517289.
[HSTP-VID-WPOM] ITU-T Technical Paper. 2020. HSTP-VID-WPOM — Working practices using objective metrics for evaluation of video coding efficiency experiments.
