Measure performance of your own streaming player with Mux
The first article after a long pause!
I was having this assignment from my MSc module on Performance Modelling and Analysis to analyze the performance of some kind of a system. So I was checking on different systems and finally chose video streaming for the performance analysis.
When I was checking on different types of streaming players that are opensource, I found two streaming players that are very easy to integrate. Video.js and HLS(HTTP Live Streaming). There are more streaming video players that you can explore! Plyr, AfterGlow, MediaElement.js, jPlayer, Cloudinary, JW Player, Kaltura video player are some of them.
Video player
First we’ll have a look at the video player implementation. For this I use a simple React.js application. Any application that can be hosted and accessible from outside will be okay, but here I’m using a simple reactjs application. The important part of this application is the video player. You can embed this video player into your existing application easily. In the below gist I have added few comments. The comments 1–4 are for the video player integration. In step 3, you can get any of the freely available streaming videos for testing purpose or you can have your own streaming url there. Keep in mind that the type is ‘application/x-mpegURL’ for m3u8 videos. Comments 5,6 are applicable for the performance measuring part. Read the rest to find it :D
After creating this application, if you go to your browser and load the page, it will look like below.
Performance Metrics
If you are going to measure the performance, then first thing you need is set of metrics that you can use to measure your performance. As my system id video streaming, below are some of the most useful metrics that I could find.
- Mean bitrate: Bit rate is the number of bits of data sent per second. When the bit rate is high, it implies the video is high quality.
- Bitrate fluctuation: If the above-mentioned bit rate is fluctuating time to time, it will be affecting the quality of the video.
- Rebuffering percentage: Rebuffering percentage is the amount of time video spent rebuffering, as a percentage of the total time video was requested
- Rebuffering frequency: Rebuffering frequency is how often the video rebuffered per unit time of video that the viewer attempted to watch.
- Rebuffering Duration: Rebuffering Duration metric is how long in seconds the viewer had to wait for their video to reload.
- Rebuffering count: Rebuffering Count is the number of rebuffering instances during videos views.
- Delay
- Packet loss rate
- Rendering quality
- Buffer Fill: Buffer fill is the time spent to fill the buffer prior to the start of video.
- Lag Length: Lag length is the sum of all the times that was waiting to fill the buffer.
- Play Length: Play length is the actual seconds, minutes, and hours of content that have been consumed by your users.
- Lag Ratio: Lag ratio is the waiting time over watching time.
This is a long list of metrics! Now, how can we measure all of these? :O No worries :) We have some tools that are freely available ;)
Performance measuring tools
The next thing I was looking was some tools to measure my performance. There were so many tools popping up if you google ‘Performance measuring tools for video streaming’. Among all them have chosen Mux.
Mux
If you are going to use mux, first you need to create your Mux account. After you create your account, login to your account and go to Environments and then click on ‘Add Environment’ button. After creating my environment it looked like below.
Here you have got two options, Host your streaming video or measure video performance. I go with the second option, track video performance. When you select the second option, there will be a popup similar to below.
From these options, I’m going to choose video.js plugin. It will redirect you to a documentation page, you can follow the steps given in that page to get the performance measurement recorded at Mux. It is explained very well and it’s easy to understand. In the above gist, I have added the mux plugin into my video.js player, by checking comments 5,6 you can understand it. Point: You need to change the env key accordingly for your env key.
If you open your network tab in the browser that your player run, it will show a load of calls going with the initiator videojs-mux. So you can find that your data is not going to mux. Now let’s go to mux and check our performance data.
Above is the Mux dashboard with performance data. You can run your application multiple times using your player and then check the performance via Mux. It gives following performance measures.
- viewer_experience_score
- aggregate_startup_time
- exits_before_video_start
- page_load_time
- playback_failure_percentage
- playback_success_score
- player_startup_time
- rebuffer_count
- rebuffer_duration
- rebuffer_frequency
- rebuffer_percentage
- seek_latency
- startup_time_score
- video_startup_time
You can also download the measurements as a csv. This report gives you many details about the performance of the player. In Mux you can simply create your own streaming video also. In next article I will be trying to host a streaming video and load test it.
Keep calm and measure performance!
