Experiments with the MediaDevices JS API

Recently, I have been helping a friend, Armando Magalhães, on an open-source project called Perry.

Perry is a tool for the front-end developed in JavaScript which allows you to record logs, events and errors from your web app in your browser while trying to reproduce a bug. If you want to know more about Perry, go take a look to the project page on GitHub: https://github.com/perry-js .

In order to bring a complete tool, Perry was missing a way to capture the screen (or the active tab in the browser, at least) of the user for providing visual assets to the report (more details can be found in this issue: https://github.com/perry-js/perry/issues/15).

As everybody already knows, JavaScript can do anything !

Based on this statement, we deep dived into the APIs documented in the MDN to look into how could we access a media (screen, camera or microphone) from JavaScript. And we found it. The API is called MediaDevices and it is the topic of this article (lucky you).

The MediaDevices documentation can be found here: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices

During this article, we will focus on 1 method and two types: getDisplayMedia, MediaStream, MediaRecorder. We will build a script which stream a media (the screen) and record it in order to create a video you can download.

Let’s start !

We create at first the UI of our experiment. We need a button to trigger and stop the record. We also want to see something more than the cursor on our video so let’s add a text input for adding some details to the video.

You can use the following HTML script:

Now that we have a UI ready, let’s implement our script.

At first, we need to retrieve the media devices available on the machine. For doing so, we use getMediaDevices. The function takes one argument that defines the constraint(s) for the stream. The constraint can be about the video or the audio. The function returns a Promise fullfiled with a MediaStream object in case of success. The stream grants us the access to the tracks composing the video.

Here is the code performing the actions described above:

If you execute the following function in your console, the browser will prompt a modal asking you which device can be used for the stream.

If you decline or cancel, getDisplayMedia will throw an exception. More information can be found here: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia .

For Edge, getDisplayMedia is accessed via the navigator instead of the mediaDevices API (navigator.getDisplayMedia instead of navigator.mediaDevices.getDisplayMedia).

Now that we have our stream, we need to create a record of it in order to create the video. For achieving it, we create a new MediaRecorder object. Unfortunately, at the time I am writing the article, this object exists only in Chrome and Firefox. For more info about MediaRecorder: https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder

The constructor for the object takes a stream as argument. Once we have created the object, we need to tell it when we start recording and stop recording. While recording, what we do is save the tracks coming from the stream to the recorder.

Let’s re-use the previous code and add the steps I have just described:

There are three things to notice in this code sample. First, we use the event handler ondataavailable. Each time that a track (audio or video) is available, the recorder triggers an event. The data property of this event contains the track. The second thing is that we need to go through all tracks to stop the recording entirely (removing the toolbar appearing on the bottom of your screen in your browser). The third noticeable thing is the way we wait for the end of the recording. The Promise will be fulfilled in case the onstop event handler is called. It rejects in case an error is thrown by the recorder. The onstop handler is called after we executed the stop method.

The last part remaining is to make the video available for download. In order to achieve that, we transform the array containing the video into a Blob. Then we use a FileReader to encode the video in base64 and make it available for download via an anchor.

Here is the code implementing the last steps to implement:

We are now done. Our script can record the screen and allow to download the video created by the record.

I hope you enjoyed this short introduction to the MediaDevices API of JavaScript.

Keep in mind that this API is currently available in few browsers and it might not be yet ready for production.

Enjoy !

Edit: you can find here a working example: