Create a remote control for Black Mirror: Bandersnatch with PubNub
While watching Bandersnatch with a group of friends I encountered an unexpected issue. Often, the group was unable to come to a consensus before time ran out. To make watching Bandersnatch with a group easier, we’ll create a remote control designed specifically for Bandersnatch so that everyone can vote for their favorite option from their phone. This makes for a fun thematic twist when watching with others — nobody is completely in control of what path the group follows.
This project consists of a Chrome extension and a web app. The Chrome extension interacts with Netflix and counts votes while the web app collects them. We’ll write the Chrome extension in JavaScript and run it through Babel with Webpack to ensure it works with older versions of Chrome. Then, we’ll write a web app with Vue.js that handles collecting votes and sending them to to the extension. If you haven’t used any of these tools before, don’t worry. We’ll walk through the process of setting everything up and go through the code to understand how it all works.
If you want to try out the final product first or would like to watch Bandersnatch now (there are spoilers later on), you can try out a live version of the project here. The full source for this project is available on GitHub.
Spoiler Alert! There are some unique pathways which required the development of special cases. These pathways are mentioned throughout the project.
General Overview
PubNub allows us to send device-to-device messages instantaneously using a publish/subscribe model. A Chrome extension detects the available options while Bandersnatch is playing and publishes them to a channel where the web app is subscribed. The web app displays the options it receives from the extension and waits for a selection. When an option is selected, it publishes it to a channel where the extension is subscribed and the extension records the vote. When time runs out for the choice, the extension will select the most popular option go back to looking for options.
Setup
Before we can start the project, we need to get our development environment set up, add some configuration files, and sign up for free PubNub keys.
Setup your development environment
I strongly recommend using Visual Studio Code for this project. It has syntax highlighting for JavaScript, git support out of the box, and allows you to use the command line from within the editor. If you go with Visual Studio Code, the Vetur extension will add syntax highlighting for the Vue.js single file components we use later on.
You’ll also need Node.js so we can use NPM (Node Package Manager) to download some dependencies. The latest version is available for download here.
To test the project you’ll need a computer with Chrome and a Netflix account. Optionally, a mobile device will allow you to test the scanning feature of the web app.
To organize everything, start by creating a directory named bandersnatch. This will contain all of the code, assets, and build tools for the project.
Get your PubNub keys
- Sign up for a PubNub account or login to an existing account from the PubNub dashboard
- Click the big red button labeled
CREATE NEW APP + - Enter a name for your app (I went with “Bandersnatch PAC”) and click
CREATE - Select your new app and then select the
Demo Keyset - Create a file named
keys.jsonand paste in the snippet below. Replace thepublishKeyandsubscribeKeywith your keys from the PubNub dashboard
Specify a deploy path
Our extension and app will need to know where the website is hosted for generating and processing QR codes. Create deploy-details.json and paste in the JSON below. When we deploy the web app at the end we’ll update this to reflect the publicly accessible URL.
Building the Chrome extension
We’re going to start with our extension because it handles most of the logic and interacting with Netflix.
Install dependencies
Create a directory named ext in the project directory. This is where we will put everything for the extension. In this directory, create package.json with the following code and then npm install. This might take a few minutes.
Our package.json has 3 important parts.
The dependencies are packages that we use in our own code. We’re using pubnub for real time messaging and qrcode to generate QR codes.
The devDependencies are packages that we use to build our code. webpack does most of the heavy lifting of bundling our files together and allowing use to include packages from NPM in a browser application. babel handles compiling modern ES6 JavaScript (which has poor browser support, but is easier to read) into JavaScript that older versions of Chrome will understand. We also have prettier to format our code and keep it readable.
The scripts is a list of commands for building the extension and formatting the code. npm run build will build and package the extension and npm run lint will format the code with Prettier.
We also need to create webpack.config.js so Webpack knows how to build our source code into the extension.
The rules tell Webpack how to process files. We only have one rule, which tells Webpack to use Babel to process all of our .js files. Babel makes sure our fancy ES6 JavaScript is compatible with older browsers.
The entry specifies the entry points for the project. Webpack starts at each entry point and bundles it with all of the imported modules. Each entry point corresponds to one output file.
The plugins contains a bit of custom setup for working with extensions. The CopyWebPackPlugin copies files from the src directory into dist. We use this to copy the HTML for the page action, images, and the extension manifest. There is also ZipPlugin which creates a zip of the dist directory. This is helpful because the Chrome web store requires a zip of the extension to submit for publication.
Chrome extension elements:
The extension manifest contains metadata like the name, icons, permissions, and scripts required for the extension.
Content scripts run in the context of a tab. They can interact with and mutate the DOM, but can’t use the features of the chrome.tabs and chrome.pageAction APIs.
Background scripts orchestrate the extension’s behavior and can access all of the extension APIs, but do not run in the tab’s context so they can not access the DOM.
Page actions provide a popup when the extension icon is clicked. They allow the extension UI to be separate from the website it is interacting with.
Extension manifest
We’ll need to create a src directory to put all our code and assets.
The extension manifest tells Chrome what tabs to inject content scripts into, where our files are, and the basic metadata. Our manifest defines a background script, a content script, and a page action. We’ll work on developing these next. Create manifest.json and place it in src.
Icons
Chrome needs the icons provided in various sizes to support different screen resolutions. Create an img directory and save the images from the repository. Alternatively, you can create your own icon in Illustrator and export with the create subfolders option checked using scales of 512px, 256px, 128px, 64px, 32px, 24px, and 16px to create all the necessary versions.
Helpers
In order to make writing the extension easier I have provided some helper functions. Copy both files into ext/src/shared.
These files provide a few useful functions.
Background script
The background script does not run in the tab’s context, so it is best to use it for networking in order to avoid potentially impacting video playback. It’s the first part of the extension to run, so it also handles some house keeping like injecting content into the Netflix tab and creating a channel code.
For our background script we’ll need a file named controller.js.
At the top of our file we want all of our imports. This pulls in some helper functions as well as PubNub and our PubNub keys.
The first thing we want to do is create a channel code. This is a unique identifier that will be used so that users can connect to the extension from the web app. It will also be the channel name used for PubNub. On startup, we’ll check to see if there’s already a code in localStorage, if there isn’t we’ll generate a new code.
Then, we can use the helper functions to start a port for communicating with other tabs.
We also want our background script to handle requests to inject content scripts because our other scripts lack the necessary API access.
Now that we can talk to other tabs, we want to add a listener that will enable our page action when a tab requests it.
The most important task for the background script is recording and tallying votes. We can do this by putting the votes in a JSON object and then finding the most popular when time runs out. We have deal with our first special pathway here. In one scene there is a 5 digit phone number instead of a binary choice. This means we’ll get an array of 5 choices from the app instead of just 1. We need to handle this slightly differently by encoding the votes as a string during the vote counting process so we can use them as a key.
We’ve got all our vote counting figured out, but we still need to connect to PubNub and subscribe to our channel so we can actually get votes.
When we get a message that is a choice we want to count the vote immediately. In the special case where we get a multichoice our options will be an array instead, so they need to be stringified first.
Our content script will send two different types of messages to pass to the web app. A options message with the available pathways and an end message with the timestamp where voting will close. For both of these we want to immediately pass the message from the content script to PubNub.
Content Script
The content script runs in the tab’s context which gives it access to the DOM. This allows it to detect changes in the page and extract the options to send to the background script. It also allows the extension to interact with the page to select options.
First we want to create a start.js content script. This will check if the tab is actually Bandersnatch (not another title on Netflix) and then inject the main content script.
Our main content script goes in interaction.js. Just like with the background script, we’ll start by importing some helper functions
Then, we’ll start a port to talk to the background script.
We also want the page action to be enabled, so we’ll ask the background script to enable the tab.
The challenging part is scraping the page to extract the information we need for the web app. For poking around in the page Chrome’s developer tools are invaluable. You can open developer tools with ctrl/cmd+shift+i or by clicking the three dots in the top right and going to More tools>Developer tools. ctrl+shift+c allows you to inspect elements as you hover over them so you can see how Netflix renders the choices. ctrl+\ or F8 will pause script execution, which allows you to pause the countdown and look through the DOM with out a 10 second limit. Adding breakpoints to events like mouseup and mousedown in the Sources panel is also useful for debugging click interactions.
The first thing we have to figure out is how to determine the options that are on screen. It turns out that all the choices have the class BranchingInteractiveScene--choice-selection. This means a simple query will return all the choices. Most of the time the choices are just div elements with text inside. In one scene, the options are images instead. In this case, they instead contain another div with the class choiceImage which has a background image instead of text. Knowing this, we can support choices that are text and choices that are an image and adjust our behavior accordingly.
In addition to extracting the options, we want to be able to select the winner after voting. We can do this by faking a click on one of the elements and adding a listener to our port. Because of how Netflix built their UI, a simple element.click() will not select an option. Instead, a mousedown followed by mouseup will simulate a click.
At this point, we aren’t running getChoices yet because we don’t know when there will be choices. Instead of running our function over and over (and slowing down the page) we can use a MutationObserver to watch for changes to the DOM. Our mutation observer will watch for any changes to the element with the class nfp (Netflix player) and run our function onPlayerChanged. We can wait 5 seconds to register this mutation observer at the start in order to not impact the initial player load.
Our onPlayerChanged function will determine if we should check for the options. When ever options become available an element with the class main-hitzone-element-container is added, so we can wait for this mutation to getChoices. In onPlayerChanged, we want to get the options and then add a temporary mutation observer to look for the data-time-remaining attribute which Netflix uses to store the duration of the decision. This comes as much as a second or two later (because the options are added to the DOM before they are visible) so by sending it as a separate message we can ensure that the options are not delayed.
Page Action
The page action presents the user with the domain of the remote app, the channel code, and a join link in the form of a QR code when the extension icon is clicked. This requires an HTML file and a small script.
Create index.html. All we need is some white text on a black background and a canvas element for our QR code.
In page-action.js we’ll get the channel code and use the qrcode module to paint a QR code to the canvas in the page action.
… and we’re done with the extension!
To test it out you’ll need to build using the webpack command. This should be run from /ext, not /ext/src. The built extension will be saved to /ext/dist. Open Chrome and navigate to chrome://extensions. Make sure Developer mode is turned on in the top right corner. Then, click Load unpacked... and find your /ext/dist directory. If you go to Netflix and start watching Bandersnatch the icon should turn red, and clicking it should open the page action.
Head back to the PubNub dashboard, select your project and then the Demo Keyset. Select the DEBUG CONSOLE on the left side and click the settings icon. Go to Channels and paste the channel code from the page action for the Publish Channel and Subscribe Channel, then UPDATE CLIENT. As you watch Bandersnatch, you should start to see options come in to the console shortly before they appear on screen, followed by a message with the end timestamp.
You can vote from the console by sending the choice command.
Building the web app
The second part of the system, the web app, handles displaying options and collecting votes. We’ll write the web app in Vue and use Vue Router to handle routing between the landing page and remote.
Setup
To make creating a web app with Vue easier, we can use the Vue Command Line Interface (Vue CLI). npm i -g @vue/cli will install it globally on your machine. With the CLI installed, vue create web from the project directory to make a new project.
For the preset choose Manually select features and enable Babel, Router,CSS Pre-processors, and Linter / Formatter. When prompted, select Use history mode and choose Sass/SCSS (with node-sass) for the CSS pre-processor and ESLint + Prettier for the linter / formatter. Don’t change the lint on save setting and place config In dedicated files. You don’t need to save these settings as a preset.
It may take a few minutes to install all the dependencies and generate files for the project. Once the project has been created cd web & npm i pubnub & npm i jsqr to install the rest of the dependencies for the web app.
Before we work on our UI, we need to adjust the generated project a little bit for our purposes. First, replace the contents of router.js. This will change the routing paths for our app so that it defaults to using /views/Join.vue for the default path and /views/Remote for the /remote path. It also will extract the channel as a parameter from the URL. For example, navigating to /remote/1234-5678-9012 will pass us 1234-5678-9012 in this.router.
We also need to adjust the App component which contains the views from the router. We don’t have any page navigation that stays between views so our App component just needs to hold the router view.
To make styling our app easier, we need a theme file that includes the colors and fonts for our app. The theme file should go in the assets directory.
To be able to use these fonts we need to update our index.html to load them from Google Fonts.
Place logo.png in the views directory and favicon.ico in the public directory and we’ll be ready to get started with writing our views.
Join page
Join.vue contains the logic and UI for connecting to the extension. For Vue components there are three sections, the template (HTML), script (JS), and style (SASS).
The template for our Join component has 3 div elements which contain the sections of our UI. The topbar contains the logo for the app and is always visible. The scanner holds the video output from the camera, but is only enabled when we set useCamera to true. When useCamera is false, the manual-entry component is rendered instead.
Syntax note: In Vue, the v-if directive only renders an element if the expression evaluates to true. An element with v-if=”false” would never be rendered and an element with v-if="true" would always be rendered.
Understanding Vue: Expressions in our template are always evaluated in our Vue. This means that function calls will run a function from the methods of our view and variables will come from the data and computed fields of the Vue instance.
For the scanner, all we need is a canvas to draw the camera output to and a message for the users.
To accept manual entry, the markup is a bit more complex. All of the input is going to go into a form. Our form has event listeners so that we can respond to backspace/delete, and the left and right arrows. The form contains input elements for accepting digits and a button to submit. The input is the most interesting element here. The inputs are rendered from a list with v-for to keep our markup simple. Each is bound to a variable in our Vue assistance and has properties so that the numeric keyboard will show up on phones.
Syntax note: In Vue, the @ directive is shorthand for v-on. This adds an event listener using a function from the methods on the Vue instance. For example, @keydown.delete="backDigit" runs the function backDigit whenever a keypress for delete or backspace occurs on the element.
Syntax Note: In Vue, the : directive is shorthand for v-bind. This binds a property or piece of content to a piece of data on the Vue instance. This way, the DOM will update whenever a change is made to the data from our scripts.
Syntax Note: In Vue, the v-for directive allows list rendering. The element with the v-for directive is rendered once for each item in the iterator and gains that item in its context. For example, v-for="(digit, index) in digits" iterates through the array digits and renders the element each time. For each element that is rendered, the digit from the array and the index in the array are available to be bound to.
Syntax Note: In Vue, the v-model directive performs two way data binding. Like with v-bind, the value will update when the model is changed. Additionally, if the value of the input is changed the model will change as well.
Most of the work as far as handling the form and scanning happens in the script.
As always, we start by importing the dependencies that we need (this should go in a script tag in your component).
All of the logic goes inside the Vue instance. There’s 4 pieces worth looking at.
The data is the heart of our Vue. The properties we put there become reactive, causing the markup bound to their values to re-render when anything changes. To start, we have an array for 12 empty digits which are used to render the input fields with v-for and are bound to the values of the input with v-model. This way, when the input value changes or we change the value of the data, they are connected and change together. The other fields are current, which tracks the current position in the 12 digits and useCamera which dictates whether the camera or manual input will be used. The data is accessed inside our Vue through this.$data.
The methods fields contains functions for our app to use. This is where we will put our event listeners for key presses and DOM events (like focus and click). The first method we need is formCode which navigates to the next page when it is submitted. To accomplish this, we take the values of all the input values and trim them to 1 digit, then add them to the path and insert a — where needed.
The rest of the methods deal with treating our 12 separate inputs as 1. By listening to key presses and focus events from the form with v-on we can move the cursor through the inputs as if we were dealing with a single field.
We also need a watcher. Watchers are a function that run whenever the piece of data on our Vue is updated. Our watcher will trim and validate any input and advance to the next input to make the transition between inputs seamless.
The most complex portion is the mounted function which runs when the template has been added to the DOM. At this point, we can start dealing with the camera and render its output. Because not all devices have cameras, we need to check for one first. If there is no camera the video output is disabled and manual entry is used instead by setting useCamera to false. When the camera is available, each frame is fed into jsQR to be processed for a QR code. When a code is found, it navigates to the next page.
Finally, we can add some styling using the color theme we created earlier. We don’t need to do anything particularly fancy except add some responsive breakpoints so the inputs will scale with the screen.
Unfortunately, we can’t test our app out quite yet because we’re missing the Remote component.
Channel (remote) page
The Remote component is pretty simple, and once we finish it up we’ll be ready to try everything out.
This time our template isn’t quite as complex. At the top there is a countdown bar which mirrors the countdown on screen. Below, it renders items from the options and depending on whether the option.type is text or an image it displays the option.data as text or uses it as the URL for an image. In the case of scenes where multiple selections or required, it displays the selected options as the user enters them.
The script for our component handles connecting to PubNub and sending and receiving messages. When a message is received with options it, updates the Vue data. When a message with a timeout is received, the countdown is started. Using a click listener, the selected option is sent back the the extension and input is disabled to prevent one user casting multiple votes. There’s nothing too interesting going on here, so it’s not worth digging into the code.
Finally, we can add some styles to mimic the Bandersnatch UI.
With all our components complete, the web app is ready to test with npm run serve. Navigating to http://localhost:8080/ should take you to the join page. Unfortunately, we can’t access the camera without HTTPS, so we can only use manual entry. To try out code scanning, we’ll have to deploy the app behind HTTPS. Ee’ll use Netlify for the sake of simplicity, so we don’t have to deal with servers or hosting.
Deploy with Netlify
Before we deploy, we can add a few commands to our package.json files that will handle tasks like installing dependencies and building our source to make development easier in the future.
In the project directory, run npm init and follow the prompts to create an empty package.json.
Then, add these fields to the scripts section. They allow us to lint and build the extension and the web app from the project directory. They also add a netlify command which we will need to deploy our site.
In the scripts section of ext/package.json add these scripts to lint and build the extension. They aren’t required to deploy the web app, but they’ll make rebuilding the extension more convenient in the future.
Finally, to complete our netlify command, we need to create the script that is run by make:keys to generate keys.json. netlify-keys.js takes the environment variables during a build and writes them to keys.json. Using this script, we can insert our keys into our Netlify build even though keys.json will be ignored by git.
To deploy to Netlify we need to setup version control with git. In the project directory add a .gitingnore file. This tells git to not commit the built versions, npm packages, and keys.json because we don’t need this files tracked in git. and run git init. Then, login to your preferred git provider (GitHub, GitLab, or BitBucket) and create a new repository. They should give you URL for your repository that ends in .git. Run git remote add origin https://provider.com/repository.git, replacing https://provider.com/repository.git with the URL from your git provider. Then, git add --all & git commit -am "initial commit" & git push to add all the files to your repository.
Now, login or sign up for a Netlify account. Click New site from Git, select your git provider, and then link your account. You should see a list of repositories. Select the repository you created for this project. For the Build command enter npm run netlify and for Publish directory enter web/dist. Open the Advanced options and click New variable twice. For one, make the key publishKey and for the other make it subscribeKey. Fill in the values with your keys from keys.json. Because we don’t commit our keys.json, it needs to be recreated on the build server from these environment variables. This is handled by the netlify-keys.js script we created earlier. Click Deploy site to put it online. You should get a domain where you project is hosted and be able to visit the web app in your browser.
Finally, replace localhost:8080 with the Netlify URL in deploy-details.json and commit the changes with git commit -m “update public path" & git push origin master. This allows the QR code to be scanned by any QR code scanner instead of only working with the scanner within the web app.
Anybody to can access the site now, and can join in if you share the code from your extension. To give others access to the extension you’ll need to share a the zip from ext/dist with them. You can also submit it to the Chrome webstore, but the submission process is out of the scope for this write up.
With your site deployed, you can open the web app from your browser. If you enter or scan the code from your extension the app will connect and your can start controlling Bandersnatch from your phone.
Wrap Up
In this project, we developed a Chrome extension using JavaScript and Webpack that can detect the options and time while viewing Bandersnatch and calculate the winner of a vote. We also built a modern web app with Vue.js that allows users to connect to the extension, displays options and collects votes. Using PubNub, we were able to connect the two together with no visible latency. At the end, we made our web app public using Netlify for serverless deployment.
