Your phone as Bluetooth controller for Web Applications
How to turn any smartphone into a Bluetooth enabled motion controller for HTML5 / JavaScript Web Applications
Why use a Bluetooth controller for Web Applications?
It’s a valid question and there are certainly easier and more “traditional” ways to control a Web Application. But there are scenarios where including the User’s phone as a (motion-) controller creates an exciting new way for them to interact, such as:
- If you are creating an exhibition or retail space let users download your controller App and allow them to interact with your applications on-site.
- With Bluetooth users don’t need to connect to your Wi-Fi network first to exchange data with on-site applications.
- Due to its low latency Bluetooth is suitable for fast gaming-style applications.
Of course you are not limited to using the phone as a controller only. Bluetooth connections work bi-directional so you could even turn your phone into a “second screen” for displaying additional content.
This all sounded like very promising ideas so I went ahead and implemented several working examples of this concept.
In this article I will share with you the concepts and technical details of my implementation. Also I made the source code for all components available on my GitHub repository:
If you’re not interested in the details feel free to just download and play with the examples. However if you want to follow the entire development journey then let’s get started with some Bluetooth basics.
Bluetooth Low Energy (BLE) basics in 30 seconds
To learn about Bluetooth you can read an in-depth introduction of the Bluetooth LE protocol or even the full specification document — but I thought I’ll save you the pain and summarize the most important bits needed for this project:
The Bluetooth LE protocol describes data communication between a Central (sometimes also called “Master” or “Server”) and a Peripheral (“Slave” or “Client”).
Peripherals provide one or more Services identified by unique Service Identifiers or Service UUIDs. Each Service then exposes one or more Characteristics which are like ports for reading and writing data. Same as Services, Characteristics are also identified by UUIDs.
Peripherals can advertise the services they provide and typically a Central will try to find and connect a matching Peripheral by scanning for nearby devices which advertise a Service known to the Central. Finally, after connecting to the Peripheral, the Central will read and write data from and to the Characteristics exposed by the Peripheral’s services.
With this (extremely basic) knowledge we can move on to design the components for our application.
Designing a Bluetooth enabled Web Application
As of today it is not possible to pair a Bluetooth Central or Peripheral with an HTML5 / JavaScript based Web Application. There is an experimental Bluetooth API included with newer versions of Chrome but it is far from production ready. Means we need another solution.
If we can’t pair a Bluetooth device with a Web Application we will need a component which translates the Bluetooth input to something the Web App can work with. Did anybody just say “WebSockets”?
Translating Bluetooth to WebSocket
My solution is to create a “Bluetooth to WebSocket” translator and put it between the Bluetooth Controller and the Web Application. The translator implements a Bluetooth Peripheral on the controller facing side and a WebSocket Server on the application facincside.
In this concept the Controller App (Central) will connect to the receiver (Peripheral) and transmit the controller data (buttons, tilt angles) via Bluetooth. The receiver decodes the data, encodes it to JSON and forwards it to the Web Application via a WebSocket.
The illustration below shows how we plan to put all the components together:
With this structure in mind we can go ahead and start implementing the individual components. But before we start writing the actual code, let’s take a minute to think about how we actually transmit the controller data between the Controller App and the Bluetooth receiver.
Defining the controller layout
For this project I decided to go with a traditional 8-bit style controller layout, featuring a D-pad and 4 buttons. Just imagine your good old NES controller with START, SELECT, A and B buttons.
In addition to the “physical” buttons the controller will also provide the current X (Pitch), Y (Roll) and Z (Yaw) tilt angles from the phone gyroscope. These will be the raw input values for enabling the motion controls.
Encoding the controller data
According to our controller definition we have to encode controller data for the following configuration:
- Eight digital input values (D-Pad + face buttons)
- Three analog input values (pitch, roll, yaw)
The 8 digital buttons conveniently fit into a single byte. We define one fixed bit for each button and set it to 1 if the button is pressed:
In the example above we’re holding the D-pad to the left and press the A button at the same time, resulting in an input byte value of 68. We can later conveniently check for each input button using the bit-wise AND operator:
// assuming "value" is our input value
if((value & 1) > 0) { /* UP */ }
if((value & 2) > 0) { /* DOWN */ }
if((value & 4) > 0) { /* LEFT */ }
if((value & 8) > 0) { /* RIGHT */ }
if((value & 16) > 0) { /* START */ }
if((value & 32) > 0) { /* SELECT */ }
if((value & 64) > 0) { /* BTN A */ }
if((value & 128) > 0) { /* BTN B */ }Encoding the analog tilt angles works a bit different. The WebView’s raw readings for pitch and roll range from -180 to 180 while (oddly) yaw uses 0 to 360 degrees. I decided to first “normalize” all values to 0–360 degrees, then compress each of them into single byte with values from 0 to 255 (8 bit unsigned CHAR). This way the whole controller data could fit into a single 32 bit integer (UINT32) which is convenient to work with.
Note: Obviously I’m sacrificing a bit of angular precision here but I found it was still more than enough for all my use cases. If you need a higher precision you can of course encode the raw floating point values in 16 bits or more.
Creating the Bluetooth Controller App
Now that we defined the controller layout and decided how to encode the controller data we can start working on the controller App. For this tutorial I created a hybrid App for IOS and Android based on Ionic 4 and Cordova. The full source code and instructions how to run this app on your phone are available on my GitHub repository.
Why Ionic 4 / Cordova? Simply because it was the fastest way for me to create an App which most readers can use. You are of course free to implement the controller in any language you are comfortable with, following the general ideas I’m demonstrating here.
Creating the D-pad and face buttons
The D-pad and face buttons on an actual controller are switches with on/off state. Let’s create a similar interface with 8 buttons arranged in a typical controller layout:
In addition to the buttons we need two methods which handle pressing / releasing the buttons and setting the 8-bit value for the button status accordingly:
export class HomePage { public buttonState = 0; public onPressButton(idx: number) {
this.buttonState |= (1 << idx);
console.log(this.buttonState);
} public onReleaseButton(idx: number) {
this.buttonState &= ~(1 << idx);
console.log(this.buttonState);
}
}
The above methods set and unset individual bits of the buttonState value. Wire up each button to the onPressButton / onReleaseButton methods with its appropriate index:
<ion-button size="large" color="secondary"
(press)="onPressButton(0)"
(pressup)="onReleaseButton(0)"
(panend)="onReleaseButton(0)">U</ion-button>Note: The panned event is not strictly necessary but I added it for usability reasons as releasing your finger from a “button” is sometimes interpreted as pan event.
Adding motion controls
Getting the tilt angles (pitch, roll, yaw) is equally easy. As Ionic Apps are at its core are Angular Apps we can directly obtain the values from the enclosing WebView using this code:
export class HomePage { public pitch = 0;
public roll = 0;
public yaw = 0; @HostListener('window:deviceorientation', ['$event'])
public onDeviceOrientation({ alpha, gamma, beta, absolute }: DeviceOrientationEvent) {
this.pitch = Math.round(beta);
this.roll = Math.round(gamma);
this.yaw = Math.round(alpha);
}
}
Gyroscope readings are often returned as raw velocities which need to be converted to angles first. Using
deviceOrientationevents of the WebView conveniently returns the pre-converted angles. When implementing your own App, make sure you get your reading as angles.
Next we implement the Bluetooth connection to the receiver (peripheral). This requires the following steps:
- Scan for a Bluetooth device advertising our service UUID
- Connect to the discovered device
- Write the controller data to the device
Connecting the Bluetooth receiver (peripheral)
As I’ve explained in the quick BLE introduction a BLE peripheral can advertise the UUID for services it provides. This allows us to scan for all BLE devices nearby and automatically filter the results to only include devices which provide the requested service UUID. In my example I’m using F0BA as the service UUID.
import { BLE } from '@ionic-native/ble/ngx';
import bleConfig from '../config/ble.config';export class ControllerService { private peripheral = null;
public isConnected = true; constructor(private ble: BLE) { } public connect() {
this.ble.startScan([bleConfig.SERVICE_UUID]).subscribe(
(device) => { this.onScanDiscovery(device); },
(error) => { this.onScanError(error); }
);
}
}
The above code shows how the scanning process works. Upon calling the connect method the App will start scanning for devices with the service UUID defined in bleConfig.SERVICE_UUID. When a device is discovered it will invoke the onScanDiscovery callback passing discovered device or report errors using the onScanError callback.
For simplicity our App will only expect a single device to be found and will automatically try to connect once a matching device was discovered. Therefore the onScanDiscovery callback can be implemented like this:
private onScanDiscovery(device) {
// connect device
this.ble.connect(device.id).subscribe(
(peripheral) => { this.onDeviceConnected(peripheral); },
(peripheral) => { this.onDeviceDisconnected(peripheral); }
);
}Connecting to the device returns an Observer. The Observer provides two callbacks, the first one being invoked upon successful connection to the device, the second one invoked on disconnect. We implement these callbacks using the onDeviceConnected and onDeviceDisconnected methods:
private onDeviceConnected(peripheral) {
this.isConnected = true;
this.peripheral = peripheral;
}private onDeviceDisconnected(peripheral) {
this.isConnected = false;
this.peripheral = null;
}
All these methods do is set / unset the connected peripheral as a member variable we can refer to later on when sending data to the device. And that’s exactly what we will do next!
Preparing and writing the controller data
We want to fit the controller data into 32 bits (4 bytes). The button values are already encoded into a single byte but the each angle value still needs to be normalized and compressed into a single byte. The result is then combined into a 4 byte array buffer:
public sendControllerData(buttons: number, x: number, y: number, z: number) { const adj = 255 / 360; // x and y (pitch, roll) range from -180 to 180
// -> normalize to 0 - 360
// z (yaw) range is already 0 to 360 - no adjustment necessary
const data = new Uint8Array(4);
data[0] = buttons;
data[1] = Math.round((x + 180) * adj);
data[2] = Math.round((y + 180) * adj);
data[3] = Math.round(z * adj); this.sendData(data.buffer as ArrayBuffer);
}
Finally we write the controller data to the device — or in more correct BLE terms: “We write the data to the input characteristic of the device’s controller service”.
The method writeWithoutResponse is used to minimize lag as we’re not expecting any response for each controller data packet. Think of it as a kind of UDP connection.
private sendData(data: ArrayBuffer) {
this.ble.writeWithoutResponse(
this.peripheral.id,
bleConfig.SERVICE_UUID,
bleConfig.CHAR_UUID,
data);
}With the controller up and running we can now move on to building the receiver (peripheral).
Implementing the Bluetooth receiver (peripheral)
Following our initial concept the Bluetooth receiver will be implemented as a Peripheral and output the incoming controller data as JSON data via a WebSocket.
Creating a WebSocket server is more or less a one-liner and I won’t go into the details in this article. If you need an introduction to creating a WebSocket server I recommend this great tutorial by Martin Sikora. The more interesting part is creating the Bluetooth Peripheral which our Controller App will connect to.
In my “30 seconds Bluetooth tutorial” I described a Peripheral as containing one or more Services which then contain one or more Characteristics for reading and writing the actual data. Luckily we don’t have to implement all the low-level stuff by ourselves but can instead use the Bleno library for NodeJS which makes creating Bluetooth Peripherals super easy.
Make sure you read and follow the Bleno installation instructions for your OS pretty much by the letter! Depending on your system and hardware, getting this library to run can be a bit tricky otherwise. Also if you’re using Windows just get a cheap USB Bluetooth dongle and don’t use the built-in Bluetooth controller.
Creating a Bluetooth peripheral with Bleno
I found the easiest way to create a peripheral is by implementing its characteristics for data in/output first. So let’s create a characteristic to receive the controller data:
var util = require('util');
var bleno = require('bleno');var InputCharacteristic = function(controllerClient) { // reference to the Websocket server (aka "Controller client")
this.controllerClient = controllerClient // configure characteristic
var charUUID = '1337',
descUUID = '2901',
desc = 'Controller data port'; bleno.Characteristic.call(this, {
uuid: charUUID,
properties: ['writeWithoutResponse'],
descriptors: [
new bleno.Descriptor({
uuid: descUUID,
value: desc
})
]
});
};util.inherits(InputCharacteristic, bleno.Characteristic);
This code will define a new characteristic with UUID 1337 and a description “Controller data port”. The description is optional but it is good practice to include it.
Except for the UUID and description we also need to define how connected devices may interact with this characteristic. In this example we add the property writeWithoutResponse which defines the characteristic as writeable but does not send a response after the write operation has finished.
If you wonder about the controllerClient variable here, it’s a reference to the object which manages both the Bluetooth peripheral and WebSocket server (see source). We will use this reference to pass back the translated data from the characteristic in the next step:
InputCharacteristic.prototype.onWriteRequest = function(data, offset, withoutResponse, cb) { if(data.length != 4) {
cb(this.RESULT_INVALID_ATTRIBUTE_LENGTH);
return;
} // read the digital and analog values
var byteBtn = data.readUInt8(0);
var byteX = data.readUInt8(1);
var byteY = data.readUInt8(2);
var byteZ = data.readUInt8(3); var adj = 360 / 255; // convert the input data
var controllerData = {
dpad: {
up: ((byteBtn & (1 << 0)) > 0),
down: ((byteBtn & (1 << 1)) > 0),
left: ((byteBtn & (1 << 2)) > 0),
right: ((byteBtn & (1 << 3)) > 0)
},
buttons: {
start: ((byteBtn & (1 << 4)) > 0),
select: ((byteBtn & (1 << 5)) > 0),
btn_a: ((byteBtn & (1 << 6)) > 0),
btn_b: ((byteBtn & (1 << 7)) > 0)
},
axis: {
x: Math.round(byteX * adj - 180),
y: Math.round(byteY * adj - 180),
z: Math.round(byteZ * adj)
}
}; // pass the data back to controller client
this.controllerClient.onData(controllerData);
}module.exports = InputCharacteristic;
Here we implement the onWriteRequest method of our characteristic. Means every time data is written to the characteristic this method will be called to process the data.
Inside the method we first check if the data is actually 4 bytes long. Remember we use one byte for the button states plus 3x1 byte for each axis. So if the data packet length is anything else than 4 bytes the data is most likely invalid and should not be processed.
If the data looks okay we can continue to extract the single bytes from it using the readUint8() method at the index where we expect the data.
Next we process the button byte with a bit of “bitwise-arithmetic-foo” to identify individual button presses. Also we expand the previously compressed angle values and revert the normalization.
Finally we create a nice and readable JSON data structure which we will pass back to the controllerClient instance which can forward the data via the WebSocket server.
Implementing the service
In our example the service is pretty much just a dumb container holding the characteristic:
var util = require('util');
var bleno = require('bleno');var InputCharacteristic = require('./InputCharacteristic');function ControllerService(controllerClient) {
bleno.PrimaryService.call(this, {
uuid: controllerClient.serviceUUID,
characteristics: [
new InputCharacteristic(controllerClient)
]
});
}util.inherits(ControllerService, bleno.PrimaryService);module.exports = ControllerService;
All this code does is defining the service and adding a single characteristic to it — the one we implemented above. That’s all, move on, nothing to see here.
Wrapping it up — Creating the peripheral
The peripheral doesn’t do much either. It‘s a container for the services and also handles certain events like powering the peripheral on or off. The most important task here is handling the “Advertising” which will announce the available services to other devices:
var util = require('util');
var bleno = require('bleno');var ControllerService = require('./ControllerService');// create services
var cs = new ControllerService(this);// install services
bleno.setServices([cs]);// handle peripheral power state
bleno.on('stateChange', (state) => {
if(state === 'poweredOn') {
bleno.startAdvertising(this.peripheralName, [this.serviceUUID]);
} else {
bleno.stopAdvertising();
}
});
As you can see from the code bleno is a singleton representing the peripheral. We add the service and tell the peripheral to start advertising the service once it is powered on.
I suggest at this point you check out the source code of my receiver and try it out by yourself. Once you have installed all dependencies you can fire it up with node bin/receiver and use the integrated console debugger to verify it’s working correctly with your Controller App:
If you see the output above — Congratulations, your receiver is working!
The final step — Building a simple Web Application
We’re almost done now. All that’s left to do is build a simple Web Application which connects to WebSocket server and reads the translated Bluetooth Controller data.
You can directly download a couple of examples from my GitHub repository.
Since we’re dealing with a simple WebSocket server here we can use the following code to connect:
let socket = new WebSocket('ws://localhost:1337');Now you can read the incoming controller data by implementing the onmessage event of your WebSocket client:
socket.onmessage = function(event) { let data = JSON.parse(event.data); // data structure is expected to look like this:
// {
// dpad: {
// up: true,
// down: false,
// left: false,
// right: false
// },
// buttons: {
// start: false,
// select: true,
// btn_a: false,
// btn_b: true
// },
// axis: {
// x: 110,
// y: -23,
// z: 258
// }
// } // do with the data whatever you like \o/
}
As you can see from the code above you will receive your controller data as structured JSON data. D-Pad and face button values are returned as boolean values while the axis angles are returned as their original integer values.
If you made it this far — Congratulations. You are now ready to create your own Bluetooth enabled Web Application 👋 👋 👋. Cheers to that🍺!
This is the end of my tutorial. Feel free to use my source as a base for your own applications. If you have any comments, questions or suggestions please start a conversation in the comments.
❤️ Thank you very much for reading! And I’d love to see all the cool stuff you come up with! ️❤️
