Communicating between a Particle Photon and HoloLens
Over the past week I’ve been researching how to connect a microcontroller to the HoloLens. After a lot of trial and error and experimentation I’ve finally managed to receive sensor data and use that data to change something in a holographic environment. This was a huge milestone and I’l be outlining how I did it, and the hurdles I ran into along the way, below.
Deciding on a microcontroller
In an earlier post I outlined what I thought would be the best ways to transfer data between an Arduino and the HoloLens. I ended that post saying that I thought Bluetooth LE would likely be the best way to get data from a microcontroller and the HoloLens, as both have BLE capabilities and it would avoid creating a REST API to send, store and retrieve the data from. Since writing that post I’ve come across the Particle platform which has a its own API and its Photon microcontroller comes with inbuilt WiFi functionality. I ended up ordering a Photon and began experimenting with sending some basic sensor data to the Paricle cloud.
Begin by setting up a basic circuit using a Particle Photon, a micro-breadboard, a photoresistor and a 220 ohm resistor. I’ve connected the photoresistor’s pins to the A0 (input) and A5 (output) pins on the Photon and I’ve got the resistor running from the A0 pin to Ground.
Write the following script in the Particle IDE:
This script receives data from the photoresistor via the Photon board every 1 second and publishes that data to the Particle cloud. It also exposes the value received as a variable called lightValue which we’ll be using in our GET request later.
We’ll need to get our own custom URI string to access data from the variable we exposed in the Particle code. URI stands for Unique Resource identifier and is how our GET request knows which data we want to retrieve. To do this we need to find the device ID for our Photon and the our unique access key and insert them into following string:
For me, that looks like this:
To find your device ID go to the Particle IDE and log in. Click on the devices icon to reveal a list of your connected devices. Click on a device to see its ID:
There are a few different ways to get your access key but the easiest way that I’ve found is to go to the Particle Console and go to Logs and copy it from the link they provide you:
Setting up the Unity project
Create a new 3D project in Unity and import the HoloToolkit Unity package. This has a few tools that make it quick and easy to get Unity projects HoloLens-ready. After importing the Holotoolkit package there should be a new item in the menu bar called “Holotoolkit”. From that menu you should see a “Configure” sub-menu which automates the Scene, Project and Capability settings required for holographic projects. Under the Capability setting make sure you have InternetClient selected as we’ll need this to communicate with the Particle API.
Next up, download and import the JSON Object Unity plugin as we’ll need this to parse the data we receive from the Particle API.
Note: Depending on which version of Unity you’re using you may need to go in and change a line of code in the JSONTemplate.cs file that comes with the plugin. I was using Unity 5.5.2 and ran into build errors before changing this. Try building your project and if you’re thrown a cast error change the following line of code (line 19):
FieldInfo fieldinfo = obj.GetType().GetFields();
IEnumerable<FieldInfo> fieldinfo = obj.GetType().GetFields();
Add a cube to the scene by navigating to the Create drop-down menu above the Hierarchy panel followed by 3D Object -> Cube. Change the cube’s scale to 0.25 in each axis, its Z position to 2 and rotate it by 45 degrees in each axis. Your scene should look like this:
For simplicity’s sake we’ll just be remapping values we receive from the photoresistor and using that to affect the scale of the cube. With the cube selected, navigate to the Add Component button and click on New Script. Name this new script whatever you like and open it up in Visual Studio 2015. Copy the script below:
Understanding the data we get back from our GET request
If you open up a browser and paste in the URI you’ve used for your web request you should see something like this:
This is the JSON object we receive back from our GET request. To get the photoresistor value we want we need to access the “result” key’s value. This is done in line 34 of the code snippet above:
lightData = lightData["result"];
This gives us the value that corresponds to the “result” key, in this case that’s 33.
Press Play and the cube should re-scale itself every 1 second depending on how much light the photoresistor is receiving. If this is working you’re ready to deploy to the HoloLens. This process is fairly straightforward if you’ve deployed to the HoloLens before, just make sure you have the InternetClient capability checked in your Player Settings as I mentioned earlier. If you forget to do this and have already built your project once make sure you delete the folder you previously built your application to as Unity doesn’t rebuild every file. If you haven’t deployed an app to the HoloLens before, check out my other post about deploying an app to the HoloLens.
Now that I’ve established a basic data pipeline from a Photon to a HoloLens I can start exploring more interesting ways of using this data to create engaging interactions. The next step is to go the other way and trigger events on the Photon, such as turning on an LED, from the HoloLens.