So half way through the semester came the mid-term project for Physical Computing. Jenil, Weiting and I started thinking of ideas of how we can combine what we have learned into making something fun.
The first thought was to create “Clappy birds” — Flappy birds that you can play by clapping your hands, but our professor, Eric Forman thought the interaction was “too simple” for the midterm. So then came the second idea. How about playing Mario with your hands and legs? How about playing it with your friends?
The idea was to interface gestures with digital actions.
So what actions could you do to play Mario? Quickly we came up with a few thoughts and then started sketching them.
An important aspect of the project was making it wireless, so the first step was to figure out how to use an Arduino. I found a really cheap module called ESP8266 (ESP-01) that could give Arduino wireless capabilities. But since it was cheap, it wasn’t that easy to work with. It had various problems, like it only works with a 3.3V power supply and you can’t use the 3.3V supply from the Arduino as it does supply enough current for the ESP8266. So we got an external power supply for the ESP8266 but it still couldn’t communicate with the Arduino. Eventually we gave up and started looking for alternatives and finally found this, ESP8266 ESP-12E. This variant of the ESP8266 is a fully functional development board which not only handles the voltage regulation and other crap but also has GPIO pins to work with, which basically means it can work standalone without an Arduino, unlike the ESP-01 which is a module.
The ability of it to work as a standalone chip was a cherry-on-the-top for us, since it was a very small board, we could stitch it on to the glove that we had in mind.
Gloves, Steppers & Box
Now that the Wifi was sorted, we started working on the glove and box. Since the hands need to move freely, we had to think how to make gloves that are flexible and light enough to play the game without any difficulties. We sewed metal parts with conductive threads.
Since the hands need to move freely, we had to think how to make gloves that are flexible and light enough to play the game without any difficulties. We sewed metal parts with conductive threads.
We had 3 micro-controllers, 2 for the hands and 1 in the box for the jumping and running. Each of the controller was running a tiny web server that read the pin values and served them on a web. The NES emulator which was running on the laptop would hit the web address of each controller and get the readings. That’s all, these reading would then make Mario go left, right, jump or run.