A Journey With NAO Robots and Arduinos Towards Eyebrows

Heather Kemp
15 min readMay 13, 2018

Introduction

While looking around online for NAO robot pictures to show a friend, I found an interesting GIF of a NAO robot with eyebrows.

The GIF that started it all

Intrigued, I decided to follow the link where this GIF lead to, and learned about a research project to give NAO robots 3D printed eyebrows in order to enhance their ability to express emotions (which you can read about here).

I reached out to Albert De Beir, one of the lead researchers on the project, and was lucky enough to get my hands on the 3D models they used for this project, along with some basic tips on how to get started. I was certain that I would be able to replicate this project in no time at all, thanks to this, but of course, I had no idea how wrong I was.

In this article, I’d like to outline several of the challenges that I faced while interfacing the NAO robots not only with the headset, but also with the Arduino, and making a program for Choreograph itself, in hopes of allowing others to jump straight past these hurdles while working on a similar project.

If you’re interested in seeing a basic project with an in-depth tutorial that resulted from these lessons learned, you can check out my other article, Interfacing NAO Robots with the Arduino (In-Depth Tutorial):

3D Printing Difficulties [The Printer]

The first issue that we ran into in regards to this project was getting things 3D printed. We decided that we would use the printing services available on campus, of which the fastest and most accessible ones were the MakerBot 3D Printers. This printer also seemed to be fairly affordable and commonly found. As one of our goals was to make this project easy to follow along with your own materials, it just seemed like the best choice overall. However, on our campus, these printers were only usable with PLA Filament, which turned out to be very brittle during printing. Furthermore, the MakerBot 3D printers didn’t have enough fine control to get the details that we needed, especially in regards to the parts for the eyebrows. The back pieces for the eyebrows couldn’t be printed with the correct mechanisms to allow the eyebrows to rotate, which prevented us from working with the original device as intended. Furthermore, when we would attempt to put pieces on, it required a large amount of bending to fit on the NAO robots head, which broke over half of our printed models, due to the brittle material, despite extensive care being used with the placement.

One of the broken headsets

At the end of the day, printing this way also took a lot of time. The pieces themselves already took several hours to days to print, and the printers we were using were a shared service on campus, and so we weren’t always guaranteed to get the pieces that we needed on time.

3D Printing Difficulties [The Printed Models]

On top of the logistics of the printing itself, the resulting piece tended to have several complications as well. Due to the lack of fine detail in the printing itself, the printed headset and eyebrows often had excess material, especially in tiny crevices. To cut such pieces loose, which was required, as the pieces had to fit together exactly, we had to use an exacto knife to slowly cut the excess away. However, this was very dangerous, as we either broke the piece, again due to its brittleness, or managed to cut our hands when our grip slipped.

Trimming the excess filament and support materials

With PLA, pieces tend to be shrunk during as printing as well. To make the pieces fit together nicely, we often had to find a way to expand them all. Simply increasing the model’s size turned out to be counterproductive, however, as doing so increased the width as well, making the model an even tighter fit instead of a looser one. As a result, we needed to do extensive prototyping to attempt to get the pieces to work together (which will be covered next).

Prototyping

Receiving the models for the parts that were finalized in the research project was a huge weight off my back, as 3D modeling isn’t my strong suit. We quickly learned that we couldn’t use the models as they were given, though, for a variety of reasons.

The original model

The first issue that we wanted to address was the brittle nature of the material. We couldn’t expect everyone to have a high quality printer, and so we wanted to find a way for this Makerbot 3D printed design to work. This involved many iterations, including removing the front and back of the headpiece separately, and then printing the front, back, and both sides on their own before trying to reattach them. Most of these variants still broke, and furthermore, were not able to be reattached, as the screw holes in the headset were not retained, meaning functionality ended up being lost. After several headsets, we finally settled on a hybrid from the original, which removed a majority of the back of the headset in favor of a single bar to keep the piece stable. It still was prone to breaking if it was put on too fast, but the piece was the most resistant to breakage while providing all of the functionality needed.

The resulting headset for addressing the brittle material

Then, by looking closely at the example videos, we were able to determine that the original model we received was equipped to use Turnigy TGY-245 servos, a discontinued line of servo motors. At the time of this research, no purchasing links were available, but now, as seen by the link provided, there are some available for purchase, but at an extremely costly price for one servo. We wanted this project to be affordable and available for anyone, and so we wanted to make a model that more robust for different servos, while servicing a main servo that we could recommend for use. This was the TowerPro Micro Servo 9g (SG90), which was $4 for one versus the $30+ of the original servo (though we bought ours in bulk from eBay, so we got them for even cheaper). After much debating and modeling using Play-Doh, we decided to use a simple cube on the back of the headset with a hole hollowed out for the servo. Then, the servo would fit through easily up until its wings, and then be secured to the headset using an an adhesive like Sugru. This design seemed to work the best for keeping the servo from colliding with the back of the NAO’s head, as we could angle it heavily without impacting the ability of the servo to reach the wires for the eyebrows.

The resulting headset for holding servos

Finally, in the original project, a custom circuit board was used, which allowed for the board to remain plugged into the NAO and handle connections to the servos without additional breadboards. However, as we, again, wanted to make this project as accessible as possible without additional skills/learning, we wanted to use purchasable and/or 3D printed materials instead of custom made ones. As a result, we needed to use a breadboard bit to allow us to connect two servos to the Arducam circuit board that we used. However, due to the shrinking of the PLA and the fact that we modeled this before soldering the Arducam (which increased the size of the circuit board), we needed to model this piece several times to get it to fit just right. Luckily, Arduinos (and thus the Arducam, as an Arduino clone) as popular enough in the 3D printing community, so we could find a base with which we could work off of, though the breadboard bit did require custom modeling. This holder would be taped or otherwise adhered onto the NAO’s back, like a backpack, to account for the long length of the USB cable and the wires to the servos in order to keep them out of the way of the servo actually functioning.

The final Arducam circuit board and breadboard bit holder

Choreograph Libraries

Due to some special libraries that Albert De Beir et al. used, I was not able to get direct access to see what they did in their Choreograph project. A considerable amount of time was dedicated to trying to figure out exactly how they managed to get an Arduino to communicate with the NAO robot.

If you look at the documentation by Aldebran (currently known as Softbank Robotics), you’ll see that while they mention the Arduino being capable of communicating with the NAO, the link to the article leads just to the websites homepage, and there is no link to said tutorial. Using the Wayback Machine, you can still gain access to the article in question, though, and download the ZIP.

You can import the library as follows:

On a Mac, go to Applications -> Aldebran Robotics -> Choreograph Suite 2.1, and then, right click on the Choreograph application itself, and select “Show Package Contents”. Once you see the package contents, navigate to the folder specified in the tutorial (Contents/Resources/share/choreographe/libraries/box) and copy the ArduiNAO package you unzipped to this directory, authenticating with your computer’s credentials as needed. The next time you open Choreograph, make sure you are the aforementioned folder, and ensure you have Box Library Directory (Directories) selected as a file type, and the ArduiNAO package will show up to import.

Accessing the box library directory

However, these is no documentation any further than this, and the code itself was crashing in Choreograph Suite 2.1, despite extensive debugging and attempts to fix the code.

Eventually, I found another video documenting Arduino NAO communications with a different library (which can be seen here), which again gave access to packages to download.

While this package gave a more in-depth tutorial, it only worked for NAO’s on version 1.x, while we were using 2.x NAOqi. We attempted to fix the code to be compatible with newer NAOqi versions, but it still crashed, no matter what we fixed, and seemed to have some bugs with installing behaviors on the robot without deleting them.

Around the time I felt like giving up, I found this article by Emotion Robotics. It included code snippets and screenshots, along with some set up advice, and I figured I would give it one more shot. Unfortunately, it still wasn’t working, and so, I decided to reach out to them on a whim, not really expecting a response, as I hadn’t received a response from the other people I had reached out to.

Thankfully, Carl Clement from Emotion Robotics responded. I can’t thank him enough for the guidance he gave me while working on this project when all else was lost. It was thanks to him that I was able to realize the major problem we were encountering, that our Arduino wasn’t the right specifications for the NAO without additional driver support (which will be covered in the next section). He also helped me get the serial connection established with the NAO robot, and gave many helpful tips on interfacing it with different hardwares, which I’m now passing on to the rest of you all through this article.

The basic code and project which we managed to create with Carl’s help can be found in the tutorial listed at the start of this article.

Arduino Hardware Specifications

The Arduino Uno and the Sparkfun RedBoard

At the start of the project, we were using a SparkFun Inventor’s Kit (SIK), which we could get immediate access to from the library until we could get the parts ordered that we needed. This kit was very helpful, as it had all of the components we needed to get started right away, such as servos, wires, and resistors. However, as the kit was advertised to be an alternative to the Arduino starter kit, we assumed that the SparkFun was compatible with the same things that the Arduino was. According to SparkFun:

“The Arduino Uno uses an ATmega16U2 loaded with custom firmware to convert between USB and serial. The RedBoard uses the FTDI FT231X. This difference is only really prevalent when installing drivers because each requires a different driver file.”

While both connections were listed as being serial, the NAO robot only is compatible with FTDI serial communication. If anything, this makes it seem like only the RedBoard would be compatible.

Due to our faulty assumption that the RedBoard was compatible, we spent much time trying to get Choreograph libraries to work, thinking the problem lied solely in the Choreograph program (see Choreograph Libraries above), when it was in reality a mix of the Choreograph program and the RedBoard we were using. After we realized that the NAO robots are equipped to deal with 5V FTDI serial communications as done by the Arduino devices and borrowed an Arduino Uno from another professor, we had to go back and retest all of our Choreograph library options to see if they truly worked with a proper connection. This was a very time consuming process of debugging for a simple fix of using the right hardware the first time.

This problem could have alternatively been solved by installing the drivers needed for the communication with the RedBoard, using the method described in the tutorial in the beginning of this article, but we couldn’t find the appropriate tar ball to pull over to the NAO, and it seemed much simpler to just use an Arduino compatible device, such as the Arducam, which was compatible immediately with the NAO while still being affordable.

It should be noted that we attempted other forms of Arduino clones aside from the Arducam alongside the Arduino Uno, and several of these clones couldn’t even be recognized by the Arduino Software (IDE) on a computer. This, perhaps, may be due to the lack of the custom firmware mentioned by SparkFun, and thus, caution should be used when purchasing Arduino clones for project such as this.

Soldering

When we purchased the Arducam, the board did not come with any pins attached. While we could make due by holding the pins in ourselves, we came to the realization that we would need to solder pins to the board, a thing we had no experience in. Luckily, we had a professor on campus who was able to help us get started with soldering, who was able to guide me through step by step and provide all the resources I needed.

Not everyone can have such a resource though, and as such, I’ve found a helpful tutorial that should help anyone else get started with soldering:

Misc.

There’s always things that don’t fit nicely into one category that are still worth noting, and so I want to take the chance to mention those things now.

The first of which is that when working with the NAO’s USB port, you need to be aware that you need to plug the device in after the NAO is fully booted. If you plug the device in before the NAO is booted, then the device won’t be recognized until you unplug it and replug it back in. Make sure you do this to avoid time wasted during debugging.

The second thing of note is that the Arduino, or any similar device, will only send and read one character at a time in unicode format. It is possible to write a program to concatenate these things into one string. In fact, this is what the group from the origin research paper did, which I managed to find while looking around GitHub, as the link in the paper was broken. However, even the code they used was prone to error and required several end of lines to process, which ended up ruining the input to the NAO robot. As such, we decided to not go about the route of sending full words, and instead established codes based on unicode characters.

The third thing of note is less related to the project itself and more related to our resource gathering. In the folder we received from Albert De Beir, many of the videos and images of the project were embedded in a PowerPoint, which made things extremely grainy and hard to read. This was especially crucial when we needed to reference what was used in the original project for things like the servo. We spent a lot of time aimlessly trying to pick a screenshot that we could read the most from and then try different readings of the letters on labels. Later on, I learned that it is actually possible to convert a PowerPoint into a zip folder simply by renaming the file with a .zip extension instead of a .ppt or .pptx extension, you can then unzip the powerpoint and gain access to all media inside the PowerPoint unchanged in the {$YOUR_POWERPOINT}/ppt/media directory. Obtaining the high quality versions of the images and videos was extremely helpful in this project as a whole.

On the topic of obtaining things that seem to be lost, the fourth thing to note is that broken website links aren’t always a dead end. A lot of time was spent trying to find alternatives and other information about broken articles, when in reality, you can use the Wayback Machine to obtain the content of those pages, and sometimes more than you bargained for, such as the case of us being able to get the full zip files and extra examples for one of the Arduino-NAO libraries.

Finally, what many of the libraries and articles failed to mention is that pyserial is required for serial communication for the Arduino (if you’re using Python, that is, which we were). Unfortunately, the NAO robot wasn’t something that we could just open up a browser on and download what we needed to install such a library. It took a lot of trouble shooting and searching, but eventually we found out that you can install pyserial by downloading the tar ball (a .tar.gz extention) instead of the .zip, then transferring it to the NAO via mounting a USB, and finally undoing the tar ball and installing the python file. This was a much more involved process than we thought, as we had to try many different alternatives, and thus, we made sure to document this extensively in the tutorial mentioned at the start of this article.

Overall Lessons Learned

At the end of this project, I found that the lessons learned best boil down to the following:

Plan in advance — things take time. Especially when using 3D printing or ordering things from online, you can’t expect materials to be there right on time for you to start working. You should try to devise a way to be able to work ahead, even if it isn’t ideal, such as using Play-Doh instead of printing every variant of a design, and using a borrowed Sparkfun Kit and Arduino Uno while waiting for ordered Arduino clones to arrive.

Make sure you give yourself ample time to fully understand how something works. For example, we spent a lot of time working on the eyebrow mechanism itself without realizing that our eyebrow pieces were printed wrong and were missing a slot for the wire to fit into. This mean we did lots of work attempting to get the pieces to work, and tried many different alternative styles for eyebrow movement, none of which worked, without truly understanding what could have been done differently. This is only heightened by the fact that we were using models from another project which didn’t thoroughly document its exact mechanism for the working project. As such, we had no idea what materials they used or how they put things together to get it to work, which resulted in a lot of guessing and checking, as we didn’t get responses from Albert De Beir after the original communication.

Be prepared for changing your plans if things don’t line up with your original timeline. Furthermore, don’t spend too long on debugging something and trying to get it to work no matter what. It may be better to try something else, or even to take a step back and get a new look at it later when you have a fresher set of eyes.

Don’t be afraid to reach out for help. The worst someone can say is no, and the best thing is they guide you or direct you towards someone who can. I received the best help and insight from people I reached out without expecting much more than an “I can’t help you”. This will also help you with the previous bit of advice, as they can help you get past roadblocks with a fresh look on things you may not have thought of.

--

--