Wrangling robots: An inside look into managing a robot engineering project
The engineering history of Misty
We connected the 9-volt battery, and the system sprang to life. The single arm on the left side of the robot rotated wildly, barely attached to the vestigial gear that we couldn’t get off the shaft. The wheels struggled to move the wooden crate body forward. “Maybe we need to add another 9-volt,” my brother suggested.
We were eight or nine years old, working in our shop, and confident that we were close to completing the world’s first combat robot. I can look back now and say that, from an engineering standpoint, our project was a mess. But to me it will always be magic.
In the years since, I’ve worked on virtually every kind of software project, as well as various devices and sensor networks. I’ve also been part of a lot of startups, because I really enjoy the frenetic, fast-paced environment. When I was brought into Sphero to lead the software engineering team for what was then known as “Home Robot,” if it had just been yet another startup, it would’ve been easy. But this time it was my childhood dream come true. A real, autonomous robot. There may not have been plans for battle gear, but it was close enough.
Not Just Another Hardware Project
You’d think that building a robot would be like any other hardware project, but it turns out to be a completely different animal. I’d done a lot of device-based projects before, but never had I tried my hand at something that needed to strike this kind of delicate balance — a finely tuned constellation of moving parts.
As I plunged myself into this world, I quickly learned that we weren’t going to do anything revolutionary by modeling what we were building after the bots that had come before us. At Sphero, people knew control systems inside and out. We could transform a microcontroller and motors into magic. When we started working on Home Robot, we didn’t exactly know where we were going, but we knew we had to take that same Sphero magic to level 50. We wanted something amazing — a cognitive robot that was both highly capable and felt alive.
Truth be told, I don’t know if we ever created a formal list of capabilities for Home Robot. We knew we were trying to appeal to a developer audience, and that meant the robot needed to be packed with the latest technology. We knew that developers are creative and will find ways to use things that we would never anticipate. We wanted to deliver a very real level of sophistication, while making it accessible to people who didn’t know much about robots. Home Robot was an ongoing work in progress.
The Challenges: Consolidation, Integration, Iteration
As we transitioned from Home Robot to Misty (and from Sphero to our own company), the tide of development kits continued to roll in. We sifted through sensors, actuators, radios, and cameras to identify parts with exceptional specifications. Vendors sent us evaluation boards, stacked with complex circuitry, and fueled by bespoke software systems.
We spent months integrating components into subsystems, then testing them in the crucible of our expectations. We built, tested, and eliminated chassis after chassis trying to create a mobility platform that balanced capability and manufacturability. We lived our days by a motto, “ABP” — “Always Be Printing.” I would wake up at night, imagining the sound of the Z18 calling me to a newly finished print. It was truly crazy finding our way through the mire.
As we narrowed down the choices to a set of components that met our demanding scenario, we met with two very real problems.
Firstly, this orchestration of development kits had to be installed into the inside of a robot. I was fairly confident that it might fit into Optimus Prime, but we were really hoping for a smaller, more approachable industrial design that didn’t offer the volumetric opportunities of an Autobot leader. All of our wonderful components we had painstakingly selected would have to go through a massive amount of consolidation if they were to ship in something smaller than a semi truck.
The consolidation process is not for the faint of heart. Ridiculous amounts of time go into each board revision, and anything reasonably complex takes weeks to turn and costs thousands of dollars to have made. When new PCBs (printed circuit boards) arrive, it feels like Christmas, only scarier. As we opened the first box and saw the three shiny new boards we’d made from the quick-turn facility, we realized there was no room to let anything bad happen during the bring up. We handled each board like a newborn baby, carefully checking voltage rails and breathing softly as to not disturb the new PCB. At the end, there was only one that even remotely worked, and it looked like a spiderweb of green wires.
Secondly, nothing quite worked together. Misty’s primary processor was running Windows IoT Core and worked surprisingly well, but it drove a MIPI-DSI (display serial interface) protocol, and the screen we’d selected only accepted RGB. The microphones shipped with a variety of packages that supported a variety of platforms, but each was different in actual capabilities and all had to be picked through painstakingly. There was a feature set that worked in Android, and a feature set that worked in a particular, specific Linux distro, and all they had in common was that they both ingested audio streams.
Everything was about the details, and each decision affected all the decisions that came after. The time of flight sensors required microsecond timings to get correct readings, and the SLAM solution worked best with a desktop-class processor. These things represented a subset of hard problems that we had to solve before we could even think of delivering the first piece of valuable functionality to an external developer.
From the beginning we focused a huge chunk of our effort on our differentiating features. We didn’t just want a basic, functional robot. We wanted to build out a whole platform, as well as support for a third-party ecosystem around that. That big vision was core to the product concept. Other things, like converting DSI to RGB, were merely context — things that had to work, but that didn’t make the product unique.
The thing is, with a robot, whether it’s a major feature or an incidental, the complex integration challenges never stop. We were dealing with them while we were part of Sphero designing the Home Robot prototypes, and we’re dealing with them now at Misty Robotics, working on Misty II and beyond. Architecting a robot from the ground up — not just taking an off-the-shelf tablet and giving it some wheels — is like designing an entirely new phone, except that the phone has motors that give off serious heat and, incidentally, have to be isolated from the microphones.
The Rewards: People. And Magic.
So there’s got to be easy stuff with building a robot, right? Well, maybe. I will say that I got lucky in hiring. Some of the engineers I hired came packed with the special knowledge necessary to deal with our crazily specific problems. Other problems were a total unknown to all of us, so we had to adapt quickly to dealing with new technical challenges on what seemed like a daily basis. For my part, as a manager, I might have no idea how to process multiple audio streams through a DSP, but I’ve gotten really good at finding an expert who does. That kind of sourcing of human resources is a skill I never thought I’d have to develop.
Now I can think back to the crate-based robot that my brother and I had built as children, and about the Raspberry Pi robot I built several months prior to starting at Sphero. Looked at from the perspective of building an autonomous robot, they were paper airplanes compared to our aspirational SR-71. The challenge of building a compelling robot, capable of doing multiple tasks is very real. It’s even harder to build that robot for other people, each with unknown use cases, varied backgrounds, and almost certainly unrealistic expectations of robots.
The breadth and complexity of this kind of project easily eclipses what an individual can do in any reasonable amount of time. So, of all the things I’ve built, I’m most proud of the engineering team here. We’re working on solving some very hard problems — problems that really do require a team of people to solve. I’ve learned that only by assembling a group of passionate experts will we take robotics to the next level.
The other big reward of my job? The thing that makes me want to come in to work every day? It’s the magic. Robots are just as much of a thrill to me now as they were decades ago. And even better now, because the magic is real.
