Reflecting on AR

Four year ago today, I was at a large equipment dealership in Las Vegas, fine tuning an Augmented Reality application I had built for a prominent corporation to show at one of the largest machine expos in the world. To say I was thrilled would be an understatement. To add that I was nervous, would be accurate. I was alone (for this trip, but not for the expo to come), as well as a soloist on the development of the project.

To think back of how it arrived on my desk sounds mad. I had been doing some R&D for a local startup on the potential of using 3D models in an AR project. The prototype was very early stage, but it was exciting. So much that it was instantly shown to anyone who would see it, and it spread like fire. Granted, this was at a time when the technology was still young, and few people had really experienced AR in this capacity.

It wasn’t long after, that I was approached with an actual project. A big one. Bigger than any of us realized. A worldwide large equipment manufacturer had a new truck. This truck had a number of accessories. The idea was to use AR to display these accessories as if they were attached to the back of the 30-ft. vehicle in real-time, on the actual truck; using an interface to select between the options. This was amazing. How could I say no?

There were, of course, some challenges to consider:

  1. The prototype was roughly the size of a phone, hovering over a sheet of paper. This was an actual 30-foot truck. Not a photo… the real deal.
    Being a real truck, anchor points needed to be established in a 2-dimensional way, for the tracking software to capture and hold.
  2. Capabilities of the iPad camera needed to be verified to ensure that there weren’t any catastrophic distance limitations that would prevent someone from standing far enough away to actually view the whole thing (and not end up inside it).
  3. Oh, by the way, we won’t have a truck to work with until just before the actual expo. So, some improvisation may come into play.

Those four items were the start of many sleepless nights. I was confident I could pull it off, but it was still a bit terrifying. So, I had to plan a way through it, very carefully.

First, the approach to solving the tracking on a real vehicle wasn’t too difficult. I just needed a known, fixed set of points that could act as a marker for the application. A recommendation was made to create an informational sign that would be attached to the back of the truck, in a way that would be covered by the 3D model when the app was active.

Next, there was a lot of testing done in various lighting conditions, and at various distances, to determine how well an iPad Air could track a marker at distance. The sweet spot ended up as a 4 foot square target that could hold an object solid to nearly 30 feet away. Nice!

Finally, we didn’t have a real truck to work with.

Wait, what?

How can you create something if you don’t have something to work with something?

Okay, breathe deep. You don’t need the actual truck, Anton. You just need the precise specifications of the truck. Every detail. Mounting brackets. Measurements. The works. A tall order, considering the level of trust I would be asking for. Equipment manufacturers do not hand out such information lightly (rightfully so). But ask I did, and access was granted. I got my numbers. There may have been some dancing involved. Maybe.

Right away, I began marking bits of tape and notes on a wall. These served as a simulation for me to work with as I created the application. Eventually I got there, and it looked pretty good. Granted, the room I was in limited space a bit, so you sort of dropped into the objects as they first loaded, but you could easily walk around and see that it was working. All that was left was to try it out on the real deal.

Fast-forward a bit to my first trip to Vegas.

I showed up at the dealership and was led to a workshop. Possibly the largest I had ever seen. Toward the back rested my prize, a freshly-cleaned, big, beautiful yellow truck, waiting for me to do my work. Thrilling.

In the end, I think it all turned out to be a big success. There were some odd little things along the way that had to be worked out, and even during the show, the three of us that were there had to stay on our toes to keep things running smoothly with the sales reps.

Reflecting on that time, it remains as one of my favorite professional opportunities to showcase my ability to research, adapt, and learn. To take a concept as simple as the prototype I created, and turn it into a product worthy of an international equipment expo.

With four years now between myself and this project, I’ve seen the changes in technology, and have picked up a few new tricks that could be used. If I ever were presented with this sort of project again, with what I’ve seen and learned? It would be amazing.