Designer as User: Testing in the Wild

Photograph by Darren Hull

In this big, complicated world, there are many processes in need of redesign. From the operating instructions for a smart toaster to the crosswalk at a busy urban intersection, designers are tasked with the important job of improving these experiences and removing pain points wherever possible. Of course, this is easier said than done. Even a common experience with which a designer is personally familiar can be a challenge to understand in a real-world setting. Because of this, research and desk-based design thinking alone cannot solve problems. Designers must take their hypotheses out into the field to test with real users and whenever possible, designers must become the user to truly understand their problems and needs.

The Problem

Imagine you’re walking into a conference centre, ticket in hand, ready to absorb all of the delicious knowledge that the speakers will soon impart. You’ve been waiting for this day to come and you want to get a good seat for the first talk, but you note the long line at the check-in area. Instead of nabbing a spot at the front of the conference hall, you’ll be stuck in this line for the next 20 minutes.

To provide a full registration experience for event planners and their attendees, I wanted to find a better way. My aim was to design a product that would create a smooth experience for attendees and provide useful information for planners. It would keep check-in lines short so attendees could get to events on time while allowing planners to keep track of attendance and supply analytics for them to improve future events.

After some desk-based design thinking it became apparent that the problem wasn’t as straightforward as I initially thought. I am not an event planner myself and had never worked a check-in table at an event, so there were gaps in my understanding of the problem. There is really no better way to understand one’s users than to become one, so the solution was to validate my assumptions at a real event by checking people into an event myself.

Seems comfortable, but how much can one really discover at a desk?

Initial Research & Testing

Upfront research was integral to a better understanding of the problems both event planners and attendees experience during the check-in process. Upon investigating competitor products, it became apparent that events are inherently different from each other and require different toolsets to accomplish goals. For example, the physical check-in space of an event varies depending on the physical layout as well as the amount and types of attendees expected at the event. The choices a planner makes about the check-in experience relate directly to the feature set they would need in their check-in product.

I built out high-fidelity mockups with which user testing would be conducted, as per the usual design flow at EventMobi. We have a top-notch staff of event experts at EventMobi, so internal user testing was sufficient to answer my initial questions.

The glorious check-in design.

My goal was to ensure that this first iteration of the feature set would be enough to check-in attendees smoothly, as well as allow the planner to customize the view of the check-in list to fit their needs at the event.

We use a variation of the rainbow spreadsheet for our usability testing.

Product development tends to focus on the digital experience without considering the physical space and the full context in which the product will actually be used. It can be easy to forget this from time to time, which sometimes leads to designing in a vacuum. I could easily have sat at my computer adjusting, questioning, answering, changing. This is an undeniably comfortable place to be but there were questions that could only be answered by real users at real events. At this point, I took my design out into the world and tested it in a true event setting.

Into the Wild

Equipped with the check-in prototype, I headed to a mid-sized Toronto event (around 1,500 attendees) to see my designs in action. The event planner at this conference gave me the chance to check-in attendees myself, which was an extremely eye-opening experience. I immediately confirmed my theory about the experience being as much a physical one as a digital one.

The lineup for registration: it’s so hectic that you can’t even see the tables at the far end of the hall.

The event organizer expressed many advantages she experienced while using our check-in product. Whereas this conference had used pen and paper to check-in their attendees in past years, we were able to prevent a repeat bottleneck by using a digital search to find people. I heard more than a few attendees comment on the ease with which they were checked in, given their badges, and sent on their way to do what they had come to do. Furthermore, through the analytics we had collected with the check-in product, the planner could predict trends for her next event and make data-driven decisions to continually improve the event year over year.

My research and testing provided me with a great starting point, but using the product onsite at this event revealed many insights that I couldn’t have discovered with desk research. As I checked attendees into the event, I took note of pain points to address in the design of the product.

Digital search functionality was a speedy solution to the registration bottleneck of previous events, but some of the attendees I met were simply not on the check-in list. In future, we would need a way to add a person into the system manually so that they could be accounted for in post-event analytics reports. Contrastingly, in a case when the wrong attendee was mistakenly checked in, it was difficult to find them in the system again. I realized it would be beneficial to see a feed of all of the checked-in attendees so that the mistake could be easily rectified. Facilitating correction of human error is a good feature for any product to have.

Due to the physical setup of the registration area, it was difficult for attendees to know which line to join. The system was alphabetical but attendees could not easily see that there was a separate line for guests. While this issue could be fixed with proper physical signage, it would have been helpful for the check-in staff to know that these people were in the wrong line (and that their badge was at another table). Therefore an easy win would be to add a ‘ticket type’ column to the interface so that we could send the attendee to the proper line where their badge was located.

These features are all immediately proven because I have first-hand experience that they would solve problems I experienced myself. On top of that, they all carry a good ratio of low-cost/high-reward, and we can add them to the next iteration of the check-in product.

Perhaps most importantly, going onsite to test the check-in product at a real event showed me that events are as unique as they are numerous, and factors like event spaces and ticket types play a factor in how the physical check-in space is set up. In turn, the check-in product should provide enough customization to match the setup of the physical space (as opposed to the product dictating the space).

Testing Designs in Context

When designing a product which is part of a physical process, there are limitations to how well a designer can understand the problem they are trying to solve. To combat this, designers need to take advantage of opportunities to use their product in the setting for which it was intended. This provides invaluable context that cannot be accessed through the screen of a laptop. It’s important for designers to avoid the complacency of designing in a vacuum, and take the leap of faith to push their product out to test with real users. Learning from mistakes and incorrect assumptions in a contextual setting helps designers to continually improve products and create better experiences for users, no matter which side of the check-in table they’re on.

Thanks for reading! If you’d like to join a team of the smartest, funniest, and kindest people I’ve ever worked with, where you are empowered to disrupt and design your own work processes, EventMobi is hiring.

Check out more of my writing at blog.chloesilver.ca.