In-Person Usability Testing at the RaiseMore Beta in Oklahoma City, OK

In September of 2015, I worked with the team at RaiseMore to help run a two day feedback summit in Oklahoma City. We went through 4 feedback sessions. The rounds of one-on-one interaction with real people were incredibly helpful. The Group discussions on Day 3 discussing the product roadmap were very insightful. The weekend would inform a lot of the next several rounds of product features and iteration. Among our team, there was a feeling that we really had our finger on the pulse of the industry and in the end, that was a bit of a liability. While we did have a good handle on software design — we did not, in fact — have a good handle on the needs of our fundraising users.

I wanted to create this post to document some of the things we looked at in case those things are at all helpful for others. Some of it’s dated but it might have some helpful things to at least think about when you begin a software project. Especially one for users that you’re new to working with.

Here are a Few Takeaways*:

• Up to 80% of our users would be using Windows computers. Up to 30% of those might use touchscreen technology in some capacity.
• Our collective use of Apple hardware caused us to miss some glaring usability issues. We missed some hover states, color inconsistencies and responsive issues on smaller screens. We missed these things because we were seeing the product through tech focused lens.
• Visual design inside the dashboard caused a lot of confusion. Defaulting to ghost buttons and a flat UI pattern was often and easily missed by our testers.
• Our internal bias toward aesthetics vs. experience was exposed. This surfaced some internal frustration on our team between UX and Visual design imperatives. The dialogue we had here was tense but positive. If people can’t accomplish the product tasks, we will have a hard time making them customers.
• The things we thought were going to be super important were a lot less important. One of our assumptions was that the person using the app would start at the feed page. This would allow people to see the status and relevant data points of all their teammates. During testing, our groups let us know they would opt for a view based on the health of their event(s) instead.
• An interesting pattern emerged here. The money raised is important but not on such a granular level. The overall event health was a more desirable metric. From there our partners could dig deeper and address individual coaching needs. There was a very “leader-leader” structure to the way our partners managed events.

The big takeaway here for me is that wee live and die by our assumptions and hypotheses. We’d have never known most of this information without getting n the trenches with real users and seeking to understand their work and their needs. Never underestimate the value of human beings when making computer software.

RaiseMore Beta Day 1 & 2 Videos

*Numbers extrapolated to identify total user-base from user-base who was able to attend the beta retreat.