Information architecture testing: Branching off with Treejack

We’re building the new Mass.gov with tasks and services as a focus. Keeping what our constituents want to accomplish in mind, we opted for verb-first, actionable language for our top-level navigation menu: Living, Working, Learning, and Visiting & Exploring.

View of the top-level navigation for the redesigned Mass.gov

We researched what other state government websites were doing, had card sorting workshops to organize our most popular content, and eventually developed a condensed menu structure. But we knew not everything fit into a single category cleanly, and that users always know best — so we needed to validate our hypotheses in the real world.

Testing our information architecture

To figure out if Mass.gov users felt comfortable and were successful navigating the information architecture (IA) we developed, we had them take it for a test drive.

We used a tool called Treejack from Optimal Workshop to recreate our proposed IA and gave testers 10 services to find based on some of our constituents’ most common user journeys.

Visualization of our overall test results

This testing showed us how many people could find what they needed (success), how many got to the right page without taking a wrong turn (directness), and how long it took (time taken).

Overall results

  • Success — 77%
  • Directness — 68%
  • Time taken — 6.4 minutes to complete all 10 tasks

On most tasks, our testers were really successful (85–95%). But on a few, we found people looking for information somewhere other than our target destination. For example, on a question asking “You’re moving to Massachusetts. Where would you find school enrollment information for your 2 children, ages 7 and 12?” people were just as likely to look in “Living” as they were “Learning,” where we thought they would go to find school information.

Multiple journeys to the same service

Similarly, when we asked our testers to check the license status of a nurse who is tending to a family member, the majority chose “Living” instead of “Working.”

Results of user paths for nursing license task

Checking a nursing license might be important for hospitals screening new hires or for nurses making sure their licenses were renewed. These audiences may look under “Working” for license information. However, the persona we asked our testers to adopt wasn’t a professional one, so they didn’t view license information as a work-based service.

Another task we asked people to complete is about as universal as it gets: Where do you go to file personal income tax? Our original target for this service was in the Living tab. However, about 4 in 10 people expect taxes to be under Working.

Results of user paths for personal taxes task

What we learned

We suspected going into this project that there’s no “perfect” IA for Mass.gov. Our test results bear that out. It’s clear that the perspectives of the millions of people who visit Mass.gov every year are incredibly different, even when they’re looking for the same service.

Our job is to balance our users’ need to access pages through multiple paths while keeping the IA as simple as possible.

In light of these results, we made some decisions to surface topics in several areas on the site. But we also determined that what we have now is insufficient in a broader sense. We need more ways to categorize and display our information than the current IA supports.

Stay tuned for more on that effort, how our research led to the creation of the service page, and how data will help us continually monitor and improve user experience as we move forward.

Like what you read? Give EOTSS Digital Services a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.