Making the bigger picture even bigger
From analytics across pages to analytics across sites
If you interact with the Massachusetts state government over the internet, odds are that you pass through more than one web property. For example, if you want to buy a hunting license, you might begin on Mass.gov, but you’ll eventually land on MassWildlife’s MassFishHunt. If you want to check on your tax refund, you might start on Mass.gov and then move to the Department of Revenue’s MassTaxConnect.
Our mission at Massachusetts Digital Service is to improve constituents’ interactions with government. However, we can’t improve services unless we understand constituents’ experiences–and we can’t do that unless we measure the full scope of digital content.
To develop this bigger, broader vision, we’ve partnered with other agencies across the Commonwealth. These agencies are experts in their services, know their audiences, and provide guidance on what interactions are meaningful to track and to measure. This is the blueprint we followed when we developed custom dashboards to empower agencies to make data-driven improvements to Mass.gov content.
This time, though, the challenge is bigger and more complicated. We’ve been measuring at the page level, and now we need to measure at the site level. We’ve been gathering and reporting web analytics data, and soon we want to work with business process data.
To protect constituents’ privacy, Massachusetts Digital Service doesn’t capture any information that would let us link a Mass.gov session with a specific person. Instead, we look at trends in pageviews and page interactions, and we measure how these trends shift — or don’t — when content changes.
Our Progress So Far
In July 2018, we rolled out first-iteration web analytics dashboards. These dashboards, which are embedded in the Mass.gov Content Management System (CMS), provide hundreds of content authors and managers a quick way to assess how their pages are performing. Example indicators include where traffic was coming from, how easy the content is to read (by grade level) and a summary of responses to the site’s user satisfaction survey.
But measuring page performance has its limits. If you come to Mass.gov to learn about recycling or about how to respond to a jury summons, you probably visit multiple pages, and it’s how these pages work together (or don’t) that determines if a service’s digital content is well-designed. That’s why we’ve joined forces with 4 other state agencies to pilot a new set of metrics and indicators that summarize how all the pages that make up a service are performing. For example, we’re measuring what percent of link clicks within all the pages in a service move users toward and away from their goals. This information should help content managers understand if their Mass.gov content succeeds or fails in bringing constituents to, for instance, another web property.
These pilot dashboards are currently semi-automated: We use a Google Analytics plugin-in to automatically fill out a Google Sheet each time we run a report. Once we’re sure these dashboards are good enough to help authors and decision makers, we develop them in a visualization platform (such as Superset).
These service-level dashboards are a trial feature on our journey to understand constituents’ full digital journeys. The next step, which we’re already working on, is cross- (and sub-) domain analytics.
Cross-domain analytics — mapping the constituent user-journey across our different government web properties
Think of a time when you were shuffled back and forth between organizations–maybe between a doctor’s office, the hospital’s billing office, and a health insurance customer service line. Someone determined to improve your experience would be more likely to succeed if they knew something about all 3 interactions. We want to develop this capacity so that our partners can use it to measure and improve service delivery.
Agencies already have good questions about their digital content: Do constituents follow the paths we predict to access services? Do they have an easy time accomplishing their goals? Are apps as accessible as possible? At Digital Service, we wanted to capture the data that would inform the answers to these questions.
When implementing cross-domain tracking with our first pilot partners, we started with the basics, such as finding the percent of mobile users that make up an application’s audience and what days of the week traffic is heaviest. As engagements evolve, we might use Google Tag Manager to add tags or custom dimensions to make it easier for agencies to summarize and analyze data.
Some questions turn out to be valuable enough to ask on a monthly or weekly basis. For these, we create dashboards using Google Data Studio or Superset. For more complicated questions, we might combine separate data sources in a spreadsheet or database. This kind of reporting helps agencies measure if their campaigns are working, satisfy federal reporting requirements, and help subject matter experts see the impacts of content changes.
This work presents new challenges: There’s a lot more data to manage and many new business processes to track. There’s a lot of knowledge to share, from what we know about web analytics data to what partners know about business process data. But it’s combining this knowledge that we hope will lead to genuine realizations about how to improve services–how to make it easier to renew a driver’s license, start a farmer’s market, or perhaps resolve conflicts with aggressive turkeys, as efficiently and seamlessly as possible.
(We’re not sure there’s any business data for resolving conflicts with angry avians, but if so, we’d love to see it.)
Interested in a career in civic tech? Find job openings at Digital Service.
Follow us on Twitter | Collaborate with us on GitHub | Visit our site
