Designing with our open data publishers: a handbook as product

Written by Jenn Hu and Jason Lally

For the unfamiliar, publishing open data can be a daunting and intimidating process. “Where do I start?” is the question that arises time and time again for new publishers.

So we set out to better understand the California Open Data Portal publisher experience from start to end. Based on what we learned, we needed to create a more user-friendly Publisher’s Handbook. We think a simpler handbook will result in more consistency across open data publishing over time.

We were helped by a team of Stanford students (including Emily Bunnapradist, Jenn Hu, and Sejal Jhawer) as part of Stanford’s course CS 184: Bridging Policy and Tech Through Design. Jenn continued on as a volunteer helping with usability testing and content and is co-author of this post!

Using design thinking to map the publishing journey

To improve the handbook, we used the Design Thinking Approach, which consists of five iterative phases: empathize, define, ideate, prototype, and test.

Through these five phases, we were able to gain a holistic understanding of the publishing process, integrate our findings into the drafting of the new Publisher’s Handbook, and then refine the handbook based on usability testing and feedback.

Empathize and define: understanding the publisher’s journey

To put ourselves in the shoes of publishers, we conducted user interviews with portal administrators, experienced publishers, and new publishers across different departments and agencies in California. The interviews focused on:

• Teasing out the steps of their publishing process
• Defining the interviewee’s role in the publishing chain
• Understanding areas of confusion and difficulty
• If applicable, gathering resources they found useful when they first started publishing

From the user interviews we gathered the following insights:

• The publishing journey broadly consists of five phases (later four after testing)
• Lack of familiarity with metadata terminology was a concern for first-time publishers
• Seemingly conflicting guidance from different resources confused publishers
• Many users lacked documentation from predecessors, having to start their open data journey from scratch

Ideate, prototype, and test: putting it under the microscope

Based on these insights, we reorganized existing publisher resources to reflect the publishing process, guiding users through the process. It includes resources recommended by interviewees, outlines best practices, and provides support helplines, with the aim of detangling and demystifying the open data publishing journey.

On the left, the Stanford student team captured stages of publishing from their interviews. On the right, they converted that into a graphic flow for the handbook. Content was then organized around this framework for additional testing.

After drafting the updated handbook, we put it in front of new and experienced publishers, from a range of technical and non-technical backgrounds, to test its usability. Specifically, we evaluated the draft for:

• Simplicity — Are users able to understand the concepts and guidelines in the handbook?
• Organization — Is the content organized in a way that makes sense?
• Comprehensiveness — Does the handbook cover the most important parts of the publishing process?
• Call to Action — Is it clear at each step what the user needs to do to successfully publish data?

On top of posing interview questions, we also asked interviewees to highlight useful words and phrases in light blue, and confusing or unfamiliar words in magenta. This allowed us to gain insight on the readability of the handbook.

Left shows an example of a marked-up Google Doc as part of the first pass usability testing. Right shows that same section in its current form.

Overall, users were overwhelmed with the amount of information in the original draft and wanted a more succinct checklist for publishing. Users also felt that the handbook glossed over the iterative nature of the publishing process.

Iteration: one step forward

To address concerns raised in usability testing, we reflected iteration visually in the process flow. We also collapsed a step and updated the pre-publishing checklist to equip publisher’s with the information they need when they get started. We hope this allows users to more accurately estimate the time and effort it will take to publish.

We updated the publishing steps and simplified content of the handbook based on results from usability testing.

We also simplified the checklists in the handbook, moving additional information to the references section and linking them to the main body. In this way, users can still access these resources, but they would not take away from the key points of the handbook.

Launch and feedback

Iteration forms an essential part of design thinking. We will continue to solicit feedback and refine the handbook after launch. The handbook is not meant to be a static document; it will change and adapt to new findings as we continue to interact with more publishers of different backgrounds and needs.

If you are a California open data publisher or are looking to publish soon, check out the new Publisher’s Handbook and leave us feedback by filling out the brief survey included in the handbook. Your input will help us improve during our next review and iteration cycle!

P.S. More opportunities to help us shape open data!

We are also taking the design thinking and human-centered approach to develop a new user experience for the California Open Data Portal in partnership with the California Department of Technology.

You can sign up to participate in future user research interviews and surveys today!