**30,000 content creators*, 2800 publications, 2 UX designers. One website.**

How we are using UX to create empathy and influence content design, systems and processes in government

Bridget Kelly

Published in

Digital Government Victoria

9 min readMar 22, 2019

What do you do when user needs and government content processes collide?

Our change journey had started with a single question: how do we design better government information and services?

That one question has led to many more in our mission to improve online experiences of the Victorian government.

Our initial research on vic.gov.au, the online entry point to the Victorian Government, told us that it was hard for people to find the information they needed and complicated for us to manage.

So, we spent a lot of time last year doing more research with citizens and Victorian Public Service (VPS) staff to see how we could improve experiences of both front-end users and back end content authors on vic.gov.au.

One of the biggest flies in the ointment of content migration has been publications.

Little known fact, “How do you solve a problem like publications?” was the original wording of the song from the Sound Of Music

Why publications are bad

Publications are PDF, Word or Excel documents that exist on government websites. They can’t be scanned by Google, are not responsive or mobile first and need to be downloaded to be viewed.

I am new to government and so I have an outsider perspective on some of what we do. There have been parts of government that are inspiring and make me feel warm and fuzzy. Publications do not bring the warm and fuzzies.

This pug also heard about publications and ended up needing a blanket to remain warm and fuzzy.

Publications are notoriously difficult to use, especially on mobile devices. We also know that most people access government websites usually when they are trying to solve a problem. This means using publications creates greater stress for people by providing a difficult experience right at the moment when they need the information the most.

Publications make finding information for blind and low vision users more difficult than digital first content. Best practice dictates that content needs to be digital first to serve the user. However, knowing all this doesn’t stop them being created (in and out of government). On over 27 government sites alone there is more than 2800 publications. More than 2800!

So, when the UX team was asked to build a publication feature on the new vic.gov.au website, it was a tricky request: design a best practice module to incorporate a format that is not best practice.

While the request was a publications module, once we dug under the surface of that what we needed to do to improve the UX for publications we found we needed to:

Stop or reduce publication creation
Support accessibility and user experience of historical publications that exist

So, the request may be simple, but the problem was complex, and the solution had to be far reaching in impact and influence.

No easy feat, if I do say so myself.

What we did

Once we understood the outcomes we needed to achieve, we articulated our discovery into two key questions:

Why so many publications?
For historical publications that will sit on the new vic.gov.au, how can we make them accessible and easy to use?

To understand these questions, we held a workshop with communications teams from across the Victorian government to scope their understanding of the world of publications. Then we conducted in-depth research on how content is produced in government. Finally, we reviewed the web for best practice design solutions.

Discovery

Discovery step 1

What we did

We held a workshop where we asked communication teams members about:

What publications are being created?
Who is the audience for these publications?
Purpose?
Process?

What we found

Because there isn’t a lot of end use data, many believe that nearly all publications are for citizens
Sometimes publications are suitable: regional communities with reduced digital access, low digital literacy communities, or communities with low general literacy.
Communications teams do advocate for digital first content when they can
While most understand that html is a better user experience there is not enough evidence to advocate for change with the actual producers of content.

Discovery step 2

What we asked

During in-depth interviews we asked communications teams across government to detail their publication workflows, including:

Who is involved?
Audience needs
Business needs
Time scales
Inputs and outputs.

What we found

Publications are created in a de-centralized way by different roles within a team or department.
Subject matter experts are the starting point for content production
Publications are produced mainly because it is the familiar format for content creators
Comms teams are only consulted towards the end of the process.
Short timelines mean that comms teams often can’t influence document format

Discovery step 3

What we did

We reviewed the websites of other Victorian Government department websites, adjacent state government departments, federal government, private industry and best practice guidance for UX design blogs to understand the theory behind potential design solutions.

What we found

Publications (Word and PDF) are not accessible to most screen readers, even when claimed they are. Vision Australia indicates that they often do not meet accessibility guidelines.
Industry wide, digital first (html) is used as it has the best accessibility, indexation in search and ease of use in mobile first content.
Other Victorian Government departments have moved to creating landing pages for publications, particularly long form (i.e. annual reports).
This format allows the content to make sense in its publication format, with improved accessibility and ability to be indexed by Google.
Downloading content is still available in digital first content via a button to download or print at the start or end of important content

A review of best in class publications on equivalent government websites

Design

Once we had a good understanding of our problem, we were able to ask two core questions about our solutions:

How can we support web first content and create empathy for the user when producing publications?
How can we translate old publications into html while retaining comprehension in content and layout without needing to be re-written for web?

User experience testing was conducted in a UX laboratory in South Melbourne.

Design step 1

What we did

We created interactive prototypes of the different potential solutions. We used a piece of content that was already in existence because it was:

Originally created as a publication
Live as a html version on the web
Available as a PDF to test design features, navigation and sentiment to our potential design solution.

Design step 2

What we did

We did UX testing with end users in the general public who had looked up Victorian Government content before and who had previously downloaded publications.

Five males and five females
Seven tech savvy users, three less confident technology users
Five android familiar and five iPhone familiar
Three blind/low vision participants
One English as second language participant
Six participants with no accessibility requirements

What we found

We used a system usability score (SUS) as an overall measure of performance as the SUS measures the usability of a website.

The publication version got 3.14/5 and the prototype version got 3.75 on average.
For the blind and low vision, publication version got 0/5 and html got 4/5.
This is because even though the publication was designed to be accessible it was still not usable. All our blind and low vision participants would have to get someone with vision to help them use the Publication.

There were 3 functional areas of UX difference between the web first and publication versions.

Navigation within the publication was hard and made searching for information difficult. All participants said they would exit using the PDF on their mobile. They would send it to themselves via email then find a desktop computer to access the PDF or print it.
Most participants’ first instinct was to find a search box if they were looking for a specific piece of information rather than scan through all the content. When this search option wasn’t available, they would exit and search in Google. This was true for hmtl and publications.
Content was not read. It was scanned and skipped due to large sections of content not being relevant, often called “government speak”.

“Government speak” is particularly problematic for blind and low vision users, as they will have the first paragraph on each page read out loud to them by their screen reader as it should inform them what’s on the page or direct them on where to find the relevant information. When it doesn’t do that, they may have to exit the document, ask for support or spend an hour having the whole page read out to them.

Did we stop or reduce publication creation?

To create empathy with content producers in government, we needed to present our findings in a compelling way. We wanted to show the impact of publications on end user experience in a way that stirred not just the mind but also the heart.

In particular, we wanted to demonstrate the impact of publications (and bad html design) on blind and low vision users, who had ZERO success using the publication version (even though it was technically an accessible version).

We created a video that included a 10 second video clip at the end of each UX testing session and

screen recordings from the UX testing sessions to show the impact of poor UX as a story, rather than as data points.

We included comments from blind and low vision participants who indicated they would need to ask a partner, friend or parent to help them use the publication. We were able to show how our use of publications in government creates barriers to access that forces these users into a dependency that they are embarrassed to have.

Success?

This video was shared across our Department, including senior management
We used Google Analytics to provide evidence of data and direct our colleagues on who they can consult to find Google Analytics data to prove that the Publication usage is lower than is expected within departments.
Together, these two data types (rich, emotional and empathetic qualitative data and statistical, big data) have shown that this is not a small problem, and we have a responsibility as public servants to address.
We have worked with our content team to make it easier for public servants to create html content. From the first release of content 20 out of 21 documents were made into web pages. On the old site only 1 of 21 of these documents had a html version
Our grants and programs process is also moving away from multiple Word and PDF documents to having most content in html in a structured order

Are we supporting accessibility and user experience of historical publications that exist?

We understood that the HTML first version performed better
The landing page version for large publications worked — it included jump navigation, which was greatly desired
Search box would have to be further investigated for indexing
Providing options to print, email or download publications was still appropriate
A module for the publication to sit within includes its date of publication, author and short summary — as well as an option to read online (html) or download (Publication).

Success?

We were able to reduce the number of PDFs for online grants application content, just by using html.
We’ve increased the number of html alternatives to PDFs on vic.gov.au
Our publications module is queued for development
Our findings have been shared with other departments so that we can continue to improve user experiences across government
We continue to work to build awareness about our research to support the digital first model, with empathy for the user at the core.
Our content team is working with content owners who have publications work coming up in the second half of the year. This means everyone is working towards a digital first model at a pace that suits them.