Usability Testing Blog
If you recall, at the beginning of May, we shared in “Always Improving: Making the Contribution to Repos Better” the lessons learned in UX testing. We are always looking to continually improve the user experience of both the Code.gov website and our repositories. That is why we are here, once again, following another round of usability testing to better understand the average contributor experience. This blog shares the key takeaways, as well as the updates our engineering team has already implemented from previous user interviews conducted in March.
We sent out an email to our Listserv and recruited seven participants with diverse backgrounds in engineering in order to see whether our core target audience would be able to accomplish typical tasks including navigating to the Code.gov repositories from our website, reviewing issues and issue tags, cloning a repository and running it locally, and making a hypothetical style change. From these tasks, we were able to see how they interacted with our repositories in real-time, which helped us to better understand when participants encountered problems or experienced confusion. Much of the feedback we grouped into one of two categories: documentation and community.
In terms of documentation, many participants gave praise noting that it was obvious that our team had spent time on our documentation and people appreciated the detail included (e.g., being told what environmental variables are optional, having issue templates, and including actual commands to run). When it comes to improvement to repos, we are prioritizing updates to our documentation that helps meet the needs of users who want to jump in and get their hands dirty (e.g., adding quickstart information to the readme, and if Docker is available, list it as an alternative pathway at the top so users can jump to it; linking to or developing GitHub 101 for government guidance; and including basic tests so users can validate that configuration is correct). We also aim to standardize our documentation across repositories, including items like our issue templates. Lastly, one participant noted their surprise at seeing linked repositories within a repository, which was a first, and while eventually able to make sense of the content, we want to update our documentation so the relationships between the different repositories is more easily apparent.
All of our data may sound as if we are focusing on technical matters at Code.gov, but we are also looking to improve our community and how to better cultivate it. A majority of participants were familiar with Code.gov and the Federal Source Code Policy, but a few were unaware that the Code.gov website itself is open source or that the team is open to external contribution. One participant asked why someone would want to contribute to a website they don’t manage and thought that it would be useful to provide incentive or reason on why external people should contribute. As we encourage and welcome external contribution, we hope to remedy this by including language explicitly welcoming external contribution on our website and in our repositories. We hope to provide more incentive to contribution by better communicating our project goals and showing contributors their potential impact by sharing an updated strategic plan once finalized.
Overall, the usability testing itself was a useful way to gain valuable feedback on content and design, as well as a way to engage with our community. We appreciate and thank our participants and aim to schedule regular usability testing in the future. As noted above, our engineering team has already begun implementing feedback from the previous user interviews and this most recent usability testing. Notable updates include identifying and standardizing issue tags across repositories, setting up automated testing for the front-end and style-guide repos, and calling out kudos to our contributors in a public way in our release notes (for more information, please see our recent Developer’s Notes.
We have received from these various testing sessions thoughtful feedback and ideas that we hope to implement in the long-term. This includes potentially submitting an opportunity for a specific feature through GSA’s Open Opportunities, or participating in Google’s Summer of Code or Season of Docs. We are even exploring the potential idea of creating how-to videos to include in the documentation and organizing regular office hours for a developer Q&A. If you have any other ideas, we’d love to hear from you! Visit us on Twitter or LinkedIn, and ask questions. We want to know how we can make Code.gov a better experience on all levels.
Code on.
