Alberta Health Services & Ontario Digital Service Shared Self-Assessment Tool
Authors: Spencer Daniels, Senior Product Manager, Ontario Digital Service; Lauren Lombardo, Master in Public Policy 2021, Harvard Kennedy School
In early 2020, when governments and digital service teams were at the forefront of the coronavirus crisis, Alberta Health Services (AHS) created a COVID-19 self-assessment tool to prevent emergency services from being overwhelmed and reassure the population. The product launched with great success, and within 24 hours over 300,000 people were using the tool, according to internal metrics. The high rate of adoption — in a province with about 4.3 million people — was partly via social media channels, with news of the product quickly going viral on Twitter.
Two AHS employees, Kass Rafih and Ammneh Azeim, were soon contacted by the Ontario Digital Service (ODS), which was looking to develop a similar self-assessment tool. In less than 24 hours, AHS and ODS had agreed on a way to share the tool’s source code. Within five days, ODS launched its own version of the self-assessment tool.
What Is the Alberta Health Service Self-Assessment Tool?
AHS began working on a self-assessment tool just two days after the first local case of COVID-19 was reported. The news of this first case prompted Albertans to seek medical advice, which quickly resulted in AHS’s Health Link experiencing unprecedented call volumes. AHS began researching how other health jurisdictions were providing resources to the public.
One striking example was the United Kingdom’s National Health Service (NHS) self-assessment tool. AHS decided to develop something similar to increase the public’s understanding of COVID-19 and decrease the number of calls Health Link received. With these goals in mind, AHS worked closely with Health Link to codify official health assessment questions and guidance into an easy-to-understand online self-assessment. AHS also spent time testing the language and user interface with Albertans to ensure that the community members using the tool would find it useful and empowering.
In just a few days (of nearly 24/7 work), the first version of the product launched. Since then AHS has continued to enhance the tool with regular updates. These updates include similar tools specifically for health-care personnel and first responders, a referral form for testing, more symptoms in the assessment, and testing recommendations for asymptomatic people. With each additional feature, user testing remained an integral component of the development process.
AHS’s Decision to Share Source Code
The day after AHS launched the initial version of the self-assessment tool, the Saskatchewan Ministry of Health and Ethos and the Ontario Digital Service (ODS) both reached out to learn more. These conversations led to a decision to share the self-assessment tool source code with both ODS and Saskatchewan. This, in turn, led to governments around the world (including New York City, Ireland, and the Asian Development Bank) contacting AHS about repurposing the tool.
Some of these relationships involved just information-sharing, while others opened a channel of two-way communication that has lasted throughout the pandemic. ODS, in particular, has formed a real friendship with the AHS team. ODS started the conversation about posting the source code on GitHub, and the teams have continued to discuss their individual changes and updates to the tool.
The code-sharing between AHS and ODS was informal. The relationship could be summarized as information-sharing between friends, with the code eventually shared through Google Drive. Part of what made this process easy was that, at the time, the self-assessment code was a flat file — a simple website that assisted decision-making and did not connect to any underlying applications or databases. The fact that the tool didn’t collect any personal information meant other government agencies, like ODS, didn’t need to do a threat risk assessment or privacy impact assessment for the source code.
Ontario’s Implementation
ODS knew it needed to get information and resources to Ontario residents quickly. But the team knew the tool would be more successful if they took the time to understand and prioritize users’ needs.
Through user interviews, the digital service discovered that some of AHS’s language didn’t translate well to the Ontario context. The language was too focused on a medical diagnosis and ignored what ODS believed to be an important emerging social context. For example, Alberta’s tool included a risk warning for those who had trouble waking up in the morning. However, through user interviews, ODS found that in the midst of a public-health crisis turned economic crisis, users had plenty of reasons unrelated to COVID-19 infection to want to stay in bed.
By talking to Ontario residents, the team radically simplified the tool’s language, both improving its adoption rate and making it more flexible, so that it could respond to multiple, rapidly changing needs brought about by the pandemic.
Selections from Versions of the Ontario Digital Services Self-Assessment Tool
ODS improved the tool by independently adapting the AHS source code to better fit Ontario’s needs. At the time of launch, the ODS tool used 30 percent of AHS’s original code. The changes made focused on providing shorter and more actionable information, explaining coronavirus symptoms, offering examples, and delivering clear guidelines to users.
Ontario’s Diverging Self-Assessment Tool
By October 2020, Ontario’s self-assessment tool had been visited seven million times and had helped Ontario residents complete nearly five million assessments. The tool was widely popular because it quickly became the default place for Ontario residents to find situational information about COVID-19.
ODS released dozens of significant updates and feature changes as public health guidance shifted dramatically in the early months of the pandemic. ODS leveraged new public health information to make sure the self-assessment met a variety of use cases. Now, Ontario residents can receive a robust series of results screens, with eight different possible results based on their symptoms and demographics.
Through user testing and feedback, the digital service found many residents saying, “I’m fine, but I need proof for my employer, or my school, or my business.” This led the team to develop a new feature through which residents can complete the self-assessment as a “healthy” person and get a results screen, with a timestamp, that they can show to their employer to help verify their self-assessed health status.
ODS also saw a need for different types of assessments for different people or use cases. One example is the school and child-care screening tool. Ontario’s COVID-19 protocols treat children differently from the rest of the population; they are assessed for different symptoms and slightly different risk factors. By October 2020, the school screening tool had helped students and teachers complete over one million assessments while providing a more customizable result indicating whether or not it was safe to go to school based on the assessed symptoms.
A similar tool was created for Ontario courthouses. The courthouse self-assessment tool helped keep courthouses operating by asking necessary screening questions, letting residents know whether they could enter a courthouse, and providing a results screen to be shown upon entry.
As of the time of publication, the ODS team continues to work on new self-assessment services and features. What’s next? One goal is adapting the screening tool to support workplaces and businesses, so that Ontario residents can specifically assess whether they can work or shop in person. After this, the team will combine these multiple streams — school and child care, courthouses, workplaces, and businesses — into a single, end-to-end experience with multiple paths and result screens. This iteration will make it easier for residents to understand their results and make safe choices to protect their health.
Ontario’s Historical Relationship with Open-Source
One big advantage ODS enjoyed is that the use of open-source code had long been normalized within the Ontario Public Service. The argument for using and producing open-source code was fought and won well before COVID-19, starting with the team that successfully built Ontario.ca using open-source software.
This norm around open-source code was codified in the Simpler, Faster, Better, Services Act of 2019. This act codifies a commitment to openness, which the ODS team has defined as building with open-source tools and making Ontario code available on GitHub. Now, ODS code is posted on GitHub by default, making it easy for the team to share its adapted version of the self-assessment tool.
This matters because ODS did not have to spend precious time working with lawyers and policymakers to assess whether it could use AHS code. Many of the traditional barriers to reusing or sharing code had already been removed, allowing the team to focus on delivering a rapid solution for Ontarians.
Reflecting on the Partnership
ODS quickly published its version of the self-assessment tool on GitHub with consent from, and credit to, AHS. This move piqued the interest of the AHS team, which could no longer keep up with the incoming requests to share its source code. Inspired by ODS, AHS then also shared its own code on GitHub so that anyone could use it.
AHS’ source code gave ODS the boost it needed to develop a highly effective self-assessment tool in just five days. In turn, ODS’s practice of sharing its code on GitHub helped make the entire project open.
This was able to happen because the AHS team was willing to share its source code informally (i.e., no memorandum of understanding was signed). By making this code easily accessible, AHS enabled ODS to move much faster towards its own version of the solution. The informal relationship also meant ODS could quickly adapt and change the code to fit its local context, without any obligation to get those changes approved or communicate them back. For its part, the AHS team avoided any expectation that it would act as a consultant for other governments’ adaptations of the self-assessment tool.
Outside of the Ontario example, AHS’s original code has now been reused and adapted for other use cases, including by the Saskatchewan team mentioned above. ODS’s changed version of the code has been re-used and adapted, too, including as an eligibility checker for businesses expecting financial help from the government.
AHS has continued to prioritize its own feature enhancements while its willingness to informally share its source code has helped other governments respond quickly to the COVID-19 pandemic.
