Did we really get people talking about AI?

Reflections on running AI for Healthcare: Equipping the Workforce for Digital Transformation

AI for Healthcare: Equipping the Workforce for Digital Transformation is one of the many educational and training programmes that The University of Manchester is delivering to prepare the healthcare workforce to deliver the digital future. The course was created in response to The Topol Review published in February 2019. The Topol Review explores how to prepare the healthcare workforce for a digital future through education and training and to make the most of new and innovative technologies that will improve and transform the NHS. The report also recognises the need to support the workforce’s digital skills and data literacy to prepare for that digital future. Therefore, in collaboration with Health Education England (HEE), we developed a course that looks at how artificial intelligence (AI) is transforming healthcare and how best to equip the workforce for this digital transformation. The online course gave us the opportunity to go beyond the usual silos so we wanted to take advantage of this openness and encourage everyone to join the debate. As well as healthcare professionals, we also wanted to hear from the public, patients, academia and commercial fields such as pharmaceutical and IT companies.

This evaluation will reflect on Run 1 (February — September 2020) of AI for Healthcare: Equipping the Workforce for Digital Transformation. Run 2 will commence 1 February 2021.

What were we trying to achieve?

We wanted to:

• open up the debate on AI to the workforce
• explore how healthcare professionals can realise their own potential in a digital world
• open up the potential challenges and benefits of AI
• de-myth the current ‘Black Box’ connotations surrounding AI

A key aim was to get rid of the jargon surrounding AI and to give clarity and build a shared understanding of the issues as well as offer an opportunity to practice these new skills.

Why FutureLearn?

FutureLearn is an online platform that delivers flexible online courses to learners anywhere in the world. There are lots of similar platforms to FutureLearn online, however, FutureLearn is built on the pedagogy of social learning, where conversation and discussion are at the centre of its design. Each course consists of steps and each step has the in-built functionality to comment directly on the step, thereby showing the conversation alongside the content and placing the conversation within the context of the content. This is a powerful way of sharing and debating ideas with fellow learners as well as an effective mechanism for listening to all views on the potential impact, benefits and challenges of AI in healthcare in the UK and effectively moving beyond the silos mentioned earlier.

So who came to the course and what did they talk about?

4,280 Enrolments

• The majority of learners were from the UK (2,278) and after that, we had a top ten enrolments from all five continents.
• Of those, 31% were classed as active social enrolments, where the learner has viewed at least 2 steps and commented on at least 1 step.

4980 Comments

• We received 4980 comments on over 80 steps. This equated to on average, over 60 comments per step.
• Of all the comments, only 2 were marked as spam or libel demonstrating the power of peer learning and the value of the conversation.
• Of all the comments posted, nearly 3% got 5 likes or more.

93% (of 201) said they gained new knowledge or skills

• Of the 186 people who had gained new knowledge, 73 had applied what they had learned in the course (36%).

70% (of 201) said they shared their learning

• Importantly, 70% of everyone who answered this question had shared their learning with others in some way, which demonstrates the impact of the course beyond the course run.

Over 300 wanted to carry on the conversation after the course had ended

• 331 people indicated that they wanted to know more about AI at the end of course survey.
• Suggestions included more short videos, case studies, glossaries and wikis as well as 20% wanting a space to share thoughts.

What worked well?

Looking at the surveys on the platform, it looked like we had met the learners' needs. During the course, 83% of learners who participated in the weekly sentiment survey left positive feedback. In the post-course survey, 89% said the course was better (37%) or had met their expectations (52%) (n=210).

However, five key things stood out that we mention here:

1. Collaboration with Health Education England

Health Education England (HEE) supports the delivery of healthcare and health improvement to the patients and public of England through the education of its workforce. Through collaborating with Patrick Mitchell (Director of Innovation and Transformation) and other colleagues at HEE, we could build up insight into skills needed for the workforce. Importantly we could focus our course around real learner needs and incorporate existing materials to embed into the course, such as links to the Digital Literacy Framework.

2. Design approach — Conversational framework

The course was designed using the ABC Learning Design (ABC LD) curriculum development approach, which is based on Laurillard’s conversational framework. The hands-on 90-minute workshop results in a visual storyboard that is generated through group dialogue and input, which is perfect for a multidisciplinary team creating content. The ABC LD workshop allows the team to make sure that there is a balance of resources and activities across the online course that meet the course learning outcomes.

3. Real-world case studies

One of the key learning activities in the course is the authentic use of case studies to learn about the current uses of AI in healthcare. The case studies look at different ways projects have used an AI approach to diagnose diseases such as cancer, osteoporosis and bipolar as well as the challenges and opportunities of using new and innovative technology to support the workforce in treating these diseases. In the post-course survey, 54.1% of the respondents (n=177) valued the case studies and wanted to see more added to the course.

4. Multidisciplinary team creating content

This course is a collaboration between The University of Manchester and HEE with educators and learning technologists coming together to design, build and run the course. This approach to co-creation resulted in a richer, more inclusive course that benefited from the skills and expertise of the wider team.

5. High completion rate

Compared to other courses, AI in Healthcare had high completion rates:

• 19.0% of learners completed ≥50% steps (n=555)
• 12.2% of learners completed ≥90% steps (n=357)

It’s difficult to say why for certain that the completion rate for the run was so high. However, it is likely that the multidisciplinary approach to design contributed to this as we were better able to understand the training needs of healthcare professionals. The run was also linked to the Topol Review and was, therefore, a topical course. Additionally, we had designed the course so that some of the tests were only available if you upgraded. This suggests that some learners saw enough value in the course to upgrade.

What did learners want to learn next?

The post-course survey shows that learners are eager to learn more about AI. For example, some learners wanted to know more about the technical side of AI (algorithms) as well as more practical examples of how AI is being applied in a real-world context. This provides useful feedback for future developments and collaborations with other schools, divisions and external partners to meet future learners’ needs. This is particularly useful for emerging topics, such as AI, where developments are constantly evolving. For example, in the next run, we are including additional topics on how AI has been used in the fight against Covid-19. However, this needs to be balanced with feedback from learners who found that some aspects of the course were too technical. This broad range of feedback can be expected in a course aimed at the healthcare workforce as well as the general public.

What will we change for the next run?

AI for Healthcare: Equipping the Workforce for Digital Transformation will start its next run in February 2021. This will run on FutureLearn’s Always Available mode. This means that the course will always be open for anyone to join at any time. Although there will be no facilitation, FutureLearn’s platform, which is built to support the pedagogy of social learning, encourages conversation on each step, where learners can read, like, reply and bookmark each others’ comments and ultimately learn from one another. We are looking into linking the course to a professional body so that clinicians can get CPD credits for completing the course. We are also looking at creating a space to carry on the conversation after the course has ended to link up healthcare professionals with an interest in this field. This will help us to understand the impact and results of the learning on this course more broadly.

When is the next run?

The next run will take place on 1 February 2021. Why not join us?

Authors

Fran Hooley

Fran Hooley is a Lecturer in Technology Enhanced Learning at the University of Manchester. Fran is Deputy Director for the PG Cert in Clinical Bioinformatics and has over 15 years’ experience of technology enhanced learning roles in Higher Education and the private sector. Fran led the development of the new FutureLearn course Artificial Intelligence for Healthcare: Equipping the Workforce for Digital Transformation

Cath Wasiuk

Cath Wasiuk is a Learning Technologist at The University of Manchester in the i3HS Hub. She has over 10 years of e-learning experience within various roles across the Higher Education sector in the UK. In her current role, she supports healthcare professionals to develop reusable (open, CPD and credit-bearing resources), innovative, learner-centric and multi-discipline learning materials.