What we’ve learned from the Independent Reviewers Panel
Professor Donal O’Donoghue, ex-Chair of Independent Reviewers Panel, registrar of the RCP, consultant renal physician at Salford Royal Hospital and professor of renal medicine at the University of Manchester:
When AI research company DeepMind created a team to work on healthcare, they made an unusual decision — to invite a group of external advisors to form an Independent Review panel to scrutinise their work with the NHS in the UK. For the last three years, a group of independent experts with experience in the NHS, technology, patient engagement and beyond, have been working as part of a unique governance structure in healthcare technology. Mustafa Suleyman should be congratulated on this initiative and for sticking with it even when it felt these efforts were sometimes misrepresented.
The role of the Independent Reviewers (IRs) was to provide an innovative accountability and transparency mechanism for DeepMind Health. The IRs were asked to act in the public interest by conducting independent and impartial public scrutiny of DeepMind Health work, and by holding DeepMind Health to high standards of behaviour and operation. We had an eye to reporting the learnings and our recommendations in a fashion that might have wider system applicability. The twelve principles we published in our 2018 report provides a useful framework for such scrutiny.
In November, DeepMind announced that the team behind its Streams app would be transferring to Google’s new health effort to help scale its work in healthcare internationally. As part of that, the UK-focused DeepMind Health IR panel won’t be the right fit for this international effort. Whilst I understand that rationale I am disappointed that the IR experiment did not have the time to run its course and I am sad to say goodbye to a project I’ve found fascinating.
As part of winding down this process, I wanted to share some details of the lessons we’ve learned along the way. I know many organisations in and around healthcare are considering models for introducing external feedback and oversight to their work, so here’s some things for people to consider if they are thinking about doing this.
Lessons learned along the way:
1. Clearly define and agree the purpose of the panel
When we first met as an Independent Review panel, we realised that different members had slightly different interpretations of the role — ranging from a medical advisory board, to something more like non-executive directors. Once we had established a clearly defined purpose, the panel could focus on addressing real-world issues, and providing appropriate recommendations. We spent a good deal of time developing principles which we felt could apply to any company working in this space. These are set out in our 2018 report alongside the company’s response to how they felt they matched them.
2. Understand commercial and legal constraints
Our panel didn’t sign Non Disclosure Agreements, allowing us free access to information with no constraints. This allowed us to make all of our recommendations and findings public, transparency which we believed was vital. However, we also recognise that this may not be appropriate for other organisations with different legal and commercial sensitivities. Any organisation looking to set up a similar oversight body should carefully consider this from the outset to make the right choice for them.
3. The panel must be independent
From the outset, the independence of this panel was extremely important. A transparent recruitment process helps, and we also decided no one should serve on the panel too long and become ‘institutionalised’. We found that a phased renewal of panel membership, including the Chair, was effective.
4. Select diverse members
Members of the panel should be diverse, both in terms of skill sets relevant to the organisation, but also in social demographics: diverse thinking is just as important as diverse expertise. This meant that the panel were able to mount effective challenges, and were not afraid to ask questions, since their backgrounds were varied and each member brought a different and valued perspective.
5. Be clear on expectations of commitment
Understandably, we found that different members of the review panel were able to commit different amounts of time and work to DeepMind Health. Some members of the panel engaged with particular issues which resulted in them spending a significant amount of time at DeepMind Health — while others attended only the quarterly meetings. Setting expectations on the expected commitment from the outset should prevent significant differences in engagement and ensure that the whole panel contributes and benefits from the experience.
6. Think carefully about operational and financial support
At first, it was felt that the review panel shouldn’t be reimbursed for their time as it may compromise independence. However, in order to ensure a diverse panel, it was agreed that reimbursing panel members for their time would allow us to attract a more diverse membership. Similarly, we thought at first that operational support shouldn’t be provided, but the work of the panel was hindered without a secretariat. So an independent administrator was appointed. But this is not a simple issue and, in other situations, it may not be appropriate to provide any financial or operational support to an oversight body. Each organisation and each oversight body must look at their particular circumstances when determining the right level of support.
7. Consider how you will communicate your work
The requirement to publish an annual report was perhaps the key feature of the arrangement and an important part of what made us independent. The reports were publicly received and reported on in what seemed a pretty polarised way. They could be perceived as an attack by some or a whitewash by others when in reality they were neither.
In the end an arrangement of this sort, being entirely voluntary, is only as good as the company is willing to make of it. Those that see the greatest potential in subjecting themselves to such scrutiny would probably be good corporate citizens without it. While those that would most benefit from the scrutiny are least likely to seek it. Transparency is good in theory but can be painful in practice. Therefore on both sides there needs to be a degree of robustness and courage.
This was a fascinating exploration into how a new governance model could be applied to such an important area such as health. It’s hard to know how this would have developed over the years but the examples given below give a good indication of how we made a tangible difference. What is clear to me is that trust and transparency are of paramount importance in healthcare and I’m keen to see how Google Health, and other providers, deliver this in the future.
Dr Dominic King, Lead, DeepMind Health:
When DeepMind Health was set up over three years ago, we approached a number of independent respected public figures to take part in a new approach to oversight of our work in healthcare. Since then, this group of Independent Reviewers has met formally four times a year and engaged widely with our team and partners to scrutinise our work with the NHS, and publicly issue annual reports about their findings.
As we announced in November, that review structure — which worked for a UK entity primarily focused on finding and developing healthcare solutions with and for the NHS — is not the right structure for a global effort set to work across continents as well as different health services.
However, we are very proud and grateful for the positive impact that the Independent Reviewers had on the work of DeepMind Health while in place. It’s no exaggeration to say that we wouldn’t have achieved so much or be in the position we are without them. I’d like to take a moment to not only thank them, but also to call out some specific examples of the impact they’ve had:
Public and Patient Involvement and Engagement (PPIE)
In its first annual report, the group highlighted the need for greater — and earlier — involvement from patients and the broader public. We embraced patient engagement wholeheartedly from that point on and indeed have become advocates for it. For example, we involved patients in our breast cancer research project from the outset and these patients have been instrumental in changing several aspects of the project.
Human Factors
‘Human factors’ work involves building an understanding of the interactions between humans and other elements of a system, in order to design systems that are resilient to unanticipated events. The IR Panel commissioned a number of independent reports on the Streams app from human factors experts. The impact was that real improvements were made. One clear example was the colour scheme used to signify different alert levels on the app. This might sound a trivial point, but in the UK, there are approximately 3 million people who are red/green colour blind and so the traditional ‘green = fine, red = warning’ system poses issues for them. The alerts in Streams could be life critical, and so an alternative was found to resolve this problem.
Navigating the complexity of the NHS
The IR Panel, and particularly its clinician members, was particularly effective at helping us understand the challenges of working with a nationally funded, locally administered, publicly accountable service.
The IR Panel also played an important role in helping DMH to focus on those issues which were of the greatest concern to the health service and in line with its strategic and clinical priorities. It also helped us to think through some of the implications of bringing technological advances to the NHS as well as steering out thinking about what areas we should focus on, for example patient deterioration in hospital.
These are just a few examples of the positive impact of the Independent Reviewers over the past three years and I want to thank them for their commitment, rigor and insight in carrying out this work. As our projects move into a new phase of its development as part of Google Health effort, I hope that they are also proud of the impact they have had on its development.
