AI in Global Health: The View from the Front Lines
By Azra Ismail & Neha Kumar, summarizing our CHI 2021 paper.
The streetlight effect is a widely recognized phenomenon, dating back to a fable by Mulla Nasruddin, Turkish poet/philosopher. It alludes to a type of observational bias, where one looks for things where they are easiest to find, such as keys under the streetlight, not where they are likely to be found—perhaps in the dark. Mark Moritz has also written about this being a tendency with big data, where he says, “Without checking facts on the ground, researchers may fool themselves into thinking that their big data models accurately represent the world they aim to study.”
It is this effect that we uncover also in our paper analyzing the advancements of AI in global health. We started working on this project, in the summer of 2019, following repeat encounters with AI enthusiasts (in/from the Global South), pushing to deploy data-driven approaches with what we considered insufficient regard for realities on the ground, much as Moritz alludes to the problem with big data. On the one hand we found ourselves in conversations that revealed an unfettered and unwarranted enthusiasm about the impact AI could have in the world of frontline health, and on the other we were shaped by our (collective) ethnographic research since 2014, introducing us to the everyday struggles of health workers in highly risky and physically taxing jobs, their unaddressed demands for better pay and working conditions, and their evolving perceptions and use of mobile technologies.
The field of Human-Computer Interaction (HCI)and frontline health have not intersected very much thus far, but this intersection becomes extremely relevant when we consider the COVID-19 pandemic that has mercilessly and impartially turned our lives upside down. Efforts to advance AI in frontline health have only multiplied in this last year, just as it has become more important than ever to address concerns and skepticism around use of such AI (and data-driven approaches, more broadly), e.g., algorithms that determine who gets vaccinated, to cough-based diagnostic tools, to disease forecasting, and more. Our paper argues for a much more careful and directed approach when applying AI in frontline health. We recognize that outright refusal of such efforts brings its own complications and challenges. But we do make the case that resources available for frontline health (and development or social good, more generally) are constrained to begin with, and it could be prudent for said AI enthusiasts to direct their attention to ensuring that the critical human infrastructure we have on the front lines is honored and preserved.
To gain a comprehensive understanding of the current AI landscape in global health, we did an extensive review of 347 academic papers, articles, white papers, and other grey literature to identify applications being proposed, predominantly in the Global South. We then considered these applications in light of ethnographic data we collected over three years with Accredited Social Health Activists (or ASHA workers, engaged on the front lines) in an underserved, Muslim-majority region of Delhi that has been determined to be at high risk of disease by the state government.
Our analysis revealed that most applications of AI (over 70%) were centered around disease surveillance and forecasting, or diagnosis and screening. Other applications included risk assessment, chatbots, behavior prediction, data quality control, data digitization, and some others. Despite our concerns around Western algorithmic colonization, we found that the development of these applications was largely led by Global South researchers, but rarely in engagement with other local non-tech actors. This led to several gaps and assumptions about local “on-the-ground” contexts that we highlight through our ethnographic data.
Some gaps that our data highlights were consideration for the broader political and organizational context these technologies are situated in, acknowledgement of the existence of competing knowledges such as complementary and alternative medicine (CAM) and religious practices, the complexity of patient choice and agency in seeking care, and the many invisible workflows in healthcare. We centered our discussion of invisible workflows around frontline health, where ASHAs are frequently called upon for quick response and to fill in the gaps in the healthcare system (as happened during the pandemic). They also spend time planning their workflows and balancing family commitments, attending to the sociocultural dimensions of health particularly when working in patriarchal societies, conducting extensive (frequently opaque) data work, supporting each other emotionally and in filling knowledge gaps, and advocating for system reform.
Additionally, the region that we studied has been the center of protests in India against the Citizenship Amendment Act (CAA) and the National Register of Citizens (NRC), motivated by concerns that they would strip many of citizenship status on the basis of religion, and disproportionately impact lower socioeconomic groups who may not have documents required to demonstrate citizenship. This leads to salient ethical concerns around data collection by the government in a Muslim-underserved region with a high migrant population.
So where do we go from here?
Our paper lays down considerations for the design of AI systems that target social good more broadly, situating our analysis in the context of frontline health work and workers. We draw on literature in Human-Computer Interaction for Development (HCI4D), post-development critique, and transnational feminist theory. We summarize lessons for AI developers below — necessarily simplified for length — and encourage you to read the paper for a deeper analysis.
- Foreground user agency, e.g. expect targeted “users” to not use these systems at all, use them in unexpected ways, or work around the systems altogether.
- Question the unit of scale, e.g. focus on hyperlocal deployments, and use AI to maximize use of fixed time of health workers on care work rather than introducing new workflows.
- Move towards a pluralist worldview by supporting self-determination by communities on the development and use of AI.
- Develop sustainable interventions that check extraction of resources, e.g. streamline and leverage existing data flows rather than introducing new ones, build systems that require less data, be directed about collecting data when required, compute less on the devices of the underserved, and build for lean computation and power consumption.
- Engage with diverse disciplines by reckoning with the source of discomfort and choosing to engage meaningfully and not only performatively, when ontological and epistemological differences arise.
- Translate systems across contexts — focusing on common struggles (such as the global struggle of frontline health workers for better pay and working conditions) — including from the South to the North.
We dedicate this paper to the frontline health workers who put their lives at risk everyday to enable healthier futures for their communities, and who have been on the front lines throughout the ongoing COVID-19 pandemic. Our hope is that this work will lead to productive dialogue across researchers and practitioners in HCI, ICTD, AI, and Global Health.
Azra Ismail and Neha Kumar. 2021. AI in Global Health: The View from the Front Lines. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ‘21). Association for Computing Machinery, New York, NY, USA.
