Federated Learning for Medical AI

Unlike conventional machine learning which works by bringing data to the code, Federated Learning brings code to the data. In healthcare where the available data is never enough, Federated Learning can do wonders by enabling AI models to learn on private data without compromising privacy.

Viraj Kulkarni
Dec 12, 2019 · 4 min read
Image for post
Image for post

Conventional machine learning works on the paradigm of bringing the data to the code. The data used to train the models is centralized in a single storage either on a local disk or on the cloud at a data center. Data samples are loaded from this centralized store while training the model.

Many valuable datasets however are highly decentralized in nature. Take, for instance, patient data lying with hospitals. The data in custody of every hospital forms a data island, and each such data island has characteristics that it does not share with data islands from other hospitals. Hence, a model trained on data from one hospital might show poor generalization on data from another hospital. The conventional approach of pooling data from all hospitals and building a single model on it presents several challenges. To begin with, hospitals are uncomfortable sharing patient data, and governments policies such as GDPR, HIPAA, CCPA etc. now make it virtually impossible for multiple organizations to pool their private patient data. Secondly, the data itself is in a state of constant change. This pooling exercise, if it has to be successful, must be repeated frequently, and this is even harder than doing it just once.

What is Federated Learning?

Federated Learning enables machine learning models to learn on private data without moving the data and compromising privacy.

In an FL scenario, there are multiple users — in our case hospitals — with each of them owning a private data store. The data never leaves the hospital’s computers. A central server maintains a global shared model which is sent to all hospitals (in practice, only a random subset of hospitals) where the clients individually update the model based on their private data. Each client then communicates only the model updates back to the server. The server aggregates these updates and makes one final update to the global model. This constitutes one round of federated learning. This process allows the users to reap benefits of the rich private data every user has without the need of centralizing the data in one place and compromising on privacy.

Image for post
Image for post
Image Credit: Towards federated learning at scale: system design, Keith Bonawitz, Hubert Eichner and al., 2019

The single biggest obstacle to developing AI models for medical applications is the lack of sufficient training data. Sufficient data exists, but it is heavily fragmented and owned by different institutions. Federated Learning allows multiple healthcare institutions to share their data to build a global model while still guaranteeing privacy.

Implementation Challenges

  1. Unreliable connectivity: Some clients may not have reliable Internet connectivity and may fail to respond midway through a round.
  2. Lockstep execution across devices with varying availability: Every round proceeds in a stepwise fashion. As the number of participating clients grows, it becomes increasingly important that the implementation is robust across all points of failure.
  3. Limited storage and compute resources on the client: The clients are typically light in storage and compute power. Especially when the clients are mobile devices, it is important that the FL implementation kicks in only when the device is available (it is plugged in and on an unmetered Internet connection)
  4. Different data distributions across devices: Clients typically house highly private data which may vary widely from one client to another. In our example, the data from one hospital may have nothing in common with the data from another hospital.
  5. Security: Inference Attacks on a client or the server where the model weight updates are exploited to reveal properties of the original data or Model Poisoning where misbehaving clients intentionally contaminate the global model need to be detected and thwarted.

Platforms

Conclusion

If you liked the article, check out my other pieces on Medium, follow me on LinkedIn or Twitter, view my personal webpage, or email me at viraj@berkeley.edu.

DeepTek

Leveraging Artificial Intelligence to Revolutionize Radiology

Viraj Kulkarni

Written by

Equal superposition of quantum computing, machine learning, and philosophy.

DeepTek

DeepTek

DeepTek harnesses the latest research in deep learning to develop AI solutions that provide patients faster and more accurate diagnosis, reduce radiologist burden and boost productivity, and ultimately democratize healthcare by reducing costs.

Viraj Kulkarni

Written by

Equal superposition of quantum computing, machine learning, and philosophy.

DeepTek

DeepTek

DeepTek harnesses the latest research in deep learning to develop AI solutions that provide patients faster and more accurate diagnosis, reduce radiologist burden and boost productivity, and ultimately democratize healthcare by reducing costs.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store