Federated Learning On Raspberry Pi

3 min readAug 15, 2019

Federated learning

First proposed by Google, federated learning is described as an alternative to centralized artificial intelligence (AI) training where a shared universal model is trained under the coordination of a central server, from a combination of participating devices. The various devices in this model can add to the model’s training and knowledge and simultaneously keep most of the data in the device. Figure 1 below shows a general outline of how federated learning works.

The approach to federated learning is described by Google in four straightforward bullet points:

· A subgroup of existing clients is chosen, each downloading the present model.

· Based on their local data, each client in the subgroup computes an updated model.

· The central server receives the model updates from the chosen clients.

· The models are then aggregated by the server to develop an updated global model.

Figure 1: A general representation of how federated learning works. A. An individual’s mobile personalizes the model locally, based on the user’s usage. B. Several user’s updates are aggregated (usually by averaging). C. A consensus change to the shared model is constructed and the process is repeated. (Image is adapted from Google AI Blog: Federated Learning: Collaborative Machine Learning without Centralized Training Data)

Raspberry Pi

A Raspberry Pi (RPi) is a low-cost basic computer originally aimed at educating students in computing and increasing the accessibility for computing education. Today, RPis are globally used to learn programming languages, build hardware projects, perform home automation and even used in industrial applications.

The RPi runs Linux and carries a set of general purposed input/ output (GPIO) pins that enable us to control electronic components for physical computing and explore the Internet of Things (IoT). As shown in Figure 2 below, the RPi is contained on a single circuit board that features ports for HDMI, USB, Ethernet, Composite Video, Audio, Power and SD Card.

Figure 2: Raspberry Pi 3 B+ with all major components labeled. (Image adapted from SparkFun Electronics)

Project

This project aims to use federated learning to assemble a model in the RPi which allows the model to be personalized and trained locally and removes the need to send the data to the central server.

The first part of the project involves installing and setting up PySyft, a python library for private deep learning, on an RPi.

The second part involves training a recurrent neural network (RNN) on RPis for text categorization through federated learning. In other words, our neural network will be trained to classify an individual’s surname to its most probable origin language.

Federated learning can process several batches of data from a cluster of RPis while avoiding sharing any data between the RPis and the server. A study conducted in 2018 involved arranging a set of six RPis to communicate with each other and collaboratively train a neural network to recognize the Fashion Mnist data set. Performance tests with RPis generated results which demonstrated that federated averaging outperforms single-node training due to 100x reduction in bandwidth requirement and 4x decrease in training time (compared to distributed training). This is because a single RPi is deficient in the processing power to effectively process the entire data-set. Through distributing the data-set over six RPis the workload is decreased for each RPi and the training is sped up.

All in all, federated learning helps bring about smarter models, reduced lower power, lower latency all together while maintaining privacy. Another immediate advantage of federated learning is that along with updating the shared model, the improved model sent back to the mobile is ready to use immediately, powering experiences based on the individual's usage.