Run a FeatureCloud app in the testbed

Federated Learning is the collaborative training of models on private data without moving or even invading the data owners’ privacy to countervail hurdles of centralized local training. With the emergence of edge technologies and mobile devices in the last decades and the prevalence of such technologies throughout the world, data can be found everywhere, which broadens the chances of tackling data scarcity challenges. With such opportunities come responsibilities. Data privacy was almost always one of the biggest concerns Machine Learning and data science community, and in recent years with enacting laws like GDPR by the European Union, the bars are higher than ever, and developers are obliged to address privacy concerns in their applications. Featurecloud is one of the first comprehensive Federated platforms that address different general challenges in FL while it is a cross-silo platform. This story explains how the FeatureCloud testbed can be used to execute apps.

App execution in FC

There are two ways to execute apps in the FeatureCloud platform:

• Workflow: FC workflow is designed to support running a chain of apps in a row, where data owners (Hospitals, institutions, etc.) can collaborate in a federated manner while respecting the data privacy laws to conduct data analysis and/or machine learning tasks.
• Testbed: On the other hand, developers, who want to provide the community with a specific app to tackle federated learning problems, need to test their apps to be sure it works as a stand-alone app. For that purpose, FC provides developers with a testbed that simplifies the app development process.

Prerequisite

1. Sign-in into featurecloud.ai
2. run FC controller
3. Built FC app

FeatureCloud Testbed

In this story, we are focused on running a stand-alone app in the FeatureCloud testbed. Before running an app, developers should build it and put the clients’ data and generic files, with the acceptable format, in the FC controller’s data directory. FeaureCloud testbed is available in https://featurecloud.ai/development/test where developers can create tests by providing the app information. Testbed can contain multiple test-runs simultaneously, while each test-run can be in one of initializing, running, or finished states at a given time.

Figure 1. Testing

App image name

All FC apps are dockerized, and their images should be pushed into the FC docker repository. Consequently, each app image name starts with the address of the docker repository address featurecloud.ai/fc- and ends with a unique name for the app in the repository. It should be available locally to run an FC app in the testbed. Here, we use flimma as an FC app to exemplify the testbed.

Testbed configuration

Currently, the FC testbed supports ten clients in Front-end configuration; however, technically, it is possible to use the testbed for running an app for more than ten clients is possible in a test-workflow, which is another story! Practically, each client should have its own data, provided in a specific sub-directory in the controller’s data directory(which is in the same path that you run the controller in). Here, the clients’ data are in:

/home/username/data/

directory, while putting clients’ data in c1 and c2 sub-directories of flimma.
Some apps require all clients to access the same generic data, which should be provided in a separate sub-directory. For instance, most apps require some config.ymlfile to run, which is generic among clients.

Testbed Communication settings

Federated apps require communication rounds now and then, and it is upon developers to decide whether they want to use the internet or local channels for that purpose. Meanwhile, the FC controller needs to check periodically for possible communication requests and conduct them if there are some. Accordingly, developers can decide about query intervals.

Results directory

Like input data, once FC apps produce the output, it should be stored in the FC controller’s data directory. Again, it is up to developers about the convenient sub-directory.

Running the testbed

Once the testbed configuration is completed, after pressing the start button, app execution starts, and there will be panels to inform developers of the status or outputs of the app instance for each client. Also, there are tools in place to manage the testbed.

Figure 2. Testing: panels

Managing the test-run

FC testbed provides three panels, configuration, overview, and traffic, to show the necessary information about the set-run to users. Based on such information as the app’s status or operational or logical states, users can decide either to continue with app execution or to interrupt it in the test-run. To interrupt the test-run, users can stop the test run or go back to the list of created apps and directly delete it.
If the user stops the test run, it kills the app docker container; however, the test run will still be available in the controller. After the test-run is stopped, users can delete it in the testbed tests list.

Another alternative to interrupt the test-run is directly deleting it. Regardless of the app state, the test-run will be first stopped and then deleted from the controller.

configuration

The configuration panel shows the testbed config details.

Figure 3. Testing: configuration panel

Overview

In the overview panel, users can find out about different clients’ statuses:

• Container: the ID that can be used to interact with the Docker engine.
• State: Logical state of the app
• Message: Specific message from the app, at the time, to the end-user.
• Progress bar: progress of the app.
• App’s Front-end access: some apps provide a specific front-end for end-users, which can be used for interaction with end-users or be used for logging, reporting, or visualization purposes.
• General log sub-panel: observing the app’s live logs during the execution or downloading them afterward.
• Downloading results.

Figure 4. Testing: overview panel

Traffic

During the app execution, the FC testbed provides traffic information for each communication in the platform. Communications in the testbed are recorded based on the chronological order, while each record includes:

• client ID: The ID is assigned by the controller and shows the data sender.
• Time: The time that communication was completed, based on the coordinator’s machine time.
• Size: data size in bytes.
• Type: The communications in the FC platform are categorized into Gather, Broadcast, or SMPC. The detail is another story!

Figure 5. Testing: traffic panel