Executing Federated Machine Learning Workflows in FeatureCloud

Federated machine learning as a privacy-aware alternative to central data analysis

Machine Learning (ML) has become an essential tool in research and industry. While the models and technology to create accurate prediction models or infer knowledge from these models already exist, the major hurdle in machine learning is still data accessibility. In many fields, such as the healthcare system, data is usually scattered across different institutions. Collecting this data in central storage is connected to high bureaucratic efforts due to stringent privacy regulations such as the General Data Protection Rule (GDPR) in Europe.

Federated Learning (FL) tackles this problem by allowing the training of ML models on geographically distributed data. In FL, the raw data stays at each participating site, while only insensitive model parameters are shared with a central aggregation instance. After the training, each participating site receives the “global” machine learning model trained on the complete dataset. This global model has been shown to provide highly similar accuracy compared to central ML in many areas.

Figure 1: Federated Learning: Each participating institution trains its local ML model (AI) and shares its insensitive model parameters with a central aggregation entity. The central aggregation entity aggregates the model parameters to a global ML model and broadcasts this model to each participant.

FeatureCloud: a unified platform for federated learning in biomedicine and beyond

While many algorithms and approaches for FL already exist, it requires a much more complex infrastructure to train models. FeatureCloud aims to reduce this complexity to a minimum and offers a unified platform to run and develop FL workflows. In this post, we will focus only on how to run federated workflows with FeatureCloud and show the practicability with an easy example workflow.

FeatureCloud offers the whole process needed for FL:

1. Algorithms: The FeatureCloud app store contains various machine-learning algorithms specially designed for FL
2. Project Management: Users can create projects, assemble workflows from different apps, and invite collaborators to participate in an FL workflow
3. Federated Workflows: FeatureCloud provides the infrastructure needed to run federated workflows and ensures that every participant receives their results.

Requirements: The only technical requirements are a web browser (we recommend Google Chrome), the FeatureCloud Controller, and Docker to run it locally.

User Roles: There are two different roles in FeatureCloud: The coordinator is the site that creates the project, assembles the workflow, and invites other participants. The coordinating site is the central aggregation entity in an FL workflow. Participants can join projects via a unique token they received from the coordinator and choose the data they want to contribute to the analysis. Be aware that the coordinator is also a participant at the same time!

In the following sections, we will guide you step-by-step through an example FL workflow that you can try with a friend or collaboration partner using our sample data. In case you have trouble, please check our know-issues page. The listed solutions for the issues usually solve most of the problems.

1. Data

Machine Learning models are only as good as the data. It is, therefore, essential that data is harmonized across all participants. Before each FL workflow, the participants should ensure that their data is aligned, meaning that feature columns are equal. In this example, we will analyze the UCI ML Breast Cancer Wisconsin (Diagnostic) dataset in a federated classification workflow. You can download the data in a three-client scenario here. You can also run the workflow with one or two clients only.

Besides a CSV file containing the actual dataset (rows=samples, columns=features), FeatureCloud expects a config.yml that explains the data for each app in the workflow and specifies parameters. This is the config file that each participant is going to use in our example:

fc_cross_validation:
  input:
    data: "data.csv" # Input file name
    label_column: "target" # Name of the label column
    sep: "," # Seperator of the file
  output:
    train: "train.csv" # Name of the output train set
    test: "test.csv"  # Name of the output test set
    split_dir: "data" # Name of the directory containing the splits
  cross_validation:
    n_splits: 5 # Number of splits
    shuffle: true # Shuffles the data before splitting
    stratify: true # Stratifies the data using the label column
    random_state: 42 # Sets a random seed for reproducibility

fc_normalization:
  input:
    train: "train.csv" # Name of the input file for training
    test: "test.csv" # Name of the input file for testing
  output:
    train: "train_norm.csv" # Name of the resulting normalized input file for training
    test: "test_norm.csv" # Name of the resulting normalized input file for testing
  format:
    sep: "," # Seperator of the file
    label: "target" # Name of the label column
  split:
    mode: "directory" # 'directory' if CV app was used, else 'file'.
    dir: "data" # 'split_dir' of the CV app if CV was used,else '.'
  normalization: "variance" # type of normalization

fc_logistic_regression:
  input:
    train: "train_norm.csv" # Name of the input file for training
    test: "test_norm.csv" # Name of the input file for testing
  output:
    pred: "pred.csv" # Output file containing the predictions of the test file
    proba: "proba.csv" # Output file containing the predicted probabilities of the test file
    test: "y_true.csv" # Name of the input file for testing
  format:
    sep: "," # Seperator of the file
    label: "target" # Name of the label column
  split:
    mode: "directory" # 'directory' if CV app was used, else 'file'.
    dir: "data" # 'split_dir' of the CV app if CV was used,else '.'
  algo:
    max_iterations: 100 # max number of iterations for the logistic regression

fc_classification_evaluation:
  input:
    y_true: "y_true.csv" # File containing the true labels of the test set
    y_pred: "pred.csv" # File containing the predicted labels of the test set
  format:
    sep: "," # Seperator of the y_true and y_pred files
  split:
    mode: "directory" # 'directory' if CV app was used, else 'file'.
    dir: "data" # 'split_dir' of the CV app if CV was used,else '.'

2. Choose your workflow algorithms

A workflow in FeatureCloud consists of several “apps”. An “app” is a certain kind of algorithm, for example, performing a federated logistic regression or a federated normalization. In our example, we want a workflow that performs normalization and logistic regression inside a 5-fold cross-validation (CV) and finally evaluates the CV. Additionally, there is the possibility of using a predefined workflow to create a project directly. For that, select the “Workflow templates” tab and press the green button of the appropriate workflow.

Figure 2: The FeatureCloud App Store

Before being able to add apps to your library, you need to create a user account on FeatureCloud. After that, you should also create a site. A site is an entity that identifies you in a federated workflow. by that, individual users are not visible to other participants. After that, all published apps are visible and searchable through the FeatureCloud app store (Figure 2). As many apps exist in the app store, users should add their favorite apps to their library. This can be done by clicking on a particular app and pressing the +add button. (Figure 3).

Figure 3: App details page.

All apps that should be executed in the workflow need to be added to the personal library first. Each app provides details about the algorithm and its usage configuration (config.yml file) on its details page.

2. Initiate a FeatureCloud project

To create a project, the coordinator goes to the projects page and presses the “Create” button. Enter the project name and a detailed description, and press the “Add” button. Afterward, you assemble the workflow based on the apps in your library and press “Finalize” when ready. Figure 4 shows the assembled workflow consisting of a Cross-Validation, Normalization, Logistic Regression, and Evaluation (Classification) app. Pay attention: after pressing the “Finalize” button, you cannot change the workflow anymore.

Figure 4: Assembling a workflow in FeatureCloud.

3. Invite collaborators or join a project

Now that the workflow is assembled, you can see the project details page. In the top section, you can see some details about the project. You see the workflow overview in the middle section; in the bottom area, you can invite other collaborators. Press the “+Token” button to create a new token. This token can then be shared with other collaborators.

Figure 5: Inviting collaborators to a FeatureCloud workflow.

Other collaborators can quickly join the project by pressing the “Join” button on the projects page. Here, you can enter the received token and press the “View” button to see an overview of the project. Finally, press the “Join” button to join the project.

4. Run the FeatureCloud controller

To run the FeautreCloud controller, first, you need to run Docker on your computer/server. Afterward, you can use the FeatureCloud CLI to run the controller:

pip install featurecloud
featurecloud controller start

An alternative way is to download the “start controller” script (macOS/Linux, Windows) from our website and run it with a double-click on windows or macOS/Linux using the following command:

sh start_controller.sh

5. Choose your local data and run the workflow

After the FeatureCloud controller runs, you can press the “Start” button and upload the data using the “Add file…” button. After adding the config file and CSV file, press the “Continue workflow” button and confirm with the “Yes” button to start the workflow. When all participants are ready, the workflow is started automatically.

Figure 6: Workflow progress overview.

6. Check your global results

After the workflow has finished, every participant can download the logs and results for each intermediate app and the final results of the workflow by downloading the results of the last app in the workflow.

Conclusion

FeatureCloud allows easy execution of FL workflows on geographically distributed data. You can invite other participants by sharing tokens and analyzing your data. Check it out now at featurecloud.ai. If you have any questions feel free to contact us on our Slack channel.