Swash
Published in

Swash

sCompute Architecture

With sCompute, Swash provides a way for data scientists to perform computations on the data without needing to purchase it. The data itself remains private and is not sold or moved.

General architecture

You can see an overview of sCompute general architecture below. This image shows how the main components of this system communicate with each other to conduct computation based on predefined sCompute requirements.

Authentication layer

Registration, authorisation, authentication and session management are the most important services of this layer.

Registration flow

Login flow

Session management

Stateless interaction

The client’s valid Ethereum wallet could also be used for authentication. It means that a client’s unregistered wallet address could be used for authentication and interaction purposes. However, the signature generation method mentioned in the login flow is completed for each interaction.

Pipeline management layer

This is the main component of sCompute. As previously mentioned, sCompute allows sClients, such as data scientists and sApps, to perform computations on raw Swash data. The pipeline management layer is responsible for providing this capability by controlling access to data and accounting. Let’s start by taking a look at the pipeline concept.

Swash pipeline

sPipeline is a work flow of computation. This concept consists of a series of interconnected steps that are defined as follows.

  • Training
  • Evaluation
  • Prediction

Access control module

This module defines users’ activities based on their allocated resources. Running a pipeline requires some resources such as machines with CPU power for processing, and storage for saving results and log files. Therefore, the access control module communicates with the billing module and resource management layer to define user access based on dedicated resources to their own computation operation.

Accounting module

This module checks the user’s limitation and controls the user’s access based on these limitations. The accounting module should be able to answer a question about the current user:

Running pipeline flow

Resource management layer

As mentioned previously, dedicating resources to each pipeline and releasing them at the end is the most prominent responsibility of this layer. The resource management layer helps the pricing system to calculate the price of requested resources. Also, limiting usage and extending resources could be achievable via this layer.

Limit control layer

One of the biggest concerns in our system is controlling the output results. This layer helps us to apply some limitations over the output results. This layer is also responsible for filtering and sanitisation. For example, generating logs while the pipeline execution conducts other outputs. This layer controls the logs to prevent information leakage. Removing and replacing important information, such as IDs, paths, IPs etc, are included as part of this prevention action.

About Swash

At Swash, if it’s your data, it’s your income.

  • Businesses access high-quality, zero-party data in a sustainable and compliant way.
  • Developers set up and build systems within a collaborative development framework with ease.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store