Cutting ‘edge’ — the technology powering our AI Briefcase

How we made our AI Briefcase a reality for disconnected, secure and edge scenarios. Spiral Data’s Chief Technical Officer, Chris Jansz, details the process to date.

Spiral Data’s IoT to AI/ML platform has matured in the cloud through several years of feature development, user experience optimisation and processing complexity enhancements. It is helping to solve a variety of complex problems using Big Data and data science across large and complex physical assets.

The platform is now at a stage where we are diverging its roadmap to create both a Cloud and Offline version. The latter is known as our ‘AI Briefcase’. In this article I will expand on our process and the technical details that matter.

Point of origin

The need for a disconnected edge device with IOT to AI capability stems from client challenges and their requirements, which are focused on three clear goals from the technology:

• Provide like-for-like AI capabilities (MLOps) to a cloud version, but within a secure environment that does not require an internet connection
• Maintain full software development lifecycle capabilities (DevOps); including IoT data ingestion, compute without latency compromise and significant storage
• Allow for site-specific deployments of the AI Briefcase that includes harsh and challenging environments

Our vision was ‘zero compromise at the edge’, to meet the challenging use cases and empower the teams involved.

Modular MLOps

This diagram illustrates our AI Briefcases’ working stack, which provides end-to-end modules from data ingest to analytics.

Our modules are thus:

Ingest Module

• REST API: Fetch data securely and efficiently from RESTful endpoints.
• MQTT: Gather real-time data using the MQTT protocol.
• Flat Files: Easily read and process data from various flat file formats (JSON, Text, csv, etc).
• Relational Databases: Connect and extract data from any database (MSSQL, MySQL, PostgreSQL, Oracle, etc).
• REST API Endpoint — Where external applications and devices can use to push data into the AI Briefcase Data-lake

Data Lake Module

• Effortlessly store structured, semi-structured, and unstructured data for analytics and machine learning. Scale data lake to accommodate massive data sets with ease.

AIML Workbench Module

• Explore data using a wide array of data exploration and visualization tools such as Jupyter, Bokeh, Pandas, etc.
• Discover valuable insights through descriptive and diagnostic analytics.
• Develop and Train New ML Models: Utilise popular IDE such as VSCode + Jupyter to build and train machine learning models.
• Leverage a wide range of algorithms and libraries to experiment with data.
• Deploy trained ML models into your production environment.
• Use the deployed model inference in applications deployed on the platform as well as your already implemented internal applications

Full-Stack Workbench Module

• Develop, Test, and Deploy Apps using popular IDE such as VSCode
• Create scalable, and secure applications which are needs dependent
• Develop in languages such as Python, NodeJS, and JavaScript
• Offline Mode — DevOps at the edge

Analytics Module

• UI/UX such as Dashboards, Reports, etc
• REST API for accessing inferences from deployed ML Models

Support Module

• Manage application components (Apps, ML Models..etc) and deploy applications with ease to the built-in docker swarm (managed by Portainer)
• CI/CD and Version Control using GitLab CE
• Docker Registry — pull all commonly used images from the cloud one time and start using from within the local environment

All software tools and services used are OpenSource. With the cloud version of the IoT to AIML platform utilising a number of web services online, it was necessary to integrate equivalents for offline functionality to ensure an equivalent feature set.

For example, the Cloud version uses AWS’ Elastic Container Service for hosting of the front end of the application. To facilitate the same functionality offline, we switched to Docker-based containerisation of the front end, with no feature limitations.

You will note the Optional Full-Stack Module, which offers versatility to a team at the edge point. Depending on their capability the team can develop, test and deploy their own custom apps through the platform, extending the use case potential while maintaining the same hardware and existing ML/AI feature set.

Security and data ingestion considerations

The AI Briefcase storage is encrypted by default which means everything stored is encrypted, from the Virtual Servers, Containers, Data-lake etc. When the AI Briefcase is powered on each-time it needs to be unlocked using a key and manifest file before it can be access or used. In addition, only hardwire connection is allowed, no WiFi.

Hardware solution suitable for challenging environments

The briefcase hardware is the AWS Snowball Edge device for disconnected Cloud capabilities. These are highly proven devices, used by the US military for mission-critical computing and data storage. We particularly like the balance of portability (the devices can be placed in check-in luggage on a flight) and power (20+TB of storage, 104vCPU and 400GB of usable RAM) which means developing a data lake at edge locations is achievable.

The Snowball Edge provides the following benefits for teams using AI Briefcase:

Fast: Network adapters with transfer speeds of up to 100 Gbit

Compute: Unparalleled local storage and computational power

Encryption: Enforced, protecting data at rest and in physical transit with 256-bit encryption keys

Model output: Snowball Edge devices have endpoints available, enabling programmatic use cases for AI/ML and analytics models

Reliability: Snowball Edge devices can be clustered for local storage and compute jobs to achieve data durability across 5–10 devices to locally grow or shrink storage on demand

Secure: Once the job of specific data transmission is completed, AWS ensures that secure data erasure is performed on Snowball

Technical updates and feature releases

The underlying OS-level updates will be released by AWS and Spiral Data will carry-out the updates, while the Platform itself is maintained and updated by Spiral Data.

The result — AI Briefcase

In removing the dependencies of connectivity and centralisation of data we have created a different paradigm for how AI is used as a decision advantage.

Our AI Briefcase can be deployed as close as necessary to where data is created in order to deliver intelligent, real-time agility, while being disconnected from the internet.

Read more on AI Briefcase use cases.

Contact Spiral Data to discuss deploying AI Briefcase with your teams.