Propeller: The Future of WebAssembly Workload Management

How do you seamlessly run WebAssembly (Wasm) workloads across cloud, edge, and IoT devices while ensuring security, speed, and scalability? Propeller, built by Abstract Machines, is a next-generation Wasm orchestrator that simplifies deployment and management across diverse environments. Powered by SuperMQ, our cutting-edge secure messaging platform, Propeller enables fast, flexible, and secure execution of Wasm applications, bridging the gap between cloud-native and embedded systems. Whether you’re optimizing performance in data centers or running lightweight workloads at the edge, Abstract Machines delivers the technology to make Wasm orchestration effortless and scalable.

The way we run workloads in the cloud and on edge devices is constantly changing, and making that process efficient, portable, and secure is more important than ever. WebAssembly (Wasm) is gaining traction because it’s lightweight, fast, and secure, but managing Wasm workloads across different environments — whether powerful cloud servers or limited edge devices — is still a challenge. That’s where Propeller comes in. It’s a distributed WebAssembly orchestrator built to simplify deployment and management, making it easier to run Wasm applications anywhere. It extends from high-performance cloud servers to resource-constrained edge devices, integrating flexibility, security, and performance.

The Challenge: Orchestrating Wasm Across Heterogeneous Environments

Kubernetes has emerged as the de facto standard for container orchestration, its architecture is not inherently optimized for orchestrating Wasm workloads across diverse platforms due to differences in execution models, resource management, and runtime integration.

SpinKube represents a significant advancement in integrating Wasm into Kubernetes (K8s), refining previous approaches such as WOK and Krustlet. It leverages the runwasi interface to enable Wasm execution within containerd. The above integration facilitates seamless deployment alongside conventional containers, yielding substantial performance enhancements. By maintaining full compatibility with existing Kubernetes tools and workflows, SpinKube provides a scalable, cost-effective, and developer-friendly solution for orchestrating Wasm workloads within Kubernetes environments. However, SpinKube does not fully address the need for a comprehensive and adaptable orchestration framework capable of managing Wasm workloads across a broader range of computational platforms. For example, SpinKube requires the relatively complex and resource-hungry Kubernetes control plane, and does not work on non-Linux platforms such as the Zephyr real time operating system.

In contrast, Mechanoid, an open-source framework, is specifically designed for building and executing WebAssembly applications on embedded systems and IoT devices. Its primary objective is to facilitate the development of secure and extensible applications while leveraging the latest advancements in both WebAssembly and embedded computing. However, Mechanoid cannot orchestrate Wasm workloads across heterogeneous platforms.

Propeller addresses the above limitation by offering a unified and adaptable framework for managing and deploying Wasm workloads efficiently across heterogeneous platforms. It is designed to bridge the gaps by providing a unified, flexible, and scalable approach to Wasm workload orchestration. While it is not intended as a direct replacement for Kubernetes, it functions as a complementary solution that enhances the management and deployment of Wasm applications across diverse computational infrastructures.

Key Features of Propeller

Propeller’s architecture is built around several core features that make it a powerful tool for Wasm workload orchestration:

1. Cloud-Edge Orchestration: Propeller enables seamless deployment and management of Wasm workloads across cloud and edge devices. High-performance cloud infrastructures handle large-scale computations, while resource-constrained microcontrollers at the edge execute lightweight tasks efficiently.
2. Fast Boot Times: With the fast boot times of Wasm, workloads achieve near-instant execution, making Propeller particularly well-suited for latency-sensitive applications such as real-time analytics and interactive services.
3. OCI Registry Integration: Propeller integrates with OCI-compliant registries, allowing for efficient storage, retrieval, and distribution of Wasm workloads. This ensures interoperability with existing containerized ecosystems, streamlining deployment processes.
4. Secure Communication: Security is a fundamental aspect of Propeller’s architecture. By enforcing strict workload isolation and end-to-end secure communication channels, Propeller mitigates risks associated with untrusted execution environments, ensuring confidentiality, integrity, and reliability in Wasm-based workloads.
5. SuperMQ Integration: Propeller integrates with SuperMQ, a modern, scalable, and secure open-source IoT cloud platform. SuperMQ provides seamless protocol bridging, supporting HTTP, MQTT, WebSocket, and CoAP, enabling flexible communication between users and devices.

System Architecture

Propeller’s architecture is composed of several key components:

• CLI (Command-Line Interface): Provides a user-friendly mechanism for interacting with the Propeller system. It supports task management, resource provisioning, and system configuration.
• Manager: The central coordination unit within the Propeller system, responsible for orchestrating task execution, managing proplets, and facilitating communication between system components.
• Proplet: Lightweight execution environment for running tasks. It integrates with MQTT for real-time messaging and supports both internal and external Wasm runtime environments.
• Proxy: Acts as an intermediary between HTTP-based registries and the MQTT-based communication framework. It fetches Wasm files from OCI registries and streams them to proplets in chunks.
• SuperMQ: A scalable and secure messaging system that facilitates communication between the Manager, proplets, and other system components.

Use Cases

Propeller’s versatility makes it suitable for a wide range of applications:

• Real-Time Analytics: Propeller’s fast boot times and low latency make it ideal for real-time analytics and interactive services.
• Industrial Automation: The Embedded Proplet’s support for embedded devices and secure communication ensures reliable operation in industrial settings.
• Smart Sensors: Propeller’s efficient execution on resource-constrained devices enables deployment in IoT networks, making it a perfect fit for smart sensors and other edge devices.

How it Works

Set up Propeller On Your Environment. Make sure you have make, go and docker installed

Clone propeller repository

git clone https://github.com/absmach/propeller.git

Build propeller service then install the binaries to your GOBIN

make all && make install

Start supermq

make start-supermq

Provision manager and proplet

propeller-cli provision

Users interact with Propeller through the CLI or API

propeller-cli task create demo

curl 'http://localhost:7070/tasks' \
-H 'Content-Type: application/json' \
-d '{
    "name": "add",
    "inputs": [
        10,
        20
    ]
}'

curl 'http://localhost:7070/tasks' \
-H 'Content-Type: application/json' \
-d '{
    "name": "add",
    "image_url": "docker.io/mrstevenyaga/add.wasm",
    "inputs": [
        10,
        20
    ]
}'

The CLI sends an HTTP request to the Manager, which creates the task with a unique ID and timestamps for traceability. The task is associated with a Wasm module, which can either be uploaded directly or pulled from an OCI-compliant registry.

You can view a task by either using the CLI or HTTP API

curl 'http://localhost:7070/tasks/<task-id>'

propeller-cli tasks view <task-id>

You can update the task using HTTP API.

curl -X PUT 'http://localhost:7070/tasks/<task-id>' \
-H 'Content-Type: application/json' \
-d '{
    "name": "add",
    "file": "AGFzbQEAAAABBwFgAn9/AX8DAgEABwgBBG1haW4AAAoJAQcAIAAgAWoL",
    "inputs": [
        10,
        20
    ]
}'

You can also upload a file to the task using the /upload endpoint

curl -X PUT 'http://localhost:7070/tasks/<task-id>/upload' \
-F 'file=@"postman-cloud:///1efb9331-db42-42b0-818c-ea887cc3579a"'

The Manager is the central orchestrator. It maintains a registry of available proplets (worker nodes) and uses a round-robin algorithm to distribute tasks evenly. The Manager monitors the liveness of proplets through MQTT subscriptions. Proplets periodically send “alive” status updates to the Manager to confirm their availability. Once a task is created, it can be scheduled by the Manager assigning it to an available proplet.

curl -X POST 'http://localhost:7070/tasks/<task-id>/start'

The Manager publishes the task assignment to the MQTT topic /control/manager/start. The payload includes:

• Task ID: Unique identifier for the task.
• Wasm Module: Either a Base64-encoded Wasm binary or a reference to an OCI registry URL.
• Input Parameters: Execution parameters passed to the Wasm function.

The assigned proplet subscribes to this topic and receives the task payload.

If the Wasm module is not included in the payload (e.g., only an OCI registry URL is provided), the proplet requests the module from the Proxy. The Proxy fetches the Wasm module from the OCI registry in chunks to handle large binaries efficiently. The proplet reassembles the chunks locally and prepares the Wasm module for execution.

The proplet executes the Wasm module using one of two runtime options:

• Wazero Runtime: An embedded Wasm runtime for lightweight execution.
• Host Runtime: An external Wasm runtime for more complex workloads.

The proplet invokes the specified Wasm function with the provided input parameters. During execution, the proplet monitors the task and ensures it operates within resource constraints (e.g., memory and execution time).

Once the task completes, the proplet compiles the results into a structured JSON payload. The results are published to the MQTT topic /control/proplet/results. The Manager subscribes to this topic and receives the results for further processing or storage.

curl -X POST 'http://localhost:7070/tasks/<task-id>/stop'

If a task needs to be stopped, the Manager publishes a stop command to the MQTT topic /control/manager/stop. The proplet receives the command, terminates the running Wasm instance, and frees up resources. If no matching task is found, the proplet logs a warning and continues operation.

For embedded devices, the embedded proplet integrates with the WebAssembly Micro Runtime (WAMR) on Zephyr OS. Wasm modules are Base64-encoded and transmitted over MQTT to the embedded proplet. The proplet decodes the Wasm binary, loads it into WAMR, and executes it in a sandboxed environment. Results are published back to the Manager via MQTT, ensuring seamless communication even in resource-constrained environments.

The Proxy acts as a bridge between HTTP-based OCI registries and the MQTT-based communication framework. When a proplet requests a Wasm module, the Proxy:

1. Authenticates with the OCI registry.
2. Fetches the Wasm module in chunks.
3. Streams the chunks to the proplet via MQTT.

The proplet reassembles the chunks locally for execution.

SuperMQ is the backbone of Propeller’s communication infrastructure. It handles:

• Authentication: Ensures secure connections between the Manager, proplets, and Proxy.
• Message Routing: Facilitates task assignments, status updates, and result publication.
• Chunked File Transfers: Enables efficient transmission of large Wasm binaries.
• SuperMQ supports Quality of Service (QoS) levels, message retention, and session persistence to ensure reliable communication even in unstable network conditions.

The Manager incorporates middleware for logging, tracing, and metrics collection.

• OpenTelemetry provides distributed tracing, offering visibility into execution flows across the system.
• Prometheus collects performance metrics, enabling operators to monitor system health and resource utilization.

Conclusion

Propeller represents a significant step forward in the orchestration of WebAssembly workloads. By bridging the gap between cloud-native ecosystems and edge computing, Propeller enables efficient, secure, and reliable deployment of Wasm applications across diverse computational environments. Its innovative features, such as WAMR integration, OCI registry support, and SuperMQ communication, make it a powerful tool for managing the next generation of cloud and edge applications.

As industries continue to adopt WebAssembly for its portability, speed, and security, tools like Propeller will play a crucial role in ensuring that these workloads can be deployed and managed effectively. Whether you’re working on real-time analytics, industrial automation, or smart sensors, Propeller offers a unified, flexible, and scalable solution for all your Wasm orchestration needs. The future of workload management is here, and it’s powered by Propeller.

To learn more about Abstract Machines, and how we’re building the world’s most sophisticated open-source IoT platform, visit our website and follow us on Twitter!

If you liked this article, click the👏 below so others will see it here on Medium.