Programming the Web with WebAssembly.
Strings in WebAssembly
I wrote an article called “Strings in WebAssembly” a few months back. At the time of writing the article I was attempting to perform the obligatory “hello world” example in a new language; WebAssembly (also referred to as just “Wasm”).
A lot has happened since then!
Moving towards programming the Web with WebAssembly
Imagine having a tiny web-enabled hand-held hardware device (such as RaspberryPi or Arduino) which you could use to summon answers to problems that require immense computation (memory and storage far beyond what that physical device in your hand could offer).
This article describes a free, cloud agnostic ecosystem which provides safe and private storage and computation, all accessible via remote HTTPS request/response model. Not only can any web-enabled device partake. You can also write your own custom programmable server-side logic (Rust/Wasm), deploy it and then call it remotely via your hand-held device.
In the beginning I knew that Wasm offered a lot of advantages such as speed, efficiency, portability and security. However, as the aforementioned article explains, WebAssembly in its pure form has a few limitations which come into play when developing applications. For example Wasm does not fundamentally handle strings and does not have a string data type. Another limitation is in relation to storage. Specifically, there is no mechanism inside a Wasm Virtual Machine (VM) to store application state. Wasm VM’s are stateless. This article demonstrates ways in which these limitations have been lifted, for those who want to go ahead and create applications that incorporate Wasm.
This article, amongst other things:
- shows you how server-side Wasm can be executed remotely by a thin client
- demonstrates, storing data remotely and making that data available to server-side Wasm functions
- outlines example applications i.e. how to use Wasm to find the average temperature from a bunch of remote sensors and then execute a callback to another Wasm function which will convert the temperature from Celsius to Fahrenheit
- outlines ideas for new applications i.e. server-side image processing where images are fetched and watermark remotely (requiring almost no bandwidth or computation from the client who is initiates the image processing)
SecondState Inc.
Now is a good time to mention that the team at SecondState have been continuously churning out Wasm related software and tooling, since my last article. The demonstrations in this article are largely based on SecondState’s latest Free and Open Source (FOSS) Wasm infrastructure.
We will be covering a lot in this article and hopefully you will find something that you can quickly and easily implement to enhance one of your software products or projects and/or business ideas.
Strings
Firstly, a quick recap about strings. SecondState’s build tool SSVMUP (which is derived from the open source wasm-pack software) now caters for all of our string needs. Let’s get a quick “hello world!” example out of the way so that we can dive in and discover some new and exciting opportunities in the Wasm space.
Function as a Service (FaaS)
The following Curl command allows us to execute Wasm bytecode via a simple HTTP request.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/1/say' \
--header 'Content-Type: text/plain' \
--data-raw 'world!'As you can see, this POST request passes in the text world! and the Wasm function returns the text hello world!
If you would prefer not to use the command line (i.e. a Curl command), below is an example of how you could perform this HTTP request using a GUI request client like Postman.
This is of course a very simple high-level demonstration of how a Wasm VM can perform the hello world demo.
If you would prefer to dive a bit deeper into how this works, you can go ahead and write your own Wasm functions. So let’s do that now.
How to create your own Function as a Service (FaaS)
Here is a simple Rust function that you can create, compile to Wasm and launch, in as little as 5 minutes.
Install Rust
Firstly, if you haven’t already, please follow these official instructions to install Rust on your machine.
Add Wasi
rustup target add wasm32-wasiInstall SSVMUP
curl https://raw.githubusercontent.com/second-state/ssvmup/master/installer/init.sh -sSf | shCreate your Rust project
Create a new project
cd $HOME
cargo new --lib double_number
cd double_number/Configure your Rust project
Edit the config i.e. vi Cargo.toml and append the following [lib] and [dependencies] sections to the end of the Cargo.toml file (after the [package]section which already exists)
[lib]
crate-type = ["cdylib"]
[dependencies]
wasm-bindgen = "=0.2.61"Write your Rust source code
Open the /home/ubuntu/double_number/src/lib.rs file and ensure that it contains only the following Rust code
use wasm_bindgen::prelude::*;#[wasm_bindgen]
pub fn double(num_1: String) -> String {
let parsed: i32 = num_1.parse().unwrap();
let answer: i32 = parsed * 2;
answer.to_string()
}
Compile your source code to Wasm bytecode
ssvmup buildLaunch the Wasm bytecode
Please ensure that you don’t use tilde (~) to provide the path to your home directory. Instead, use $HOME (in double quotes) or be explicit and use /home/your_username i.e.
--data-binary "@$HOME ..."or
--data-binary '@/home/your_username ...'With that in mind, please paste the following into your command line. Noting the file path in the data binary section which we just talked about.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/executables' \
--header 'Content-Type: application/octet-stream' \
--header 'SSVM_Description: Double a number' \
--data-binary "@$HOME/double_number/pkg/double_number_bg.wasm"The launching of your Wasm bytecode will return a result like the one below.
{
"wasm_id": 40,
"wasm_sha256": "0x7cb3accab2bcc758891d8ec9e1f1c876bf885cb41d083a417577aa99cfce9e95",
"SSVM_Usage_Key": "00000000-0000-0000-0000-000000000000",
"SSVM_Admin_Key": "bab9f00a-ee36-4f89-bacd-617371e03dc9"
}Please save the data i.e. wasm_id and SSVM_Admin_Key as you will need these. Specifically, you will be putting your own wasm_id into the next command (our wasm_id for this demo is 40, what is yours?)
Call your Wasm bytecode via HTTP
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/your_wasm_id/double' \
--header 'Content-Type: text/plain' \
--data-raw '22'The function call above doubles 22 and returns the correct answer of 44.
Access control
If you were thinking that the above procedure would allow anyone to execute your function, you would be correct.
Public
Until now, we have just uploaded a publicly available Rust/Wasm function that anyone (capable of making a HTTP request) can access.
Private
If you would prefer your Rust/Wasm function to only be accessible by a select few (or perhaps just yourself) just go ahead and generate anSSVM_Usage_Key when originally launching your Wasm bytecode for the first time.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/executables' \
--header 'Content-Type: application/octet-stream' \
--header 'SSVM_Description: Double a number' \
--header 'SSVM_Create_Usage_Key: true' \
--data-binary "@$HOME/double_number/pkg/double_number_bg.wasm"We just launched the Wasm executable and this time we have a new wasm_id of 41. During this launch the addition of theSSVM_Create_Usage_Key: true in the request header has produce a new SSVM_Usage_Key for us, as shown below.
{
"wasm_id": 41,
"wasm_sha256": "0x7cb3accab2bcc758891d8ec9e1f1c876bf885cb41d083a417577aa99cfce9e95",
"SSVM_Usage_Key": "323161a1-8724-4ce3-ac9e-db143b777d65",
"SSVM_Admin_Key": "8aa4915e-48e4-4270-afac-564d79766d62"
}Once an SSVM_Usage_Key is associated with the Rust/Wasm executable each request to execute the function will be required to pass in that SSVM_Usage_Key as part of the header like this.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/41/double' \
--header 'Content-Type: text/plain' \
--header 'SSVM_Usage_Key: 323161a1-8724-4ce3-ac9e-db143b777d65' \
--data-raw '22'Success will result in the correct answer of 44. However, failure to provide the SSVM_Usage_Key will end up in a response like the one shown below.
{"error":"Wrong usage key ... 41 can not be accessed."}If you would like to remove the access control, you can use the power of your SSVM_Admin_Key to return to a blank (zeroed out) key like this.
curl --location --request DELETE 'https://rpc.ssvm.secondstate.io:8081/api/keys/41/usage_key' \
--header 'SSVM_Admin_Key: 8aa4915e-48e4-4270-afac-564d79766d62'The above DELETE verb returns the following
{"SSVM_Usage_Key": "00000000-0000-0000-0000-000000000000"}
Your Wasm executable is not completely open and public again.
Going from public to private after initial launch …
If you:
a) forgot to create an SSVM_Usage_Key at the launch
b) want to return to having private access control after making it public
c) want to refresh an existing already-private SSVM_Usage_Key
You can simply perform the following PUT verb at the same endpoint.
curl --location --request PUT 'https://rpc.ssvm.secondstate.io:8081/api/keys/41/usage_key' \
--header 'SSVM_Admin_Key: 8aa4915e-48e4-4270-afac-564d79766d62'With a little help from the SSVM_Admin_Key as proof of ownership, the above PUT verb returns a fresh SSVM_Usage_Key
{"SSVM_Usage_Key": "acb88052-ae7c-43e8-94ee-74d884f73513"}
Storage
One of the key benefits of SecondState’s Wasm execution environment is its storage capabilities. Strictly speaking containerised execution, like that of Wasm Virtual Machines, is stateless. For example, each function call (HTTP request) instantiates a brand new VM instance which executes the Wasm bytecode in a brand new stateless environment.
In these scenarios, nothing is ever stored in the execution environment; the function’s caller is just receiving separate pieces of data each time the VM returns the function’s return data.
SecondState provides short term (ephemeral) storage as well as permanent storage. Let’s firstly talk about ephemeral storage.
Fast and flexible storage
This fast and flexible storage is ephemeral. It quickly provides a way to buffer information on the client side (where the HTTP requests are being made). The following examples show how you can quickly and easily store and retrieve ephemeral data via HTTP requests.
The following POST verb:
a) creates a new storage location
b) stores the JSON data
c) returns your private collision resistant access key (for future use)
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/ephemeral_storage' \
--header 'Content-Type: application/json' \
--data-raw '{"storage_example_1": 25}'The return value is as follows
{"key":"cd22a3a1-9416-4511-9678-a49f595bd156"}Using a PUT verb, the data at this key’s location (your key will be different to this one of course) can be updated, with any valid JSON (which will simply override the existing data).
curl --location --request PUT 'https://rpc.ssvm.secondstate.io:8081/api/ephemeral_storage/put_your_ephemeral_storage_key_here' \
--header 'Content-Type: application/json' \
--data-raw '{
"storage_example_2": "some text",
"more_data": [1, 2, 3, 4, 5]
}'Reading/fetching the data is done using the GET verb with the key as part of the URL (via a simple HTTP request with no headers and no body) like this
curl --location --request GET 'https://rpc.ssvm.secondstate.io:8081/api/ephemeral_storage/cd22a3a1-9416-4511-9678-a49f595bd156'The above GET request returns the following JSON data
{
"value": {
"storage_example_2": "some text",
"more_data": [1, 2, 3, 4, 5]
}
}If the data is not updated within 60 minutes then it is automatically disposed of.
If you want to deliberately delete the data at the key then you can simply perform the DELETE verb, like this.
curl --location --request DELETE 'https://rpc.ssvm.secondstate.io:8081/api/ephemeral_storage/cd22a3a1-9416-4511-9678-a49f595bd156'Once the data is gone, any requests will return an error saying that the key is not found, like this.
{"error":"Key not found"}The ephemeral storage is arbitrary. It is not tied to any WebAssembly executable and there is no access control mechanism for ephemeral storage. You can store as much or as little arbitrary application data as you like by simply taking care of the ephemeral storage keys
This ephemeral storage is perfect for situations where individual client-side sensors (i.e. Raspberry Pi units) are able to pass in their raw data via HTTP request.
A quick sample use case
In this scenario, none of the Raspberry Pi devices nor the web application are required to store any data, the data from each of the individual sensors is always available (in real-time) via HTTP requests to/from the ephemeral storage endpoint.
The data is accessed by the temperature gauge application. The temperature gauge application is the entity that collates the raw data (from the ephemeral storage endpoint) and in-turn physically passes that raw temperature data into the Rust/Wasm function[s] as input parameters.
In this case the gauge application calls two functions. One that calculates the average temperature and another that converts the calculated result from Celsius to Fahrenheit. We will talk more about performing multiple function calls using callbacks a bit later in the article.
Permanent storage
SecondState also provides permanent storage. The permanent storage differs from the ephemeral storage in one very special way. The permanent storage is accessible inside the Rust/Wasm code. This means that the client does not have to pass the actual raw data into the Rust/Wasm execution environment.
Instead the client can simply pass in a storage key and let the server side do all of the heavy lifting. The permanent storage option is perfect for storing and processing larger data objects on the server-side i.e. large images and videos.
In this scenario the calling client only needs to store and pass around keys to the data (not the data itself).
Let’s go ahead and create another custom Rust application which demonstrates how to permanently store a String and receive a key to the String’s location as the response.
Create a new project
cd $HOME
cargo new --lib store_a_string
cd store_a_string/Then add the following code to the src/lib.rs file
use wasm_bindgen::prelude::*;
use rust_storage_interface_library::ssvm_storage;#[wasm_bindgen]
pub fn store_string(s: String) -> String {
let storage_key: i32 = ssvm_storage::store::store(s);
storage_key.to_string()
}
The permanent storage is implemented inside Rust via the rust_storage_interface_library package and therefore you will need to do a little more with the Cargo.toml this time i.e. make sure to also include this rust_storage_interface_library dependency along with the rest i.e.
[lib]
crate-type = ["cdylib"]
[dependencies]
wasm-bindgen = "=0.2.61"
rust_storage_interface_library = "^0.1"We can then go ahead and build this; producing the Wasm file.
ssvmup buildOnce compiled, we submit this new Wasm bytecode (as we have done in the previous examples).
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/executables' \
--header 'Content-Type: application/octet-stream' \
--header 'SSVM_Description: Permanently store a string' \
--data-binary "@$HOME/store_a_string/pkg/store_a_string_bg.wasm"We get back the following response.
{
"wasm_id": 43,
"wasm_sha256": "0xdb7b82360a770a9ba4aca37549d87071f2e5f25a27059e2bead237a8fb222dda",
"SSVM_Usage_Key": "00000000-0000-0000-0000-000000000000",
"SSVM_Admin_Key": "bd73637d-7191-4e23-b73d-8735104aec8e"
}We can then go ahead and store some data via a HTTP request.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/43/store_string' \
--header 'Content-Type: text/plain' \
--data-raw 'string to store'This returns a key as plain text; a key which we can use in the future to access the ‘string to store’ data which we just saved.
The Rust code to retrieve the string inside the Rust/Wasm application’s execution is as follows
let my_loaded_string = ssvm_storage::load::load_as_string(storage_key);These storage features, such as the Rust Native Storage Library demonstrated above, are custom features which have been developed by SecondState.
Please get in touch via this GitHub link if you want to provide feedback or if you have any specific questions about how to implement these services into your own applications. Your feedback is greatly appreciated.
Permanent storage of other high-level data types
Storing a Rust Struct
Aside from just high-level strings, you can also store custom structs.
Here is an example of the Rust code that you would use to store and load a custom high-level struct.
For this to work you will need to make Serde a dependency, so please go ahead and modify the Cargo.toml (as we have demonstrated previously in this article), this time adding Serde as well.
[lib]
crate-type = ["cdylib"][dependencies]
wasm-bindgen = "=0.2.61"
rust_storage_interface_library = "^0.1"
serde = { version = "1.0", features = ["derive"] }
The Rust code (in the src/lib.rs) to store a struct will need to be similar to what is shown below. As you can see we firstly, declare a struct, then we instantiate and finally store the struct.
use wasm_bindgen::prelude::*;
use serde::{Serialize, Deserialize};
use rust_storage_interface_library::ssvm_storage;#[wasm_bindgen]
pub fn store_struct() -> String {
// Declare a struct
#[derive(Serialize, Deserialize, PartialEq, Debug, Clone, Default)]
struct TestStruct {
a_vec: Vec<u8>,
a_i32: i32,
a_u8: u8,
a_bool: bool,
}
// Instantiate the struct
let test_struct1 = TestStruct {
a_vec: vec![134, 122, 131],
a_i32: 4,
a_u8: 4,
a_bool: true,
};
// Store the struct
let storage_key: i32 = ssvm_storage::store::store(test_struct1);
storage_key.to_string()
}
We can compile this Rust example (using ssvmup build as we have demonstrated previously)
ssvmup buildOnce compiled, we can go ahead and upload that via another HTTP request
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/executables' \
--header 'Content-Type: application/octet-stream' \
--header 'SSVM_Description: Permanently store a struct' \
--data-binary "@$HOME/store_a_struct/pkg/store_a_struct_bg.wasm"This will return the new wasm_id and keys etc.
{
"wasm_id": 45,
"wasm_sha256": "0xbd13bbbe64a9600c30242a1983a7c25d841c644ba8f06a8afd5ac97c34445201",
"SSVM_Usage_Key": "00000000-0000-0000-0000-000000000000",
"SSVM_Admin_Key": "d4d08c54-d224-432b-8e78-fe6dd3651316"
}We can then go ahead and call the function to store the struct permanently.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/45/store_struct'Notice, in this case (just for demonstration purposes) we are not sending any input parameters to the Rust function (because the Rust is creating the Struct data internally) and therefore we do not require Content-type in the headers or any raw data as the body, in order for this request to succeed.
Loading a Rust Struct
// Declare a custom struct
#[derive(Serialize, Deserialize, PartialEq, Debug, Default)]
struct TestStruct {
a_vec: Vec<u8>,
a_i32: i32,
a_u8: u8,
a_bool: bool,
}// Instantiate the struct, just using the default (no actual data necessary, this is just a placeholder)
let struct_skeleton = TestStruct::default();// Fetch the struct
let my_loaded_struct: TestStruct = ssvm_storage::load::load_as_struct(struct_skeleton, storage_key);
The newly created variable called “my_loaded_struct” will now contain the following data.
TestStruct {a_vec: [134, 122, 131], a_i32: 4, a_u8: 4, a_bool: true}This next section is super interesting. Not only can we call Wasm functions via the Web (HTTP requests).
WebAssembly and the programmable Web
SecondState’s infrastructure facilitates a host of functionality and brings Wasm execution into the programmable Web. Aside from the features we just mentioned above, programmable Web features include:
- Post execution callbacks
- Pre execution HTTP POST requests
- Pre execution HTTP GET requests
- Wasm execution using multipart/form-data and more
Custom Callbacks (Web hooks)
This is an awesome feature worth explaining/demonstrating. As we will show you shortly, the calling client is able to specify post-execution HTTP requests in several ways.
Callback object per Wasm executable
Issuing a PUT verb to the following endpoint
https://rpc.ssvm.secondstate.io:8081/api/callback/3with the following body
{"hostname": "rpc.ssvm.secondstate.io","path": "/api/run/wasm_id/function_name","method": "POST","port": 8081,"headers":{"Content-Type": "text/plain"}}will load a permanent callback to that specific Wasm executable (which in this case has the wasm_id of 3). Here is the complete HTTP request example.
curl --location --request PUT 'https://rpc.ssvm.secondstate.io:8081/api/callback/3' \
--header 'Content-Type: application/json' \
--data-raw '{"hostname": "rpc.ssvm.secondstate.io","path": "/api/run/wasm_id/function_name","method": "POST","port": 8081,"headers":{"Content-Type": "text/plain"}}'From this point onward this callback will be automatically executed every time a function (of that Wasm executable ‘3’) is executed.
Callbacks in the HTTP Body
If a callback is placed in the HTTP request’s body it will be executed.
Please note: callbacks in the HTTP body override the permanent callback we just mentioned.
{
"SSVM_Callback": {
"hostname": "rpc.ssvm.secondstate.io",
"path": "/api/run/1/say",
"method": "POST",
"port": 8081,
"headers": {
"Content-Type": "text/plain",
"SSVM_Usage_Key": "83f02dd3-6440-482f-983f-78127ed6f943"
}
},
"Function_Parameters": [1, 2, 3, 4]
}Pay special attention to the explicit convention of the “SSVM_Callback” name, and also the fact that this “SSVM_Callback” object is accessible at the top level of the body object.
If a blank callback object is placed in the body then no callback will take place at all during that call’s execution. This a great way to negate the permanent/automatic callback (if you wish for no callback to be run during your current request).
Callbacks in the HTTP Header
If a callback is placed in the HTTP header it will be executed (instead of the callback in the body and the permanent/automatic callback). Take the following example.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/1/say' \
--header 'Content-Type: text/plain' \
--header 'SSVM_Callback: { "hostname":"rpc.ssvm.secondstate.io","path": "/api/run/1/say", "method": "POST","port": 8081,"headers": {"Content-Type": "text/plain"}}' \
--data-raw 'world!'The function above will return the following
hello hello world!Let’s take a closer look at how a callback works …
The first part of the request executes looks like this
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/1/say' \
--header 'Content-Type: text/plain' \
--data-raw 'world!'The first part of the request returns the following data
hello world!The callback takes this data (from the first request) as input (to the upcoming callback request) and then executes the callback (in this case the same function again), which of course results in the following data being returned to the original caller.
hello hello world!As just mentioned, if an empty callback object is passed (in this case, to the HTTP headers) then no callback will take place during this call’s execution. Here is an example.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/1/say' \
--header 'Content-Type: text/plain' \
--header 'SSVM_Callback: {}' \
--data-raw 'world!'This will override all other callbacks and will just return the following data
hello world!Remote fetching of data
Server-side remote fetching via HTTP POST request
This is another great feature that lets the calling client process large files/data (like images or videos) without the need (for the calling client) to upload/download the files/data. The heavy lifting is all done on the server side (where the Wasm execution also takes place). Great for handheld devices, mobile phones and thin-client application development.
If the thin-client’s calling code includes an SSVM_Fetch header key (with a header value that is a HTTP POST request object) as part of the original request. Then the HTTP POST request will be executed up front. In this scenario, the response value from the aforementioned HTTP POST will actually become the input which will now be sent into the upcoming Wasm function’s execution.
This allows the calling client to initiate a function call without having to physically provide the data itself.
Consider the following example.
As you can see below, we are again using the same simple text/plain words like “body”, hello and world here for brevity. You could/would fetch very large amounts of data remotely i.e. image frames for video processing and so forth.
curl --location --request POST ‘https://rpc.ssvm.secondstate.io:8081/api/run/1/say' \
--header ‘Content-Type: application/json’ \
--data-raw ‘{
“SSVM_Fetch”: {
“body”: “body”,
“hostname”: “rpc.ssvm.secondstate.io”,
“path”: “/api/run/1/say”,
“method”: “POST”,
“port”: 8081,
“headers”: {
“Content-Type”: “text/plain”
}
}
}’The above code returns the following data
hello hello "body"Where the result of the pre-execution SSVM_Fetch is
hello “body”This pre-execution fetch, when passed into the Wasm function, returns
hello hello “body”Server-side remote fetching via HTTP GET request
This feature is a little bit simpler (when compared to the HTTP POST example above). In this case, the value of the SSVM_Fetch header key only has to be a URL (not an HTTP POST object).
This also allows the client to initiate a function call without having to physically provide any data itself. You could/would fetch any data that is available via a HTTP GET request. Including raw data from a URL, as we demonstrate below.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/1/say' \
--header 'Content-Type: text/plain' \
--header 'SSVM_Fetch: https://raw.githubusercontent.com/tpmccallum/test_endpoint2/master/tim.txt'In this example, the URL provided for the SSVM_Fetch (pre-execution HTTP GET request) simply contains the text/plain of “Tim”. Therefore the end result which is returned from the client’s call above is as follows
hello TimMixing SSVM_Fetch and SSVM_Callback
What if we used both the SSVM_Fetch and the SSVM_Callback in the header? Let’s give it a try.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/1/say' \--header 'Content-Type: text/plain' \--header 'SSVM_Fetch: https://raw.githubusercontent.com/tpmccallum/test_endpoint2/master/tim.txt' \--header 'SSVM_Callback: {"hostname":"rpc.ssvm.secondstate.io","path": "/api/run/1/say", "method": "POST","port": 8081,"headers": {"Content-Type": "text/plain"}}' \
Yes, you guessed it! The result would be as follows
hello hello TimMultipart form-data
Working with multipart/form-data is actually pretty straight forward. There are a couple of key things to remember.
Firstly when providing a multipart/form-data key please ensure that it contains a suffix which conforms to the following pattern; underscore followed by a sequential number i.e. _1 or _2.
The reason for this underscore number (_1) is because (as with all programming languages) the function that we will be calling requires that its input parameters are in a specific order. For example, if we write a function as shown below.
fn process_two_inputs(input_a: String, input_b: String)Then the calling client would need to provide this function with the correct arguments when the client performs the HTTP request. Here is an example
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/wasm_id/process_two_inputs' \
--header 'Content-Type: multipart/form-data' \
--form 'first_input_example_1="one"' \
--form 'second_input_example_2="two"'As mentioned a second ago, all multipart/form-data keys in the request will require this sequential numbering.
Secondly, a prefix which conforms to the following pattern; the text “fetch” followed by an underscore i.e.
fetch_Will allow the caller to inject results (from remote Web requests) as inputs into the function which it is calling.
When a multipart/form-data key starts with the aforementioned prefix of fetch_ the value associated with that key will be treated as a fetch-able resource which can be obtained via a HTTP GET or HTTP POST request.
Please note, this key will also need to have the sequential number convention applied to it.
fetch_third_input_example_3The reason being, this fetch-able resource will be fetched and then used as input into the Wasm function which is being called. Let’s see an example of how this would be coded up
Here is an example of a Rust/Wasm function (now with 3 input parameters)
fn process_three_inputs(input_a: String, input_b: String, input_c: String)Here is an example of the client request to call that function
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/run/wasm_id/process_three_inputs' \--header 'Content-Type: multipart/form-data' \--form 'first_input_example_1="one"' \--form 'second_input_example_2="two"' \--form 'third_input_example_3=https://raw.githubusercontent.com/tpmccallum/test_endpoint2/master/tim.txt'
If the client is wanting the server-side execution to perform a HTTP POST (and use that response as input) then the call would have a HTTP request object (instead of just a URL), as shown below.
curl --location --request POST 'https://dev.rpc.ssvm.secondstate.io:8081/api/run/67/add' \--header 'Content-Type: multipart/form-data' \--form 'first_input_example_1="one"' \--form 'second_input_example_2="two"' \--form 'third_input_example_3={"body": "body","hostname": "rpc.ssvm.secondstate.io","path": "/api/run/1/say","method": "POST","port": 8081,"headers": {"Content-Type": "text/plain"}}'
It is important to note that each of the multipart/form-data inputs will take a different amount of time to become available. Say, for example, a HTTP POST to an unknown endpoint takes a few seconds longer than expected. This will not become an issue. The reason being, SecondState’s endpoint waits for all of the inputs to be returned and in addition (thanks to the numbering convention _1, _2 etc.) the arguments are always provided to the Wasm function in the correct order.
Future opportunities
We have discussed a plethora of functionality which should open the floodgates of new ideas. If you are not already bursting at the seams with ideas, here is an image processing idea to whet your appetite.
Image watermarking with native Wasm
In the appendix of this article, you will find the Rust source code to an image watermarking function that I whipped up. You will notice from the source code that it is very light on third-party dependencies :)
You will also notice that the image data being passed into the function is in the form of the byte array, and that the function actually returns the modified image as a byte array.
This design allows any 3rd-party image software to open the image ahead of time. This Rust/Wasm function only processes the raw pixel data as a byte array and does not even know the image file type i.e. JPG, PNG etc.
This low-level design also allows for this particular Wasm executable to be integrated into Function as a Service (FaaS) architecture and reused by many different calling clients.
Scenario …
Perhaps another service has already converted a bunch of images to byte arrays ahead of time. This would depend on the scenario’s business use case and so forth. These broader application details here are hypothetical for inspiration sake :)
Now let’s say that the client (who is calling this function, utilises HTTP Headers and) adds the SSVM_Fetch key and a URL to the raw pixels (as part of the original call). This would allow the raw pixels to be fetched remotely on the server side. Which is fantastic because, for example, we could be dealing with over 2 Million (2, 073, 600) pixels per frame if we were processing stills or video of 1920x1080 resolution. The advantage here is that the calling client would use absolutely minimal bandwidth and storage when executing this functionality.
In addition, because the pixel processing is also done by SecondState’s Wasm VM (on the server side) the calling client would endure almost no processing/computation.
As feedback to the calling client, perhaps the client could be sent a small thumbnail when the watermarking process is complete. Take the example below (which was created by the Rust code in the appendix).
As mentioned at the start of this article, we are now able to execute functionality from one Web entity to another. In this regard, perhaps the thumbnail resize could be another separate Rust/Wasm function which could be executed separately after the prior watermarking. Remember our SSVM_Callback option?
Programming the Web with WebAssembly is now starting to make sense!
If you have been thinking that the architecture covered here today fits in nicely with the term “cloud native” you are spot on! As we know, cloud native is essentially container-based computation.
We have shown here that we can have the best of both worlds.
On one hand we have immense flexibility, functionality and safety. On the other hand we have our discrete functions which are available for execution the form of services; packaged into containers. The containerised execution environment that underpins everything we have used today is called SSVM. Each time we have been executing our Wasm functions today, we have also been spinning up a new stateless instance of the SSVM in an unnoticeable fraction of a second.
SecondState on Product Hunt
SecondStates WebAssembly VM is a high performance and enterprise-ready WebAssembly (WASM) Virtual Machine for cloud which was recently featured on Product Hunt and received 312 upvotes in 2 days.
What’s next?
This article has provided details about infrastructure which will allow you to build almost anything with WebAssembly. The infrastructure demonstrated here can be part of a broader solution, or make up your entire solution.
What applications would you like to see deployed in the Wasm space? If you would like to ask any questions or get some assistance in building your own Function as a Service (FaaS) solution using Wasm and the programmable Web, please get in touch via the details below. Thanks for reading.
Twitter: https://twitter.com/mistermac2008
Email: mistermac2008@gmail.com
Appendix
Image watermarking source code
The following Rust source code can be compiled to Wasm, see below.
use std::collections::HashMap;
use wasm_bindgen::prelude::*;
#[derive(Debug)]
struct Pixel {
r: u8,
g: u8,
b: u8,
t: u8,
}
#[wasm_bindgen]
pub fn watermark_single_image(
_image_width: String,
_image_height: String,
mut _image_pixels: Vec<u8>,
_watermark_width: String,
_watermark_height: String,
mut _watermark_pixels: Vec<u8>,
_watermark_pos_width: String,
_watermark_pos_height: String,
) -> Vec<u8> {
// Convert the String inputs to u32 integers
let image_width: u32 = _image_width.parse().unwrap();
let image_height: u32 = _image_height.parse().unwrap();
let watermark_width: u32 = _watermark_width.parse().unwrap();
let watermark_height: u32 = _watermark_height.parse().unwrap();
let watermark_pos_width: u32 = _watermark_pos_width.parse().unwrap();
let watermark_pos_height: u32 = _watermark_pos_height.parse().unwrap();
let mut pixels = HashMap::new();
// Adjust width so that watermark placement does not exceed image width
let mut width: u32 = watermark_width;
let mut height: u32 = watermark_height;
if watermark_pos_width + watermark_width > image_width {
width = image_width - watermark_pos_width;
}
// Adjust height so that watermark placement does not exceed image height
if watermark_pos_height + watermark_height > image_height {
height = image_height - watermark_pos_height;
}
//println!("_watermark_pixels.len(): {:?}", _watermark_pixels.len());
// Map out the watermark pixels as a struct
for w_h in 0..height {
let height_start_pixel = ((width * 4) * w_h);
for w_w in 0..width {
let width_start_pixel_at_byte = (height_start_pixel + (w_w * 4));
//println!("width_start_pixel_at_byte: {:?}", width_start_pixel_at_byte);
let temp_pixel = Pixel {
r: _watermark_pixels[width_start_pixel_at_byte as usize],
g: _watermark_pixels[width_start_pixel_at_byte as usize + 1],
b: _watermark_pixels[width_start_pixel_at_byte as usize + 2],
t: _watermark_pixels[width_start_pixel_at_byte as usize + 3],
};
let watermark_key =
format!("{}{}{}", w_w.to_string(), "_".to_string(), w_h.to_string());
//println!("key {:?}, pixel: {:?}", watermark_key, temp_pixel);
pixels.insert(watermark_key, temp_pixel);
}
}
let mut w_counter: u32 = 0;
let mut h_counter: u32 = 0;
let height_limit = watermark_pos_height + height;
let width_limit = watermark_pos_width + width;
for i_h in watermark_pos_height..height_limit {
let height_start_pixel_2 = ((image_width * 4) * i_h);
for i_w in watermark_pos_width..width_limit {
let width_start_pixel_at_byte_2 = (height_start_pixel_2 + (i_w * 4));
// We have the start byte here so we can update in place
// Create a key to obtain the watermark data from the pixels array
let image_key = format!(
"{}{}{}",
w_counter.to_string(),
"_".to_string(),
h_counter.to_string()
);
// Fetch the watermark data base on its key
match pixels.get(&image_key) {
Some(pixel) => {
_image_pixels[width_start_pixel_at_byte_2 as usize] = pixel.r;
_image_pixels[width_start_pixel_at_byte_2 as usize + 1] = pixel.g;
_image_pixels[width_start_pixel_at_byte_2 as usize + 2] = pixel.b;
_image_pixels[width_start_pixel_at_byte_2 as usize + 3] = pixel.t;
}
None => println!("Not found"),
}
w_counter = w_counter + 1;
}
w_counter = 0;
h_counter = h_counter + 1;
}
w_counter = 0;
h_counter = 0;
_image_pixels
}Once compiled, using ssvmup build the Wasm file can be launched.
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/executables' --header 'Content-Type: application/octet-stream' --header 'SSVM_Description: Watermark' --data-binary "@/home/ubuntu/server_side_watermarking/pkg/watermark_bg.wasm"Once launched, we can call the watermark_single_image function via HTTP request, as shown in the Curl command below
curl --location --request POST 'https://rpc.ssvm.secondstate.io:8081/api/multipart/run/wasm_id/watermark_single_image/bytes' \--header 'Content-Type: multipart/form-data' \--form 'image_width_1=1024' \--form 'image_height_2=1024' \--form 'fetch_image_pixels_3=https://public-wasm-files.s3-ap-southeast-2.amazonaws.com/globe_image' \--form 'watermark_width_4=256' \--form 'watermark_height_5=256' \--form 'fetch_watermark_pixels_6=https://public-wasm-files.s3-ap-southeast-2.amazonaws.com/watermark_image' \--form 'watermark_position_width_7=10' \--form 'watermark_position_height_8=10'
