Decentralized Storage for Files, Websites, and More: LiquidStorage Walkthrough

If you’ve been through any of the DAPP Network walkthroughs, you know how to get started — by installing Zeus SDK (if you haven’t) and unboxing a sample box to investigate. LiquidStorage is no different:

$ npm i -g @liquidapps/zeus-cmd  //if you don't have Zeus SDK
$ zeus unbox storage-dapp-service
$ zeus test -c  //if you'd like
$ cd storage-dapp-service/contracts/eos/storageconsumer
$ nano storageconsumer.cpp  //or, as always, with your fave editor

The sample storageconsumer contract uses LiquidAccounts. What shows below is the default storageconsumer.cpp, and the bold represents what we will remove in order to disable LiquidAccounts. We’ll be focusing exclusively on the parts related to LiquidStorage.

Don’t worry — the stripped-down version with LiquidAccounts removed will follow shortly.

/* ALLOWS FOR QUICKER ACCESS TO USER DATA WITHOUT THE NEED TO WARM DATA UP */
#define VACCOUNTS_DELAYED_CLEANUP 120

/* ADD NECESSARY LIQUIDACCOUNT / VRAM INCLUDES */
#include "../dappservices/ipfs.hpp"
#include "../dappservices/multi_index.hpp"
#include "../dappservices/vaccounts.hpp"
#include <eosio/singleton.hpp>

/* ADD LIQUIDACCOUNT / VRAM RELATED ACTIONS */
#define DAPPSERVICES_ACTIONS() \
  XSIGNAL_DAPPSERVICE_ACTION \
  IPFS_DAPPSERVICE_ACTIONS \
  VACCOUNTS_DAPPSERVICE_ACTIONS
#define DAPPSERVICE_ACTIONS_COMMANDS() \
  IPFS_SVC_COMMANDS() VACCOUNTS_SVC_COMMANDS()

#define CONTRACT_NAME() storageconsumer

CONTRACT_START()

/* THE storagecfg TABLE STORES STORAGE PARAMS */
TABLE storagecfg {
  // all measurements in bytes
  uint64_t max_file_size_in_bytes = UINT64_MAX; // max file size in bytes that can be uploaded at a time, default 10mb
  uint64_t global_upload_limit_per_day = UINT64_MAX; // max upload limit in bytes per day for EOS account, default 1 GB
  uint64_t vaccount_upload_limit_per_day = UINT64_MAX; // max upload limit in bytes per day for LiquidAccounts, default 10 MB
};
typedef eosio::singleton<"storagecfg"_n, storagecfg> storagecfg_t;

/* SET PARAMS FOR storagecfg TABLE */
ACTION setstoragecfg(const uint64_t &max_file_size_in_bytes,
                     const uint64_t &global_upload_limit_per_day,
                     const uint64_t &vaccount_upload_limit_per_day) {
  require_auth(get_self());
  storagecfg_t storagecfg_table(get_self(), get_self().value);
  auto storagecfg = storagecfg_table.get_or_default();

  storagecfg.max_file_size_in_bytes = max_file_size_in_bytes;
  storagecfg.global_upload_limit_per_day = global_upload_limit_per_day;
  storagecfg.vaccount_upload_limit_per_day = vaccount_upload_limit_per_day;

  storagecfg_table.set(storagecfg, get_self());
}

/* THE FOLLOWING STRUCT DEFINES THE PARAMS THAT MUST BE PASSED FOR A LIQUIDACCOUNT TRX */
struct dummy_action_hello {
  name vaccount;
  uint64_t b;
  uint64_t c;

  EOSLIB_SERIALIZE(dummy_action_hello, (vaccount)(b)(c))
};

/* DATA IS PASSED AS PAYLOADS INSTEAD OF INDIVIDUAL PARAMS */
[[eosio::action]] void hello(dummy_action_hello payload) {
  /* require_vaccount is the equivalent of require_auth for EOS */
  require_vaccount(payload.vaccount);

  print("hello from ");
  print(payload.vaccount);
  print(" ");
  print(payload.b + payload.c);
  print("\n");
}

/* EACH ACTION MUST HAVE A STRUCT THAT DEFINES THE PAYLOAD SYNTAX TO BE PASSED */
VACCOUNTS_APPLY(((dummy_action_hello)(hello)))

CONTRACT_END((hello)(regaccount)(setstoragecfg)(xdcommit)(xvinit))

Besides LiquidAccounts, this contract also includes some sample code (the large bold section at the bottom with the hello action). This example code demonstrates the use of payload structs for action data, rather than lists of parameters — because LiquidAccounts requires payload structs. There is no reason not to use payload structs, even if we’re not using LiquidAccounts, but this sample struct dummy_action_hello and void hello() can be safely cut.

Here’s what we end up with:

#include <eosio/singleton.hpp>

#define CONTRACT_NAME() storageconsumer

CONTRACT_START()

TABLE storagecfg {
  uint64_t max_file_size_in_bytes = UINT64_MAX;
  uint64_t global_upload_limit_per_day = UINT64_MAX;
};
typedef eosio::singleton<"storagecfg"_n, storagecfg> storagecfg_t;

ACTION setstoragecfg(const uint64_t &max_file_size_in_bytes,
                     const uint64_t &global_upload_limit_per_day
) {
  require_auth(get_self());
  storagecfg_t storagecfg_table(get_self(), get_self().value);
  auto storagecfg = storagecfg_table.get_or_default();

  storagecfg.max_file_size_in_bytes = max_file_size_in_bytes;
  storagecfg.global_upload_limit_per_day = global_upload_limit_per_day;

  storagecfg_table.set(storagecfg, get_self());
}

CONTRACT_END((setstoragecfg))

Much shorter! Since the actual contents of files and sites kept in LiquidStorage are not ever kept in state, we can do without vRAM in our contract.

If we’d like to test this, we can remove hello from our test file in the test folder since we’ve removed it from our contract. Then, run $ zeus test -c. Despite the evisceration we performed on our consumer contract, storage tests should still run with no problem.

OK, let’s take a look at what we’re actually doing in the storageconsumer.cpp code above. We don’t use the smart contract to store anything; rather, we use it to configure our dApp’s storage limits, so that users cannot overwhelm our infrastructure with massive amounts of uploaded content.

Currently, the following limits are supported:

1. max_file_size_in_bytes. This is the limit per file; the default is 10 megabytes. In this code, we set it to UINT64_MAX, which is an insanely large number. In practice, this is unlimited. Once our upload sizes are pushing a yottabyte — the largest data size with a standardized name — our descendants can work on enlarging this.
2. global_upload_limit_per_day. Also in bytes, this is the amount a single EOS account can upload to LiquidStorage each day. Its default is 1 gigabyte. Which, if we’d like to keep our perspective on UINT64_MAX, I should note is equal to 0.000000000000001 yottabytes.
3. vaccount_upload_limit_per_day. If LiquidAccounts are being used, this sets the limit on what a virtual LiquidAccount can upload per day. The default here is 10 megabytes. The reason for this low default is that if a dApp allows easy creation of LiquidAccounts with no spam protection, users may try to abuse the storage. (We removed this option when we stripped out LiquidAccounts.)

These limits are not hardcoded in this sample; they can be updated by executing thesetstoragecfg action with the appropriate new values.

For example, if our contract is deployed to Kylin, we could push the following command to set our max file size to 1 megabyte and our total daily limit to 10 megabytes. This example uses dfuse’s Kylin endpoint; and, of course, you’ll need to modify YOUR_ACCOUNT_HERE:

$ cleos -u https://kylin.eos.dfuse.io push transaction '{"delay_sec":0,"max_cpu_usage_ms":0,"actions":[{"account":"YOUR_ACCOUNT_HERE","name":"setstoragecfg","data":{"max_file_size_in_bytes":1000000,"global_upload_limit_per_day":100000000},"authorization":[{"actor":"YOUR_ACCOUNT_HERE","permission":"active"}]}]}'

Deploying the Contract and Activating the Service

The account for the contract meant to use LiquidStorage must be staking to one or more DAPP Service Provider LiquidStorage packages.

Deploying a contract and staking to services was covered in this walkthrough. For examples using the LiquidStorage package ID largestorage, see the LiquidStorage getting started page under the “Deploy Contract” and “Select and stake DAPP for DSP package” sections.

Modifying LiquidStorage Configuration

It’s fairly easy to update the limitations as conditions change. In order to set the maximum file size to 1 megabyte and the global sizes to 10 megabytes each, we would send the following command:

cleos -u https://kylin.eos.dfuse.io push transaction '{"delay_sec":0,"max_cpu_usage_ms":0,"actions":[{"account":"YOUR_ACCOUNT_HERE","name":"setstoragecfg","data":{"max_file_size_in_bytes":1000000,"global_upload_limit_per_day":100000000},"authorization":[{"actor":"YOUR_ACCOUNT_HERE","permission":"active"}]}]}'

If we had LiquidAccounts enabled and wanted a 10 megabyte global limit for virtual accounts, as well:

cleos -u https://kylin.eos.dfuse.io push transaction '{"delay_sec":0,"max_cpu_usage_ms":0,"actions":[{"account":"YOUR_ACCOUNT_HERE","name":"setstoragecfg","data":{"max_file_size_in_bytes":1000000,"global_upload_limit_per_day":100000000,"vaccount_upload_limit_per_day":100000000},"authorization":[{"actor":"YOUR_ACCOUNT_HERE","permission":"active"}]}]}'

Uploading a File to LiquidStorage

Now that the LiquidStorage service is set up in our smart contract, our app can take advantage of it to upload and retrieve files. Here’s a bit of example Node.js code. As in the above examples, we’re using Kylin.

This example uses the dapp-client library, whose getClient() function makes several DAPP Network operations much easier to work with. Run $ npm i -g @liquidapps/dapp-client in order to use dapp-client.

const { createClient } = require('@liquidapps/dapp-client');
const fetch = require('isomorphic-fetch');
const endpoint = "https://kylin-dsp-2.liquidapps.io";
const getClient = () => createClient( { network:"kylin", httpEndpoint: endpoint, fetch });

(async () => {
    const service = await (await getClient()).service('storage', "YOUR_ACCOUNT_HERE");
    const data = Buffer.from("a great success", "utf8");
    const key = "YOUR_ACTIVE_PRIVATE_KEY_HERE";
    const permission = "active";
    const response = await service.upload_public_file(
        data,
        key,
        permission
    );
    console.log(`response uri: ${response.uri}`);
})().catch((e) => { console.log(e); });

This will return and console log for us an IPFS URI (accessible, as you can see in the code, at response.uri). Now, we can easily access the file we’ve stored with LiquidStorage at that URI by passing the URI to service.get_uri.

Working with files in LiquidStorage is straightforward thanks to dapp-client:

• Use upload_public_file for regular accounts — orupload_public_archive to unpack a .tar file and upload all of the included files and/or folders.
• For LiquidAccounts, use upload_public_file_from_vaccount.
• Use get_uri to retrieve a file by its URI.
• Use unpin_public_file to delete files held in LiquidStorage.

You can examine examples for each of these methods here.

Developers — Use LiquidStorage to build a decentralized storage dApp that enables the end-user to choose from a range of redundancy and encryption levels, and you could win a bounty worth up to 250,000 DAPP tokens! Learn more about our DAPP Drive Bounty.

DAPP Drive Bounty

Follow LiquidApps

Website | Twitter | Telegram | LinkedIn | Github

Please click here to read an important disclaimer.