SUPERALGOS HOW TO

Introducing Postgres To Superalgos

How to setup Postgres with Superalgos

Photo by Tobias Fischer on Unsplash

Introduction

Here we will install a new Postgres database server setting up a new user and database for the Superalgos network to use. We will then learn how to create a profile and run a network server with the new database backing it.

This is currently only applicable to the develop branch as these changes have not been merged into the master branch yet.

Why add a database?

The app stores a lot of data, particularly user data and websocket connections in memory, at the moment the user base is not huge so the memory footprint is not big, but as the project grows this footprint will increase. Using a separate database will allow the app to offload some of this memory use and be able to retrieve the data it wants on demand.

A good use case for this is for signal broadcasters, in order to distribute signals the network service will read all the registered user profiles into memory and each profile has its token balance fetched from the blockchain which is then stored in memory. When a user registers to follow a signal, the user has their balance checked, their ranking assigned and their websocket connection all stored in memory in order to distribute the signal correctly.

With this data stored in a database, the app can fetch the user balances in order, saving on having to create and maintain an in memory ranking — the fetching of balances previous ran on the same thread as the signal distribution, the introduction of using a datastore (file or postgres) has allowed this fetching process to operate on a separate thread, leaving allowing the distribution thread to operate with a more singular focus. The websockets connections can also be stored in the database with a connection status and a link to a user profile, allowing the network to pull all active connections when a signal is ready to distribute. The upcoming release of the social trading app would also benefit from the use of a database to allow for easier organisation of the models used to organise a distribute information.

Install Postgres

Option 1: Install on your system

https://www.postgresql.org/download/ has installers for all the major operating systems, find the right one for you, download and install.

Now you have Postgres installed you will want to create a database and username/password for Superalgos to use. The Postgres docs are very thorough so I will skim through the steps, if you have issues or want to know more then please read the docs.

# Create a database named 'superalgos'
createdb superalgos

# Create a user named 'superalgos' 
# the user will have superuser privileges (-s), if you want to restrict this that is for you to investigate
# then you will be prompted to enter a password (-P)
sudo -u postgres createuser superalgos -s -P

You will now have a database, username and password to connect with Superalgos.

Option 2: Run as a docker container

Like with the system installation above Postgres has a nice blog post which is pretty comprehensive on using the docker image, so I will be quite brief.

We want to be running these commands from your Superalgos directory, this way we can mount a volume so the database storage is persisted into the Platform/My-Data-Storage directory

docker run -tid \
--name superalgos-postgres \
-e POSTGRES_PASSWORD=superalgos \
-e POSTGRES_USER=superalgos \
-v $(pwd)/Platform/My-Data-Storage/pgdata:/var/lib/postgresql/data /
-p 5432:5432 \
postgres:15

docker ps 
# this will list all the running docker containers
# you should see the above container in the list

The above command will pull down the latest stable version (15) of the Postgres docker image, and using the 2 environment (-e) arguments we can set a password and create a user/database combination when the container spins up, it will also expose the port 5432 to your system and continue to run as a daemon process.

Creating a suitable profile

Now we head back into Superalgos. Here we can create a new profile, if you have not used profiles in Superalgos before then not to worry it is quite simple. A profile is a set of parameters that can be used to override some of the default settings. If you have not got one, in your user home directory create a new directory ~/.superalgos and inside that create a new file profiles without a file extension. Open the file in your favourite text editor and we will create a JSON array with our new profile object inside.

[
  {
    "name": "network-db",
    "database": {
      "type": "database",
      "database": "superalgos",
      "host": "localhost",
      "password": "superalgos",
      "port": 5432,
      "user": "superalgos",
      "users_table": "users"
    }
  }
]

Above is the minimum you need to setup to run the network service with your new Postgres database, if you wanted to run this with a specific network in you user profile, for demonstration imagine I have a network called ‘My-Profitable-Network’ then I would update the config to look like this:

[
  {
    "name": "network-db",
    "p2pNetworkNodeSigningAccount": "My-Profitable-Network",
    "database": {
      ...
    }
  }
]

Now we have our profile setup we can either run it using a service file, pm2 or manually from the CLI.

The manual CLI approach is a good idea to test it all works:

Superalgos$ PROFILE_NAME=network-db node network

If you want to run this in your service file or via pm2 then you just need to add the PROFILE_NAME variable to the environment variables for that process.

At the time of writing we are just storing user balances, but as time progresses we will start adding more tables to the database.