Keeping up with your Oasis node — Monitoring with PANIC Oasis

Francesco Cremona
Aug 20 · 4 min read

PANIC for Oasis is a lightweight yet powerful open source monitoring and alerting solution for Oasis nodes by Simply VC. The tool was built with user friendliness in mind, without excluding cool and useful features like phone calls for critical alerts and Telegram commands for increased control over your alerter.

Image for post
Image for post

What we’ll need:

  1. Node exporter on each node to export system statistics.
  2. An Oasis API server agent running beside each node.
  3. PANIC for Oasis running on a separate VM.
Demo of setting up PANIC for Oasis

Set up dependencies on both Oasis and PANIC hosts

$ sudo apt-get update
$ sudo apt-get install python3.6 -y
$ sudo apt-get install python3-pip -y
$ sudo pip3 install pipenv
$ curl -sSL https://get.docker.com/ | sh
$ sudo apt install docker-compose -y

Node exporter is used to export metrics related to each system’s hardware metrics, for alerting and monitoring of them through PANIC.

Create a Node Exporter user for running the exporter:

sudo useradd --no-create-home --shell /bin/false node_exporter

Download and extract the latest version of Node Exporter:

wget https://github.com/prometheus/node_exporter/releases/download/v1.0.1/node_exporter-1.0.1.linux-amd64.tar.gz
tar -xzvf node_exporter-1.0.1.darwin-amd64.tar.gz
sudo cp node_exporter-1.0.1.darwin-amd64/node_exporter /usr/local/bin/

Give the Node Exporter user ownership of the executable:

sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Perform some cleanup and create a Node Exporter service with the below contents:

sudo rm node_exporter-1.0.1.darwin-amd64 -rf
sudo nano /etc/systemd/system/node_exporter.service
[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target
[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

Reload systemctl services list, start the service and enable it to have it start on system restart:

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
sudo systemctl status node_exporter

CURL your node exporter port to confirm that it is not erroring:

curl http://127.0.0.1:9100/metrics

Install and Run the Oasis API Server

In order for the API to be able to run correctly, Prometheus metrics should be enabled in the Oasis node’s configuration file config.yml in the /serverdir/etc/ directory which was set during the installation of the Oasis node. To enable the metrics append this to the end of the file and restart your node:

metrics:
mode: pull
address: 0.0.0.0:9090

Change :9090 to the port you want prometheus to be exposed at.

Configuring the API

Configuring the API on the host of your Oasis VM involves setting up three config files.

Start off by cloning this repository and navigating into it:

git clone https://github.com/SimplyVC/oasis_api_server
cd oasis_api_server

You can run the Python script run_setup.py which will guide you through the configuration process, and ask you to enter the values one by one. This can be done by running the following, starting from the root project directory, oasis_api_server . Please note that sentry data does not need to be set up.

pipenv sync
pipenv run python run_setup.py

Installing the API

Run the following commands to build the Docker image:

cd oasis_api_server
docker build -t simplyvc/oasis_api_server:latest .

Running the Docker Image

Now that the Docker image is on your machine, and you have written configurations for it, you can run it as follows, where:

CONFIG_DIR for eg. /oasis_api_server/config

INTERNAL_SOCK_DIR for eg. /serverdir/node/internal.sock

PATH_IN_NODE_CONFIG for eg. /serverdir/node/internal.sock

Now that the Docker image is on your machine and you have written configuration for it, you can run it as follows:

docker run --network="host" -p 127.0.0.1:8080:8080 --mount type=bind,source=<CONFIG_DIR>,target="/app/config/" --mount type=bind,source=<INTERNAL_SOCK_DIR>,target=<PATH_IN_NODE_CONFIG> -d simplyvc/oasis_api_server:latest

Confirming the API Works

If you wish to make sure that the API is running, the following should return {"result":"pong"}:

curl -X GET http://localhost:8080/api/ping

If you wish to check the API’s connection to a node, you can run the following for some node <NODE>, being the identifier of the node to be tested.

curl -X GET http://localhost:8080/api/pingnode?name=<NODE>

Running PANIC

Start by cloning the panic_oasis repository and navigating into it:

git clone https://github.com/SimplyVC/panic_oasis
cd panic_oasis

At any stage, you can use the command docker ps to confirm that the installed component (Redis, MongoDB, the UI and alerter) is running

Now that you have it, set up your configuration files for PANIC’s components, including Redis, Mongo and UI authentication, togeter with the nodes to be monitored:

pipenv sync
pipenv run python run_alerter_setup.py
pipenv run python run_ui_setup.py

Turning on the alerter

To deploy Mongo, Redis, PANIC UI and the PANIC alerter, use the following command inside the panic_oasis directory:

docker-compose up -d --build

You should now be able to access the UI on a browser with your authentication details.

Image for post
Image for post

Stay tuned to Simply VC for more Oasis related guides and documentation:

Simply VC

A team of security, cryptocurrency & business experts…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store