Photo by kazuend on Unsplash

Codiusd 1.2.X: Upgrading and WebSocket support

As the Codius network continues to grow, it becomes imperative that the hosts available on it are held to a number of expectations. This helps the network become more distributed, granting uploaders the certainty that the host they are uploading to will be able to support their apps.

Many web apps are designed using WebSockets, which requires additional features on top of HTTP. Due to their pervasiveness, we expect Codius hosts to have support for them to be part of the network. In order to encourage this, a self-test has been added with Codius v1.2.0 that runs at startup. This self-test includes testing for WebSocket compatibility, and failure to pass this or the other tests will bar the host from being discovered by the rest of the network. For many Codius hosts, explicit support for WebSockets will need to be enabled in their proxy configurations.

Many tutorials, such as the inaugural one by Stefan Thomas, used Nginx as a proxy. As such, this post will explain the steps necessary to upgrade a Codius host that’s using Nginx. If your host is set up in this manner, read on.

Note: This article assumes Codiusd is installed as described in the original Medium post or in a similar manner.

Updating the Codiusd Daemon

First, let’s make sure that Codiusd is up to date. Recent versions of Codiusd (starting with v.1.2.0) introduce a self-test feature on startup to ensure it has a proper configuration for participating in the network. Outdated versions of Codiusd will make no such checks, which may cause uploaders to become wary of such hosts.

Determine Codiusd’s package manager

Codiusd is hosted on NPM, meaning that there are a number of package managers that pull from it. Depending on the method by which Codiusd was installed, the package management portions may have been abstracted from you. We’ll go over the ways to check your installed packages using NPM and Yarn, two of the most popular NPM package managers.

Using NPM

npm list -g --depth=0 codiusd

Using Yarn

yarn global list --pattern "codiusd" --depth=0

If Codiusd does not appear, check your installation method to discern the installation method. Assuming that Codiusd’s package manager was either NPM or Yarn, it can be updated with the following:

Using NPM

npm update -g codiusd --unsafe-perm

Using Yarn

yarn global upgrade codiusd

The latest versions of Codiusd contain tests that are performed on itself whenever it starts up to ensure it is suitable for participation in the network. Aside from the obvious pod upload test, it also introduces a test for WebSocket support. Failure to pass these tests prevents the host in question from engaging with the rest of the Codius network. More on dealing with test failures later.

Enabling WebSockets in Nginx

To enable WebSocket support, a few additions will be made to the Nginx configuration. First, locate the Nginx config file that contains the configuration for the server listening on port 443. This may be in /etc/nginx/conf.d/codius.conf as per the original tutorial, but it may also be the main config file (/etc/nginx/nginx.conf ). In any case, it should contain a block that looks like this:

Unimportant config lines have been truncated with ellipsis for brevity.

server {
  listen 443 ssl;
  ....

  location / {
    proxy_pass http://127.0.0.1:3000;
    .....
  }
}

First, we’re going to add a couple of aliases to the configuration:

map $http_upgrade $connection_upgrade {
  default upgrade;
  '' $http_connection;
}

This block should be located outside of the server block, like so:

map $http_upgrade $connection_upgrade {
  default upgrade;
  '' $http_connection;
}
server {
  listen 443 ssl;
  ....
  location / {
    proxy_pass http://127.0.0.1:3000;
    .....
  }
}

Finally, we need to add some headers when the connection is proxied to Codiusd from the / path.

map $http_upgrade $connection_upgrade {
  default upgrade;
  '' $http_connection;
}
server {
  listen 443 ssl;
  ....
  location / {
    proxy_pass http://127.0.0.1:3000;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_buffering off;
    .....
  }
}

Note: The additional proxy fields MUST come before any other fields in the block!

These changes will be sufficient for your Nginx proxy to support WebSockets. Restart Hyperd and Nginx, then restart Codiusd and ensure it passed the startup tests:

systemctl restart hyperd
systemctl restart nginx
systemctl restart codiusd && journalctl -u codiusd -f

If all is well, the startup logs should report no errors and your Codius host will now have support for WebSocket connections.

Addressing Self-Test Failures

The self-test that is run on host startup mimics a pod upload request from a client. The pod that is uploaded to the host runs a WebSocket server, which the self-test then attempts to connect with. When these operations are successful, peer discovery is activated and the Codiusd process continues.

In the event the tests fail, the reason can be discerned from the logs. First, we’ll need to make sure the logs only output the information we need. To do this, make the following changes to your codiusd.service file. It is typically located in /usr/lib/codiusd/codiusd.service, but its location can be determined from the output of systemctl status codiusd. Inside the service file, locate the line containing Environment="DEBUG=*" or some variation thereof. Replace (or create it if it isn’t there) the line with Environment="DEBUG=codiusd*,ilp*,-*Peer*,-*trace*" in the [Service] section as shown:

[Unit]
Description=Codiusd
After=network.target nss-lookup.target
[Service]
ExecStart=/usr/bin/npm start
Environment="DEBUG=codiusd*,ilp*,-*Peer*,-*trace*"
Environment="CODIUS_PUBLIC_URI=https://codius.feraltc.com"
WorkingDirectory=/usr/lib/node_modules/codiusd
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=codiusd
User=root
Group=root
[Install]
WantedBy=multi-user.target

Then be sure to reload the service file before restarting:

systemctl daemon-reload
systemctl restart codiusd && journalctl -f -u codiusd

From here, the reason for the failure can be found. All messages regarding Self Test status can be found in the log lines that contain SelfTest and SelfTestCheck. The host will continue to run, but will not accept uploads externally, broadcast itself to the network, or accept network messages other hosts. Instead, any requests made to the host will return a message that gives the state of the Self Test, so it can be used for debugging.

The Self Test performs three tests that it reports on in case of failure:

• An upload test
• An HTTP connection test to the uploaded pod
• A WebSocket connection test to the uploaded pod

You can view the statuses of these tests by either visiting your host in the browser while it is in its failed state, or by using a tool like curl in the command line.

Final Words

The Codius network is growing fast, with an astounding number of hosts being launched since the announcement in June. But without consistency among hosts, it invalidates a portion of that achievement. To ensure that growth in the network is meaningful, we must enforce proper configuration for all members of the Codius network.

If you have any questions, feel free to ask them in our Gitter.