Self-hosting a CTF Platform (CTFd)

The challenges page from csictf 2020.

Introduction

This article is a part of a series describing the work that went into setting up the infra for csictf 2020. In this post, we’ll be going over how we set up the CTFd platform. Since this is a CTF, we also talk about some extra tips to configure:

1. Rate limiting requests on the server, to mitigate brute-force/DDOS attacks
2. A firewall to only allow connections on some ports
3. SSL certificates (https) for the server
4. Logging requests correctly to trace back illegitimate activity

The CTFd Platform

CTFd/CTFd

CTFd is a popular open-source platform used by many CTF events. It’s easy to use, and has a featureful admin panel that shows useful statistics during the CTF, and also allows you to perform common user/team management tasks.

Admin Panel stats from csictf 2020

There is also a fork of CTFd available which supports logging into your CTF through CTFTime using OAuth. For our CTF, we initially planned on using the fork, but later found that it was outdated compared to the upstream repo, so we created our own fork which is a bit more up to date and provides the CTFtime OAuth feature as well. Regardless, the instructions in this post should work with both our fork and the mainline CTFd repo.

Login with CTFTime

csivitu/CTFd

Running CTFd on your Server

There are two ways to deploy CTFd on your server:

1. You can clone the repo, manually install its dependencies using pip on your server, and also configure MySQL and Redis databases manually.
2. You can use the docker-compose.yml file present on the CTFd repo to conveniently deploy each component on your server in separate containers.

We highly recommend option 2, as it helps later to also easily migrate your CTF to another server if at all, you need to.

So let’s start setting up the platform.

First, you want to start by ensuring your server has docker and docker-compose installed. There are instructions available in the official docker docs, but here is how we usually set it up on a server running Ubuntu 20.04/18.04/16.04 anyway:

Now, clone the CTFd repo on your server:

git clone https://github.com/csivitu/ctfd

cd CTFd

If you want to setup CTFTime authentication, you’ll need to add the SECRET_KEY , OAUTH_PROVIDER , OAUTH_CALLBACK , OAUTH_CLIENT_ID , and OAUTH_CLIENT_SECRET environment variables in the docker-compose.yml file in the repo we just cloned. (Not that this will only work if you cloned our fork that supports CTFtime OAuth in the previous step!)

Here’s our docker-compose.yml file (don’t forget to replace the values where prompted):

Another important option to note in the above yml file is the WORKERS option. This defines how many (gunicorn worker) instances of CTFd will be running. As a general thumb rule, the no of workers you need is proportional to the amount of traffic you expect. The value that we used throughout csictf was 10 workers. (For stats on how much load csictf experienced, and how to test what value works right for you, there is another article on statistics coming soon :))

To startup CTFd now, cd inside your CTFd repo, and run:

docker-compose up -d

Be patient, this will take a while the first time, but it should eventually startup CTFd running on port 8000 on your server.

Navigate to port 8000 on <YOUR_SERVER_IP>:8000 or if you have linked a domain to your server, then at <your domain>:8000 on a browser.

Complete the initial setup for CTFd as prompted on the browser page, and you should be able to now access the Admin Panel to manage your CTF!

Setting up a Firewall, Nginx and Rate Limiting

NGINX | High-Performance Load Balancer, Web Server, & Reverse Proxy

Nginx is a reverse proxy server, i.e, its job is to accept incoming connections to your server, and route them to another server running on a machine. We will be setting up Nginx and configuring it to do the following things:

1. Previously, you accessed your server using <your domain>:8000 we will instead route <your domain> to CTFd running on port 8000 automatically.
2. We’ll set up rate limiting to limit both the no of requests per second to CTFd and also the maximum no of simultaneous connections from a single host.
3. (optional) If you’re using Cloudflare, we’ll also reconfigure Nginx to correctly log the original user’s IP address, instead of logging only Cloudflare IPs in Nginx logs.

First, start off by installing ufw (a firewall service) and nginx on the server:

sudo apt update

sudo apt install nginx ufw

Now, allow ssh, HTTP, and HTTPS through the firewall:

sudo ufw allow 'Nginx Full'

sudo ufw allow 'OpenSSH'

And finally, enable the firewall :)

sudo ufw enable

At this point, visiting your domain should show the default Nginx page, let’s reconfigure it to show CTFd instead.

Create a file at /etc/nginx/sites-available/mydomain.com (replace mydomain.com with your domain) with the following contents. This sets up rate-limiting at 10 requests per second, and a max of 10 simultaneous connections per IP address at a time, and also tells Nginx to route requests to mydomain.com at port 8000.

Important Note: If you are using Cloudflare, replaces $binary_remote_addr in the above snippet with $http_cf_connecting_ip . This is done because we want to limit requests on a per-user IP basis, not on a per Cloudflare server IP basis.

Create a symlink to the file you created in the previous step in /etc/nginx/sites-enabled and reload Nginx, and we’re done!

sudo ln -s /etc/nginx/sites-available/mydomain.com /etc/nginx/sites-enabled/mydomain.com

sudo nginx -s reload

Try visiting mydomain.com in a browser, it should now show the CTFd page, and also rate limit your requests!

BONUS (Only if you’re using Cloudflare)

Currently, the default Nginx configuration will log every request in /var/log/access.log but there is a slight issue: the origin IP of each request is going to be a CF server (if you are using Cloudflare). This is bad because if we want to trace back illegitimate user requests during the CTF, we won’t be able to trace it back to the user.

We can fix this by making a small change in the http section in /etc/nginx/default to log the real User IP instead:

Run sudo systemctl restart nginx and now, Nginx should log the real user IP instead of CF IPs!

Setting up https:// using certbot and LetsEncrypt

Certbot

SSL Certificates normally cost money $$, but we’re gonna use Let’s Encrypt, a service whose primary goal is to provide free certificates to websites on the internet.

certbot is a great CLI tool to automatically provision Let’s Encrypt certificates for your website, and also handle automatic renewals of certificates.

You can refer to the amazing documentation at certbot’s official site for instructions, but here is how you can set it up on a Ubuntu machine:

Once you have certbot installed, run

sudo certbot --nginx

This should prompt you to pick which domains you want to setup nginx for. Select the domain we setup with nginx before for CTFd, and follow the series of questions certbot asks. Make sure you ask certbot to always Redirect to HTTPS, we don’t want anyone accessing the website over HTTP!

Once you have answered all the prompts, try going to your domain, you should see it redirect to https://.

Your browser should now show that the connection to the site is secure

There’s more…

You should now have a stable platform set up for users to register, login and view challenges/submit flags on your CTF.

But this article was just the tip of the iceberg in what goes into setting up efficient, fast, and more importantly a stable CTF.

If you liked this article and would like to know more about behind the scenes in conducting a successful CTF, click the link below for other articles in this series, where we cover other aspects, like load testing, reference traffic statistics from our CTF, recommend machine sizes, containerizing/deploying/autoscaling challenges with Docker and Kubernetes, CI/CD and more!

csictf

csivitu/ctf-challenges