Review of Discourse deployment strategies

An analysis on strategies for providing a Discourse forum service.

Evaluating four different strategies, implemented in three of the major cloud players, providing data with pros, cons and costs of each.

All the interesting investigation work and primary authoring credit goes to Luis Fernando Laimer

Introduction

Many companies rely on a good forum tool in order to improve communication with customers and communities. A forum enables people to create discussion topics and collectively contribute with ideas, votes and comments in order to improve the topic.

Although a simple concept, a good forum application requires features in order to enable a good communication flow and sharing of ideas. These features include, identity management, content management, tagging, analytics, attaching, notifications, a good UI/UX, community moderation, social media integration, digests and many other features that significantly increase the complexity of this kind of tool.
As most companies do not have the bandwidth to develop a tool with that many features, adopting an existing tool becomes almost a natural path for a great majority. Also, maintaining a forum server requires maintenance, such as updates, and managing backups which can be overwhelming for smaller organizations.

There are a considerable number of forum apps available in the market. Discourse, Flarum, Kialo, NodeBB, FluxBB, MyBB and several others, most of it open-source and free.

To make it even better, some of the alternatives also include a paid SaaS version of the forum, so companies do not have to manage infrastructure and maintain the tool.

For the purpose of this post, we will take a better look at the Discourse as a forum product, since it is the most widely used on the web3 ecosystem, and will do a deep dive into a few different deployment scenarios for this tool.

About Discourse

Discourse is one of the most complete forum solutions available in the market. It is open-source, but also has the option to be purchased as a SaaS. It covers multiple languages, SEO, an impressive number of administration features, a good user interface, lots of flexibility through different plug-ins available. Overall it is a great solution.

Discourse is written in Ruby and uses PostgreSQL and Redis. Redis is used specifically for live page updates, Rails cache, distributed rate limit and internal semaphore system, while PostgreSQL is widely used for meta, configs and content storage. MailHog is also used to interface with email servers (this one needs to be set up by the user).

All these components are normally configured through a bash script that packs them into a Docker container. The script also has the option for creating two Docker images to both execute on a single Docker client.

To improve performance using a CDN, it’s a standard CDN configuration with a single entry in the Discourse YAML config file.

Deploying Discourse

Although Discourse can be easily deployed using the setup script, there are a few important points of attention when running this installation:

• There is no redundancy on the system. If your machine, your Docker or your container go down, your forum will be unavailable.
• There is no native monitoring in the solution deployed via script

If availability is something crucial for your forum service, and you have a high SLA associated with it, it might be a good idea to start to think about redundancy.

I can think in a couple of options that can be explored when looking for a high availability architecture:

• Deploy Discourse into Kubernetes.
• Deploy Discourse using multiple hosts and an Application Load Balancer.
• Purchase SaaS service from Discourse.

In this post we will take a look at each one of these scenarios and compare the pros, cons and costs (as of Aug 2022). We will also compare with deploying to a single host using the provided configuration script.

(* denotes the provider recommended email solution)

Strategy: Kubernetes

Kubernetes is an open-source container orchestration system for automating software deployment, scaling, and management. Kubernetes dramatically increases the availability of container based applications. In order to deploy Discourse using Kubernetes, we require a Kubernetes cluster, a Network Storage and a Database server (PostgreSQL)

Pros:

• Extremely reliable solution
• Very easy to maintain
• Easy to implement CI/CD using Infrastructure as Code
• Simplified monitoring and logging solution.
• Email service can be deployed without extra cost (i.e. Mailu)
• No limits on views

Cons:

• Kubernetes is a very complex system and requires specialized knowledge to operate
• Cost can be a lot higher if you are not sharing this cluster with multiple applications
• Kubernetes was designed to work with microservices. Anything that maintain state on disk can be a challenge and will require better tunning to extract good performance
• Still have the cost of the external database.

Cost comparison:

In order to add some perspective into costs for this scenario, we have an estimate of the scenario in each of the main Cloud Services (AWS, GCP and Azure)

AWS:

• EKS Managed service
• 2x workers 2vCPU 16GB
• RDS PostgreSQL 2vCPU 7.5GB

Price/month: U$547.84

GCP:

• GKE Cluster Management
• 2x workers 2vCPU 16GB
• CloudSQL PostgreSQL 2vCPU 7.5GB

Price/month: U$452.42

Azure:

• AKS Management
• 2x workers 2vCPU 16GB
• Azure Database for PostgreSQL

Price/month: U$480.10

Strategy: Multiple hosts

Another valid alternative would be to create multiple hosts, in different zones and use an application load balancer to distribute traffic.

Pros:

• Good availability
• It is possible implement CI/CD
• Less complex approach than Kubernetes
• No limits on views

Cons:

• Requires external Postgres and Redis
• Not as flexible as Kubernetes
• Slightly more expensive than using a Kubernetes solution
• Requires monitoring and centralized log solution
• Requires email service

Cost Comparison breakdown:

AWS:

• 2x vm 2vCPU 16GB
• Amazon MemoryDB for Redis
• RDS PostgreSQL 2vCPU 7.5GB
• Elastic Load Balancer
• Simple Email Service (SES)

Price/month: 588.37

GCP:

• 2x vm 2vCPU 16GB
• CloudSQL PostgreSQL 2vCPU 7.5GB
• Memory store for Redis
• Cloud Load Balancing
• 1x VM 2vCPU 8GB RAM (Sendgrid, Mailgun or Jetmail images available for free)*

Price/month: U$401.80

Azure:

• 2x vms 2vCPU 16GB
• Azure Database for PostgreSQL
• Azure Cache for Redis
• Load Balancer
• SocketLabs Core subscription (for email relay)*

Price/month: U$583.51

Strategy: Discourse SaaS

The Discourse offers a paid product as a SaaS.

Pros:

• Very low complexity to set up
• Low maintenance overhead
• Very good availability
• Multiple tiers, which makes it the cheapest option

Cons:

• Data hosted outside your environment
• Increase in use may incur in a larger price

Cost comparison:

Standard:

• 100k month views
• 20GB storage
• 5 staff accounts

Price/month: U$100.00

Business:

• 500K month views
• 100GB storage
• 15 staff accounts

Price/month: U$300.00

Strategy: Single host

In this case, a single machine is used to run the containers from Discourse.

Pros:

• Low complexity setup
• Data ownership
• No limits on views

Cons:

• No fail-over architecture, which means that if machine, docker or discourse goes down, forum will become unavailable
• Requires monitoring and centralized log solution
• Requires email service

Price comparison:

AWS:

• 1x VM 4 vCPU 32GB
• Simple Email Service (SES)

Price/month: U$175.17

GCP:

• 1x VM 4 vCPU 32GB
• 1x VM 2vCPU 8GB RAM (Sendgrid, Mailgun or Jetmail images available for free)*

Price/month: U$180.96

Azure:

• 1x VM 4 vCPU 32GB
• SocketLabs Core subscription (for email relay)*

Price/month: U$ 244.45

Summary

Each proposed scenario has strengths and weaknesses.

As an example, if you already own a Kubernetes cluster running other systems, this might be the best cost benefit for you to deploy Discourse, since the cost will be diluted with multiple applications.

Looking at the table below we can have a good comparison between the costs of many different strategies/providers.

    :-----------------+-----------+---------:
    | Strategy        |           | USD$    |
    :-----------------+-----------+---------:
    | Kubernetes      | AWS       | 547.84  |
    |                 | GCP       | 452.42  |
    |                 | AZURE     | 480.10  |
    |-----------------+-----------+---------|
    | Multiple Hosts  | AWS       | 588.37  |
    |                 | GCP       | 401.80  |
    |                 | AZURE     | 583.51  |
    |-----------------+-----------+---------|
    | Single Host     | AWS       | 175.19  |
    |                 | GCP       | 180.96  |
    |                 | AZURE     | 244.45  |
    |-----------------+-----------+---------|
    | SaaS            | Standard  | 100.00  |
    |                 | Business  | 300.00  |
    '-----------------'-----------'---------'

• In the case your Discourse data cannot be hosted in a 3rd party either because of compliance or because of security concerns (some compliances will require that the 3rd party have an appropriate security certification)
• In the case your forum has a huge amount of access, this would bump the price from SaaS significantly. In this case you will require to choose between multiple hosts or Kubernetes (considering that in this case, SLA will most likely be an important factor for your business)

For the SaaS product, we can have a less expensive, low complexity forum, with no maintenance costs. It is crucial here data storage management and control over crawlers and web spiders access, since this could easily bump your tier from Standard to Business.

Overall the SaaS should cover most of the needs for the majority of small to medium size organizations.

Conclusion

This study enabled the Windranger team to make a decision regarding our current choice of Discourse deployment strategy. After analyzing the data, a decision was made to transition from a single host strategy into a SaaS strategy for the forum discourse.bitdao.io, since this option was better in many different aspects such as maintenance, availability and cost.