Deploying Ruby on Rails with AnyCable using Docker & ECS

This is a guide for setting up AnyCable locally using Docker Compose, and deploying it on Amazon Web Services (AWS) Elastic Container Service (ECS). The purpose of this article is not to explain what AnyCable is, so if this is the first time hearing about this awesome Ruby gem, I recommend checking out the AnyCable documentation website.

I’m using Rails 7, Ruby 3.1.1. and Redis 4 gem in this application.

The following steps outline how to setup AnyCable in Docker-Compose:

1. Add gem 'anycable-rails', '~> 1.3', '>=1.3.4' to your Gemfile and run bundle install.
2. Run bundle exec rails g anycable:setup
3. Ensure your config/cable.yml file looks similar to this:

# config/cable.yml
development:
  adapter: <%= ENV.fetch("ACTION_CABLE_ADAPTER", "any_cable") %>

test:
  adapter: test

production:
  adapter: <%= ENV.fetch("ACTION_CABLE_ADAPTER", "any_cable") %>

4. Add the config below to config/environments/development.rb and production.rb located in the same folder directory.

config.after_initialize do
  config.action_cable.url = ActionCable.server.config.url = ENV.fetch("CABLE_URL", "ws://localhost:8085/cable") if AnyCable::Rails.enabled?
end

5. If you are using cookie-based authentication, make sure your cookies are accessible from both the HTTP server and WebSocket server domain:

  config.cache_store = :redis_cache_store, {driver: :hiredis, url: ENV.fetch('REDIS_URL', 'redis://redis:6379')}
  config.session_store :cache_store,
    key: "_session",
    domain: :all,
    compress: true,
    pool_size: ENV.fetch("REDIS_CONNECTION_POOL_SIZE", 5).to_i,
    expire_after: 1.year

6. Create websocket and anycable services in docker-compose.yml

docker-compose.yml

We’ve already done enough to actually run AnyCable locally using docker-compose. Go ahead and give it a try, docker-compose up and check out the logs. They should be similar to below:

docker-compose websocker service logs

docker compose anycable service logs

However, actually deploying AnyCable is a slightly different story, and we had much more of a struggle, or fun depending on your point of view. We deployed it to AWS ECS.

I’d like to give a huge shout-out to Connor Reid and Joshua Christensen, the two eXp-Realty DevOps geniuses I am blessed to work with. They invented the architecture I’m about to present, and below is a diagram illustrating the solution we ended up implementing.

For the sake of clarity, I skipped some AWS services here, like VPC, subnets, availability zones, etc..

It starts with a client hitting either our Rails’ apps’ endpoint or a Websocket service endpoint (specified in CABLE_URL ENV). An Application Load Balancer listener is set up in order to forward HTTPS requests to our Rails app and websocket requests to a websocket service. A Websocket service is just an official AnyCable-go docker image packed in a container.

The Websocket Service needs to communicate with the AnyCable Service. Due to encountering similar issues when setting up the Application Load Balancer — as described in this blog post — we also switched to a Network Load Balancer. Unlike an Application Load Balancer, which works on Layer 7 of OSI model (Application Layer), a Network Load Balancer works on Layer 4 (Transport Layer). It is not familiar with the concept of HTTP; listener options are limited to only TCP, TCP_UDP, TLS, and UDP.

We set it to listen on TCP port 50051 and forward traffic to the RPC service target group. Adding a health check here was as simple as specifying the protocol and port on which the load balancer will send requests. It’ll check if the service responds.

One last piece was to create a private hosted zone on Route 53 and set an alias to point to our Network Load Balancer. We saved the private hosted zone domain name as an ENV ANYCABLE_RPC_HOST in the WebSocket service.

Given that all your services’ security groups are set up to allow traffic from each other (such as RDS and ElastiCache to allow inbound traffic from AnyCable), it all should work. Is there something you would do differently, have a suggestion, question or point of clarity? Please let us know in the comments!