Are Ruby on Rails applications scalable?

Ruby on rails is a great framework for rapid web application development, which makes it an ideal tool for a startup. But we often hear people talk about Rails scalability issues when a startup project grows too large. Being a Java developer myself I too had my set of reservations on ROR scalability. I have seen many a discussions around Java vs Ruby and inadvertently someone pointing out that ‘Twitter moved to scala from ROR’ and hence ROR is not scalable. So does ROR really doesn’t scale? Do we need to ditch this framework if our app goes really big?

Well not really. There are many popular large scale companies which have scaled ROR applications. Shopify, Airbnb, Github and many others using advanced Rails application have scaled quite well for many years in a row and continue to do so.

What Is Rails Scalability?

The scalability of an application is a measure of the number of users it can effectively support at the same time. An application built with any framework should be able to grow and manage more user requests per minute (RPM) in the future. It’s incorrect to talk about framework scalability or Ruby on Rails scalability because it’s not the framework that must (or can) scale, but rather the architecture of the entire server system.

So let us look at how a simple rails application looks and how we have evolved our system to handle large traffic.

Rails deployment architecture

Simple Rails Setup

One Rails instance handles all requests. Rails is single-threaded: There is only one concurrent request.

Typical Rails Setup

• A load-balancer distributes the incoming requests
• Some load-balancers will deliver static requests themselves
• Several Rails instances handle all requests

Number of concurrent requests equals number of Rails instances.

Horizontal scaling using Application server (Phusion Passenger)

We can scale a Rails application horizontally similarly to how we scale many other frameworks. Horizontal scaling means converting the single server architecture of your app to a three-tier architecture, where the server and load balancer, Rails app instances, and database instances are located on different servers. In such a way, we allocate equal and smaller loads among machines. By adding Passenger app server we can span multiple rails processes in a single server instance.

Moving towards SOA and micro services.

So far the architecture we used is monolithic. This type of architecture has some problems:

• Main controllers and models have a lot of logic
• Merge issues arise in big team
• Lots of contributors and no ownership
• Difficult deployments with long integration cycles
• Tests are not green, it’s really hard to support stable test quality

Solution was to divide the application into small and independent pieces of service.

So our product not long before was a monolith application. With the growing product, it was imperative that we divide the application into multiple independently managed services.

Considerations of Microservices:

• Loosely Coupled: You must be able to deploy a single microservice independently.
• Small & Focused: Microservices need to focus on a unit of work and are small.However, there are no rules on how small a microservice must be.
• Language Neutral: Microservices do not need to be written using the same/specific programming language.
• Bounded Context: Particular microservice does not “know” anything about underlying implementation of other microservices surrounding it.

The current architecture looks something like:

• So CDN acts as a proxy and redirects to the appropriate load balancer
• Some of the services are built on JRuby(More on that in a later post)
• Services interact with each other through clearly defined API contracts

Performance Optimizations

Rails application performance can be improved in number of ways. Some of the optimization we did include:

• Caching — Both in memory and distributed shared cache.
• Solving N+1 query problem.
• Code refactoring and optimizations
• Async jobs using resque workers.

Fine tuning Apache and passenger

Moving to mirco-service architecture helped us in scaling the application and build modular components. But our core service was still running on too many servers. It was time to address growing infrastructure usage and cost.

Background:

Our CPU usage up until now has been less than 20% and memory usage of less than 35% even during peak traffic hours. It is not efficient in terms of usage as well as cost if we don’t fully utilize our compute power. Keeping that in mind we did the following config changes to minimize the infra usage which will intrinsically bring down the cost.

• Enabled apache KeepAlive. This will ensure that http connections are kept alive for certain configured time. It will reduce the overhead of establishing connection with each request.
• Upgraded Passenger to latest 5.1.2 version
• PassengerMinInstances is set to 8. This will ensure that at any given point of time at least 8 passenger process are always running.
• PassengerMaxPreloaderIdleTime set to 0. Preloader handles spawning of new passenger processes. As long as the preloader is running, the time to spawn a Ruby application process only takes about 10% of the time that is normally needed.
• PassengerStatThrottleRate set to 300. This setting tells Passenger how frequently it has to performs several filesystem checks(auto detection of apps, restart app etc.)
• PassengerMaxRequests set to 100000. Max number of requests each rails process will handle. This prevent memory hogging by single process.
• db pool size set to 30.
• Scheduled based Auto Scaling configuration.

Future Improvements Scope:

We plan to do further improvements to improve the system efficiency.

• Move from Apache prefork-mpm to worker-mpm.
• Move db writes to master db and db reads to salve. I have already made the code changes for this. Need to do a separate launch to monitor.
• Keep identifying and improving our existing APIs.

Results:

So the service which was running on 10 servers before, after the tuning is now running on 3 servers. All this without having any impact on the performance of the application.

System Usage

Cost Analysis

The server cost was around 130 USD/day till the month of March 2017. After the fine tuning the cost has been reduced to around 50 USD/day. That is whopping 62% reduction in server cost.

Cost Per Day

Conclusion

Like any language and framework, Ruby on Rails based applications can be scaled. To repeat what Shopify said several years ago at a conference dedicated to Ruby on Rails scalability: “There isn’t any magic formula to scaling a Rails application”. However, depending upon the characteristics of the applications, by doing infrastructure & performance optimizations and fine tuning the deployment configuration scalability of ROR applications can be achieved.