Autoscaling and High Availability

Cloud infrastructure is a boon to both providers and consumers of applications.

Providers can spend much less time fiddling with operational tasks like provisioning and configuration. While consumers (aka end-users) can expect applications to be available when they need it and to perform well under most conditions.

One of the key mechanisms provided by any cloud platform is the ability to very quickly provision new compute nodes (aka servers). With a click of a button, operators can spin up one, two or even fifty new nodes. Each of which will be consistently provisioned from a ‘golden image’ of an operating system.

Configuration management tools like Chef and Puppet allow for these nodes to be quickly and consistently configured. Each being made ready for an application or service to run within seconds or minutes.

After application deployment, the new node (or nodes) are ready to rock and can begin handling traffic from end-users.

The ability to rapidly and consistently build, configure and deploy enables two extremely important patterns: High Availability and Autoscaling

High Availability (HA) means that an application is operating in such a way that it can withstand severe conditions that might bring down an application not built with HA in mind.

The most basic form of the HA pattern requires that each node in the application architecture has a duplicate node ready to accept traffic. This redundancy means that a particular node can fail at any time and a backup node will automatically take its place.

HA can be improved by ensuring that redundant nodes exist in a physically separate environment, such as another physical server, rack, room or building. You can also increase HA by adding more redundant nodes, which will allow the application to continue functioning even after multiple nodes have failed.

Many cloud providers also enable improved HA by automatically restarting or even rebuilding failed nodes, such that an operator doesn’t need to intercede should a particular node fail.

Autoscaling is all about efficiency. Autoscaling is a mechanism by which a set of application nodes can be horizontally scaled up and down, in realtime, in response to a particular stimulus.

The most common use of autoscaling is for applications where usage varies greatly over a particular time period. For example, a system that is involved in quarter close activities is probably pretty quiet most of the time. The exception being at those four times per year when the flow of traffic jumps to process invoices and orders, only to slow again once the next quarter is under way.

During the times of increased load, the autoscaling system would add more nodes to the cluster until each of the nodes in the cluster is running at a reasonable load. Once the activity decreases, the autoscaling system will start to shrink the cluster by eliminating nodes until each of the remaining nodes is again running at a reasonable load.

Without autoscaling, the application would need enough servers to handle the peak load during quarter close. While during the lightly loaded in-between times, most of the servers would sit idle. Autoscaling can be a massive cost savings toolwhen averaged out over the course of a year.

HA and autoscaling are certainly related, but are not identical. The key difference is that autoscaling requires that an application be architected such that a node can be taken out of service at any time, without interrupting end-users. Whereas with HA, it is understood that a node dropping out of service is a sign of a severe error and thus some interruption is expected.

Autoscaling is more difficult to implement than HA, but offers many more benefits.

Both HA and autoscaling are patterns that are enabled by any reasonably robust cloud provider. But neither is available unless an application has been carefully developed to support them.

The HA and autoscaling patterns greatly improve the application experience of end-users by making applications more reliable, operators by reducing outages and performance issue, and to application owners by greatly reducing costs!