Linear Scalability
An insight into building forever scalable systems.
Thanks to the growth of ‘the cloud’, most large scale applications are built in a world where you can provision systems somewhat limitlessly (I suspect there is a point where AWS would run out of EC2 instances, but I wouldn’t want to guess how much you’d be spending…)
As cloud offerings have evolved, however, we’ve moved from buying units of ‘metal’ to buying ‘functionality’; or, more specifically, ‘throughput’. Within the AWS ecosystem that manifests itself in a database you provision by reads-and-writes (Dynamo/Aurora), a streaming system paid for in flow (Kinesis) and an execution model bought by units-of-effort (Lambda).
What’s important about these ‘throughput’ services is that they allow us to build ‘infinitely’ scalable services.
If you can realise your application out of these building blocks, then all you need to do to make it scalable is keep your functionality within the the constant time performance of the service interfaces.
Consider an application that processes a stream of data for a user. If you build your processing function so that it takes less time to execute than the flowrate limit of the stream (i.e. 1000 records/second in Kinesis), then it doesn’t matter how many users you get, because you can just keep growing your shard count (and concurrent lambda executions).
This approach of stitching together scalable services has a lot of advantages. Firstly, it’s a very easy win to grow a system that will scale without much effort. It also gives you a clear performance target- so long as your code runs within the interface limit, your system will scale; something which can be easily tested. And thanks to the pricing models of these services- you also get a constant(-ish) cost per unit-throughput.
Of course, there is a limit to how much the connected services will scale (A single Kinesis stream tops out at 80k records/second, for example). At which point you’ll need to start doing your own second-order scaling. However, as limits are advertised, you’ll be in the enviable position of knowing exactly when this will happen, and can monitor and prepare accordingly.
Just don’t look at your AWS bill…
