An approach to the cloud — tips on scaling infrastructure
Tips on building your company’s infrastructure at different stages.
The speed of tech advancement right now is mind boggling. To thrive, businesses need infrastructures that can adapt quickly and efficiently to the newest tech developments — in other words, they need to be able to scale. Whether your company is a brand new start-up or an age-old corporation, predicting technology’s future (and more importantly, preparing for it) is virtually impossible. So how can businesses, big and small, embrace this change for the better? Surprisingly, the answer to that question doesn’t live in glitzy new AI or data applications. It lives somewhere that many might think is a bit more mundane: in server technology. This is where the next big stage in tech transformation has already begun to take place, and to understand what this really means for you it’s crucial, first, to understand its evolution.
Made by IBM during the sixties and used mainly by governments and corporations, big budget, monolithic applications were mostly written in one language with each programme and piece of code entangled. This meant that if one part of the system failed, everything failed and scaling up simply meant getting bigger and fatter. In those days agility wasn’t even on the horizon. Now, however, it’s totally different. When it comes to building your company’s infrastructure there are countless options available. Knowing which one to go for is an art in itself, so whether you’re just starting out or you run a corporate enterprise, here are a few tips to help you navigate this maze…
Let’s say you’re running a brand new startup: you’re in seed stage so completing your proof of concept (POC) is top priority. As money and time are the most important factors at this point, your infrastructure will naturally play a pretty marginal role. All you need is an architecture that’s simple and quick to set up. The best option is to create one relatively small instance on a Platform as Service (PaaS) of your choice, you might choose Amazon’s EC2 for example. Using this platform you can create a small database to support your POC (a relational database structure [RDS] is a good choice, as this will create your database for you). If things go well and your company grows fast, you should consider using a continuous integration deployment system to save time tinkering with code later on.
So, your POC was successful. Now what? Well, your company is still in the early stages so you’ll be focussing on your minimum viable product (MVP). Time and money are still the biggest factors, but your product needs to perform even better than before. It’s a good idea to run it on one medium size instance. Stick with an RDS or Cloud SQL fully managed solution. Quality assurance and continuous integration are a must.
Post-MVP is where it can get tricky. Up until now you won’t have had many (if any) paying customers, and you certainly won’t have had users in different countries. Now, you’ll likely have both. To deal with this growth as seamlessly as possible, your infrastructure will need two things; redundancies and load handling. Naturally, to deal with the extra workload you will need to increase your instances (say, two medium to large servers) and connect them on a load balancer. As a safety measure — and because you really don’t want 10,000 unhappy customers — you ought to create replicas of both your database and your application. Combine this with a Content Delivery Network and you’ll achieve low latency in your product’s performance, allowing it to perform well under pressure; an essential factor when your company is beginning to attract attention.
When you’re past the 10,000 customer mark and heading somewhere towards 100,000, you need to start being diligent about where and how your content is being distributed. Start shifting your static files onto simple storage devices instead of instances (Amazon’s S3 is a good option) and create redundancies in your infrastructure’s different available zones. And wherever you can, start caching. This will save a lot of precious time in the future.
Now scaling. When you’re somewhere between 100,000 and 500,000 users, your product really needs to able to adapt. Start integrating monitoring systems (e.g. AWS Cloud Watch or Stackdriver monitoring) and connect those to auto-scaling and log analysis systems. For even more users you’ll need to have a deployment orchestration that runs updates on the widespread distribution system. This architecture will be caching your database regularly, making it extremely reliable and cost effective.
For bigger and older companies, the challenge is different. With infrastructures that can date back twenty odd years, corporations are starting to see their ambitions curtailed by legacy systems. For those wary of updating old systems, the option is there to maintain legacy software by building on existing infrastructure (probably what Microfocus will do after acquiring a piece of HP for £6.6billion). It’s obvious that a new infrastructure is needed to keep up with demand, and big, successful companies like Adobe, Spotify and many others have all started to adapt their infrastructures for the better.
Whether or not this is the right move for you, entirely depends on where you’re at. In all sectors going forward, all services and products will require tech systems that are intrinsically flexible, scalable and cost-effective. Standing still is not an option — the way your company moves forward however, is entirely up to you.
