A Startup’s Guide to “the Cloud”
At Rho AI, we constantly evaluate the latest & greatest technologies to see what works best and where we can improve our products. A shift over the last several years has been the explosion of ‘managed services,’ which we define as cloud providers offering SaaS products for everything from databases to computational tasks.
It is essential that technical organizations continuously evaluate what technologies they use, why they are using them, and who is managing them. While it is important to avoid shiny object syndrome and modify your tech stack too frequently, it is equally as important to ensure you’re not leaving money and/or valuable engineering time on the table simply to support legacy choices.
Below are Rho AI’s thoughts on the current landscape, along with two checklists your organization can use. The “Cloud Provider Evaluation Process” can be used to determine which cloud provider is best for you and the “Managed Service Evaluation Checklist” can be used both when a new component of your tech stack needs to be chosen and used periodically to evaluate your existing tech stack.
Cloud Provider Evaluation Process
Although it’s fun to say, ‘cloud agnostic’ and there are some circumstances under which you want to use multiple clouds, the majority of use cases will reside on a single cloud. Therefore, it’s important to try picking the best option up front.
- All of major cloud providers are going to offer similar services at similar prices.
- Typically, you will find very specific use cases that are more, or less, cost effective on each provider based on total compute power, storage requirements, network load, etc.
- Most frequently, you can smooth out the cost differences by shifts to your architecture and efficiently using the provider’s levers for saving money.
- For the majority of businesses, the cost differences will be in the noise if you’re taking advantage of standard cost saving techniques.
- All of the major providers are very responsive to each other, so it doesn’t take too long for one provider to offer an identical alternative when a new service is offered.
- Ultimately, for a startup, it’s most important that the development team is comfortable with the provider and is able to deploy applications quickly and securely.
- People get very skittish about ‘vendor lock-in’ but, generally, the fear is overblown if a team follows some common-sense procedures when choosing where and how to deploy application services (see the “Managed Service Evaluation Checklist” below). In many cases it’s highly cost-effective to use a managed service (which creates an element of vendor lock-in) to begin and then migrate to a self-managed implementation as you scale if necessary for cost reduction and/or vendor flexibility.
1) Determine what your architecture/workload/general product profile looks like.
- Will your infrastructure be generally static (which can be optimized using reserved instances) or ephemeral (which can take advantage of spot markets)?
- Do you have network intensive tasks that will require very fast networking and/or cheap network transfer costs?
- Do you require a geographically distributed infrastructure that requires a lot of options for data centers?
- Is your application/product storage heavy such that it that requires very cheap long-term storage?
- Do you have specific security and compliance requirements?
- Is there a particular type of computing you are doing that requires special hardware to run? One clear example of this is whether you’ll need GPUs for computation.
2) What does your team look like?
- Does anyone on the team have proficiency in any particular cloud?
- Is your team large enough to handle self-hosting most services or are you small enough that you’ll want to take advantage of a lot of managed services for databases, etc.?
- How much control does your team want/need over the intricacies of the underlying network?
- Who gives you the most up front money?
- Does any one of the providers have a managed service that you already know you want/need?
Most typically, the responses to 1) and 2) fall somewhere on a spectrum of options such that there’s no perfect provider that matches everything. Broadly speaking, we feel that AWS is the clear leader and will continue to be. The other providers in the space are playing catch-up and have their own niches they play up, but in almost every case AWS has an answer.
Managed Service Evaluation Checklist:
Regardless of which cloud provider you choose, you’ll then need to constantly evaluate whether to self-host or pay for a managed service for the various components of your product(s). This relates to pretty much everything (your log aggregation system, databases, web servers, file storage, etc.). The results of this checklist vary greatly, which is why there’s no set procedure such as “always start with a managed service, then move to self-hosting at scale”.
We recommend you follow this checklist any time you are adding a new component to your tech stack and periodically use this checklist to audit your existing systems.
- What self-host options exist, if any?
- How mature are the self-host options in terms of stability, support, etc.?
- How much time will it take to deploy this service manually?
- How much time will it take to maintain this service once deployed?
- How much will it cost to self-host this solution?
- What managed services exist (both on my cloud provider and from third-parties)?
- How much time will it take to deploy the managed service?
- How much time will it take to maintain the managed service?
- How much will it cost to use the managed service?
- What SLA is attached to the managed service?
- How difficult will it be to migrate off of this managed service if I want/need to?
The landscape for cloud providers and, more so, managed cloud services continues to change rapidly. If you review your technology choices often and with a critical eye you can often realize big gains (both in terms of performance and your bottom line) by optimizing what technologies you use and who manages them.
by Gilman Callsen • @gcallsen • Founder & CTO of Rho AI
Note: Rho AI has evaluated Azure and Google Cloud services in non-production environments. We have never deployed any of our production workloads to either cloud.