The Cloud, Anything-as-a-Service (XaaS) and Infrastructure as Code (IaC)
[Article 6 of 6 in series Software Delivery, DevOps and CICD for the uninitiated]
In this article, we will cover:
- the various types of cloud service (Software, Platform, Infrastructure)
- why their use can accelerate product delivery by shedding extraneous burdens.
- Automating cloud based Infrastructure with Infrastructure-as-Code to offer speed and provide environmental consistency.
Introduction
Delivering and providing software requires all sorts of technological components. So far in these series we have discussed many of the CICD tools needed (Source Code Repository, Binary Repository, Automated Test Execution Engines, Code Quality Inspection, Configuration Manager), but there’s also the components that make up your application itself (e.g. servers for Database, Web, Application, Authentication, Notification, etc.) as well as for protecting it (e.g. Firewall), monitoring it (e.g. Alerts, SLA Dashboard) and planning/managing it (e.g. Bug tracking, Customer Service) etc.
All this technology requires:
- physical hardware; infrastructure such as servers, networks and storage.
- human resource; high skills and lots of time
Sourcing and managing both of these yourself gives great control, but can require high levels of upfront, and ongoing investment, as well as being slow; both during initial setup, and subsequently when wanting to react in the face of changing circumstances. What happens when you need to scale up operations in response to surges in demand, support a new geography, support a new target environment because of a new operating system release, or create new test environments to facilitate specific testing e.g. performance or resilience?
Thankfully, there’s an alternative.
Cloud Services
The term ‘the cloud’ is often misunderstood as being a synonym for the internet itself. More precisely, ‘the cloud’ refers to the provision of services over the internet, as opposed to services which are provided locally or on-site within an enterprise. Recurring features of this type of model is that the consumer:
- needn’t worry about the physical hardware supporting that service
- needn’t worry about setup or maintenance of that service
- can get going at the drop of a hat
- can scale up or down quickly or even automatically based on peaks/troughs in demand
- can benefit from near constant availability
- can avoid network latency thanks to multi-region support
The catch, is that the consumer pays for how much they use the service (measured in terms relevant to that service e.g. per database operation, per user, per api call etc.). At first glance such prices can seem expensive, but when compared to the costs of recruiting, training and managing skilled employees, to investing in hardware that needs and to the business drawbacks in terms of speed, the proposition becomes more interesting.
Cloud services can be particularly compelling to startups who might not have the bank balance for high sunk costs, and might not yet have high volumes. Many cloud service providers also offer a free tier for those just getting started, which can be compelling for hobbyists. The main benefits, speed and simplicity, should be a priority for hobbyists, startups and large scale enterprises alike, and as such we should look for opportunities to make use of these kinds of services for our software project or organisation. It is possible to compose major parts of your overall technology needs from Cloud Service components— so what’s on offer?
‘XaaS’ is a shorthand notation for ‘Anything-as-a-Service’; referring to a model whereby any kind of service is provided over the internet. Conceptually, there are three main groups to consider, providing increasing levels of abstraction from the underlying complexities:
- Infrastructure as a Service (IaaS) — virtualized computing resources and infrastructure components managed over a web interface
- Platform as a Service (PaaS) — a set of tools and hardware aimed at supporting software development and hosting managed and used over a web interface
- Software as a Service (SaaS) — ready-to-use software components used over a web interface
Examples
In the remainder of this article I will list various examples of cloud services, as a sort of shopping list: the more of our needs that we can push to the cloud, the lower our setup and maintenance burden!
Source Code Repository
Your source code is hosted in the cloud, and retrieved or changed via a git client over the internet. The provider gives a web interface to view and manage projects, permissions and changes.
Examples: GitHub, GitLab, BitBucket
Entire CICD Pipeline
The CICD process flow we established in Continuous Integration (CI), Continuous Delivery (CD) and Containers: Enhancing CICD and Operations empowers organisations to deliver high quality changes to their application quickly. But since it is constructed from various tools, integrating them to work together is not necessarily straightforward.
By contrast, many cloud service providers offer significant parts of an equivalent CICD pipeline, if not the whole thing, all-under-one-roof. Their unified solutions avoid much complex setup.
Examples: Atlassian Bitbucket+Bamboo, AWS CodeCommit+Code Deploy+CodePipeline+EC2 Container Service, GitLab CI
Error Tracking
By embedding a library into your application, errors are reported to a centralised server, where they can be inspected, aggregated, and manged over a web interface. Information can be reported by and to the application’s users, to help assist investigations and keep customers happy.
Example: Sentry
Database-as-a-Service
Provides users with access to a database without the need for setting up physical hardware, installing software and configuring for performance, reducing the burden on Database Administrators (DBAs).
Examples:
- (Relational): AWS RDS, MS Azure SQL Database
- (No-SQL): AWS DynamoDB, Azure Document DB
QA as a Service
Rainforest aims to provide ‘Quality Assurance as a Service’. Unlike oothe examples on this page, this service includes humans.
Clients of rainforest place their application in a Rainforest virtual environment, and express, in natural langauge, the tests they wish to be carried out. Rainforest agents log in and carry out the required tests. The clients can see the results of the test in real time, and agents can provide screenshots. Rainforest manage the resourcing for the testing.
This service allows organisations to undertake high volumes of testing in short periods of time, facilitating CICD and business agility.
Project management
Examples: JIRA Software, Trello
Messaging
Examples: Slack, Skype
Platform-as-a-Service (PaaS)
For our purposes, we will assume that we will support our application by managing the key parts of its underlying infrastructure ourselves. As such I won’t attempt to describe the subtleties of Platform as a Service, but intrigued readers can look to the following examples:
Examples: Google App Engine, AWS Elastic Beanstalk, Microsoft Azure, Heroku, Cloud Foundry and Engine Yard
Infrastructure-as-a-Service (IaaS)
So far on this journey to uncover and define the tools and processes we’ll employ to deliver our software application, we’ve described the need for various environments (e.g. Develop, Build, QA, PreProd and Prod) in which to run our application, the need for redundant copies to facilitate disaster recovery and zero-downtime releases as well as a host of CICD tools to help us create it. Assuming we choose not to go with a Platform-as-a-Service (PaaS) solution, where it is all supplied to and managed for us, we have ourselves quite an infrastructure problem.
Running any infrastructure such as servers, networks and storage ultimately requires physical hardware somewhere. Traditionally an organisation needs to purchase or rent the hardware, hosted in a data centre. These well trodden paths have either high setup or high ongoing costs (or both) and neither are supportive of business agility. Instead, we turn to the cloud, where all the benefits of cloud servicing discussed previously, be it speed, upfront cost, availability or simplicity, apply just as well.
Examples: Amazon Web Services (AWS), Google Cloud, Microsoft Azure Cloud Services
Infrastructure as Code (IaC)
Even with the extra help the IaaS provider tools offer, managing infrastructure by hand would be to treat the infrastructure like pets: each has a special name (‘mr-jenkins’), is lovingly hand-raised, trained and cared for so that when it gets sick, it is individually nursed back to health. With all the cats (infrastructure components) we have, we’ll need a lot of cat ladies (a.k.a System Administrators!)
What if instead we were to treat our infrastructure components like livestock? Livestock are identical, have homogeneous names (‘jenkins-34’, ‘jenkins-35’,…) and when one gets sick, we… get a new one (meet ‘jenkins-36’!). All we need is an astute farmer…
J. Infrastructure Provisioner
What does it do?: Infrastructure as Code (IaC) is the practice of describing infrastructure requirements in imperative code form, which can then be interpreted by specific tools to automatically create, or update the desired infrastructure.
NB. If IaC sounds familiar, it might be because the Configuration Management tool we adopted to deploy application builds to our environments as part of our Continuous Delivery (CD) process is a prime example of Infrastructure as Code in action. In fact, we can make use of Configuration Management scripts to configure all kinds of servers that make up all the environments we need, not just those which directly run our applications. But Configuration Management scripts only manage servers which already exist…
Infrastructure Priovisioners interpret coded requirements to put in place servers from base images, and ensures the presence of storage, networking, DNS entries etc. This:
- Reduces repeat setups to seconds not hours or days, meaning individual developers or teams can at the drop of a hat create a new test environment instantly, conduct their explorations or testing, and tear it down then they’re done.
- Provides consistency so that the environments all behave the same, reducing time lost investigating divergent behaviour due to invisible environmental differences
- Gives documentation for free (the code itself serves also to document the current state of the server) and auditability of infrastructure changes.
Examples: Terraform, AWS CloudFormation, Azure Resource Manager, Google Cloud Deployment Manager
Summary
In this article we explored:
- The meaning and types of cloud service model (Software, Platform and Infrastructure as-a-Service)
- How pushing service needs to the cloud can reduce an organisation’s setup and maintenance effort and costs, to allow a businesses to focus on product development
- Other benefits of cloud services, including scale, speed and availability
- Examples of some cloud services
- How Infrastructure as Code (IaC) can supplement Infrastructure provided as-a-Service (IaaS) to offer speed and consistency.
