API services on cloud #1: Twelve steps to build a minimal AWS infrastructure for API service

Note: this is the Part 1 of the “API services on cloud” series on running API Services on cloud and learning cloud infrastructure automation. Stay in the know. There’s more of this to come!
Next >

Intro

Today I am starting a series of articles of how to run API services on cloud. There are many ways to achieve it and even more tutorials about it. In these articles I will describe the recipes I tried myself. I will try to keep articles short but informative as much as it possible.

For this part I used service written in NodeJS. The main target of this tutorial is to show how to launch the infrastructure, thus I will not provide detailed explanation of the service itself. Instead, I will highlight the key points you should remember when integrating services into infrastructure.

Infrastructure overview

Figure 1. Top level overview of the infrastructure

On the figure above you can see the minimal valuable infrastructure to run services. Of course you can manually start EC2 instance, deploy and launch service on it. But one instance is not scalable, not secure, not manageable, and many other ‘not’. That’s why I decided to start from the configuration depicted on the Figure 1.

Here all traffic goes through the Internet Gateway to Application Load Balancer. The routing is based on the requested path. Then the traffic forwarded to the relevant service which is running within the AutoScaling Group. All instances are running within the private subnets. That’s why we need NAT Gateways to provide outbound traffic from the instances.

For automation and manageability I used CloudFormation template to describe the infrastructure. If you don’t want to go through the process of creation of the template step by step, you can find the final version of the template here.

CloudFormation template anatomy

The CloudFormation template I wrote for this example consists of the four main parts:

• Parameters - allow to parametrise templates to avoid duplications and reuse same template in different environment;
• Mappings - predefined key-value pairs, later in the template I can use intrinsic function Fn::FindInMap to get parameter value by key;
• Resources - the main part of the template, all infrastructure resources defined here;
• Outputs - sometimes you need to get access to the created resources. For example you might want to know the public DNS of a load balancer to be able to send requests to it. In this section you can define all the outputs of the template.

Ok, let’s start defining the infrastructure.

Step 1. Defining parameters

Usually it’s hard to foresee all the required parameters. But there are two parameters which I always add to all CloudFormation templates: Environment and Namespace. These parameters help easily differentiate stacks in the console and filter logs in CloudWatch. For Namespace I usually take ‘project’ name. And Environment is pretty much self-explanatory. I add Namespace-Environment prefix to all resources tags.

SimpleAPIRepoUrl is the link to the GitHub repository where the service resides.

The rest of the parameters describe IP addresses ranges within VPC.

Step 2. Defining mappings

Because we are going to launch instances we need to provide the list of the AMI. Every region has it’s own optimised AMI. Please, feel free to extend/replace the mapping with the values for the region where you are going to launch EC2 instances.

Step 3. Defining VPC and Internet Gateway

Now it’s time to start building the infrastructure. The following steps will add items to the Resource part of the CloudFormation template. First, I defined the ‘borders’ of the infrastructure — VPC.

Step 4. Defining subnetworks

Then I put Public/Private subnets into two Availability Zones across the VPC.

Step 5. Defining NAT Gateways

After all subnets defined, we need to provide outbound traffic from them. Here I defined two NAT Gateways and attached them to the public subnets.

Step 6. Defining routing

After both Internet and NAT Gateway defined I could specify routing rules.

Step 7. Defining Security Groups

The next step after routing is definition of the Security Groups. I needed two groups:

1. Group to manage access to the Load Balancer
2. Group to manage access to the EC2 instances running behind LB (as you can see, instances could be accessed only via LB)

Step 8. Defining Load Balancer

Here I created Application Load Balancer and placed it into public subnets. Also, load balancer listener and default group have been created on this step. All services will attach it’s own listener rules to the listener created on this step.

Step 9. Defining AutoScaling Group

Once load balancer declared I could start defining an AutoScaling Group. The group stretches across private subnets and automatically launches four EC2 instances. Then I defined a TargetGroup and ListenerRule for the NodeJS service.

In the TargetGroup I defined port and path for the health check requests. Based on the response to that requests the AutoScaling group will launch/stop EC2 instances.

Then I defined a path to the echo endpoint exposed from the NodeJS service.

Service routes definitions

Step 10. Defining EC2 configuration

Since the AutoScaling Group defined we can start configuring instance which should be launched. Here I specified image and instance type. Assigned security groups to instance.

files section creates logger configuration. It allows to see service’s logs in CloudWatch. commands section creates the necessary directories and files to run the service. And the last command starts the NodeJS service. The default user used by CloudFormation is root. Thus it’s very important to create users with restricted permissions and run your services by service users.

Step 11. Defining EC2 instance role

Last step in instance configuration is role definition. The NodeJS service does not communicate with AWS services except CloudWatch. That’s why here defined only permissions to write logs to the CloudWatch.

Step 12. Defining outputs

And finally the last step. Here I output the most important information about created resources.

Thank you

That’s all for today. I thank you for reading till the end. Please ‘clap’ or/and comment if you liked it. If you did not like it, please comment why — so that I will be able to improve it next time.

References

• SimpleAPI — NodeJS service and CloudFormation source code.