For growing companies that offer services over the internet, there often comes the point where they need to distinguish between what services they offer to the public and what services to offer only internally.
Especially when you’re building a micro-service infrastructure, you want that most services can talk to each other, but not that the entire internet can talk to them. The most typical way of doing this, is to ask the user for some kind of authentication, e.g. a password. The first downsides that come with passwords is that you have to management (e.g. when employees are joining and leaving your company) and they don’t protect you from attacks like DDoS.
One solution that I would like to talk about here are private subnets:
You want some of your machines/services to be accessible from the public internet, e.g. your web servers, but want to shield other machines, e.g. databases form direct web access. Often it is helpful to allow services in the private subnet to “call-out”, but not to allow systems from the outside to “call in” (see the red vs green-dashed arrows in the depiction above).
Since this is a tutorial on setting up a private subnet on AWS, you should be logged into the AWS user interface. I will assume that you have broad access rights in your account (e.g. you’re an administrator) and will not go into detail which rights you will need.
If you are worried that you might break a running system that already is in your account you have 2 choices. Create a new account for this tutorial or pick a region where there are no production systems running (the latter is obviously much less work!)
Pick a region
In AWS there are many different regions and probably you have already decided for one. If not here are my 2 cents on this topic:
- You should pick a region that is close to your customers and therefore reduce latency
- Picking the “older” and more established regions often gives you early access to new product launches and in some cases less headaches (e.g. new S3 authentication mechanisms). Good candidates are:
Creating new subnets
We will create 2 new subnets in this tutorial one public, one private. If you have a completely new AWS account, you could also just use the already existing subnet as the public subnet, but I will assume you create a new one.
In the AWS Web UI go to
VPC (virtual private cloud) →
- Click the blue
Create Subnetbutton at the top of the page
- You can chose what ever name you like, I would suggest the following naming scheme:
REGION-PURPOSE-ACCESS-AZin my case this would be:
so I called it:
- Chose your VPC — typically you only have 1 at this stage
- Availability zone: I would suggest you pick
a, but in most cases it doesn’t really matter — for better fault tolerance, you should probably create at least one more subnet that is connected to
bafter this tutorial
- CIDR: This is basically the IP address range for your subnet. As a very short tutorial:
126.96.36.199/16defines the range of IP address from
/16at the end defines how many
bitsfrom the start of the IP address are fixed (the same for all addresses in the range). In this case 16 of 32 are fixed. Since IP addresses are written in blocks of 8 bits, this means that the first 2 blocks
0must be present in all IP addresses in the range, but the remaining 2 blocks are allowed to change!
Here some more examples:
To find a suitable CIDR for your case, you have to pick a sub-range of your VPC’s CIDR (it’s show a few lines earlier in the UI) and ideally follows you existing subnets. Without looking at the other subnets in your account, an easy way to find a CIDR would be to take your VPC’s CIDR, e.g.
172.0.X.0/20as your subnet CIDR. For X you can try through numbers
0, 16, 32, 64, …until the AWS UI stops complaining ;)
- Repeat all steps from 1–5 for the
privatesubnet (replacing the
privatein the name)
- After having created both subnets, select the
publicsubnet and above the list select
Modify auto-assign IP settingsand set the checkbox: Auto-assign IPv4 to Enable auto-assign public IPv4 address
Done! You now have 2 fresh subnets.
Creating a NAT
In order to allow services from your private subnet to talk to the internet, we need a Network Address Translation device. You could use an EC2 instance for this, but let’s use a preconfigured one from AWS:
In the AWS Web UI go to
Create NAT Gateway
- Select your
publicsubnet that we just created — you can type the name of it or copy&paste the id. In my case I picked:
- Create a new EIP: an Elastic IP is basically just an IP address that stays the same — so that you can reboot the machine and IP stays the same. In our case we will just create a new one (you could also reuse an existing one).
After creating the EIP, I would just that you copy and save the EIP’s identifier, in order to give it a name later, for keeping your account clean.
Create NAT Gateway
Your NAT Gateway is created! The following step is just the before mentioned cleanup.
- Go to
Elastic IPsand filter for the EIP’s id you saved. Rename it for instance to
NAT Gatewayjust to keep track what it’s being used for
Routing your traffic
Now in the last step of the setup process, we need to route where the traffic is going. For this we need to go to:
- You can reuse the existing one, but in order to reduce the risk of breaking a running application, let’s create a new one. Click
Create Route Table.
- Pick a name: In my case I just called it
- Pick the same VPC where you created the subnets
- Now back in the
Route Tableoverview, select the table you just created and in the information view at the bottom of the screen select
- Under Destination type:
0.0.0.0/0and under target select
NAT Gatewayand pick the one that you just created (typically you only have one!)
0.0.0.0/0means that when trying to talk to any address on the internet (range
255.255.255.255) your machine should ask the NAT gateway to help!
- Now back in the
Route Tableoverview, still with your new route table selected, in the information view at the bottom of the screen select
Edit subnet associations
- Select your private subnet and click
Done! If you have a normal AWS account without making any changes in this section you’re done. To be save, let’s make sure that your public subnet also knows how to talk to the internet:
- In the overview screen on the
Route Tablepage find the route table where the column
Mainis equal to
Yesand select it.
- In the information view on the bottom select
Routesand check there is a route with Destination
igw-XXXXXXX(where X can obviously mean any character)
- If this is the case, you are done here!
- If this is not the case, please follow through the steps of creating the private route table, except when choosing the Target, please select Internet Gateway instead of the NAT!
You are now finished setting everything up — but since there can always be a problem, let’s make sure it works by testing it!
Starting a server in our private subnet
Let’s start a new EC2 machine in our private subnet, by going to:
- You can launch any image, but I will assume you picked some kind of
Ubuntu Server ...and pressed
- You can pick any machine size, but I suggest a
Next: Configure Instance Details
- The only option that we have to change here is to select our
privatesubnet we created earlier. Now click:
Next: Add Storage
- Nothing to change. Now click:
Next: Add Tags
- I suggest you add a tag called
my-private-instancethis helps you finding this instance later! Now click:
Next: Configure Security Group
- In the security group section, you have to make sure to set a security group that allows ssh access. If you don’t have any existing ones, you have to create a new one. AWS already selected everything for you, I would just suggest to change the name from
Preview & Launch
- Now click:
- AWS will now ask you for an ssh key-pair. Either pick an existing one or create a new one. If you create a new one, please follow these steps:
a) Put in a name, e.g. your name :P
c) On a mac or unix machine you would now go to the terminal and run:
mv ~/Downloads/FILENAME ~/.ssh/NAME_YOUR_KEY.pem
chmod 600 ~/.ssh/NAME_YOUR_KEY.pem
These steps will put the key into the normal folder, reduce access rights so that only you can use it and tells the ssh agent (service running on your machine) that it exists
- After having dealt with the SSH Key we can finally
Launch Instancesto start the machines.
We have now started a server in the private subnet — but we should not be able to connect to it via SSH directly, since the private subnet is… private.
In order to connect to it, we must launch another instance in the public subnet. Go through the same steps as launching the private machine, except for selecting the
public subnet in step (4)!
Now you should have 2 instances running — 1 private and 1 public. The public one should also have a public ip and dns. This public server is typically called
bastion and I will refer to it as such in the next steps.
To connect to your private machine, we first have to ssh into the
ssh -A ubuntu@PUBLIC_IP_ADDRESS_OF_BASTION
After typing the above 2 commands into your terminal, you should now be logged in on your private server. To validate that also the outgoing internet connection is working, just type
ping 188.8.131.52 into the terminal and should see output like this:
64 bytes from 184.108.40.206: icmp_seq=34 ttl=108 time=12.6 ms
64 bytes from 220.127.116.11: icmp_seq=35 ttl=108 time=12.6 ms
64 bytes from 18.104.22.168: icmp_seq=36 ttl=108 time=12.6 ms
64 bytes from 22.214.171.124: icmp_seq=37 ttl=108 time=12.5 ms
64 bytes from 126.96.36.199: icmp_seq=38 ttl=108 time=12.5 ms
64 bytes from 188.8.131.52: icmp_seq=39 ttl=108 time=12.5 ms
I hope it does! → if yes, we are done! 🎉🍾
otherwise I’m very sorry and please go back though the steps above to check what may have gone wrong.
- We created 2 subnets — a public and a private one
- Created a NAT Gateway to allow internet access for the private network
- Routed the traffic in the networks to get internet access
- Validated everything is working
Obviously this is only the start of the journey, but where can you go from here?
- If you want to connect to these private subnets from your office wifi, please go through this explanation: AWS Docs
I will probably also clarify this in another Post, but until then, you can use the link above.
- You can now launch your database or other services with a much better protection layer
- You can launch private serverless (AWS Lambda) functions with regional endpoints. I will soon write a Post on this! This makes building micro-services architectures much easier!