Venture into Amazon Web Service: Deployment with EC2 and RDS Postgres from A — Z

Have you ever had that moment where the thing you’re working on is very close to being complete, but you cannot figure out the last missing piece of the puzzle? Yea, tell me about it right? I scoured the internet on how to connect EC2 to RDS, and even went on a google hangout with a person from AWS help desk, but to no avail. Each person I interacted with during this process got me one step closer to the finish line. And I promised them I would write a meaningful blog post that entails this process. So here it is, let’s begin.

In this tutorial, I want to narrow the focus down to just setting up EC2 and RDS Postgres and establishing the connection. Let’s admit it, reading the docs is not easy… or fun. If it was, then we would all be dev op ninjas. The documentation on the AWS page is great, it is very detailed. Maybe too detailed that it’s overwhelming. Where do I start? How do I begin? This tutorial is essentially a very specific stack overflow answer.

Here is the repository that we will use for deployment. This is what we will achieve at the end.

Fork it, clone it down, then npm install. The only page we will modify is queries.js in db directory, everything else is irrelevant. We have to fork it because the EC2 will be connected to the forked version.

The boilerplate we are using is a React, Node/Express, Postgres fitness application that renders exercise data in a table. The part we care about is whether we are able to query the data back from our RDS instance or not. We will begin by creating a relational database service instance.

RDS Postgres Instance

  1. Head over to the AWS console, create an account and sign in
  2. Click on service, choose RDS, then click on Launch a DB instance. Here is the visual doc on their page for this process
  3. Select PostgreSQL, pick Dev/Test (since this is practice)

We will not go into detail about what each of those settings are because they are in AWS documentation. Just know that we are selecting the choices that best fit the free tier option. For a real world application, the DB instance class will need to be bigger. Don’t forget your password.

Other than giving a name to our database and making sure port is 5432, we leave this as is. Launch DB instance and wait for its status to become available. It should take 5–10 minutes.

Connecting to RDS Postgres instance from localhost

Once our RDS instance has status available, click on it, then instance action and see detail. Below is all the information required to access the it.

In db/queries.js modify the object passed into pgp to match the instance information given above. Commit and push to origin master.

user is username, database is DB Name, password is master password from earlier, host is writer endpoint, port is port

Now we have an established connection between localhost and our RDS instance. Currently we have no data in our RDS instance so we need to insert the data from db/schema.sql.

There are two ways to connect to our RDS instance, by using pgAdmin or by psql. Here is an excellent documentation on how to do that.

I used pgAdmin the first time I got this set up, we will use psql for brevity in this tutorial. Here is what we will paste in our terminal, we will be prompted for the password. My password is “mytutorial123” from the image above.

psql \
--host=awstutorial.ctwfwbqhfx8u.us-west-1.rds.amazonaws.com \
--port=5432 \
--username=aws \
--password \
--dbname=myAWS

Once we are in, we can manually insert data. In db/schema.sql I will copy line 6 (Create Table chest) all the way to the end. Do not paste in line 1–4 because we do not need to drop or create database “training”. We are using RDS Postgres, this was left over code from my local db instance.

Once inserted check to see if data exists by using the following commands.

select * from chest;
select name from back;

Now let’s fire up our application with npm restart to bundle and start server. Go to http://localhost:8080/#/table to see the table. The main page doesn’t do anything. We can now exit the RDS instance in terminal since there is nothing else to insert.

Try fiddling around with the object passed into pgp and we will find that the connection to RDS Postgres is cut. No need to “npm restart” because we already have the bundled file. Use “npm start” to fire up server since we are only fiddling with server code. Again this is my connection object.

const db = pgp({
user: ‘aws’,
database: ‘myAWS’,
password: ‘mytutorial123’,
host: ‘awstutorial.ctwfwbqhfx8u.us-west-1.rds.amazonaws.com’,
port: 5432,
});

Congratulations, we have established the connection between localhost and RDS Postgres. We are half way done, now we need to set up our EC2 instance.

Deploying with AWS EC2

Go back to the AWS console, click on service, then EC2, and Launch Instance. Select Ubuntu 64 bit.

Select general purpose t2.micro (free tier eligible), then next: configure.

We will not modify anything on configure instance, add storage, and tag instance. Keep clicking next until configure security group.

Add the following rules to your configuration. The warning basically tells us that without security group our instance can be accessed by unknown IP address. For now, this is irrelevant so don’t mind it. We will get into security groups later. Review and Launch.

Create a new key pair and download key pair. Save the file somewhere you can easily access it. I am saving mine in the same directory as my AWS tutorial app. We need this file in order to connect into the EC2 instance. Launch instances and wait until instance state and status check is green. This will take 5 minutes.

Go to the directory that contains our pem file that we downloaded earlier. Mine is called “mytutorial.pem”.

Grab your EC2 instance’s Public DNS by going to your EC2 instance on AWS console. In the description, look on the right side and copy the value of Public DNS.

Paste the following command into your terminal.

ssh -i <filename>.pem ubuntu@<Public DNS>
// here is mine
ssh -i mytutorial.pem ubuntu@ec2-54-67-26-107.us-west-1.compute.amazonaws.com

It will ask us if we want to continue, then give us this error.

Basically we need to change our pem file’s permission to read and write. I won’t go into details about that but it will let us go access EC2 instance. If we enter ls -l into the terminal, we will see the below. If you are curious how all this works under the hood check out this video. The EC2 portion of this tutorial is possible thanks to MicrowaveSam.

total 8
drwxr-xr-x@ 14 justinTime225 staff 476 Aug 28 14:16 awstut
-rw-r — r — @ 1 justinTime225 staff 1696 Aug 28 15:41 mytutorial.pem

Enter “chmod 600 mytutorial.pem” into the terminal and type ls -l again.

total 
8drwxr-xr-x@ 14 justinTime225 staff 476 Aug 28 14:16 awstut
-rw-------@ 1 justinTime225 staff 1696 Aug 28 15:41 mytutorial.pem

Now our pem file is just read and write. Enter the shh command again.

ssh -i mytutorial.pem ubuntu@ec2-54-67-26-107.us-west-1.compute.amazonaws.com

Now we should be in our EC2 instance.

Now we need to type in a few commands to update our instance

sudo apt-get update
sudo apt-get install libssl-dev g++ make

Then go to the node website, right click on node-v4.5.0.tar-gz and copy link address.

In the terminal type the following command

// template
wget <node address link>
// here is mine
wget https://nodejs.org/dist/v4.5.0/node-v4.5.0.tar.gz

Type ls again and we should we a node file. Enter in the following command. Make sure the file name matches what you see in your file. This will unzip the node file.

tar -xvf node-v4.5.0.tar.gz

Now cd into the unzipped file and install node by typing the following command. This will take 20–30 minutes.

./configure && make && sudo make install

Next up cd out of the node file and install git to our ubuntu EC2 instance

cd ../
sudo apt-get install git

Now clone down your forked repository into the EC2 Ubuntu instance and npm install. Don’t worry if there appears to be some npm errors. This will take longer than a local npm installation.

git clone <repo URL>

This is normal, don’t mind it.

Bundle.js should already be created for us, check to make sure it’s there then start server with “node server.js”.

Check out our EC2 instance by pasting our public DNS with port 8080 on the browser.

Congrats! We now have a deployed site. If we go to the /table path then there is no data, that is because we have no connection between our EC2 and RDS.

Before we get there, lets address a few problems.

First, it seems that our EC2 instance crash if we stop the server from running. We want a way to keep it on even if we shut down the terminal. Second, our url has the port number at the end. Its not necessary but I prefer to not have the port number there.

So lets get rid of the port number first. Type in the command below.

sudo iptables -A PREROUTING -t nat -i eth0 -p tcp — dport 80 -j REDIRECT — to-port 8080

This command will allow our site to be reached at port 80. Port 80 is default so there is no trailing port number at the end. We can now reach our site with or without the :8080 at the end.

Now to wrap this up, lets have our server stays on even if our computer is off. Type in the following command.

sudo node server.js &

Try closing the terminal and see if it works. It does right? Magic!

If we want to shut down the EC2 instance we have to kill the background process. Type the “ps -ef” to see all background process.

We can end the EC2 instance by typing “sudo kill 3038”

Type ps -ef again and it is no longer there. Refresh our deployed app and we see its down. Simply type “sudo node server.js &” again to keep our server on.

Connecting EC2 instance to RDS Postgres

Above is our RDS Postgres’s detail, click on the Security Groups. At the bottom click on inbound and see the available port.

The current source is our localhost’s IP (I think). If we alter that source then we will not longer be able to connect to the RDS from localhost.

We need to add the IP of our EC2 instance to establish the connection. Check out the security group in the description for our EC2 instance.

Click on awsTutorial and check out the description for that.

The part we care about is the Group ID. Now we must go back to our RDS Postgres’s security group, go to the inbound tab and click edit, then add rule.

Choose the EC2’s group name/group ID as highlighted and save. Give it a minute and we will have RDS data flowing into our EC2 instance. The result now is that our RDS allows connection from localhost IP and EC2 IP.

Here is the final snapshot

This RDS allows connection from localhost and awsTutorial EC2

You can also allow it to accept any IP, though this can cause security issues. Keep in mind that the RDS’s password is still needed to access this DB.

That is the end of our tutorial. Thank you to everyone who gave this tutorial a shot. If there is an issue with localhost working but EC2 not, then make sure that both files are exactly the same. Meaning check to see if you have push after commit. Here is the result.

The terminal is no longer in Ubuntu EC2

When exiting out of ubuntu EC2 instance, just type “exit” and we will be back into our regular terminal. Sometimes terminating process might kill the running instance. So if our EC2 instance is offline in 10 minutes then ssh back into ubuntu EC2 again and check if the instance is running. Just exit out nicely and the bad stuff won’t happen!

One final thing to keep in mind is when deploying your own application with EC2, any bugs that result in node crashing will cause EC2 instance to disconnect. Just to be safe, remove any console.log on the server. My app had crashed before due to a console.log in the getChestExercise function. Since there is no CI in this deployment tutorial, whenever you want to update the deployment version, you would kill the running process and do a git pull.

Thats it for now, look forward to seeing Elastic Beanstalk deployment in the future!

Show your support

Clapping shows how much you appreciated Justin Nguyen’s story.