Showcase you Streamlit Web App to the world with AWS EC2

Streamlit is an amazing and powerful tool to create web apps in hours. But what is the use if you can’t deploy it for the world to see. That’s where AWS EC2 comes into the picture. Elastic Cloud Compute is an AWS service that lets you create and delete virtual machines in the cloud easily. We will basically run our Streamlit web app on an EC2 instance i.e. a virtual machine.

Btw, this is the second part of the series “Create Streamlit Web App and deploy using AWS Services ”. If you have not read the first part on how to create a Streamlit web app, then please go and read so here —

Using Streamlit to create interactive WebApps from simple Python scripts

Amazon lets you use some AWS services for free for 12 months with limited resources as a part of its Free Tier. We will be using the free tier for AWS EC2 so the deployment will be free of cost. :)

Creating your AWS account

To get started, you need to create an AWS account. You can go to this portal and create an account. After that, you need to fill your contact information, add and verify your payment information (AWS will charge a small fee of Rs. 2 which will be refunded to you), verify your phone number, select the Basic support plan, and wait for your account activation. Usually, it happens within a few minutes but it can take up to 24 hours.

Once your account is activated, log in to your AWS console and you will see a dashboard like this:

Now, search EC2 in the Find Services tab and click on Instances under Instances tab in the left side menu.

Click on the Launch Instance button on the top left. I have some instances already created as you can see in the picture. For you, it will be empty as you don’t have any instances yet.

Creating your EC2 Instance

There are seven steps to creating an EC2 Instance.

Step 1: Choose an Amazon Machine Image (AMI)

An AMI is a configuration template for your virtual machine which specifies the Operating system, application servers, or any other applications required to launch your instance. You can create your custom AMIs and use them also. For now, we will use the Amazon Linux 2 AMI. If you are choosing any other AMI, make sure to select from one of the Free Tier ones unless you want to pay. :p

Step 2: Choose an Instance Type

Instance type refers to the different combinations of CPU, memory, storage, and networking capacity for your virtual machine. Depending on your application needs you can select your Instance type. I will be selecting t2.micro as it is Free Tier eligible and I don’t require more resources for my lightweight Streamlit web app. After selecting your Instance type, click on Next: Configure Instance Details on the bottom right.

Step 3: Configure Instance Details

Instance details lets you select the number of instances to launch, Network lets you select which virtual private cloud you want to launch your instance in (default VPC selected), which availability zone you want to launch in (Subnet) and much more. Let the default values be selected for now and click on Next: Add Storage.

Step 4: Add Storage

By default, we have a root EBS volume of 8GB. If you want, you can add more but we don’t require that for this web app. So you can simply click on Next: Add Tags.

Step 5: Add Tags

Adding tags are optional and you can add up to 50 tags and it can be whatever you want it to be. I myself prefer a name tag always as you can search your EC2 instance using the name tag value in the EC2 dashboard.

You can add a tag by clicking on Add Tag and then providing the key-value pair. After you are done adding tags, click on Next: Configure Security Group.

Step 6: Configure Security Group

Security groups are a very interesting topic in AWS. I feel they are the backbone of AWS security. Security Groups act as a firewall on EC2 instances and decide what connections to allow and what to deny. By default, all incoming connections are denied and all outgoing connections are allowed.

You can create a new security group or use an existing one, I am creating a new one named streamlit-aws-deployment-security-group (you can name it whatever you want) and add these two rules:

1. SSH on port 22 for 0.0.0.0/0 (all over the world)
2. Custom TCP on port 8501 for 0.0.0.0/0 — Streamlit

You can add as many rules as you want according to your needs and select a specific range of IPs instead of allowing it for the whole world but as I want my web app to be accessed from anywhere I am putting 0.0.0.0/0 for my Streamlit rule.

Once you are done setting up your security group, click on Review and Launch.

Step 7: Review and Launch

You can review all your previous configurations in this step and once finished reviewing you can click on Launch on the right bottom.

After you click on launch you’ll get a popup to choose a key pair for your EC2 instance. A key pair creates a public key for AWS and a private key for you that together secures your EC2 instance connection. You can create a new key pair or select an existing one. If you are creating an instance for the first time, you’ll have to create a new key pair. Give your key pair name and click on Download Key Pair and make sure to store it securely. This key pair is a .ppk file and will give you permission to connect to your EC2 instance.

And tadaaa…Congratulations, on your first EC2 instance 🥳. Go back to the Instances tab, you will find your instance there. Although, it may take some time to start.

Phew! That was a lot of information..

Connecting to your EC2 instance

Once your instance is running, select it, and copy the public IP of your instance. We will SSH into our instance. SSH allows you to control a remote machine using just your command line.

Linux/Mac OSX or Windows 10 onwards

You can directly SSH into your instance if you are on Linux / Mac OSX or Windows 10 or greater versions.

Go to your command line and type the following command —

ssh -i <path to your key pair file> ec2-user@<your public IP>

Here, -i stands for identity and ec2-user is the username for Amazon Linux 2 AMI.

As you are connecting for your instance first time, it will ask for your permission. Give yes.

It will give you another error for permission too open as your private key is not secured. To solve this problem —

In Linux / Mac OSX

Run this command and run the ssh command again, it will work this time.

chmod 0400 <key pair name>
example - chmod 0400 Streamlit-app.pem

In Windows

Left-click on your security key pair and click on

Properties -> Security ->Advanced and remove any other users than yourself. If you are not able to remove, click on Disable Inheritance and then try to remove the users other than yourself again. Also, make sure you have full control access.

After this run the ssh command again and you are inside your EC2 instance.

Windows less than 10

SSH utility won’t directly work for Windows version less than 10. But we can use the free tool Putty for our rescue. Putty is an open-source SSH client that will let us SSH into our EC2 instance. You can download and install Putty from here.

1. Convert .pem to .ppk (Putty Private Key) file

The first thing we will do is convert our security key file to a format that Putty likes, which is .ppk. Open PuTTYgen, click on File, select Load private key and then select your .pem file. After importing your private key, click on Save private key and save it in .ppk format.

2. Connect to EC2 instance using Putty

Open Putty tool now and in the sidebar open Connection -> SSH -> Auth. Inside Auth click on Browse and select the .ppk file that you just created. Once that is done, go back to Session at the top of the sidebar.

In Session, put your public IPv4 address along with the username in the hostname tab, put the name of the session in the Saved Sessions tab, click on Save to save your session, and then click on Open.

You will get a warning as you are connecting to this instance with this security key for the first time. Simply click on Yes and you’re connected.

Using EC2 Instance Connect

No need to panic if the above methods for Linux/Mac OSX or Windows do not work out for you. You can always use EC2 Instance Connect from AWS to ssh into your instance. It is a browser-based ssh tool.

Go to your EC2 instances dashboard and click on your instance and then click Connect.

Select EC2 Instance Connect and put the username which is ec2-user and click on Connect.

And a new browser window opens, that lets you connect to your instance. The best part is here you do not have to take care of the private key as AWS handles it for you in the backend. :)

Setting up your instance with required libraries

As you are done connecting with your instance, you need to set the environment for this instance to run your Streamlit web app. Firstly, install python3 using the following command:

#Updating your instance quickly
sudo yum update -y
#Installing python3
sudo yum install python37

Check your python version using:

python --version

Install pip3:

curl -O https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py --user
pip3 –-version

But here when you check the version to your pip3 you might get a warning saying :

WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
pip 20.1.1 from /home/ec2-user/.local/lib/python3.7/site-packages/pip (python 3.7)

To fix this you need to add the executable path to your pip3 in the PATH variable.

The path is ~/.local/bin, you can view the hidden files using ls -a

Open your .bash_profile and edit the export command to your profile script.

$ vi .bash_profile

Load your bash profile script:

$ source ~/.bash_profile

Check the version of pip3 now and it will work fine!!

Install streamlit, pandas, chart_studio, and plotly:

These are the dependencies I require for my project but you need to install the dependencies for your project in case you are using a different web app.

pip3 install pandas 
pip3 install streamlit 
pip3 install plotly 
pip3 install chart_studio

Yayy!! You have successfully set the environment for your virtual machine and it’s ready to launch your application.

Getting your project files and running the web app

We can simply get the web app from Github by cloning it in our instance or we can opt to transfer files from our local system using WinSCP. We will discuss both ways and you can go ahead with your preferred way.

Setting up Git and cloning your project

Run the following commands to set up Git and clone your project. I am cloning my Covid-19 web app but you can clone any other project also.

#Installing git in your instance
sudo yum install git -y
#Checking git version
git version
#Cloning your repository
git clone https://github.com/nishtha697/Covid19India.git

Using WinSCP to transfer files from the local system

There might be some files that you want to transfer from your local system to the virtual machine instance. You can do so using WinSCP.

1. Download and install WinSCP from here.
2. You will need to create a .ppk file from your security key pair if you have not done it yet. You need to see the Convert .pem to .ppk (Putty Private Key) file section of this article.
3. Open WinSCP and click on New Site, in hostname enter the Public DNS of your EC2 instance.

4. In the User name enter ec2-user and click on Advanced.

5. In Advanced site settings, click on Authentication under SSH and load the .ppk private key and click on OK.

6. Click on Login and if you get a warning, click on Yes.

7. Once logged in you will get your local system on the left and EC2 instance on the right. You can select a file from your local system that you wish to upload and click on Upload.

Running your Streamlit app in AWS EC2

Run your Streamlit web app inside your instance using the same command we do in our local system.

streamlit run corona.py

And we have our web app running. You can access it anywhere from the world using the External URL 😄

Keeping our Streamlit session running

Well, our web app is running for now but what if we close the terminal? Yes, the session will stop and the web app won’t be accessible anymore. But this problem can be fixed, thanks to TMUX.

TMUX basically allows multiple sessions in a single window. And to be precise, in our case, TMUX will keep our session running even after we close the terminal. Sounds interesting right!

To install TMUX:

sudo yum install tmux

To start a new tmux session:

tmux new -s StreamlitWebApp

Now start your Streamlit web app:

streamlit run corona.py

Now the web app is available at External URL from anywhere. You can press Ctrl+B and then D to detach your session from the terminal and keeping it running in the background. You can close the terminal window now and your web app will still be running.

You can attach the same session using:

tmux attach -t StreamlitWebApp

StreamlitWebApp is just the name of the session I gave, it can be anything you want.

SSH Troubleshooting

1. If there’s a connection timeout on port 8501, it means that the port 8501 is not configured correctly in your EC2 instance's security group. Check your security group again, it should look like:

2. If you have restarted your instance, your public IP will change. So while connecting with your EC2 instance you will need to update your SSH command with the new IP address.

3. If you are not able to SSH into your instance even after having the correct security group, it means your personal or corporate firewall is blocking the connection. Use EC2 Instance Connect instead.

4. ssh command not found on Windows then use Putty or EC2 Instance Connect instead as explained in this article.

5. If you get a connection refused, it means that the security group is working fine but your ssh is not responding. Try and restart your instance or create a new one.

6. If you get permission denied during ssh, it means you are using the wrong security key or the wrong user. We are using Amazon Linux 2 in this tutorial and the user for the same is ec2-user.

7. If nothing seems to work, go ahead with EC2 Instance Connect. :)

Also, you need to run your Streamlit web app at port 8501 as we have only configured this port in AWS EC2’s security group. If your web app is not starting at port 8501 then it means another instance of Streamlit is already running at port 8501. You can find the process ID for this port and kill it.

To get the process id:

ps aux | grep streamlit

You will get a response like this:

Here, 12748 is the process Id we need to kill. We can do so by running:

kill -9 12748

Now run your Streamlit web app again and it will start at port 8501. :)

I know this was a very long article as I covered and explained each part of the process and provided you with a lot of alternatives to do things but I feel it’s important when working with AWS. You should know how things are happening rather than running a few commands and just clicking at things.

I am still not satisfied though as we don’t have a domain name and we’ll have to share the external URL only that contains the Ipv4 address and the port 8501. And it doesn’t look nice to me!!

If you feel the same then please stay tuned as I will be posting another article on how to replace this external URL with a legit hostname like a did for my web app www.corona-updates.in.

I hope this article helps the ones in need. Please leave your feedback in the comments. Thank you :)