Deploying Your Streamlit Project with a Free AWS Account
In this article, we embark on a journey through one of the most exhilarating and popular topics in technology: application deployment or distribution. Specifically, we will meticulously examine how to create a project with Streamlit, an exceptional Python programming language library accessible online using Amazon Web Services (AWS), through a series of concise, easily applicable steps.
Our article at this link provides an introduction to deploying models with Streamlit. Indeed, reading the article above first to grasp the general logic of deployment will make understanding this piece significantly more beneficial.
Why Streamlit and AWS?
Streamlit is a library that offers data scientists and engineers the ability to create user-friendly, easy, and compelling web applications. On the other hand, AWS is one of the most preferred cloud services by developers worldwide (currently leading with 34% in global cloud usage), opening the doors to quickly bringing our projects to life (a concept we elaborated on in our previous article) with its flexible, secure, and scalable infrastructure.
In this article, we will first tackle a simple Streamlit application. Subsequently, I will share, step by step, how we can deploy this application on AWS’s Elastic Compute Cloud (EC2) service. Throughout this process, I will share theoretical knowledge, practical tips, experiences, and best practice recommendations. Whether you are a data scientist, a software developer, or simply someone curious about technology, this article will provide valuable guidance on how to share your projects online. Let’s get started!
Section 1: Prerequisites
Basic Information
Before we delve into the main subjects of this article, Streamlit and AWS, I would like to provide a brief overview.
Streamlit: An open-source, Python-based library that serves as a user-friendly tool for data scientists and engineers. It creates data visualizations, machine learning models, and interactive dashboards.
AWS (Amazon Web Services) is Amazon’s cloud computing platform that offers a wide range of services. In this article, we will specifically utilize the EC2 (Elastic Compute Cloud) service, which allows us to create and manage our virtual servers. A virtual server is like a safe deposit box in a bank, which you rent and is empty initially. We will equip this space with the tools we need, i.e., Python libraries.
Required Tools
To complete our project, we will need the following tools and technologies:
Python: Given that Streamlit is based on Python, basic knowledge and installation are necessary.
Streamlit: The library we will use to create our application.
AWS Account: An AWS account is required to deploy our application.
SSH Key: We will create an SSH key to securely connect to our AWS EC2 instance.
To achieve the desired outcome by the end of this article, it is crucial to have reached the level specified in our previous article.
Creating an AWS Account
If you do not yet have an AWS account, you can easily create one by visiting AWS’s official website or clicking this link. You will be asked for your credit card information during the account creation process. However, there’s no need to worry! AWS offers new users the opportunity to use many of its services for free (with some restrictions). In this article, we can smoothly carry out our tasks utilizing the free usage tier.
In our previous article, we discussed the necessary steps to start Streamlit’s example application. Moreover, you should have successfully reached the “Streamlit Hello” page without any issues.
Now that we have reviewed the essential information and the necessary tools, we will learn how to create an EC2 instance on AWS and run our Streamlit application on this instance in the next section. Are you excited?
Section 2: Creating an AWS EC2 Instance
In this section, we will create an EC2 instance on AWS and configure it to run our Streamlit application.
Step 1: Log into AWS Management Console
Log into your AWS account and access the Management Console.
From the “Services” menu, locate and click on the “EC2” option.
Step 2: Create an EC2 Instance
Please follow the steps given in the images below in sequence. The steps were different two years ago but have now been simplified in the AWS interface. Might they change again in the future? Possibly, but the logic will remain roughly the same.
As shown in the image below, let’s begin creating our first virtual machine in our AWS account by clicking “Launch Instance.” Moving from the previous example, we are now building our safe deposit box, which is empty like a drawer in a bank.
We define the server name as “streamlit-server” for example. You can give it a different name; common alternatives include “server” or “test”.
Now, let’s select which operating system we will use on the virtual machine. Generally, we proceed with the Amazon Linux option. Since we are going with the free tier features, some options may not be available. In this article, we will build a virtual machine using Linux.
In the image below, we choose the hardware specifications for the virtual machine. Here, we will select one of the options available for free. The machine the arrow indicates will suit us because it has more stable hardware. If you desire a more powerful virtual machine, you can proceed with the paid options to experiment.
In the image below, we select the Instance type offered for free. This option is usually pre-selected by default. Other options are chargeable.
Step 3: Creating and Downloading an SSH Key
At this stage, we will generate a key file called a “Key pair.” If you already have a “key pair” file created on your computer, you can use that as well. Let’s build a new file here. We will continue by clicking “Create new key pair,” as shown in the image below.
For the Key pair name, we named it “streamlit1” in the example, and it will generate a file with that name. All other settings will be as shown in the image below.
The file will be saved to your computer as “streamlit1.pem” as shown in the image below. In the subsequent steps, we will modify its properties.
We continue to configure the settings on AWS. As you scroll down the page, new settings windows will appear. Now, let’s set up the ports. Click on the “Edit” at the top right of the image below and move to the area shown in the other picture below. Here, the existing Rule will remain, and the 22 SSH port indicated by the arrow will be approved by default. However, we will add an additional rule by clicking “Add security group rule” as shown by the arrow.
Following the settings in the images below, we will add port 8501 as an additional port. Port 8501 will be used for Streamlit.
The final adjustment or confirmation will be made with the selection in the window below. You do not need to create any additional modifications in this section. The hardware option shown, which is the minimum, will be sufficient for our work.
Finally, by clicking “Launch Instance” as shown in the image, we will have completed this section. Our instance, or machine, is now configured and ready for use. Following the example, it’s as if we have rented a safe deposit box with the desired features from the bank and put the key in our pockets.
Step 4: Connecting to the EC2 Instance
After clicking “Launch Instance” as mentioned in the last image, we will click on the “View instances” section shown in the image below.
In the image below, all the Instances that have been launched will be visible. We have created only one Instance, meaning we have rented a single machine.
To activate the instance we created, select the checkbox to the left of it. Once selected, notice that the “Instance State” indicates “Running” in green. This may require a bit of waiting. Our remote virtual machine is now operational.
Step 5: Modifications to the “streamlit.pem” File
At this stage, we need to make a configuration change to the “streamlit1.pem” key file we generated earlier. Otherwise, you will encounter an error during the next step, which involves establishing a connection between our computer and the AWS remote machine. Let’s see the steps to be taken for this file:
Right-click on the “streamlit1.pem” file and click on properties.
Transition to the Security tab and click on the Advanced option.
Among the list that appears, one will start with the name of the computer. For this computer, since the computer name is “pc,” that will remain while the others are removed. To remove them, first, we will select “Disable inheritance,” then click on remove. We will sequentially remove the two options there.
Enter the Edit option.
Only the Read-only option will remain.
Step 6: Establishing the AWS Connection
At this stage, we will connect to our remote virtual machine, i.e., our AWS machine, from our computer and continue working there. To establish this connection, we will follow the steps below.
Open the Anaconda prompt and navigate to the file's location (in this article, the "streamlit1.pem" file) by typing the necessary commands. Go to the directory where you downloaded the .pem file.
Click on Connect in the EC2 screen on the AWS interface.
Navigate to the SSH client section and click the Copy part next to the Example at the bottom left to copy the IPv4 address. Here, “streamlit1.pem” is the name of the key file you downloaded, and “your-ec2-ip-address” refers to the IPv4 Public IP address you can obtain from the EC2 Dashboard.
Paste the text copied from above into the Anaconda Prompt and answer “yes” to the prompt that appears to establish the connection.
The connection has been established if you achieve the display shown in the image below (within the rectangular area). You are now connected to the AWS machine. The line starting with “ec2” in the underlined section is the command line of the machine you will manage with Linux codes on your AWS machine.
Congratulations! You now own your EC2 instance and can connect to it via SSH!
Step 7: Installing Software on the AWS Machine
Our AWS machine is empty, equipped only with a Linux operating system, and capable of interpreting Linux commands. At this stage, we will outfit our AWS virtual machine with libraries enabling us to run our Streamlit project. Essentially, we are transforming an empty machine into one capable of displaying our project.
Execute the following commands in the prompt sequentially to update the system and install the necessary libraries for running Streamlit and for pulling files from GitHub:
sudo yum update
sudo python3 -m pip install streamlit
sudo yum install git –yWe will pull the project file from GitHub to our AWS machine. Paste the following text into the “Anaconda prompt” to clone the relevant folder from GitHub to your virtual machine, thereby downloading the files of the Streamlit project folder onto the virtual machine:
git clone https://github.com/niyaziozkan/Projects.git
By typing
lsin the Linux commands, the folders are listed, and the Projects folder downloaded from GitHub is displayed. The image shows that the Projects folder has been successfully downloaded.
Navigate into the Projects folder, and from there, enter the Car Price Prediction folder. Execute
lsagain to view the contents of the folder. The file we are looking for,app.py, is here. (In this folder, it is saved asCar_Price_Prediction_app.py).
Step 8: Running the app.py File
Having reached this point, we will now run the app.py file of our project created with Streamlit, thereby displaying the entire project as a web page. I would like to remind you that in our previous article, we tested and published the entire project on our local computer. Now, we will be going live via AWS.
Before running the app.py file, we need to install the necessary libraries (in this case, scikit-learn and joblib).
sudo python3 -m pip install joblib
sudo python3 -m pip install scikit-learnWe will run the relevant code to execute the app.py file. However, due to some update issues and related program requirements not being met recently, you may encounter an “Urllib error”. To fix this error, write and run the following code:
sudo python3 -m pip install urllib3==1.26.6
Now, let’s run the relevant app.py file.
(The app.py file in this project is saved under this name)
streamlit run Car_Price_Prediction_App.py
NOTE:
Instead of installing Python library packages in separate lines as we did here, we could specify them in a requirements.txt file. The file would contain the following lines:
scikit-learn==1.2.1
joblib==1.3.2
pandas
streamlit
Step 9: Port Change Situation
When you connect with AWS, make some changes to the Streamlit project, and try to rerun the Streamlit project, you might encounter a port error. This is because we only added port 8501 to the AWS Rules in the lines above. In a new run attempt, the system automatically increments to port 8502 (as shown in the image below), but since there is no corresponding port in AWS, we will receive an error and will not be able to publish. With every intervention in the publishing stage, you will see the port increment, like 8503, 8504, etc.
For this situation, we will need to return to the EC2 section of the AWS interface. First, enter the Security settings of the Instance, then open the Security details.
Navigate to “Edit Inbound Rules” (as shown in the image below).
Then click on “Add Rules”.
Enter the values for port 8502.
Now, the new port has been added. If necessary, other ports can also be added. Alternatively, you could have initially changed the initial port from 8501 to 8502, but adding a new port may be a more appropriate approach.
Section 3: Ensuring Continuous Operation with TMUX
At this stage, our project is running on a URL accessible to everyone, but if we close the terminal screen through which we’ve conducted all operations so far, our application will stop. To prevent this, we will use the TMUX Terminal software.
TMUX (Terminal Multiplexer) is a terminal multiplexer that allows users to create multiple terminal sessions within a single terminal window, switch between them, and control them independently. TMUX enables users to visually manage numerous tasks simultaneously. It is especially useful for maintaining session resilience against disconnections when working with remote servers, allowing users to pick up where they left off upon reconnecting.
To stop the AWS process running from the previous terminal, we will press “Ctrl + C” once. But be cautious to do it only once. Otherwise, you will also disconnect from the AWS machine. You should see a “Stopping” message as shown in the image.
First, we install TMUX on the terminal. For this, enter the following code in the terminal:
sudo yum install tmux
We create a new TMUX session named Streamlit with the following command line:
tmux new -s Streamlit
When the TMUX session is opened, you will see a green line at the bottom of the screen, as shown in the image below.
Now, in this TMUX session screen, you execute the command:
streamlit run Car_Price_Prediction_App.pyAfter running the command within the TMUX area, as shown below, you copy the External URL part and paste it into a web browser to run the project.
Now that we’ve pasted the external URL, our project is running. To ensure that our project application (the Car_price_prediction file discussed in this article) continues to run even when we close the terminal screen, we will detach from the TMUX session. This must be done carefully to achieve our goal. For this operation, press Ctrl+B and immediately press the D key (do not press the Ctrl key while pressing D!).
Even if the terminal is closed, the application will continue to run. However, this deployment will only continue if you keep the AWS machine running.
Since we’re using a free account, we have up to 750 hours of free usage. If this limit is exceeded, you might be disappointed that AWS will bill the credit card details previously provided. Alternatively, you can consider other hosting options or continue with AWS. The choice is yours. If you do not wish to exceed the free account limit after completing all steps related to the deployment process, you must terminate the Instances, i.e., remove them. This is done as follows.
Keeping your page live involves knowledge of domain processes, DNS operations, and hosting activities. Additionally, you can take advantage of the opportunities provided by Streamlit to publish your page through a domain they provide.
You can access the version of the project published via Streamlit Sharing by clicking here.
Conclusion
And that’s it! Throughout this article and our previous one, we have step-by-step explored how to develop a project through a Streamlit application, how to configure an AWS EC2 instance, and how to make our application accessible online.
Deployment, especially in cloud environments, is a broad topic that can be approached in many ways. This guide helps you with deploying your projects. I wish you success in your projects!
