Hi all. Today we’ll be seeing how we can use EC2 like you would have used a lambda function.
So this one time I wrote a scraper in Python and I wanted to host it on AWS lambda as the scraper was scheduled to run once everyday. But I came to know that lambda functions have a time limit of 15 minutes, and guess what my scraper code took more time to execute. Then I discussed this problem with a friend of mine and he suggested that I use the
paramiko python library to SSH into the instance and then execute commands.
Let’s start be importing all the libraries that we’ll use to run our code.You’ll need to install
botocore packages using pip install.
After you’ve done installing the above mentioned libraries, we’ll need to define 3 variables namely
You’ll get the
instance_id from your AWS EC2 dashboard and
aws_secret_access_key will need to be created from the IAM dashboard by creating an IAM role and assigning the proper permissions to it. After doing this let’s create a function called
ec2_ctrl which will turn the ec2 instance on or off depending on the action argument’s value.
Now we will define a function called
cmd_exec that’ll be responsible for finding the ip address of the instance that we’ll be executing code on using the instance name. Then
paramiko will take that ip address to ssh into our server and execute commands on our behalf. In the example below we are running a python script named
scraper.py located in the home directory for the ubuntu user.
Now we just need to run the above defined functions in the correct order to achieve our goal. So we type…
Just remember that all this code needs to be in one single python file. The filenames for the gists differ because for each snippet of code I had to create a new gist.
That’s all for this article and if you have any question or any kind of suggestion on how I can make this article better, do let me know in the comments below or DM me on twitter. My twitter handle is