How to host a script on cloud with Amazon Web Services
Let’s say you created a trading script, and want to test out its performance, you’d probably leave the script and computer running for a few hours while getting yourself a beer. In case you want to test longer, fine just again leave it running and head off to the beach. However in case everything goes well, you might want to run the script 24/7, and at this point you have 2 options:
- Keep running it on your computer 24/7 (Burn your computer)
- Run it on a cloud (Burn others computer 😃)
Well just to clarify burning others computer is not why cloud hosting is more preferable to me, but cost efficiency is. Also hundreds if not thousands of staff are handling the burn for the cloud service providers everyday, so let the pro handle that.
For sure its possible to build a computer designed specifically to run around the clock, but if a simple host to run a few scripts is all you need, going for a cloud service might be easier and cheaper, given that you don’t need to care about electricity cost.
Amazon Web Services (AWS) has an edge over other providers by providing a 1 year free tier to new users. Create your account now and start enjoying your free trial. Please note however, the free tier comes with usage limit, if you exceeded the limit, charges may apply. The guide below will stick with “Free tier eligible” resources, meaning that if you follow step by step, you can launch your virtual machine (or, instance) without needing to pay within your free trial.
Launching a Windows instance
Note: The operating system of the instance to launch is different from the operating system of your own computer. The two can be unrelated, meaning that you can launch a Windows instance regardless your computer is running Windows, Mac or Linux.
AWS provides an official and concise guide on this, follow it step by step and you should be able to launch your instance.
After setting up everything, you need to connect to your instance with a Remote Desktop Protocol (RDP) client
[Windows] Windows includes an RDP client by default. To verify, type mstsc at a Command Prompt window. If your computer doesn’t recognize this command, see the Windows home page and search for the download for Remote Desktop Connection.
[Mac OS X] Use the Microsoft Remote Desktop app from the Apple App Store.
[Linux] Use rdesktop.
** Within the free tier usage limit, you can at max run 1 Windows and 1 Linux t2.micro instance non-stop per month without paying, exceeding the limit will lead to charges. If you just want to test it out and don’t want to be charged unexpectedly, make sure to terminate your running instance before launching a new one. Read the section at the bottom for more about usage calculations and limits.
Launching a Linux instance
This is going to be more challenging than launching a Windows instance, mainly because there will not be a pretty UI coming along, meaning that whatever you want to do, will need to be done via the terminal. I suggest going for a Windows instance if you are not familiar with terminal commands.
However if you are clear a Linux instance is what you want, read along.
** Within the free tier usage limit, you can at max run 1 Windows and 1 Linux t2.micro instance non-stop per month without paying, exceeding the limit will lead to charges. If you just want to test it out and don’t want to be charged unexpectedly, make sure to terminate your running instance before launching a new one. Read the section at the bottom for more about usage calculations and limits.
Follow the official guide from AWS
Depending on your Operating System, there are some one-time settings to do when connecting to the instance for the first time, steps are detailed in the above guide as well. After that, you can simply connect with SSH at the command line
Use SSH to connect to your instance
Windows: ssh -i ‘c:\Users\yourusername\.ssh\MyKeyPair.pem’ ec2-user@{IP_Address} (ex. ssh -i ‘c:\Users\adamglic\.ssh\MyKeyPair.pem’ ec2-user@52.27.212.125)
Mac/Linux: ssh -i ~/.ssh/MyKeyPair.pem ec2-user@{IP_Address} (ex. ssh -i ~/.ssh/MyKeyPair.pem ec2-user@52.27.212.125)
You can tell you are connected when you see the username and hostname changed in the command line
Transferring files from/to instance
If you are going for a Windows instance, easy, just treat it as another computer and transfer files via Google drive, sending email to yourself, or whatever you can think of
For Linux instance its gonna be more clunky, one way of doing it is transferring via scp:
Uploading a file:$scp -i ~/Desktop/amazon.pem ~/Desktop/MS115.fa ubuntu@ec2-54-166-128-20.compute-1.amazonaws.com:~/data/
This command will upload a file — MS115.fa in your ~/Desktop/ folder of your laptop to folder ~/data/ on an Amazon instance. Note you still need to use the private key you used to connect to the Amazon instance with ssh. (In this example, it is the amazon.pem file in ~/Desktop/.
Downloading a file:$scp -i ~/Desktop/amazon.pem ubuntu@ec2-54-166-128-20.compute-1.amazonaws.com:/data/ecoli_ref-5m-trim.fastq.gz ~/Download/
This command will download a file /data/ecoli_ref-5m-trim.fastq.gz from Amazon instance to your ~/Download folder in your laptop.
The above examples come from the cool guide below, the author of the guide also introduced FileZilla, a more user-friendly way to transfer files:
Screen command
The screen command is insanely useful when it comes to interacting with Linux instances. If you run a script in the terminal, what happens if you close the terminal? Your script stops, of course. Or you want to run more than 1 script at a time, what should you do? Thats when the Screen command comes into play.
The screen program allows you to use multiple windows (virtual VT100 terminals) in Unix.
To start, simply enter screen in the terminal. Now you are in the “screen”, you may run your script here
To detach from a screen, press Ctrl-a d. Now that you are detached from the screen that has a script running, you can close the terminal without affecting the running script
To start another screen, simply do the above 2 steps again for starting a screen and detaching
To check your opened screens as well as the IDs, enter screen -r
To resume a screen, enter screen -d -r <your_screen_ID>
There are a lot of other commands you can use with Screen as well, checkout the link below:
Free tier usage limit
Free tier users are entitled to limits shown here
Instances fall under the “Amazon EC2” category, at the time of writing, free tier users are entitled to 750 Hours per month, it is an accumulated limit.
Illustrations:
- You keep 1 Windows instance running for the whole 30-day month, 24 hours/day x 30 days = 720 hours used, you are all good
- For some reason you want to keep 2 Windows instances running, in this case you will be using 48 hours (24 x 2) of quotas a day. That means, you can at max run them 750/48 = 15.6 days without being charged
As a closing note, AWS also has the best support in the world, its been a very pleasant experience for myself. If you are looking for cloud providers, I would definitely suggest giving AWS a shot
