M2M Day 366: Please take my money
This post is part of Month to Master, a 12-month accelerated learning project. For October, my goal is to defeat world champion Magnus Carlsen at a game of chess.
A few days ago, after waiting eight hours for my personal computer to run my chess program, I found out that I would need more computing power to have any chance of creating a usable chess algorithm.
In particular, I can’t use my personal computer to process the amount of data I need to build my chess algorithm within a reasonable timespan. I need a much much faster, more powerful computer.
Luckily, cloud computing exists, where I just upload my code to someone else’s computer (Amazon’s, Google’s, etc.) through the internet, and then let their specially optimized machines go to town.
If only it were that simple though…
I spent some time trying to get my code running on Amazon Web Services, with limited success. After the basic setup, I struggled to get my massive dataset into Amazon storage to be accessed at code runtime.
But then, while researching my options, I came across a tool called Floyd, and it’s pure magic.
Floyd is a layer that sits on top of AWS and lets me deploy my code super easily:
- I open Terminal, and navigate to my chess-ai folder on my local computer
- I run two special Floyd commands
- My code is uploaded to Floyd’s servers and instantly starts running on their GPU-optimized computers.
There are few additional things I needed to figure out like separately uploading and “mounting” my dataset, downloading output files off of Floyd’s servers, etc.
But, overall, I got things up and running very quickly.
As a result, here’s my model running now…
The Floyd computers seem about 3x faster than my personal computer, and I can run multiple jobs in parallel, so it’s definitely helping. Floyd runs at about 75% of the speed of AWS, but the ease of use more than makes up for this. It’s also only $99/mo for the biggest plan, which I’ll happily pay for something this good.
It’s always great to find a new tool that I can add to my arsenal.
Now, whenever I need to do any kind of data science or machine learning project in the future, I’ll know where and how to deploy it in only a few minutes.
This is why learning compounds over time… Today, I did a lot of upfront work to discover the best tool and learn how to use it. Now, I can continue reaping the rewards forever.
Read the next post. Read the previous post.
