I Bought a Laptop for Deep Learning and Now I Mainly Use The Cloud

This article reviews my struggle with the question — “Cloud or on-premises GPU machine for deep learning?” and conveys some of my lessons learned.

DISCLAIMER / UPDATE (06/25/2021)

Although this article conveys the pros of using the cloud vs. spending a lot of money on personal hardware, I should caution people that you need to be careful when managing cloud costs. For more information, you can read my article on this concept here. Thanks :). Now back to your regularly scheduled programming :-D ….

For this article, let’s first go back a year or so …

Photo by Aron Visuals on Unsplash

A year or two ago I was doing deep learning on Kaggle, Google Colab and a bit on Sagemaker. I was running up against timeouts on Kaggle and Colab, as well as the compute costs on Sagemaker. I thought maybe it was time to invest in a personal GPU machine. Since this was my first GPU machine for deep learning, I had the following requirements:

• A machine that works
• Doesn’t break the bank
• No waiting for it to ship
• Portable

The first requirement seems like a given, but not necessarily since my first consideration was to build my own machine. I went out and bought books on the subject, and quickly realized that building my own machine was risky and time-consuming.

Photo by Jackson Simmer on Unsplash

Next, I took a look at online custom PC shops like XOTIC PC. These options were enticing, but I didn’t like the idea of waiting for the machine to ship, and most of the options seemed expensive.

After some more research, I finally threw in the towel and I ran to my local Best Buy. After some time browsing around, I finally landed on an ASUS — ROG GU501GM 15.6" Gaming Laptop — Intel Core i7–16GB Memory — NVIDIA GeForce GTX 1060–1TB Hybrid Drive + 128GB SSD — Brushed Black for ~$1,000.

Photo by Avi Richards on Unsplash

I brought it home and quickly got to work setting it up. Setting up the machine for GPU deep learning use was a bit scary given it was my first time, and I didn’t want to mess anything up (not to mention I just dropped a ton of money on this new machine). After a few hours of reading and delicately following instructions, I was finally up and running! I had a TensorFlow environment and a running recurrent neural network training job!

Fast forward to today, and a lot of laptop deep learning jobs later…

If you could plot the amount of training time spent on my laptop vs. the cloud, the time spent on my laptop would be a downward trend.

Photo by Ussama Azam on Unsplash

So what happened? Well, a few things happened.

1. The truth is that things are moving fast, and cloud services keep getting better and cheaper!
2. Transfer learning has become more powerful and available.
3. My personal laptop only has so much compute.
4. As new frameworks and packages become available, a personal GPU machine requires time setting up environments and such, vs. the cloud which is more available and tolerant of making setup errors!

Photo by Jake Givens on Unsplash

I kind of pigeonholed myself by buying a GPU laptop with below-average compute power, but it was my first GPU machine, so no regrets there, it’s good to start small and simple.

As things progress in the world of deep learning, the cloud is really becoming a more viable option. The following is a list of cloud services I use from most frequent to least frequent:

1. Kaggle
2. Azure
3. Paperspace
4. Colab
5. Sagemaker

Kaggle is my #1 location for running deep learning jobs because it’s free and has an amazing community for collaboration.

Azure and Paperspace are tied, and it really depends on the type of job that I’m running.

• Azure spot instances are awesome for running cheap multi-GPU jobs for custom neural network architecture search. The one drawback of Azure is that you need to be careful with data storage. Lingering data can cost you a fortune, especially if you leave that data on expensive hardware.
• If a single machine with high GPU memory is needed, then Paperspace offers a 24GB P6000 GPU for roughly $1/hour.

Colab is my go-to when resource needs aren’t high and I’ve run out of computing time on Kaggle. They now offer Colab Pro for roughly $10/month which grants access to larger GPUs with no timeouts! Granted you don’t get full access to everything you might need, it’s still a nice balance between cost, ease of use, and power.

Lastly, Sagemaker has a lot to offer, but most of the time I just want to get up and running. Sagemaker is either too expensive or time-consuming. If I want to get up and running quickly, I could run my deep learning in a Sagemaker notebook instance, but this is expensive. To save money I can do my training in a Tensorflow container, but this requires some code rewiring. It’s just easier to run to another service, so I can focus on the things that matter. Maybe my view on the subject will change over time.

DISCLAIMER / UPDATE (01/19/2022)

So my opinion on Sagemaker has changed lol. Although I don’t do as much hobbyist deep learning training, I’d rank Sagemaker and Azure about the same.

In Summary …

Photo by Aaron Burden on Unsplash

If there are a few things to take away from this story, it’s this:

1. The cloud is becoming a more viable option for powerful deep learning on a budget! And as things progress, it’s less to maintain so you can spend more time doing what matters!
2. If you’re going to buy a personal GPU machine, think long and hard about what you’re not getting from the cloud, and maybe focus on that. Also, know that whatever you pick, it will most likely become obsolete quicker than you expect.

Thanks for reading and hope people get something out of this little story!