Machine Learning: AWS GPU or build your own rig? (Budget Edition)

tl;dr : Build your own machine learning rig if you are into machine learning for the long run. Get a minimum GTX1070 if you are on a budget and you should be able to handle most modelling requirements.

Embarking on Udacity’s Self Driving Car Engineer Nanodegree, I’ve come to realize the benefits that gamers have brought to the machine learning community. The development of Graphics Processing Unit (GPU) largely for video processing and gaming have also allowed developments and growth in the field of machine learning. The use of GPUs have greatly cut down experimentation time with different machine learning models. Personally, i’ve experienced 10–20x reduction in training time through the use of GPUs. Once you begin to dive into the world of machine learning, you’ll come to appreciate the time saved where experimentation time is as important as the time spent in learning the theories behind various models.

Because of the need for GPUs, there is also the dilemma between using On-demand GPUs from Amazon Web Service (AWS) or to build your own rig that incurs an initial cost. Till date, i have spent 140 hours (US$1/hour) on 2 of the projects in the nanodegree, and that works out to US$140 or SGD210! Luckily, kudos to AWS who was able to offer US$100 credit due to errors that prevented me from utilizing the $50 credit offered by Udacity in the course.

AWS offers g2.2xlarge GPUs where the GPUs are NVIDIA GPUs, each with 1,536 CUDA cores and 4GB of video memory. As you delve deeper into machine learning, you will also realize that video ram is going to be a key limitation on the models you are able to develop. With 4GB ram on AWS, I’ve run out of ram on multiple occasions especially for Convolution Neural Network (CNN). I could upgrade to the higher tiers with more RAMs but then again it’ll also come with a higher cost. Nonetheless AWS GPU instances gave me a taste of what GPUs are able to offer but with considerations of venturing further into machine learning, I reckon that it’ll be more worthwhile to setup my own machine learning rig. Hence also my search began on finding the best setup on a budget (approximately SGD1000) for a desktop machine learning rig.

Researching for the right GPU

Quora was my first stop in determining what was the best GPU for machine learning but what finally got me to settle on the Nvidia GTX1070 was the following post on reddit and a subsequent post comparing the price for performance. As i am looking to build a machine learning rig on a budget and subsequently upgrading it when necessary. GTX1070 suits me just well. Comparing 1,920 CUDA cores and 8GB of video memory in the GTX1070 with what is on offer on AWS, that is 25% more cores for utilization and twice the ram for modelling. Price was also affordable at SGD578 for a Palit GTX1070 Dual compared to other GTX1070 from other manufacturers.

Shopping around for other used computer parts

As GPU was my primary consideration and this is a machine learning rig built on a budget, other computer parts were largely sourced from the used parts market place. I found Tim Dettmers’s post on deep learning hardware guide pretty useful where it goes deep into the technicalities of choosing the right components for the rig.

By a stroke of luck on the used parts market place, my final setup for my machine learning rig was built for approximately SGD1000 and the parts list are as follows:

  • CPU: Intel Core i7–4770k
  • Motherboard: MSI Z87 Mpower
  • Ram: 8GB Corsair Vengeance 1600MHz
  • Harddisk: Samsung 750 EVO 120GB + 1TB Toshiba HDD
  • PSU: Seasonic X650 650W
  • Case: Cooler Master HAF XB
  • CPU Cooler: Akasa Venom Voodoo w/ 2x Akasa Viper S-Flow Fans in Push Pull
  • Case Fan: 2 x Stock Cooler Master
  • GPU: Palit GTX1070 Dual 8GB

I am certain the parts that i’ve listed above will not be the best parts for machine learning but my practical experience thus far tells me that this rig has been able to handle most CNN i have thrown at it and at speeds one can only dream of compared to times when i was running on a Macbook Air (2015) i7 2.2GHz.

Hope this post is able to help the same people who are like myself starting out on machine learning, and planning to deep dive into the topic to see how deep the rabbit hole goes.