Build your 1st Deep Learning Rig
Step-by-step: How to find, buy, and construct your AI sandbox
Part 3 of our 5-part series How to Build an AI Sandbox. Part 1, 2, 3, 4, 5.
Previously we defined an AI Sandbox:
The AI Sandbox is the development hardware, software, data, tools, interfaces, and policies necessary for starting an enterprise deep learning practice.
In part 2 I explained why you need an AI Sandbox. In this post I’ll explain how to build your deep learning rig, the hardware portion of your AI Sandbox.
Expectations
When I started my career hardware was a big part of working in technology. Buying, hosting, installing, and configuring hardware was just part of the job.
Times have changed and today almost all development happens in the cloud. The last time I bought and configured a linux server? Nearly 20 years ago when I built my own linux router and mail server from scratch.
So don’t be intimidated if your only hardware experience is plugging your MacBook into an electrical socket — just don’t expect a quick, painless process.
The only critical skill you need is patience.
Plan on a day to pick parts
I know you’re looking for “the easy button” — you want me to tell you “just by X from Y”. I started by looking for pre-built options from custom gaming computer makers like Xidax and others. Unfortunately I didn’t find a solution close to what I wanted. The pre-built systems using higher-end GPUs were optimized for gaming and not deep learning.
My advice is to take your time and block out a full day to research the components for your system. There is no “right” system for you because you have infinite options.
Plan on a day for construction
PC hardware manufacturers have collectively crowdsourced their documentation to YouTube. Most documentation is terrible.
Expect:
- a pile of unlabeled screws and cables.
- inconsistent and conflicting documentation and advice.
- rambling YouTube “HowTo” videos by people who know less than you.
- confusion. “Where do I plug in this fan?”
Fortunately enough Googling will get you to the right answers.
Research & References
Helpful articles
Building your own deep learning box by Brendan Fortuner
Which GPU(s) to Get for Deep Learning: My Experience and Advice for Using GPUs in Deep Learning
The $1700 great Deep Learning box: Assembly, setup and benchmarks by Slav Ivanov
Each of those link to other articles.
Google-fu
Most of what I learned came from Googling about topics. Tom’s Hardware frequently ranked high in search results and lead to good discussions by smart people.
PCPartPicker Builds
Searching through existing systems and you’ll learn a ton.
Here is my parts list. Remember that I’m NOT building a low-budget solution but a starter system for an enterprise team.
How to design your system
Get organized
Open up a new browser window and create tabs for the references above and this article. Read through each once, you’ll refer back to them constantly.
Create a free account on PCPartPicker, at that to another tab and start a parts list.
Build your parts in an order optimized for deep learning — not gaming or general computer use.
First pick your NVIDIA GPU (Video card)
Start with the GPU and ensure all other components are compatible with it.
Tim’s GPU article provides great background material on GPUs. Unfortunately this information quickly becomes dated.
A good rule of thumb is to start the fastest gaming GPU from NVIDIA to optimize price/performance.
Currently that is the 1080 Ti. Dozens of manufacturers make video cards based on these chipsets. Just pick a popular one.
Second pick your CPU
Pick a CPU compatible with your GPU AND which allows you to add additional GPUs without a loss of performance. This seems to be a common mistake. I only avoided it myself because read about completed systems before starting.
Look for 40 PCIe lanes.
Third pick your motherboard
You know this is a “mobo” because you’ve been reading about builds by gamers. Don’t you feel cool?
PCPartPicker now becomes a great resource because it filters out incompatible options.
Again, pick popular options with a lot of reviews.
Now Google-fu!
You’ve selected the 3 most critical parts of your deep learning rig: GPU, CPU, and mobo.
Spend a few minutes reading about existing builds and Googling combinations of all 3. You may also need to research potential incompatibilities flagged by PCPartPicker.
Add RAM, SSD, HDD
As you begin building more complex models you will quickly learn hacks for optimizing data placement. Having layers of storage will give you the best price/performance curve.
- RAM — According to Tim at least the same RAM size as your GPU. I got 32 GB.
- SSD — Get a Solid State Disk build with NVMe. You’ll put your Unix home directory and most active data files here to prevent data slowdowns.
A 500 GB NVMe SSD costs only $200 and will boost your productivity — it is a no-brainer.
- HDD — Get as much Hard Disk Drive space as you want. You’ll store large data files (like all of Imagenet) here. HDD’s are easy to add and cheap. I started with 5TB.
Add a CPU Cooler
These are commodities, lots of options. Your best bet is research on PCPartPicker.
Test scalability by adding additional RAM and GPUs
Future-proof your hardware by adding additional RAM and GPUs on PCPartPicker — even if you don’t but it now.
Add a second GPU and max-out the RAM for your mobo.
Pick a Power Supply fit your future system
Get more power than you need for your future system.
Case is personal choice
Read about existing builds and reviews. Pay particular attention to cable management and space.
You don’t want to waste hours cramming everything into tiny spaces!
I bought a relatively big case with lots of room — and was glad when it came time to assemble everything.
Everything else
I found a monitor, keyboard and mouse for less than $100. I only used them to configure the computer. Now they just gather dust.
Get a good screwdriver
Thank you Brendan Fortuner for this great advice. A long, magnetic screwdriver was an absolute lifesaver.
I got this one. Ignore the negative reviews about magnetism problems. Unless you try to pick up a crowbar it will work just fine.
Final review — existing builds
Finally go to PCPartPicker and review existing builds. Pick combinations of your case, GPU, CPU, and mobo to read about others’ experiences.
Buy it!
I decided to buy everything on Amazon for simplicity.
Plan on tweaking your parts list based on availability. I chose a different case and SSD because I didn’t want wait for weeks. This is another reason why you can’t just copy my parts list!
Getting everything from Amazon was convenient because I could compare my Amazon shopping cart to my PCPartPicker build list.
How to assemble your deep learning rig
Get organized. Stay organized.
Assembly is actually easier than picking parts — but it still takes awhile. Give yourself a lot of space, light, coffee, grab your new screwdriver and jump into it. about installing your motherboard into your case.
Read the manuals for your case and motherboard first. Your system is built around them.
Confusing? Of course. Watch a few YouTube videos about your case and motherboard. If you picked popular components you should find something.
Stay organized! Each box will have many extra parts. Keep everything together and you’ll save yourself a lot of agony.
Unpack and organize your case
Separate and organize the screws and cables in your case.
Give yourself an entire desk. You will be constantly turning your case around.
Follow your case instructions and then your motherboard instructions
I chose to first install the motherboard in the case and then install the CPU, cards, and RAM on it. Other people choose to install everything on the motherboard first. I think it comes down to personal preference.
I ran into problems and figured out how to overcome them. Brandon ran into different problems and overcame them. You’ll have to so the same.
It doesn’t work? Get a bigger hammer
As others have noted, sometimes you just have to press harder.
How hard is too hard? I have no idea, but my GPU and my RAM didn’t work until I REALLY pushed on them. Then they snapped into place.
It works? Now give it a cool name
My AI rig is black with red LEDs, so of course I named it Drogon.
