eGPU for Mac — Deep Learning with Tensorflow on a Macbook Pro with Bootcamp Windows 10
So, you have finally gotten frustrated with the slow training performance of your Macbook training your Deep Learning models and want to do something about it. You have watched with interest that Macs come with Thunderbolt 3, and eGPU support.
You know that:
- Tensorflow uses CUDA, which is an nVidia technology, and requires an nVidia GPU. So the key point here is Tensorflow needs an nVidia GPU if you are looking to go eGPU.
- Apple uses AMD GPU’s and doesn’t support nVidia.
- Some people have gotten eGPU’s to work with Mac and Tensorflow, and have written helpful posts to share what seems to be a bit of a painful but doable process. There seems to be a way forward.
This is my story of Mac and eGPU’s. It is worth pointing out here that my Macbook Pro is a 2018 running Catalina — 10.15.4.
After some research I find a good eGPU enclosure (Razer Core X) and a kicking nVidia GPU (RTX 2080 Ti) to get going. The installation of the GPU into the enclosure couldn’t be simpler, and I’m really impressed with the quality of the Razer box.
Now for the system setup.
The steps defined Nvidia eGPU + MacOS + TensorFlow-GPU? The Definitive Setup Guide to Avoid Headaches are my starting point.
Step 1, disable SIP, easy, Step 2 run Install Purge-Wrangler.sh — seems to go ok
Until..
No……..
The problem
Researching a little more (which I should have done earlier!), I discover that the missing piece is an nVidia driver for RTX 2080 Ti for Mac. But hang on I’m a bit confused, why do I need a Mac nVidia driver if I have CUDA loaded. Have a look below at the block diagram from this presentation (slide 10).
Note that you need the CUDA runtime and the CUDA driver. Turns out that nVidia make CUDA drivers for High Sierra, and there isn’t a driver for Mojave or Catalina. If you do have High Sierra, you should check if there is support for your card. I think from what I have read, only Pascal architecture cards are supported. Further, there is no chance that there will be one, ever! Many web posts talk about the high level animosity between Apple and nVidia. I’m in a pickle!
Bootcamp to the rescue.
There are a couple of posts from people that have resorted to running windows 10 on Bootcamp, after a little deliberation, and disappointment that there weren’ t any other options, I decided to push ahead and take this approach, having invested in the card and case.
To install Bootcamp on Mac is relatively straight forward and there are many instructions available. Here are Apples instructions.
Once you have windows up and running, it should be plug and play (make sure you use a later build — 1903 or greater which has better hot starting of Thunderbolt 3), download the nVidia drivers and install them with your eGPU connected. If your GPU has been detected ok, you will be able to install the drivers. If there is a problem, the software won’t install.
Then install the CUDA drivers from here. Again you need your GPU connected for the software to install. If all goes well, your card should be installed and ready for use.
Setup Environment
I used miniconda to setup my python environment, you can install it from here. Make sure conda is on your path, I had a plain install, so accepted the option to add to the PATH variable. From the terminal install your libraries:
conda create -n deep_learning
conda activate deep_learningconda install keras
conda install tensorflow-gpu
conda install scikit-image
And now I am up and running and can train my model. How does it look?
Before
The model I am training is a Fully Convolutional Network (FCN) in a decoder-encoder architecture. Using the Macbook CPU using Mac OSx Catalina the results for a short epoch are below.
You can see that one step took around 2 seconds, and the model trains in about 20 epochs of 1000 steps. Total train time ~ 11 hours.
After
Using the eGPU and RTX:
Running the same model resulted in a step time of ~ 70ms, around a 28 times speed up. The total train time now is around 23 minutes!
Problems
Although the eGPU works with Bootcamp, it is a bit fragile. It appears that there is enough resources to run the eGPU, but when I insert an external USB drive, the eGPU stops working. Similarly I can’t run bluetooth either. After a little more investigation, it looks like the problem is allocation of resources by windows.
For a later build of windows and a 2018 Macbook pro the eGPU is plug and play, but with no resources left for external devices. The fix is to use the iGPU which is deactivated by default when booting in to windows, and to remove drivers for the dGPU (Radeon Pro 560X) which takes up a PCIe bus. Unfortunately, this requires a bit of a hack, just my luck much harder to do in Catalina, which at this point I’m not to keen to do.
Conclusion
Using an eGPU (RTX 2080 Ti) with a Macbook Pro via Windows 10 is realtively easy to do and Bootcamp resulted in a 28 times speed up of training. This means that models previously trained in days are now trained in hours. The speed up is significant, but you must have a strong need for speed, as it is a bit of a hassel getting it all together.
I definitely recommend the Razer Core X, and the Gigabyte RTX 2080 Ti. I also love Keras and Tensorflow. I just wish nVidia produced drivers for Mojave and Catalina so Mac it was a little easier to be eGPU enabled.
Overall though, it was worth it! Hoping my experiences are useful for you!
Update: May 5.
One week + in, see my follow up article to see how things are going — the good, the bad the ugly!
