M1 Mac Mini Scores Higher Than My RTX 2080Ti in TensorFlow Speed Test.
UPDATE (12/12/20): RTX 2080Ti is still faster for larger datasets and models!
The two most popular deep-learning frameworks are TensorFlow and PyTorch. Both of them support NVIDIA GPU acceleration via the CUDA toolkit. Since Apple doesn’t support NVIDIA GPUs, until now, Apple users were left with machine learning (ML) on CPU only, which markedly limited the speed of training ML models.
With Macs powered by the new M1 chip, and the ML Compute framework available in macOS Big Sur, neural networks can now be trained right on the Macs with a massive performance improvement.
According to the recent Apple blog:
“The new tensorflow_macos fork of TensorFlow 2.4 leverages ML Compute to enable machine learning libraries to take full advantage of not only the CPU, but also the GPU in both M1- and Intel-powered Macs for dramatically faster training performance. This starts by applying higher-level optimizations such as fusing layers, selecting the appropriate device type and compiling and executing the graph as primitives that are accelerated by BNNS on the CPU and Metal Performance Shaders on the GPU.”
Since I got the new M1 Mac Mini last week, I decided to try one of my TensorFlow scripts using the new Apple framework. I installed the tensorflow_macos on Mac Mini according to the Apple GitHub site instructions and used the following code to classify items from the fashion-MNIST dataset.
First, I ran the script on my Linux machine with Intel® Core™ i7–9700K Processor, 32GB of RAM, 1TB of fast SSD storage, and Nvidia RTX 2080Ti video card.
The training and testing took 7.78 seconds.
I then ran the script on my new Mac Mini with an M1 chip, 8GB of unified memory, and 512GB of fast SSD storage. The new Apple M1 chip contains 8 CPU cores, 8 GPU cores, and 16 neural engine cores.
The training and testing took 6.70 seconds, 14% faster than it took on my RTX 2080Ti GPU! I was amazed.
The Apple M1 chip’s performance together with the Apple ML Compute framework and the tensorflow_macos fork of TensorFlow 2.4 (TensorFlow r2.4rc0) is remarkable. For now, the following packages are not available for the M1 Macs: SciPy and dependent packages, and Server/Client TensorBoard packages. Hopefully, more packages will be available soon. Adding PyTorch support would be high on my list.
Finally Mac is becoming a viable alternative for machine learning practitioners. Although the future is promising, I am not getting rid of my Linux machine just yet.
UPDATE (12/12/20): RTX2080Ti is still faster for larger datasets and models!
Many thanks to all who read my article and provided valuable feedback. Special thanks to Damien Dalla-Rosa for suggesting the CIFAR10 dataset and ResNet50 model and Joshua Koh to suggest perf_counter for a more accurate time elapse measurement. Here is a new code with a larger dataset and a larger model I ran on M1 and RTX 2080Ti:
First, I ran the new code on my Linux RTX 2080Ti machine.
Training and testing took 418.73 seconds. I only trained it for 10 epochs, so accuracy is not great. GPU utilization ranged from 65 to 75%.
Next, I ran the new code on the M1 Mac Mini.
On a larger model with a larger dataset, the M1 Mac Mini took 2286.16 seconds. More than five times longer than Linux machine with Nvidia RTX 2080Ti GPU! According to Mac’s activity monitor, there was minimal CPU usage and no GPU usage at all.
Since M1 TensorFlow is only in the alpha version, I hope the future versions will take advantage of the chip’s GPU and Neural Engine cores to speed up the ML training. Only time will tell.
I am looking forward to others’ experience using Apple’s M1 Macs for ML coding and training.
Thank you for taking the time to read this post.
Andrew
@tampapath
