GPU computing has become indispensable to modern artificial intelligence. The vast parallel processing power of graphics cards allows neural networks to train on huge datasets faster than ever before. But one company dominates this landscape — Nvidia, with its proprietary CUDA platform. AMD has struggled for years to provide an alternative with its open-source ROCm software. Where does this battle currently stand?
A Brief History
CUDA burst onto the scene in 2007, giving developers a way to unlock the power of Nvidia’s GPUs for general purpose computing. This proved revolutionary, as CUDA’s performance exceeded CPUs by orders of magnitude on parallel workloads. It sparked the GPU computing revolution that enabled breakthroughs in AI.
AMD was quick to respond with its “Close to Metal” initiative in 2008, but this proprietary technology failed to gain traction. In 2014, AMD tried again announcing the Heterogeneous System Architecture (HSA), an open standard for GPU computing. But HSA languished due to limited industry adoption.
Finally in 2016, AMD launched ROCm — an open-source platform for GPU computing on Linux. ROCm provided tools like compilers, libraries and the HIP programming language. HIP was designed as a “portability platform” — a CUDA clone that allowed developers to port their CUDA code with minimal changes.
The Current State of ROCm
Years later, where does ROCm stand in relation to CUDA? Let’s dig into the developer experience, documentation, performance and adoption.
Developer Experience
Right away, ROCm reveals a fragmented developer experience. The ROCrand documentation points developers to two different platforms — ROCm itself, and something called “HIP-CPU”.
HIP is already a CUDA imitation layer. Splitting it into HIP and HIP-CPU seems duplicative, when alternatives like SYCL and Kokkos run cross-platform from a single codebase.
The HIP-CPU GitHub page has languished in development for 3+ years. This paints a picture of AMD spreading itself thin across disparate competing platforms.
Documentation
Unfortunately, ROCm’s documentation remains quite poor. The ROCrand docs consist almost entirely of truncated function documentation copied and pasted repeatedly.
The ROCrand Python API docs are shockingly sparse — literally a single page with no detailed guidance. The C++ API docs mostly repeat the same vague information about random number generator algorithms.
This feels like documentation exists solely to “tick a box”, without any real effort to aid developers. Especially given the complexity of GPU programming, excellent documentation is crucial.
Performance
Benchmarking ROCrand against CUDA on an Nvidia V100 GPU reveals a 30–50% performance deficit on real workloads like raytracing.
Some may argue this benchmark is unfair to AMD hardware. But a simpler Philox implementation achieved parity with CUDA, suggesting the difference lies in ROCrand’s implementation quality — not a lack of GPU-specific optimizations.
Adoption
Despite being open source, ROCm has failed to achieve widespread adoption. That’s likely due to its limitations around performance, documentation and compatibility.
The latest StackOverflow developer survey found CUDA usage dwarfing OpenCL and ROCm. On the HPC side, Nvidia continues to dominate the Top500 supercomputer list.
That said, AMD GPUs have achieved high-profile supercomputing wins like Frontier and El Capitan. However, this seems more attributable to competitive procurement than developer preference.
The Long Road Ahead
While AMD has absolutely made progress with ROCm, the platform remains far behind CUDA in critical aspects like documentation, performance and adoption.
Realistically, AMD will struggle to achieve parity let alone surpass Nvidia given their massive head start. Nvidia invests billions in CUDA development and ecosystem expansion annually.
That leaves the door open for a new challenger like Intel, whose pockets rival Nvidia’s. Intel’s SYCL technology flaunts far superior documentation to ROCm. If Intel executes on the software side, they could pose a threat.
For AMD to truly challenge CUDA, they must double down on ROCm documentation, performance and compatibility. Recent events suggest a growing commitment to ROCm. But executing that vision will require major resources.
The GPU computing landscape remains dominated by Nvidia’s proprietary CUDA. AMD has a mountain to climb with ROCm. While the open ecosystem they envision is compelling, it will require immense focus to make it a reality. This battle is far from over.
