Wrangling with the GPU: Lessons Learned about Insidious Bugs
We always take for granted the systems we are familiar with, and often become used to all the boilerplate we have to write for them. To those unfamiliar to the rituals of a system, or without a comprehensive understanding of one — bugs can seem downright impossible to squash. You can often feel you are doing it “wrong”, out of your depth, or just plain stupid.
On many technical courses at universities, you are at some point required to do some kind of technical project followed by a write-up. This procrastination-generating task often takes place in your 3rd year, and is normally an important part of the course.
Here you can find some of my experiences when I programmed for the GPU for the first time during such a project at university how I learned from them. The hope is that this information will be useful, interesting, and give you an understanding of how many, many people go through the same struggles you do when in these shoes!
The Project — A Brief Overview
The project I decided on doing was a simplistic, real-time simulation of the surface of water on the GPU, implemented in the Unity game engine. You can find the project and the dissertation on GitHub. The basic premise was to take an idea known as Wave Particles, which has been used in games like Uncharted 3 and Uncharted 4, and implement it on the GPU using compute shaders.
The technique essentially works by treating the surface of the water as a 2D plane and representing disturbances to the surface using particles on that plane — by rendering a bump where these particles are, the sum of them can create convincing wave effects.
This is particularly well suited to the GPU, which are massively parallel, as each compute unit can be used to operate on each particle.
Programming for the GPU — It’s Hard
I took on this project never having written a single program for the GPU before (unless you count this). In fact, I had never used Unity or had any experience with C#, the program language Unity uses. I was diving straight into the deep-end, which is hard when you don’t even have your water simulation ready yet!
One thing that’s worth pointing out is that the difficulty didn’t come from the theoretical difficulty of the algorithms involved, but instead from the faff of setting everything up to work. The algorithms weren’t hard to understand or grasp, but getting them to run on the hardware was an entirely different matter!
The code below shows the GPU compute kernel — code that runs on the GPU — that ‘splats’ Wave Particles to the water’s surface.
Despite the simplicity of this code itself, getting it to run wasn’t the most straight forward thing in the world.
Boilerplate
This is where boilerplate code comes into play, setting everything up for this simple piece of code to run, and it isn’t pretty. I won’t go into detail about what is going on (it also handles some other GPU code beyond the above) but will highlight some of the more aggravating parts.
One of the first things that the boilerplate does is define a load of strings similar to the above. Why? Because the variables in the compute kernel (which are essentially parameters) above need to be identified by the CPU-side code somehow and string literals are the only option available! This also means that any programmers who wants to change the name of a variable that is set by the CPU needs to remember that these strings need to be edited, because you won’t get a compile-time error. That in and of itself would be relatively manageable if it weren’t for aggravating point #2:
Kernel initialization
The code above is how the variables are initialized and the compute kernel itself is invoked. Again, this doesn’t look unreasonable, but it would have been nice to have been able to do the following, complete with compile-time type-checking and boilerplate generation:
However, the real difficulty didn’t come from the code, it’s the fact that if there is any issue at all with the above, for example you forget an initialization, you don’t name a parameter correctly, or there is a type mismatch: you don’t get an error, just incorrect behavior. I was lucky in that it never bit me too hard, but I did get the following issue for two weeks with no idea as to how it was caused.
It turned out that one of the textures involved was accidentally set to use integers instead of floating points! I actually thought that was the case the entire time, but I missed the one piece of code that did the incorrect texture creation. Oops!
Why Talk About This?
Why talk about all these issues then? What message am I trying to convey by talking about all these troubles I had with a 3rd year project at university? Firstly, because I hope it’s interesting both technically and otherwise in addition to providing a brief overview of how some aspects of GPU programming may work. However, it’s mainly because I think it’s worth being reminded that a lot of people are in the same boat when doing these things — struggling day-to-day having no idea if or how things will pan out.
Although I have discussed things that were frustrating, part of me thought that maybe I was (and am) just stupid, maybe I was doing it wrong because surely this isn’t the way to do it. I still feel that now, and am a bit nervous about sharing it because then I probably will be told that I was doing it wrong the whole time. I think everyone can feel that way, and one of the things that always helps when staring at a piece of code that still isn’t doing what you want it to after the 100th edit is that everyone else is going through the same process.
However, feelings like this are an opportunity to question the way we do things and improve them. In fact, beginners can provide invaluable feedback on removing friction from systems which may seem invisible to experts that are used to it. I’m now working on a system that explores type safety across the CPU/GPU boundary as a result of my experiences with these systems as a beginner — even though now I know how to work around these issues.
If you are currently doing one of these style of projects, and feel out of your depth, a bit behind, or even just not sure if you are heading down the right path — other people have been there as well. I haven’t even discussed the process of trawling through the depths of documentation, battling LaTeX, feeling like the winter holidays were wasted, or many of the other things that happened.
However, by the end of all of this it feels so satisfying and when you get your actual first result all the difficulties become so worth it. So if you are doing one of these…
BEST OF LUCK!
Special Thanks
I would like to thank the following people who helped me on my project and writing this post!
Talea Roberts — I would not have been able to get through this project otherwise! Fantastic advice, support and morale boosting.
Huw Bowles — For providing really incredible advice and support, and originating the project idea.
Rafal Mantiuk — For agreeing to supervise me and all the valuable feedback on my project and dissertation.
Alastair Toft — For the general moral support.
Hari Prasad, Christian Silver and Jared Khan — For editing the contents of this post!
