High performance drawing on iOS — Part 1
As I was building my recently released drawing game, I explored different ways to perform 2D drawing on iOS. Throughout this 2-part series, I will share what I’ve learned by comparing the performance of 4 distinct ways to draw on the platform. All 4 techniques will look exactly the same in terms of the final drawn image, but as we will see, their implementation and performance will vary wildly.
This is by no means a definitive guide for 2D drawing. I am still learning about all of this, and I’ll be updating this article as I learn new information. If you wish to share a better way to draw, I would love to hear your thoughts in the comments section or on Twitter!
Part 1 will cover:
- Low performance CPU-based drawing
- High performance CPU-based drawing
Part 2 (available here) will cover:
- Draw(layer:ctx:) GPU-based drawing
- Sublayer GPU-based drawing (what I ended up using in my game)
Low performance CPU-based drawing
This approach was probably the simplest to implement. Here you would subclass UIImageView and use it as your drawing canvas, and then you call this drawing code inside touchesMoved:
let renderer = UIGraphicsImageRenderer(size: bounds.size)image = renderer.image { ctx in image?.draw(in: bounds) lineColor.setStroke() // any color ctx.cgContext.setLineCap(.round) ctx.cgContext.setLineWidth(lineWidth) // any width ctx.cgContext.move(to: previousTouchPosition) ctx.cgContext.addLine(to: newTouchPosition) ctx.cgContext.strokePath()}
You can view the entire canvas class here if interested.
note: be sure to set isUserInteractionEnabled = true on your UIImageView
That code lets you draw stuff like this:
This approach got me pretty far in my development of the game. In fact, I only noticed its drawbacks once I launched the game on my iPad to test the UI on the bigger screen (I spent the first month of development exclusively on my iPhone 6s 😅)
I was getting a smooth 60 frames per second while drawing on my iPhone 6S, but bizarrely, when I tried drawing on my brand-new 11" iPad Pro the frame rate plummeted to the teens! (17 fps average)
Something was terribly wrong here. I looked at Xcode, and noticed that the CPU was sitting at 100% utilization throughout my drawing
After some investigation, the culprits turned out to be:
image = renderer.image { ctx in ... }and
image?.draw(in: bounds)First, the renderer is creating a UIImage based on a set of drawing instructions inside the closure. This is a CPU-intensive operation.
Second, that closure above also includes image?.draw, which itself is drawing the entire image to that graphics context. This is another CPU-intensive operation.
Now imagine calling both of these methods hundreds of times a second, no wonder the frame rate is plummeting!
You might be wondering why the iPhone 6s, a nearly 4 year-old phone, doesn’t have this issue, while the iPad Pro does. There are actually two reasons for this:
- The 11" iPad Pro has a resolution of 2388x1668, that’s 4 million pixels, while the iPhone 6s has a resolution of 1334x750, or 1 million pixels
- The iPad Pro has a 120hz display while the iPhone’s is just 60hz. In other words, touchesMoved gets called twice as often on the iPad
Overall, we’re looking at a ~8x increase in CPU demand. The iPad Pro’s A12X CPU is certainly far more advanced than the 6s’s A9. Unfortunately, CPU-based drawing is a single-threaded operation (note the screenshot above shows 100% used out of potential 800), so the multicore setup in the A12X doesn’t give it any benefit here. And if we compare the single-threaded performance between the two CPUs, the A12X is just over twice as fast as the A9 according to Geekbench.
Now that the performance mystery has been solved, it’s time to explore a better way to draw!
High performance CPU-based drawing
This approach is centered on the draw(_ rect:) method for UIView, which you cannot call directly. Instead, you invoke that method by calling setNeedsDisplay() on the view. The idea for drawing here is that you would store the points you want to draw inside of an array (this is done in touchesMoved), and then you would loop over these points inside draw(_ rect:) to perform the actual drawing.
The initial class looks like this:
If you run this code on an iPad, you’ll see an immediate improvement over the first approach. This time you’ll start with 120 FPS, and CPU utilization will no longer be maxed out at 100%, however it’s still somewhat high, hovering around 60%. Moreover, if you continue drawing on the canvas, you’ll notice the CPU % gradually rising over time. After a short while, it will reach 100%, and soon after the frame rate will plummet to the teens. Uh oh, not good!
But fear not, there are a couple of easy optimizations we can make that will greatly improve the performance here. First up, instead of calling setNeedsDisplay, which marks the entire view as needing a redraw, we really should be calling its sibling, setNeedsDisplay(_rect:), which would mark only a small rect within that view as dirty and in need of a redraw.
To do that, we would need to calculate the area that needs to be updated within the view. This is a rectangle between the last touch location and the new one, something like this:
We should also account for the drawing line width, so the actual rectangle will be slightly bigger. We calculate it with this method:
That method takes the lastTouchLocation, so we need to retrieve that by modifying our touchesMoved method slightly. Here’s what it looks like now:
Try launching the app on the iPad again, and you’ll find that the CPU utilization has dropped all the way down to 20% while drawing now. Nice!!
However, we’re not out of the woods yet; you’ll notice that the CPU % quickly climbs back up as you continue drawing. To deal with this, we will have to flatten the image every once in a while (i.e empty the lines array and convert the drawing to bitmap.)
We will flatten the image whenever we collect 200 points, or when touchesEnded is called. A nice side effect of this is that we no longer need a multi-dimensional array of lines to save the points. A simple [CGPoint] array will do the job, since we’re no longer storing multiple lines (they get flattened now).
We need to add some convenience methods here (summary below):
To summarize what was added, we’re basically flattening the image when touchesEnded is called. We also flatten the image when we reach 200 points (this is checked in the property observer), however we leave one point in the array, to ensure no interruption to the drawing.
The draw(_ rect:) method has also changed a bit, since we need to render the flattened image now if it exists:
You’ll notice that we’re using image.draw() again (this caused a massive slowdown above), but don’t worry, because this time we’re calling it on a small subset of the view. It won’t apply to the entire size of the view.
If you run the app on the iPad, you’ll see that the CPU will now stay around 25% utilization no matter how long you keep drawing. And the frame rate is a steady 120 FPS (or 60 FPS depending on device.) Cool!
So, to recap this approach:
- In touchesMoved, we store the touch points inside of an array
- We call setNeedsDisplay(rect:) to mark the dirty area for redraw
- In draw(rect:), we loop over the array of points and perform the actual drawing
- When the array reaches 200 points, we flatten the image and empty the array
At this point we have a very workable system, and I can see myself shipping the game like this. But something still bothered me about this approach; it’s the fact that it’s single-threaded. I wanted to know if it were possible to utilize the full power of the hardware instead of just one CPU core.
In Part 2, I will talk about my attempts to leverage the GPU to perform parallel high-performance drawing that is not limited by one core. It actually turned out to be not that difficult. By the time I released the game, I had successfully managed to lower the CPU utilization to just 10%! Check out Part 2 here!
I uploaded a small app to github that demonstrates the drawing concepts described in this series. It has an FPS counter in the corner to help you compare the performance. You can either draw yourself, or tap the “Start Drawing” button to have it draw a spiral for you. You can find it here
Be sure to run it on a real device, as the simulator won’t give you accurate performance measurements.
This is the drawing game that I built based on the concepts discussed in this series:
I learned about the optimizations discussed in this article from this WWDC video; I highly recommend watching it if you’re interested in the topic:
