Optimising Image Processing

6 min readJul 2, 2018

Reducing memory consumption and time taken by 80%

Investigating old code

In my previous article, we covered how we found and solved a fixed cost of using PutObject on the AWS S3 .NET client; it was also mentioned that we transform the image before it is uploaded to AWS S3 - this is a process that is executed at least 500,000 times a day. With that in mind I decided it was worth exploring the entire code path. As always you can find all the code talked about here, and we will be using this image in our testing. Let us take a look at the existing implementation. Before we can transform an image we need to get a hold of it:-

Do you see that .ToArray()? It is evil. I do not have the statistics to hand but I know for a fact I have seen images larger than four megabytes come through this code path. That does not bode well - a quick peak at the implementation shows us why:-

The LOH threshold is 85,000 bytes this would mean any images larger than the threshold would go straight onto the LOH - this is a problem, I covered why it is a problem in my previous article. At this point curiosity got the better of me — I really wanted to know the size of images coming into this service.

This service processes hundreds of thousands images per day, all I need is a quick sample; an idea of the size and shape of incoming data. We have a job that runs at 14:00 every day to pull the latest images from one of our customers.

Hitting their service manually returns 14,630 images to be processed. Spinning up a quick console app to perform a HEADrequest and get the Content-Length header:-

Total Images: 14,630
Images Above LOH Threshold: 14,276
Average Image Size: 253,818 bytes
Largest Image Size: 693,842 bytes
Smallest Image Size: 10,370 bytes
Standard Deviation of Sizes: 101,184

Yikes, that is a huge standard deviation and 97.58% of the images are above the LOH threshold (85,000 bytes). Now that we know the spread of image sizes, we can resume looking at the rest of the existing implementation:-

I have suspicions about the two MemoryStream's knocking about but dotTrace can confirm or deny my suspicions. Running V1 with the sample image one hundred times:-

Using dotTrace we can see where the biggest costs are:-

Now that we have established a baseline — lets get cracking!

Streaming is winning

If we inline the GetImageFromUrl and remove the CanCreateImageFrom - which is not needed because we check the Content-Type earlier in the code path (not shown), we can directly operate on the incoming stream.

Stats for V2:-

Awesome, a few minor tweaks and all the metrics have dropped across the board.

Pool your streams

Using dotTrace again, we can see the next biggest cost:-

Something is happening inside of the constructor of GPStream that is costing us dearly. Luckily dotTrace can show us the decompiled source which saves us a trip to Reference Sources:-

You know when you ask yourself a question out loud and you can answer it straight away?

What?! I can’t seek on the incoming HTTP stream?

No, you can’t. Makes total sense when you think about it and because the incoming Stream is not seekable GPStream has to take a copy of it.

Okay, first thing — can we move the cost from framework code to our code? It is not pretty but something like this works:-

dotTrace shows us that we have indeed moved the cost from framework code into our code:-

Now we could use our old friend System.Buffers and just rent a byte[] array then pass that to the constructor of the MemoryStream. That would work, except two things. Firstly, Content-Length is not guaranteed to be set. Secondly, a few weeks earlier I was poking around the .NET driver for Cassandra when I came across Microsoft.IO.RecyclableMemoryStreamand it felt like it was exactly what I needed here. If you want to learn more about RecyclableMemoryStream Ben Watson has a great post on what it is and its various use cases. Jumping straight into V3:-

One of my favourite things about RecyclableMemoryStream is that it is a drop-in replacement for MemoryStream. The ArrayPool from System.Buffers requires you to Rent and then Return. Whereas RecyclableMemoryStream handles everything for you as it implements IDisposable. Anyway, enough of my admiration for RecyclableMemoryStream; stats for V3:-

That is an incredible improvement from V1. This is what V3 looks like in dotTrace:-

Goodbye System.Drawing, you will not be missed

My knowledge on System.Drawing is pretty sketchy but an afternoon reading about it leads me to the conclusion that if you can avoid using System.Drawing then you are better off. Whilst searching for an alternative to System.Drawing, I came across this article written by Omar Shahine. Huh, I never considered the overloads. Takes two seconds to try this so we might as well; v3 with useEmbeddedColorManagement disabled and validateImageData turned off these are the stats:-

A minor increase in some areas but a noticeable drop in Peak Working Set - great! A slightly more recent article by Bertrand Le Roy goes through the various alternatives to System.Drawing. Thankfully, there is a nice little chart at the bottom that shows performance in the context of time taken.

According to that article PhotoSauce.MagicScaler is pretty magic — word of warning this library is Windows only. I wonder how magic it really is? Spinning V4 up:-

And the stats:-

An insane drop in time taken and a healthy drop in Peak Working Set!

That is magic.

In the interest of transparency

Pun not intended — just before this article was about to be published I saw this interesting tweet:-

Naturally I was interested, spinning up V5 gave me similar results to V4 except a noticeable increase in time taken, peak working set, and LOH allocations. After a conversation on their Gitter I learnt that they are:-

Still working on improving their JPEG decoder — this is relevant because we process other image formats too despite only focusing on a JPEG image this article
Aware that their resizing algorithm is not as optimal as it could be — this is relevant because the main part of the transformation we do is resizing

TLDR — tell me what to do

If your image transformation process is mostly resizing and you are hosted on Windows then look into using PhotoSauce.MagicScaler with Microsoft.IO.RecyclableMemoryStream you will see a significant reduction in various performance related metrics. In our case sixty to ninety percentage reductions.

Find me Twitter, LinkedIn, or GitHub.