Optimising Image Processing
Reducing memory consumption and time taken by 80%
- Investigating old code
- Streaming is winning
- Pool your streams
- Goodbye System.Drawing, you will not be missed
- In the interest of transparency
- TLDR — tell me what to do
Investigating old code
In my previous article, we covered how we found and solved a fixed cost of using
PutObject on the AWS S3 .NET client; it was also mentioned that we transform the image before it is uploaded to AWS S3 - this is a process that is executed at least 500,000 times a day. With that in mind I decided it was worth exploring the entire code path. As always you can find all the code talked about here, and we will be using this image in our testing. Let us take a look at the existing implementation. Before we can transform an image we need to get a hold of it:-
Do you see that
.ToArray()? It is evil. I do not have the statistics to hand but I know for a fact I have seen images larger than four megabytes come through this code path. That does not bode well - a quick peak at the implementation shows us why:-
The LOH threshold is
85,000 bytes this would mean any images larger than the threshold would go straight onto the LOH - this is a problem, I covered why it is a problem in my previous article. At this point curiosity got the better of me — I really wanted to know the size of images coming into this service.
This service processes hundreds of thousands images per day, all I need is a quick sample; an idea of the size and shape of incoming data. We have a job that runs at 14:00 every day to pull the latest images from one of our customers.
Hitting their service manually returns
14,630 images to be processed. Spinning up a quick console app to perform a
HEADrequest and get the
Total Images: 14,630
Images Above LOH Threshold: 14,276
Average Image Size: 253,818 bytes
Largest Image Size: 693,842 bytes
Smallest Image Size: 10,370 bytes
Standard Deviation of Sizes: 101,184
Yikes, that is a huge standard deviation and 97.58% of the images are above the LOH threshold (
85,000 bytes). Now that we know the spread of image sizes, we can resume looking at the rest of the existing implementation:-
I have suspicions about the two
MemoryStream's knocking about but dotTrace can confirm or deny my suspicions. Running
V1 with the sample image one hundred times:-
Using dotTrace we can see where the biggest costs are:-
Now that we have established a baseline — lets get cracking!
Streaming is winning
If we inline the
GetImageFromUrl and remove the
CanCreateImageFrom - which is not needed because we check the
Content-Type earlier in the code path (not shown), we can directly operate on the incoming stream.
Awesome, a few minor tweaks and all the metrics have dropped across the board.
Pool your streams
Using dotTrace again, we can see the next biggest cost:-
Something is happening inside of the constructor of
GPStream that is costing us dearly. Luckily dotTrace can show us the decompiled source which saves us a trip to Reference Sources:-
You know when you ask yourself a question out loud and you can answer it straight away?
What?! I can’t seek on the incoming HTTP stream?
No, you can’t. Makes total sense when you think about it and because the incoming
Stream is not seekable
GPStream has to take a copy of it.
Okay, first thing — can we move the cost from framework code to our code? It is not pretty but something like this works:-
dotTrace shows us that we have indeed moved the cost from framework code into our code:-
Now we could use our old friend
System.Buffers and just rent a
byte array then pass that to the constructor of the
MemoryStream. That would work, except two things. Firstly,
Content-Length is not guaranteed to be set. Secondly, a few weeks earlier I was poking around the .NET driver for Cassandra when I came across
Microsoft.IO.RecyclableMemoryStreamand it felt like it was exactly what I needed here. If you want to learn more about
RecyclableMemoryStream Ben Watson has a great post on what it is and its various use cases. Jumping straight into
One of my favourite things about
RecyclableMemoryStream is that it is a drop-in replacement for
System.Buffers requires you to
Rent and then
RecyclableMemoryStream handles everything for you as it implements
IDisposable. Anyway, enough of my admiration for
RecyclableMemoryStream; stats for
That is an incredible improvement from
V1. This is what
V3 looks like in dotTrace:-
Goodbye System.Drawing, you will not be missed
My knowledge on
System.Drawing is pretty sketchy but an afternoon reading about it leads me to the conclusion that if you can avoid using
System.Drawing then you are better off. Whilst searching for an alternative to
System.Drawing, I came across this article written by Omar Shahine. Huh, I never considered the overloads. Takes two seconds to try this so we might as well; v3 with
useEmbeddedColorManagement disabled and
validateImageData turned off these are the stats:-
A minor increase in some areas but a noticeable drop in
Peak Working Set - great! A slightly more recent article by Bertrand Le Roy goes through the various alternatives to
System.Drawing. Thankfully, there is a nice little chart at the bottom that shows performance in the context of time taken.
According to that article PhotoSauce.MagicScaler is pretty magic — word of warning this library is Windows only. I wonder how magic it really is? Spinning
And the stats:-
An insane drop in time taken and a healthy drop in
Peak Working Set!
That is magic.
In the interest of transparency
Pun not intended — just before this article was about to be published I saw this interesting tweet:-
Naturally I was interested, spinning up
V5 gave me similar results to
V4 except a noticeable increase in time taken, peak working set, and LOH allocations. After a conversation on their Gitter I learnt that they are:-
- Still working on improving their JPEG decoder — this is relevant because we process other image formats too despite only focusing on a JPEG image this article
- Aware that their resizing algorithm is not as optimal as it could be — this is relevant because the main part of the transformation we do is resizing
TLDR — tell me what to do
If your image transformation process is mostly resizing and you are hosted on Windows then look into using
Microsoft.IO.RecyclableMemoryStream you will see a significant reduction in various performance related metrics. In our case sixty to ninety percentage reductions.