Performance Pitfalls in C++ — Why plenty of copies!!!

Courtesy: https://podtail.com/podcast/programming-and-performance-with-cliff-click/fast-bytecodes-for-funny-languages/

Regardless of how experienced C++ programmer you are, due to the complexity of this programming language, you may unknowingly commit some silly mistakes. The code may work perfectly fine, BUT when it comes to the performance, you will realize how BIG these silly mistakes can cost you especially if you are dealing with something that deserves ultra low latency level performance.

Let me introduce some of the performance specific pitfalls I came across as a programmer, mentor and a code reviewer. I really hope this series will be useful for you guys & girls.

Item 1: Exponential growth of copying in std::vector

Let me implement a simple Triangle class which has a constructor and a copy constructor.

I have simply inserted 3 instances of Triangle into it. Everything looks so clean and innocent. If you are ignorant on how vector::push_back really works, you may not expect that copy constructor to be ever executed. Anyways, let’s see what do we get the result when you execute this piece of code. Here we go!

For inserting 3 Triangle elements into this vector, constructor has executed 3 times(as expected) and the copy constructor has executed…how many times? 6 f** times!! Imagine a situation where you have to store thousands of such data points into a container. How many number of copies your looking so innocent code will generate? The implication : terrible performance.

Ok s**t happens in life. Let’s clean it up.

Optimization 1: reserve enough space in your vector

As you all know, vector maintains its elements in contiguous memory blocks like arrays. But the poor vector doesn’t know how many elements of burden it is going to carry later. It may vary from none to millions. But as a programmer, when you define a vector instance, you may know how many elements this poor vector may hold in the future. Lets help our vector friend to reserve enough space in the very beginning.

And here is the result when I execute this code:

Magic happens! Copying is reduced by 50%. Wow, I am too smart! Just by reserving enough space for our poor vector, it has done a great performance improvement of 50%. Isn’t that amazing? (If you would like to know more about vector::reserve, here are the details)

But still, I am not fully convinced. Why the heck there are still 3 copies? That’s insane, stop creating copies Mr Vector (or, Mr C++). Don’t worry, C++ is too smart, it offers a great solution here. Let’s try it out.

Optimization 2: move constructor, replace push with emplace

As you can see, the Triangle instance I have created are not named entities. In C++, these are called rvalues. If you are creating rvalue instances of your class, add a constructor in your class which can handle these unnamed entities. These constructors are called move constructors.

The second change I want to introduce is to replace vector::push_back with vector::emplace_back. As the name says, it is not creating a copy of the element to insert at the end of container. Instead, it directly creates at the end of container. You are saving a copy operation here. Here is the latest code:

And here is the result:

No more copying!!! Now the performance gain has been increased to 100% !! Isn’t that amazing? The changes we made are trivial but the result is spectacular.

So two takeaways.

1. If you are dealing with vector, always try to reserve sufficient space before you start inserting items to it. This requires a bit of planning ahead
2. Use emplace functions over push functions if your container supports it

I really hope this simple tip helps you while writing your next piece of code. Simple changes in our habits can create great results!

Thanks for reading, cheers!