premature optimization is the root of all evil.
Whether this sentence was originally said by Tony Hoare or Donald Knuth, you’ve most likely heard it during your programming life.
Quite too often in my experience, I heard people use this sentence to justify that we should not think about performance at an early stage. But those people also usually forget two major points:
- This sentence was published in 1974 which is, technologically speaking, a few light years away. Optimization had a different meaning back in the days and may not always be valid nowadays, especially for such a language as Ruby.
- Part of the quote is often left off:
We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.
As he said, we should consider these 3% which my might have a huge impact on your software. That’s where it actually matter.
3%, here I come!
I was recently going through the New Relic statistics for our production services and saw that around 3% of each request were spent in time parsing methods (yes 3%, you read it right). So I started to investigate. I had recently read Ruby Performance Optimization (which I would recommend to any Rubyist) where Alexander Dymo described how slow Date.parse could be. I’ll let you guess which method we used to parse time: Time.parse. 😉
Most of our production servers run on JRuby 1.7.x (and some roll-out to JRuby 9k are on the way). So I decided to run some benchmark on these JRuby versions out of curiosity. As far as I could remember, our input date should be in ISO8601 format, therefore I used the following benchmark:
And the results were quite unexpected:
When using Ruby 2.3.0, Time.iso8601 is 1.8 times faster than Time.parse which is understandable as the latter should be able to parse (almost) any kind of time input. This was already a significant difference. But as I said, we mostly use JRuby and this is were things get really interesting.
We can notice that on JRuby, using Time.iso8601 is 14 times faster than Time.parse. 14 TIMES!!! That’s a 1300% performance gain here. I could not understand why there was such a big difference between C Ruby and JRuby so I started to dig into the source code.
I was a bit surprised at first to see that in both JRuby and C Ruby, Time.parse basically delegate to Date._parse. Another surprise was awaiting me down the road when I figured out that the JRuby implementation of Date._parse was done in Ruby while the C Ruby implementation was done, well, in C!
On the other side, the Time.iso8601 implementation is exactly the same in both platforms and was written in Ruby:
Here you go, there’s your explanation for this huge performance gap between C Ruby and JRuby.
Even though people have observed such behavior with Date.parse and DateTime.parse classes, I don’t think that the majority of JRuby users are aware of such a caveat.
Was it worth it?
Obviously improving a code performance is good if and only if the improvement is globally worth the effort. In my case, all these interesting discoveries were extremely useful for a few reasons:
- I was able to get an overall 3% speed up by changing only ONE line of code.
- I got a lot more insight on how C Ruby and JRuby handle time internally (cause who doesn’t need a bit of C and Java in their life?).
- One thing that you might have missed while reading the benchmark results is that date parsing is faster on JRuby 1.7.x than on JRuby 9k. That only confirm what we observed when we tried to migrate some services to JRuby 9k: JRuby 9k is overall slower than JRuby 1.7.x.
3% performance gain here may seem like an unnecessary optimization at first glance. Depending on the amount and duration of requests it could have a huge impact on your system though. Let’s assume for example that you have a proxy server that would need to parse a time sent by the client (for logging purpose for instance) and forward the request to a micro service. Improving this proxy performance would mean improving all the micro services performance. In our case, this ended up being a ~3ms gain per request on services that handled millions of requests per day.
One thing I came to wonder is why we even used Time.parse in the first place. Time already provides parsing methods for different time standards such as iso8601 or rfc3339. I sincerely had no answer for that and the best my colleagues could offer me was: “We probably did not even question ourselves.”. In my opinion this may come from a behavior that devs tend to have when using Rails: let the framework (or library in that case) handle things automagically. Therefore we probably thought “Yeah! Time has a ‘parse’ method. Then it should handle things for us!”. This is fine as long as you actually know what you are doing. In our case though, it could have been error prone as we expected the input to be in iso8601 format. Therefore sending a different format could have created an issue further down the request.
As a reminder, you have to keep in mind that this optimization was helpful in our case only because we had the following requirements:
- We expect our input to be in ISO8601 format.
- We use JRuby on production.
- Time parsing had a significant impact on our requests performance.
Let’s wrap it up
What I got out of this experience (and what you should get out of this article) is:
- Always benchmark when in doubt (or simply curious), you have nothing to lose. You may waste 5 minutes of your day but you might also gain hours of performance debugging.
- Always use Time.iso8601 (or rather avoid Time.parse).
- Get to know your platform (JRuby in my case) better.
If we had known beforehand that Time.parse was so bad, we would have used Time.iso8601 in the first place. It would have been an quick and easy win. And that’s what premature optimization should be in my opinion: knowing beforehand what can significantly improve your system’s performance without having too much impact on understanding or readability.
Article by Yohan Robert
