Optimization and other surprises

Several years ago I set out to create a programming language which satisfied my particular needs. It was to be “cross platform”, so I didn’t have to write five different versions of my code. It had to be “secure”, meaning that applications written in it would be hard to “hack”. I wanted it to be easy to use, so that I didn’t have to take a long time to write apps. And I needed to be easily able to use other people’s code if necessary.

That’s how the 8th programming language was born.

One of the non-goals of 8th was that it be blazingly fast. Through the years I’ve learned that usually “fast enough” is also “good enough”. Still, measuring and testing my code to make sure things only improved seemed prudent.

It’s been an ongoing challenge to resist the temptation to optimize the hell out of everything. Mostly, I’ve met that challenge.

To ensure that 8th’s performance didn’t backslide, I wrote a bunch of benchmarking code. Most of it was pretty much “toy code”, but still indicative of general performance. Then someone asked me why 8th was so slow on a particular benchmark of his…

Since the benchmark did a lot of math, I assumed that the problem lay in how 8th does math. But because I’ve been coding a long time, I took advantage of the perf tool — and discovered to my chagrin that the slowness was almost entirely due to the central data structure in 8th, the “stack”.

Like Forth and PostScript and some other languages, 8th is a “stack based language”. That means that data are passed to and fro between functions using a “stack”. As it happens, my implementation was too conservative and by making just a few tweaks, I got rid of the problematic performance that fellow told me about, and restored my reputation (in my own eyes, anyway).

First take-away: measure your code’s performance before “optimizing”!

Unlike most every other language around, 8th doesn’t optimize the code at all (except for tail-call optimization). So if the programmer writes “slow” code, it will remain slow. That’s a feature: you get what you write.

So I had always assumed that, even though 8th is generally faster than python, ruby and other languages of that sort, it would always lag behind C++ in performance. Then I made another benchmark.

The results of this latest benchmark took me completely by surprise. My 8th code was considerably faster than the unoptimized C++ code, and quite competitive even with the optimized (-O2) C++ code. This surprise teaches some important lessons.

First: real-world performance must be measured, not assumed.

Second: the underlying data structures and memory layout of your application have immense impact on the app’s performance.

Third: spending your time optimizing unnecessarily is a waste of your time and your client’s money.

Resist temptation, and remember: “premature optimization is the root of all evil”.