The Programming Productivity Problem
Writing software systems is hard. What makes it even harder is the constant mismatch between developers and business people. Developers want one thing and business people another. At best, the debate going on between these two parties is what creates systems that are technically robust and solve actual business problems. On the other hand when either party gets dominant then the end result is inevitably going to be lacking.
Maybe the number one cause of conflict between developers and business people is productivity. Business people expect to get features which add to the value of the product quickly and with high quality. Developers are struggling to meet this demand since they have to do not just the feature but also everything under the hood that allows that feature work in harmony with the rest of the system. Without having a deep knowledge of the system at hand it is easy to assume things are easier to do than they actually are.
Maybe the number one cause of conflict between developers and business people is productivity.
One can compare this to building a house of cards. In the beginning it is easy and fast to put cards on top of each other. You get a small and basic house of cards built quickly. But as you proceed you need to be more careful with the placement of new cards so that you avoid the whole house from collapsing. If however in the beginning you spend more time thinking forward and making the foundation more robust you’ll have easier time later on. And sometimes you need to go back and change the foundations since you learned something new.
Most of the software language and library innovation is focusing solely on improving developer productivity. Languages are providing features so that developers could write less code for same functionality. Libraries and frameworks exist solely for the reason that one would not have to reinvent the wheel all the time, which is interesting since it improves productivity.
Like with all innovation you can go forward only for so long and improve your productivity only so much without making radical changes to your basic view on how you do things. You can crossbreed different horses to finally get that ultimate horse but it won’t still go any faster than what horses are capable of. If you want to go faster you need to invent a car. You need a fundamentally different approach to reach the next level.
Recently I was working with a customer who did not expect too much of us regarding productivity. When we outlined our expected deliverables they seemed incredulous. However, one month later that same person was calling me a “super-coder” since not only had we delivered but with better quality than they had expected. Our productivity was much higher than what they had been used to. But am I truly this “super-coder”?
The reason for our better productivity was focused on using technologies that are not mainstream but provide much better productivity in many scenarios.
The reason for our better productivity was the focus on using technologies that are not mainstream but provide much better productivity in many scenarios. We used Clojure to write the functionality. It is very different from mainstream languages such as Java, Python, JavaScript, C or PHP. It takes a completely different view on how to handle and manipulate data. Clojure puts data to the forefront instead of computing instructions. It is not yet another horse.
Using non-mainstream programming languages or technologies do have downsides however. There’s bound to be less libraries available due to the difference in sheer volume of developers creating those. You end up writing logic which you might avoid writing in a more mainstream language. This might be mitigated by the language by being more powerful: many things that would take maybe even hundreds of lines of code in Java can be done with just few lines of code with Clojure. It doesn’t matter if you have to write the functionality yourself at that point, it’s faster to write it yourself than to learn yet another library.
It doesn’t matter if you have to write the functionality yourself at that point, it’s faster to write it yourself than to learn yet another library.
The biggest issue with non-mainstream languages is however the availability of competent developers who know it. Hiring competent developers is hard enough as it is but hiring a competent developer for a language that half of the people haven’t even heard of is even harder. This causes a maintainability and thus a business problem. One can hire people who do not know the language and then teach them but that affects productivity negatively in short term and adds uncertainty to recruitment process which is risky business all by itself. This sometimes might cause too much risk and in such a scenario it is better to stay with the “old and tried” technologies.
However, the only way to get to the next step is to take the leap of faith and embrace the risks. Like Morpheus said in Matrix: “You take the blue pill — the story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill — you stay in Wonderland, and I show you how deep the rabbit hole goes.” Challenge the status quo, embrace the risks in a safe way and try new things.
As a final note I would like to emphasize that Clojure is not the only non-mainstream language available that provides significant productivity improvements in many scenarios. One could mention languages such as Haskell or Elixir as examples of languages which do things differently. Each language is a tool and different tools fit into different situations. Just remember that you can improve your existing process only so far: at some point you need to change it to gain competitive edge.
