Do We Need More Programming Languages and Run-Times?

First, languages are tools, nothing more. They’re a means to communicate ideas and concepts to the computer. We have tons of languages (Wikipedia’s list is fairly long: link). Unfortunately, there really aren’t quantitative metrics to decide which one is the best. Sure, there are “popularity” rankings like the TIOBE index, as well as volume of code rankings like githut. These tell us what is being used, and its approximate popularity. There are issues with volume metrics such as population bias (young programmers looking for free repositories vs. industry which prefers closed repositories). All these languages (at least the popular ones at the top of the lists) have one slight (perhaps serious) flaw which we’ll get to in a second; first a bit about modern hardware.

Modern hardware is inherently multi-core. I’ll spare the Moore’s law (perhaps conjecture?) bit since it is overdone and skip to the point. Multi-core hardware, and massively multi-core hardware is the present, and future. It’s a trend that started some time ago:

In the early 90’s the average computer had a single core, and if you were lucky, a floating point unit as well. Today, we have hundreds and thousands of cores per device with GPUs, and large multi-core processors (e.g. Phytium’s 64-core ARM). The question is, how do we best program them?

This future also involves heterogeneous multi-core hardware. By this I mean that multi-core hardware now has different processor types, memory hierarchies, etc. that have vastly different performance characteristics. The hardware’s characteristics determine the performance of the applications that run on it. So what do languages have to do with this? The one issue with many current languages (at least the popular ones on the aforementioned lists) is that parallelism is an after thought. That, in and of itself, isn’t an Achilles heel, however it is a handicap. To add parallelism to a language, developers typically rely on libraries or pre-compiler directives to extract independent operations that can be executed concurrently. The other approach is to attempt to extract it via the compiler itself (which must be conservative in its transformations). Languages could be designed with parallelism explicitly in mind, so that the user is forced to define tasks with only well defined dependencies (many of these types of languages end up with a very data-flow/stream feel to them). With this style of language the compiler and run-time can exercise as many compute cores as there is parallelism available without additional libraries/add-ons.

So what do I have against language add ons such as those provided by pthreads, OpenMP, MPI, etc.? For experts, they’re great (well, extremely usable). For novice and intermediate users, there are some issues. First, they expect the program author to know where things like variables are stored (e.g., are they only accessible to the local thread or potentially writeable by others?). These libraries expect the authors to be able to identify where race conditions could occur, and also prevent them with the appropriate application of locking or atomic variables. Further complicating matters for the programmer is that things like the memory model implicit in the architecture itself can differ (see: link). Sound difficult yet? Sure, the average programmer can hack out a single threaded program quite easily, go to multi-thread it and the real fun begins. To be performant, safe, and portable (remember the memory model?) then an integrated approach is needed.

Getting performance from parallel code also involves difficult tasks like minimizing communication cost between threads. With most current run-times this entails knowing exactly what hardware an application is to run on and what the communications patterns will be like. This is time consuming, and not likely to be understood by even intermediate-level programmers (usually this requires partitioning algorithms, queueing networks, etc.). With the cost of memory accesses quickly outweighing the cost of computation itself, this is becoming important for things like battery life. So what is the solution for all these ailments? That’s a very good question, one I hope to answer. One possibility for a “next-generation” language is a very old one: stream processing. Stream processing and data-flow programming were once synonymous terms, but lately the differences are a bit more nuanced than they were in the 1970’s when this concept was first published. I’ll cover this in my next post.

Original version posted on my blog, jonathanbeard.io/blog.