The World Might be Missing a Programming Language
We have all the languages we need — perfection has been achieved. Or has it?
A commonly expressed idea, since 1995 or so, is that the world does not need another programming language. We have all the languages we need — perfection has been achieved.
After working with a variety of languages in my career so far, I think there is at least one language missing. Always on a quest for that silver-bullet, I have spent some serious time over the winter holidays trying to find a new language to add to my toolbox. However, I am having trouble locating this mythical language.
I am looking for a language that I can use on servers, which I can also use to write mobile apps, desktop apps, web apps…and just maybe write a high-performance multi-player game with. It might be too much to ask, but I would rather have one language that I love to death and is supremely useful, than 3–5 languages, which I think are OK.
Some criteria for a useful language:
- It can be compiled/transpiled to run on multiple platforms, including UIs and servers. Easy transpilation to other languages is crucial to allow it to run in many places at the onset.
- It must compile to machine-code for performance. This of course does not prohibit the language from having a Virtual Machine as well, and being interpreted by that Virtual Machine instead of compiled to machine code (for the record, Java is compiled to bytecode then interpreted).
- A VM/interpreter may be essential, because it is sometimes prohibitive to compile each and every program. Instead, for scripting we use interpreted languages because we can run programs without having to re-compile them for every machine.
- It is structurally-typed*, not nominally-typed. Nominal-typing seems to offer few advantages to structural-typing, as far as I can tell. Structural-typing makes libraries less bloated and easier to write. FP languages are more often structurally-typed than OOP languages, and I think structural-typing is unequivocably better. Structural typing is especially useful for interoperability between libraries, otherwise both libraries need to import each other.
- Has inferred* typing. Types should be inferred as much as possible, chaining and FP seem to help with this, whereas pure OOP seems to mean a lot of duplicate declarations.
- It has no offside-rule — aka, no whitespace-as-syntax, aka no significant whitespace. Whitespace-as-syntax has to be one of the worst ideas in all language design, for obvious reasons. It is all about vanity and will end up killing you someday.
- It has automatic garbage collection and memory management.
- It avoids striving for hard-real time, instead going for soft-real-time. Hard-real time means pre-emptive or priority threading is used, which makes software development a lot trickier. Since 99.9% of software does not require hard-real-time features, this is something that a silver-bullet language can avoid, and therefore it can avoid pre-emption. Hard-real-time software might include control systems software, algorithmic trading software, etc.
- It has the best of both worlds when it comes to functional/object-oriented, aka, why not both?
- It has first-class functions. Template string literals. Anonymous functions, closures, etc.
- It has solid language-level support for immutability and immutable data structures etc.
- It has a good module system, packaging system, and dependency management system.
- It allows for more than 1 version of a dependency loaded at runtime. Java does not allow this which makes for dependency hell — for a given class only one instance can exist, that’s how the classloader works.
- The language itself makes writing 3rd party libraries easy.
- It has a widely available and supported interpreter, since this makes WORA easier to implement.
- It has easy deserialization/serialization. JSON is probably here to stay for awhile, but a lot of typed languages make it overly difficult to do I/O.
- It is technically superior so that any barrier to entry is completely necessary, which has the helpful effect of keeping away unskilled, inexperienced, or uninitiated programmers.
- I don’t think operator-overloading is much more than vanity (correct me if I am wrong), but I do think that method-overloading has advantages. Trying to find two or three or four different names for essentially the same routine can be annoying at best. Learning Golang has reminded me that not all languages have method-overloading — this makes me sad.
- This is by no means a complete list and I will add more later.
*Regarding thread pre-emption : Java has it, Node.js doesn’t. I am currently investigating async runtimes like Erlang to see if Erlang processes can interrupt each other and therefore arbitrarily change the order of execution of scheduled operations. If you know about this please add a comment. Garbage collection is one task in most runtimes that could be pre-emptive.
In asynchronous development, it is critical that the order of execution is consistent. In this case, we expect step 2 to always run before step 3, no matter what happens. In pre-emptive threading environments, it is sometimes a danger that step 3 can be run before step 2. Pre-emptive environments usually require the developer to use more locks to prevent race conditions etc, which can be burdensome and error-prone. In the Node.js community, inconsistent APIs that can fire callbacks synchronously and asynchronously are known as “releasing Zalgo” or “summoning Zalgo”.
Here is a simple example of pre-emption gone wrong:
By *structural typing, we mean that type-checking only looks at properties (the composition of an object) not at the names of classes/types. Structural and nominative typing don’t differ when you pass primitives as arguments. But when you pass an object to a function, it will compile in a structurally-typed language as long as it has the expected properties/methods. For example, with TypeScript:
By *inferred types, at the very least we mean if we pass a function to another function or method, that it will know exactly what the argument types are. Here Golang fails at this simple exercise:
Unlike Golang, if we have a method definition, and pass a callback function, TypeScript knows the types of arguments for that callback function, so you do not have to redeclare them.
Going back to the TypeScript example from above, we can see that TS has inferred typing:
How existing languages miss the mark
I don’t know that much about C++ because I have never used it extensively, but I have read about it countless times. To my knowledge, C++ is very versatile and powerful, but lacks auto memory management which makes it dangerous to use. You can write desktop UIs, but writing UIs for Android, iOS, Web is not that easy, as far as I know. Clearly, the reason is that people choose to support other languages instead of C++ to run on their platforms.
Golang was promising but it is lacking in a couple of key areas. It is not very supportive of functional programming and lacks generics and inheritance which are useful OOP features. Golang does score major points with regard to being structurally-typed, proving that a structural type system can compile very quickly, however it requires redundant type declarations (see above) which other structurally-typed languages avoid (TypeScript and OCaml being some examples). Golang also has a somewhat restrictive circular dependency issue where two files in different packages cannot depend on each other (although perhaps all compile-to-machine-code languages have this restriction — OCaml seems to). So, Golang is good for writing simple high-performance servers — but what else?
In my opinion, OCaml is the most promising of them all. However, the one thing it lacks is good concurrency. OCaml was created before the age of modern multi-core machines and never had a solid concurrency model, other than spawning another process. I believe it has threads but it also has a GIL. With ReasonML on the move, OCaml might be getting more mindshare. I see there are big efforts to improve OCaml concurrency including “multi-core” OCaml and binding the Libuv Event Loop engine used by Node.js to OCaml.
F# is promising. It is a high-performance clone of OCaml and because it is Microsoft, it can borrow a lot of the nice libraries from Microsoft / C#. The problem is that F# is nominatively-typed because it borrows from existing MS libraries. Damn it! Still, nice developer tooling with VSCode. Unlike OCaml, F# uses optional significant whitespace.
C# is nominatively typed, like Java and F#. It is perhaps better than Java, but still has nominative typing, making it bloated just like Java. Like F#, it has nice developer tooling though.
Along with OCaml, Rust is highly promising. One thing Rust does appear to lack is method-overloading — a feature that I think is useful.
As we can see, none of the languages I have discussed meet all the critieria. The hardest criterion to satisfy seems to be structural-typing — only Golang and OCaml seem to meet it.
The Big Threat
The obvious threat is that with more and more languages, mindshare is spread too thin. We need some genius to come in and create the language to kill all others, but any more mediocre languages are going to cause more harm than good. One of the biggest reasons why languages and libraries are good to work with is the community that can help when you hit a tough problem. When people stop using a language en masse it is effectively dead — a classic network effect problem.
Identifying the counter-intuitive / lesser-known ideals
- I take it as self-evident that structurally typed languages are superior. Golang provides evidence that they can compile quickly and TypeScript shows us how wonderful structural typing is.
- Why are there no compilers for interpreted languages and interpreters for compiled languages? It seems like this would allow for more Write Once read Anywhere (WORA). Obviously, this would take work, but also forethought.
- Incremental compilers like
tsc-watchfor TypeScript are fantastic, why don’t languages like Java have incremental compilers that behave the same way?
Room for improvement
- Process startup time. With things like Lambdas in the cloud, a process startup time for Node.js processes of 300ms or so might be prohibitive. To improve theprocess startup time of interpreted languages we could create compilers for them. Likewise, to get compiled languages to run everywhere, we could create interpreters/VMs for them.
- Using non-preemption at the language/runtime level. From what I can tell, most languages allow for pre-emption which can arbitrarily change the order of execution. The developer expects code to execute in a certain order but when doing asynchronous development pre-emption can change that. The developer can handle pre-emption manually with locking (although this can be painstaking and error-prone) but writing libraries for general use in multiple environments can be a lot harder when pre-emption is something that needs to be worried about. Garbage collection in Node.js.
- Incremental compilation. With Java for example, you have to recompile everything. JRebel will hot-load code but only after it’s been compiled, which remains your job. As mentioned above, I would like to see more incremental compilation tools for compiled languages. The only incremental compiler I have used is for TypeScript. That means that you have some dev server, which holds compiled code in memory and listens for changes to files.
Perhaps quantum computing will usher in this change. I would be willing to bet that only a big company will be able to get the adoption and the talent together to create a super-language.
Quantum computing aside, my untested theory is that OCaml is approaching the ideal. I have not used OCaml much yet, but it’s the next foray for me. If Java were a structurally-typed language, I think this conversation would be over. Alas, Java has a verbose type system with bloated libraries/ecosystem because the type system is nominal.
Scala is probably OK, but a bit slow to compile and not that useful outside of servers. Rust does not have automatic memory management and I believe it also has nominative typing. Lisp is untyped. Haskell too impractical. F# and C# are Microsoft and I tend to avoid MicroSoft languages because I don’t work on Windows — although it does appear that F# is a Microsoft clone of OCaml in the same way that C# was an MS clone of Java.