LLM as Compiler
I have a strong hunch that we are fast approaching the next revolution in programming languages.
A little history
In 1972, we were suddenly able to write programs in C, a watershed moment for the software industry. In 1987, we were suddenly able to write programs in Perl, followed by Python, Ruby, JavaScript, Java, and then C#. When C was released, it was considered a “high level” programming language. But when Perl and friends were released, they were considered “high level”, and interestingly, they were all written in C. The C programming language was expressive enough to allow a generation of language designers to use it to invent even more expressive and powerful languages.
Now, in 2023, we’re seeing the emergence of a technology that could reframe “Perl and friends” in the same way that they reframed C. After paying close attention to software engineering for 23 years, along with some recent conversations, I cannot shake the feeling that a specialized Large Language Model (LLM) will power the next programming language revolution.
Incepting conversations
I have two friends who got excited about AI early while I’ve been a bit of a laggard, too busy to dive deeply into the tech or implications.
The first is Kevin Solorio, who’s been in my ear since 2020 about machine learning, followed by AI and LLMs more recently. At a dinner together back in June, Kevin mentioned that he thought something like Cucumber could be a good way to communicate to an LLM what sort of application it should then generate. My summer has been a bit of a disaster, so I immediately forgot what Kevin suggested as I had bigger fish to fry in the present than to think about the future.
The second is Ryan Prinz, a software engineer with a product management background. Ryan and I took a walk together in September when he told me about his explorations into test-driven development and LLMs. This time it started to click for me, especially as Ryan asked me about the decade that spawned everything from JUnit to Selenium to RSpec, and finally Cucumber in 2008. I wasn’t heavily involved in any of those specific projects, but I’ve spent time with all of their creators. Through Ryan’s explanations, I started to see the possibility that a specialized LLM could generate code while a product developer used a natural-language-based technology like Cucumber to specify in enough detail to help the LLM understand the desired behavior.
Ryan and I considered a few people who we should talk to next, and I suggested Ward Cunningham. Ward was kind enough to write the forward to my book, Apprenticeship Patterns, so I had his contact information. He quickly agreed to chat with us. Ward is a recently-retired software pioneer. He invented and contributed to many highly influential things over his career, including the wiki, which then, incredibly, gave rise to the emergence of Wikipedia. Ryan and I had two wide-ranging discussions with Ward over the course of a week. We wanted to run these ideas past him to see what he thought. At the end of our first discussion, after he had listened for a while, he gave us a wonderful bit of encouragement:
“Nobody asked for the wiki.”
At the beginning of our second conversation, Ward said he’d been thinking about what we were describing and thought it made sense. We learned a lot from our conversations with Ward, but our biggest takeaway was that he wasn’t telling us we were off track.
Disclaimer
What I’m imagining is purely conceptual. Kevin and Ryan have both actively worked with AI-assisted technologies as well as learned far more than I have about the tech behind them. I hope that we, with a community of others, can keep discussing this and do some more thorough experimentation.
The vision
Some programming languages are compiled (C and Java) and some are interpreted (Python and JavaScript). A compiled language generates an intermediate artifact that is then interpreted by a runtime. An interpreted language has no intermediate artifact. The interpreter directly executes the specified code.
My vision for this next revolution is that we will initially have a compiled language, though it could eventually evolve to become interpreted. The programming language will be compiled via the following steps:
- Specify the high-level behavior using something like Cucumber’s Gherkin.
- Specify in more technical detail, using programming by wishful thinking, using something like Cucumber’s Step Definitions. A GitHub-Copilot-like tool could assist in writing these step definitions in one’s preferred “Perl and friends” language.
- Feed steps 1 & 2 to the specialized LLM, which acts as a compiler and generates the resulting codebase.
- The resulting codebase can be executed by the appropriate “Perl and friends” runtime.
- Manually verify the behavior of the code. If the behavior needs to change, goto step 1.
The implications
To the extent that we achieve “LLM as Compiler”, we are once again going to have an incredible boost in developer productivity, similar to the boosts with C in 1972 and Perl in 1987. A C programmer is an order of magnitude more productive than a programmer writing in Assembly. A “Perl and friends” programmer is an order of magnitude more productive than a programmer writing in C. And soon, someone who can specify the behavior of a software system in something like Cucumber, will be an order of magnitude more productive than someone writing in “Perl and friends”.
Now, there are still specific problems that can only be solved with low-level languages, and they will always be with us. People still have to be able to work completely up and down the stack. People learn and use Assembly and C/C++ every day. “LLM as Compiler” won’t eradicate the current programming languages, but it will, once again, make it easier for people to tell computers what to do.
“LLM as Compiler” is a different way of using AI to boost programmer productivity. The typical way that people are boosting their productivity is “LLM as Teammate”, which already works via tools like GitHub Copilot and ChatGPT. The Compiler vs Teammate approaches are not mutually exclusive, and are actually quite complementary.
Today, a cross-functional product development team tends to consist of a product manager, a UX designer, a UI designer, and multiple software engineers. An “LLM as Compiler” could mean that team has a single software engineer. I don’t believe that means we’ll need 80% fewer software engineers. I believe that means we’ll have 5x as many product development teams. Why do I believe that? I’ll share more in my next post.
If you’d like to subscribe to my Software Therapy newsletter, sign up here.
