The Sorcerer’s Apprentice: Software Writing Software

ChatGPT from OpenAI took the world by storm at the end of 2022. An AI platform capable of generating human-like responses to text inputs, it has been busy writing emails, college essays, and executive summaries ever since.

ChatGPT is trained on “large language models” to learn general patterns and relationships between words and phrases. Its talent for language and syntax is not limited to the English language: it can do the same thing with computer code.

Science fiction? No. In March 2023, Microsoft launched Copilot, a code autocompletion tool that uses ChatGPT to suggest code snippets as a programmer writes code. The large language model used to train Copilot is none other than Github, the world’s largest hub for open-source software with more than 100 million repositories of code and 73 million registered users. Microsoft owns Github and has invested more than $10 billion into OpenAI.

A Primer on ChatGPT (and a cautionary tale for romantics)

I used ChatGPT to write a Valentine’s Day poem for my wife this year.

Me: write a valentine’s poem for wife Jill

ChatGPT: Jill, you are the sun in my sky, The beating of my heart, the sparkle in my eye. I’m grateful for your love, your wit, your grace, And for being by my side in every place.

I thought it was pretty good. (Jill was not impressed).

ChatGPT works by breaking down user queries input into smaller parts and analyzing the relationships between them. It then uses this information to generate a response that is similar to what a human would say. ChatGPT is designed to be very fast and efficient. It can generate responses in real-time, allowing for seamless interactions with users.

GPT stands for Generative Pre-trained Transformer.

“Pre-trained” refers to the fact that GPT models are typically trained on vast amounts of text data, such as books, articles, and websites.

“Generative” refers to the fact that GPT models are capable of generating new text based on the patterns they have learned from the pre-training data. This makes them useful for applications such as chatbots, text completion, or text summarization, where the model needs to be able to generate new text that is similar in style and content to the input data.

“Transformer” refers to the fact that GPT uses Transformer neural networks to analyze text in parallel, rather than sequentially, as was done by prior neural networks for language processing. Transformer networks rely on self-attention mechanisms that attend to different parts of the input at different times, rather than processing the sequence one element at a time.

An Avalanche of Innovation

ChatGPT usage is growing faster than any technology the world has ever seen. Two months after its release, ChatGPT had more than 100 million monthly active users. It took cell phones sixteen years to reach that level. The Web took seven years, Twitter five years, Instagram two and half years.

The creators of ChatGPT are making continuous enhancements, not least of which is a nextgen large language model. On March 1, OpenAI announced ChatGPT Plus. The “Plus” means that websites and mobile apps can directly query ChatGPT, and vice versa. These interfaces turned ChatGPT into a very useful apprentice overnight. It can now (in theory) see your calendar, book flights and dinners, and pay for them from your bank account with a one simple request.

Earlier this month, a new open source project called AutoGPT emerged. AutoGPT spawns “GPT agents” that speak with one another and, through iterative dialog, complete sophisticated multi-stage tasks.

Because ChatGPT can write computer code, AutoGPT can spawn agents that write code, debug it, and optimize it. In principle, this code can interact with any website or mobile app.

The sorcerer (Open AI) has created a magnificent apprentice (ChatGPT), and the apprentice has summoned into being an infinite army of its own apprentices (AutoGPT).

Software That Writes Software: What Will it Mean?

The mind races to diabolical predictions. Every bad actor and malcontent, regardless of technical ability, will soon be able to create and launch worldwide AI-perfected cyber attacks. Spam emails and phishing attempts will seem quaint by comparison. There might be large language models to train AI, but is there a “large morality model” to guide the actions of software agents?

I will leave the existential and moral questions to others. For now, a few thoughts on the more prosaic implications for startups and investors.

It may soon be possible to generate sophisticated software on-demand. The amount of capital required for an entrepreneur to create a minimum viable product will diminish to zero. What will happen to the saas business model beloved by venture capitalists? Will a customer pay a monthly recurring fee for software that can be generated on the fly?

Insofar as software can create poetry, fiction, art, music and film, all of these can also be created on demand as well. What will be their value?

In a world where software writes software, value will be found in infrastructure (chips, data centers, communication networks) and the energy that powers it.

Value will be found in data and the systems and sensors that gather data.

Value will be found in scarcity: time, attention, uniqueness. Advertising will survive because it monetizes time and attention. Blockchain technology will survive because it guarantees the uniqueness and finiteness of digital assets.

Value will be found in systems that regulate and safeguard AI (if it’s not too late).

Value will be found in land and space. In air and water. In the elements of the periodic table. In skilled labor (for a while).

Startups and investors are well advised to ensure that their ventures are anchored in elements of fundamental value.

Déjà Vu All Over Again

In 1797, Johann Goethe penned Der Zauberlehrling (The Sorcerer’s Apprentice). It tells the tale a precocious magician’s apprentice who brings a broom to life and commands it to fetch water. The broom obeys zealously, refusing to stop bringing water even as the house floods. The apprentice cannot recall the magician’s command to return the broom to lifelessness, so he chops the broom in half with an axe, only to have both halves spring to life and continue fetching water. Goethe’s original excerpted here:

Stop now, hear me!
Ample measure
Of your treasure
We have gotten!

No, no longer
Can I please him,
I will seize him!
That is spiteful!
My misgivings grow the stronger.
What a mien, his eyes how frightful!

Brood of hell, you’re not a mortal!
Shall the entire house go under?

Disney animated the tale 150 years later in Fantasia. We are animating it once again.

Credits & Inspiration

The All-In Podcast, E124: AutoGPT’s massive potential and risk

The Evolution of Technology Adoption: From the Telephone to ChatGPT by Peeyush Sharma

Edwin Zeydel, 1955 translation of The Sorcerer’s Apprentice by Johann Goethe

ChatGPT by OpenAI for writing about 25% of this post

DALL-E 2 by OpenAI for generating the cover artwork in response to my prompt: existential angst about algorithms and humanity abstract art