To Create An Evolvable API, Think About The Protocol
Why understanding protocol fundamentals is essential to programming
This post shows how coding against basic protocols can be a great cost-effective solution to model communication between different services. It doesn't require you to build another version of HTTP or TCP, that's absurd. However, it requires you to think differently of how most people create APIs these days.
A few months ago I read an interesting piece from Farnam Street Blog entitled Yuval Noah Harari: Why We Dominate the Earth. The post shows the reason we dominate the earth as humans is due to Evolution and our ability to communicate.
We began to cooperate flexibly, in large groups, with an extremely complex and versatile language. If there is a secret to our success — and remember, success in nature is survival — It was that our brains developed to communicate.
— Shane Parrish on Yuval Noah Harari: Why We Dominate the Earth
The book Microservice Architecture, which you can read now for free, also touches the subject of communication. It shows the benefits of understanding Conway's Law: to create a meaningful impact on a software product you need to observe and change the real-life communication structure of the people who build the software.
Put simply, communication dictates output.
— Microservice Architecture, page 55
You don’t need to go very far to learn that communication is one of the most important things that exist. Yet, we don’t give it due care.
In HTTP, you can model an evolvable API in the form of a conversation where the structure of the URL doesn't matter. In that model, the server requires more orchestration from the client to give the ability for both the server and the client to evolve independently.
The reason why two computers can evolve independently is that their programmers have written code against the format of a protocol. The server provides a response containing a format that the clients know how to interpret, not a response that can never change because it's carefully crafted to the operation the client is trying to perform.
Note: If you want to know the details of why the URL structure of an evolvable API doesn’t matter, see the post “To Create An Evolvable API, Stop Thinking About URLs.”
The reason why two computers can evolve independently is that their programmers have written code against the format of a protocol
Let's take the Internet Relay Chat protocol as an example. You send a request using a specific set of characters within a particular format, such as the
PRIVMSG command for private messages. If the server supports the IRC protocol, it can understand the operations you are trying to execute as a client; otherwise, they don't.
The specification of the protocol defines the format of what you're trying to do. It has affordances, which tells the client what operations are possible. The content of the message is specified by the user, not the programmer. As a programmer, you need to know how the protocol works and write the code in a format that complies with it; the users of the system don't need to know about that format.
The fundamentals of the IRC protocol, including the format, are the same as HTTP and HTML. In that case, HTTP is the protocol and HTML is the format. While the IRC has a message interchange format optimized for a chat, the HTML is a message exchange format optimized for building websites and web applications.
As Tim Berners-Lee and Noah Mendelsohn documented many years ago:
When designing computer systems, one is often faced with a choice between using a more or less powerful language for publishing information, for expressing constraints, or for solving some problem. This finding explores tradeoffs relating the choice of language to reusability of information. The “Rule of Least Power” suggests choosing the least powerful language suitable for a given purpose.
— The Rule of Least Power, 2001
According to The Rule of Least Power: Use the least powerful language suitable for expressing information, constraints or programs.
However, it's not enough for two systems to understand the same language, like English, Russian, Spanish, HTML, IRC, etc. for two systems to understand each other effectively, they need to understand the same domain vocabulary, like appointments, e-commerce, accounting, table orders, Web UI, chat messaging, Micro-blogging, etc.
The domain vocabulary is another protocol on top of the language.
The language becomes less powerful as the communication structure between two computers becomes more specific.
If you write code to communicate between two computers, your job as a programmer is to find the balance and the most optimal language power. That’s why you need to understand protocol fundamentals.
It's not an easy task.
The protocol is the language two computers use to talk to each other. It's the equivalent of how we, as humans, communicate in the real world. Yet, by looking at the majority of the APIs out there, nobody seems to care about this.
This concept is another essential tool for your toolbelt. It can be a smart investment with a huge payoff.
According to Yuval Noah Harari, Evolution has given us the ability to dominate the earth due to our ability to communicate. However, evolution doesn't apply only to the billions of human brains that compose the world and make it work. It also applies to all the services that comprise your project and make it happen.
If they can't work effectively, they'll get to know Natural Selection.