To Create An Evolvable API, Think About The Protocol

Why understanding protocol fundamentals is essential to programming

Fagner Brack
Apr 18 · 5 min read
The picture of a woman that is checking the bill for a busy restaurant. For a restaurant to make money, the reception, kitchen, and waitress need to agree on a protocol. If they speak the same language and domain vocabulary, they can work independently and communicate with each other only when necessary.

This post shows how coding against basic protocols can be a great cost-effective solution to model communication between different services. It doesn't require you to build another version of HTTP or TCP, that's absurd. However, it requires you to think differently of how most people create APIs these days.

A few months ago I read an interesting piece from Farnam Street Blog entitled Yuval Noah Harari: Why We Dominate the Earth. The post shows the reason we dominate the earth as humans is due to Evolution and our ability to communicate.

We began to cooperate flexibly, in large groups, with an extremely complex and versatile language. If there is a secret to our success — and remember, success in nature is survival — It was that our brains developed to communicate.

— Shane Parrish on Yuval Noah Harari: Why We Dominate the Earth

The book Microservice Architecture, which you can read now for free, also touches the subject of communication. It shows the benefits of understanding Conway's Law: to create a meaningful impact on a software product you need to observe and change the real-life communication structure of the people who build the software.

Put simply, communication dictates output.

Microservice Architecture, page 55

You don’t need to go very far to learn that communication is one of the most important things that exist. Yet, we don’t give it due care.

In HTTP, you can model an evolvable API in the form of a conversation where the structure of the URL doesn't matter. In that model, the server requires more orchestration from the client to give the ability for both the server and the client to evolve independently.

The reason why two computers can evolve independently is that their programmers have written code against the format of a protocol. The server provides a response containing a format that the clients know how to interpret, not a response that can never change because it's carefully crafted to the operation the client is trying to perform.

Note: If you want to know the details of why the URL structure of an evolvable API doesn’t matter, see the post “To Create An Evolvable API, Stop Thinking About URLs.”

The reason why two computers can evolve independently is that their programmers have written code against the format of a protocol

Let's take the Internet Relay Chat protocol as an example. You send a request using a specific set of characters within a particular format, such as the PRIVMSG command for private messages. If the server supports the IRC protocol, it can understand the operations you are trying to execute as a client; otherwise, they don't.

That's it.

The specification of the protocol defines the format of what you're trying to do. It has affordances, which tells the client what operations are possible. The content of the message is specified by the user, not the programmer. As a programmer, you need to know how the protocol works and write the code in a format that complies with it; the users of the system don't need to know about that format.

Note: If you want to know more about the IRC protocol, take a look in this post from Rahat Ahmed. To dig deeper, look at the RFC 1459.

The fundamentals of the IRC protocol, including the format, are the same as HTTP and HTML. In that case, HTTP is the protocol and HTML is the format. While the IRC has a message interchange format optimized for a chat, the HTML is a message exchange format optimized for building websites and web applications.

A message interchange format is also a language. However, it’s not a general purpose programming language like PHP, Java, or JavaScript. It's a language of least power.

As Tim Berners-Lee and Noah Mendelsohn documented many years ago:

When designing computer systems, one is often faced with a choice between using a more or less powerful language for publishing information, for expressing constraints, or for solving some problem. This finding explores tradeoffs relating the choice of language to reusability of information. The “Rule of Least Power” suggests choosing the least powerful language suitable for a given purpose.

The Rule of Least Power, 2001

General-purpose languages like PHP, JavaScript, C++, or Ruby are hard to analyze. For example, it's impossible to know if a program will eventually finish running or continue to run forever; that's the Halting Problem. However, languages of least power like Jasonette, HTML, IRC, or Atom are easier to analyze because they are not complex programming languages. For that reason, it makes sense to use them to transfer information between computers.

According to The Rule of Least Power: Use the least powerful language suitable for expressing information, constraints or programs.

However, it's not enough for two systems to understand the same language, like English, Russian, Spanish, HTML, IRC, etc. for two systems to understand each other effectively, they need to understand the same domain vocabulary, like appointments, e-commerce, accounting, table orders, Web UI, chat messaging, Micro-blogging, etc.

The domain vocabulary is another protocol on top of the language.

The language becomes less powerful as the communication structure between two computers becomes more specific.

If you write code to communicate between two computers, your job as a programmer is to find the balance and the most optimal language power. That’s why you need to understand protocol fundamentals.

It's not an easy task.

The protocol is the language two computers use to talk to each other. It's the equivalent of how we, as humans, communicate in the real world. Yet, by looking at the majority of the APIs out there, nobody seems to care about this.

When developing systems using JSON, JavaScript, React, or HTML, be aware of the protocol and the underlying communication format. Instead of documenting end-points and HTTP responses, document the specification of the format for the language both systems use to communicate. That's how you decouple and allow two systems to evolve independently.

This concept is another essential tool for your toolbelt. It can be a smart investment with a huge payoff.

According to Yuval Noah Harari, Evolution has given us the ability to dominate the earth due to our ability to communicate. However, evolution doesn't apply only to the billions of human brains that compose the world and make it work. It also applies to all the services that comprise your project and make it happen.

If they can't work effectively, they'll get to know Natural Selection.

Thanks for reading. If you have some feedback, reach out to me on Twitter, Facebook or Github.

Thanks to Mike Amundsen, Roger Oliveira and Cassio Lemos for their insightful inputs to this post.

Fagner Brack

Written by

I believe ideas should be open and free. This is a non-profit initiative to write about fundamentals you won’t find anywhere else. ~7 min post every few weeks.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade