I recently attended a workshop by Computer Associates on one of their products to consolidate multiple enterprise-wide APIs into an “API Gateway”. I was again reminded that computer science is still a language only a few understand and that it is easy to be misled by the one-eyed person in the village of blind. This article is the first in a series to try and explain certain concepts in non-technical terms, or at least I’m going to try.
The world of APIs isn’t new, It has been around since punch cards. The word API stands for Application Protocol Interface, and its just a fancy way of saying it’s a way for other applications to talk to your application in a way that your application can interpret. Think of it like a meeting at the UN between different countries. Even though everyone speaks a different language, they still need to convey their message to one another without the loss of definition and substance. This is achieved by professional translators. In computer science these translators are APIs.
Just like using your aunt that had French at school to translate a conversation between you and a French colleague, the same is true for badly written APIs. It will get the job done but the quality of the conversation and misinterpretation will make you never use her again.
For that reason, software engineers have focused on the Protocol side of Application Protocol Interface over the last two decades or so. A protocol is defined as:
The accepted or established code of procedure or behaviour in any group, organisation, or situation.
One synonym for Protocol is etiquette, which I found very appropriate:
The customary code of polite behaviour in society or among members of a particular profession or group.
Not to confuse this Protocol with the physical interfacing to the system i.e. TCP/IP, UDP, which is also referred to as protocols for good reason, but not in this context. Those protocols are the Interface of Application Protocol Interface, but I digress. To achieve a standardised way of communication between different applications some workgroups were established to find and define some rules around engagement. Some of these protocols I will highlight in this article, my own selfish preference, will be:
- The ever misunderstood REST – generic to web applications
- Old faithful SOAP – also geared for web applications
- The FIX Protocol – built for the financial market to standardise communication between buy and sell side participants.
- Bespoke APIs – this can include anything from using a middle layer like SQL, building a communication layer on top of a service bus or even raw binary over TCP.
Representational State Transfer (REST) is a software architectural style that defines a set of constraints to be used for creating web services. Web services that conform to the REST architectural style, termed RESTful web services, provide interoperability between computer systems on the Internet. RESTful web services allow the requesting systems to access and manipulate textual representations of web resources by using a uniform and predefined set of stateless operations. — Wikipedia
REST is the current flavor for web and mobile software development. It makes integration into your system seamless by using uniform HTTP verbs like GET, POST, PUT and DELETE. The developer then has a choice of how they would like to receive the requested data i.e. in a JSON (Java Object Notation) or in XML (eXtendable Markup Language). The problem with rest is the definition of the available calls and the models associated with each call. Luckily, because of the wide adoption of REST there are tools that have been developed to bridge this gap. Some of my favorite tools to debug REST are:
- Swagger — This tool assists in the full lifecycle of API development bringing together collaboration on design, build, documentation, testing, and standardisation of APIs across multiple endpoints.
- Postman — Postman like swagger also facilitates the full lifecycle of API development. They took collaboration between team members to the next level by introducing team workspaces. Now your distributed team can in real-time build and explore together.
The majority of modern development languages can either spin up a RESTful endpoint or consume data into data models quite seamlessly.
SOAP (originally Simple Object Access Protocol) is a messaging protocol specification for exchanging structured information in the implementation of web services in computer networks. Its purpose is to induce extensibility, neutrality and independence. It uses XML Information Set for its message format, and relies on application layer protocols, most often Hypertext Transfer Protocol (HTTP) or Simple Mail Transfer Protocol (SMTP), for message negotiation and transmission.
After the big hype around REST the debate around which web API is the best began. The die-hard SOAP evangelists love the structured manner in which state is transferred between systems. SOAP has out of the box definition files (WSDL Web service description language) that accompany the API. These definition files can be consumed and in a lot of tools, like Visual Studio, be imported and referenced as objects. When the API change, the definition file gets updated and the developer can easily import the latest version. Because REST can facilitate the use of JSON and XML, and SOAP can only do XML, the preference to rather use REST for web and mobile related state transfers are apparent. XML has a massive overhead, depending on the complexity of the XML structures being transferred. JSON is much more lightweight and is natively supported on all browsers, so the translation and deserialisation of these JSON objects happen much quicker on the client side.
The Financial Information eXchange (FIX) protocol is an electronic communications protocol initiated in 1992 for international real-time exchange of information related to the securities transactions and markets. With trillions of dollars traded annually on the NASDAQ alone, financial service entities are investing heavily in optimising electronic trading and employing direct market access (DMA) to increase their speed to financial markets. Managing the delivery of trading applications and keeping latency low increasingly requires an understanding of the FIX protocol.- Wikipedia
FIX is a bit more specialised, but I wanted to throw it into the mix to illustrate the vast reach and importance that APIs play in our everyday dealings. FIX is different from SOAP and REST in the sense that although there are protocols that explain and dictate the message structures, the interpretation of the message structures differs from company to company. For this reason, each company that wants to send a FIX message to another company needs to first complete a “Rules of Engagement” form to make sure that the interpretation of the different messages is the same. A lot of time and development goes into the translation of these FIX messages into the native APIs of certain exchanges, but also into the mapping of fields from one broker to another. So let’s say broker A is using FIX version 4.2 and wants to send an order to buy some Microsoft shares to broker B that uses FIX version 5 SP1. The interpretation of tag “55” is the instrument code for version 4.2, but for version 5 SP1 the same tag is used to specify the volume and tag “65” is used for the instrument code. In this case, the FIX engine needs to first translate tag 55 to 65 before broker B can understand what the instruction is. It obviously gets much more involved than this simple illustration, but what I’m trying to illustrate is it’s not a simple plug-and-play exercise.
The last aspect I would like to touch on is bespoke APIs. Sometimes you just want to facilitate communication between different micro-services in the same fabric. When communication is isolated to just your system you can mess around and design your own communication layer. Having said that, it’s never a good idea to not follow some sort of generally accepted protocols, because somewhere down the line there will be a requirement for your system to integrate with external systems, and that is where good APIs truly shine.
APIs are wonderful tools that we cannot live without, but without good governance and proper planning, they can become a massive liability to your company. On-going evaluation of your strategy is key in making sure your data is safe, your data is being accessed and interpreted in the correct manner and that you conform to the on-going changes in this beautiful landscape.