Why you should decide to do API Management

Photo by Tirza van Dijk on Unsplash

A lot of IT landscapes are still very scattered to that day. By that I mean, that they have grown in a place of many singular systems, more or less capable of serving and utilizing APIs.

Especially in the domain of ERP Systems, “enriched” by custom applications one of the more or less typically situations is that there are several different ways in place how those systems and applications communicate with each other and how it came to that. And this is by no means supposed to be a complete list.

1. Direct connection to persistence, e.g. database connection
2. Direct API connection, meaning a point-to-point connection
3. Hey Joe endpoints (Hey I need data XYZ for my app, can you develop that?)
4. Existing endpoints that are in place because of a vendor choice or for any other reason, the reusable kind
5. File exchange

You get the picture. As long as it works, it works — right? One of the challenges that come up with situations like that, and in my opinion it is a very important one, is the cost of change. Whenever you want to get data that is not API available, or in the wrong format, it is going to be expensive. Whenever you need to change your data communication for whatever reason, the point-to-point dependencies will hurt, and it is going to be expensive.

But the costs start very much way earlier: How do you know where to find which data set? And which API to use? How would you try something out quickly?

And if those challenges weren’t already big enough problems there is more. What about security, accessibility, discoverability, rate-limiting, performance, analytics, protocol, formats and — well: naming?

But how did we get there in the first place? Most of the time it goes like that:

1. Some project comes along
2. Somebody decides on a web application
3. The application needs to consume data
4. The application creates and consumes APIs
5. More projects, apps, systems appear and they create and consume APIs

And quickly we find ourselves in a landscape of scattered technologies, protocols, formats and data descriptions as well as implementations. Little to know consistency whatsoever.

Apart from that extensive “stitching” together for each and every project, you will also have to read documentation, figure out how which API works and keep learning and learning things that are more or less tedious but present due to inconsistency in API “management”.

There is a multitude of things to consider when building applications with APIs. Things like security, consumptions patterns (circuit breaker, rate-limiting), accounting and analysing usage are solved differently from project to project — or to the gusto of the architect present then.

So what goals are we actually trying to achieve? Some might be (and feel free to add more):

1. Freedom to innovate
2. Speed
3. Discoverability
4. Self-Servicing
5. Technical implemented SLAs (e.g. rate-limiting)
6. Analytics on usage
7. Resilience
8. Documentation
9. Monetization
10. Creation of products and services
11. Security
12. Automated governance
13. …

How hard can it be? As it turns out, not that hard on paper, very hard in practice. A lot of those things can be helped with API Management.

An API management solutions come with a couple of things in place that are particularly useful and help speed up the process to reach those goals.

Access control enables you to put authentication mechanisms in place and bridge the gap between different auth mechanisms. Imagine the system you are trying to hook up to API management works with LDAP and only allows for basic auth, but you want to put API key-based authentication in place. You can adapt to whatever authentication challenge your organization provides and transform it into the one that helps most reach your goals.

Treat protection helps you especially with internet-facing APIs but also, depends on your security strategy against attacks from the inside (if you go by assuming compromise). That can be SQL injections, cross-site scripting or other attacks.

Throttling & quotas will come in handy whenever you are thinking about controlling the performance of your APIs. When you come across huge amounts of data being requested you want to make sure that the systems behind are protected against heavy use and you do not have to solve the issue of when to throttle for whom with which attributes in which amount of time etc. all over again.

Caching & optimization can help with the experience of API performance, especially with the nature of APIs where you basically cannot guarantee what kind of technology or client is using your data.

Transformations are in place for all the inconsistencies in the APIs you have created or in place. Instead of rewriting what exists you can transform to the agreed-upon target formats and data models. That can be basically a facade to what you are trying to publish. Uniformity in developer experience can be achieved way easier.

Insights & analytics provide you with the information on who is using your APIs, when, from where with which volume. And therefore enable you for productization and monetization as well as learning how to improve and create a more resilient API experience by detecting misuse patterns.

Catalogues & organization are the stepping stone to the governance part of your API management. Naming conventions, logical grouping and also enabling publishing and discoverability to developer portals which in return enable self-servicing. Also from the other way around it allows for controlling who can work on which internal part of an API and release new versions of it.

As it all sounds good and shiny I’d also like to emphasize that there is one puzzle piece missing. To actually enable the whole self-service aspect I strongly recommend having a developer portal in place. Where developers can discover APIs, read about how to use them, find definitions, gain access and find people to talk to.

There are always two sides to APIs. The one side is the one that wants to expose an API, and the other one is the one that wants to consume an API.

Within a company, it is very likely that both sides are developers. And that is why I recommend starting there. Which APIs are provided currently by developers and which ones are consumed? Maybe also provide a tool that makes it easy to publish APIs. A CLI tool that can be integrated into build chains. Whatever you think of, just make sure it is easy and it provides an advantage over not using it.

The second group to consider are obviously users. Which products or services are enabled and where can be a possibility to gain efficiency or revenue from it? Find out which audience you want to see reached with APIs and go from there — opposed to simply smashing everything you can find into your API management solution as you go.

To know if you are successful you could also make an effort to measure and introduce some KPIs. For example

• The Time from idea to API
• The effort from idea to API

Measuring the increase in efficiency and if you decide to go for monetization revenues created by your APIs. You could also think about creating a usage advantage. Every published API via API Gateway and defined technologies and policies does not have to go through an application security audit or review anymore, since that is covered by the API Management solution.

I’d also recommend implementing a grandfathering rule for legacy systems. Meaning: Stay as you are as long as things work, but the moment you change something in terms of change requests of features, the mandatory update hits you and you need to publish your data via the API Management solution in place. With that you could tackle the existence of “Hey Joe”-APIs.

On the API consumer side, you could start measuring how long it takes from discovering an API to including it in a hello world example (TTFHW — Time To First Hello World). How many developers are using the API (the developer portal should deliver those numbers) and how many applications are tied to your endpoints?

Other KPIs you might want to consider can be things like:

• requests per minute
• average latency
• response codes
• usage growth
• users
• …

You probably also should not forget about a team that manages the API management solution itself. This is by no means an “install-and-provide” thing that does not need operations. Change is constant, also and especially for critical infrastructure like an API Gateway, where resilience and uptime are of the highest priority.

The problem I did not address yet is one of many point-to-point connections, and the question really is: Is it actually a problem? That is up to you, to be honest. Do you want to have some bus infrastructure in between, some service mesh, or something different — up to you.

But now you have it in your hand to come from a highly scattered and maximum diversified API landscape to one that is manageable.

Because after all, it is the same with systems as it is with almost everything else. The secret is in good communication. Understanding, language, format, protocol, habit, speed, maybe standardization. And having API Management in place can get you just there.

And on top of that, you might integrate the one system, nobody wanted to talk to. By adding a gateway that can facade and transforms.

Questions to be answered are certainly how _really_ to get started, how to organize, name and finally integrate. But that should get you pointed in the right direction for a start.