359⁰ of API Design
Undoubtedly, communication is one of the most vital aspects of our life. Communication allows us, humans, to work together to achieve greater goals far beyond individual goals. Without good communication, we won’t be able to achieve what we’ve achieved so far. Nowadays, we could make a reservation, order foods, or book a flight with just a few taps. All of those things are possible because of API, the hero under the hood. In a way, API is a kind of a descendant of language which serves as a medium to allow good communication to happen.
Since API is the hero under the hood, so I guess it’s pretty much worth it to talk about, isn’t it? The topic of this article is about what are the things that needs to be considered for crafting API specification. But, before we go to that part, let’s start with the “why” question.
Why do we need to design the API with proper consideration?
What may happens if we don’t?
In my experience, there will be two major reasons why we need to design the API properly, here we go:
The first one is cost. I believe at least one of you ever seen this chart somewhere along the way. Basically, this chart is telling us that finding a bug (or defect) needs to be done as early as possible. If the defect detected at the design phase, the cost will only be 1x (we could assume that it’s only using the architect’s time). But, when the defect detected at the production phase, then the cost will be far higher since it’ll take architect’s, developer’s, QA’s, PM’s, and the user’s time.
Second, is the most important one, which is the business goal itself. If the API was not properly designed then there will be a higher chance that something won’t work as expected which leads to a higher chance of not achieving the goal.
The impact is quite big problems, isn’t it? But the good news is, we can do something to mitigate those problems. Let’s move to the next question.
How we can measure the quality of API?
The chart above is made based on my personal experience and a bit of Googling. There are at least five metrics we need to consider to measure how good an API specification is:
- # of API calls for each business scenario.
The more # of API calls for one business scenario will make the interaction between systems become more complicated and hard to implement. - Compatibility and extensibility.
A good API should be fully backward compatible when there’s an older version of API and it should be made as easy as possible to be extended in the future, in case there are business requirement changes. - Security
Of course, a good API will need to have good security to make the interaction as secure as possible. - User friendly
A good API should be easy to understand and easy to implement by any developer that already have the common knowledge. - Technical performance
How good is the performance of the API, how many TPS it could take, is there any monitoring dashboard available to monitor the healthiness of the API, is there any stop bleeding mechanism such as limiter or switch.
Now, we can differentiate between good and bad API specification. Hence, we’re ready to go to the last question.
What are the things we need to consider?
Let’s take a look a little deeper at each of the items.
Business requirement
As the one who designs the API specification, we have to really understand what are the businesses that will be supported by the API. For example, when it comes to the restaurant ordering app, we may have food rating from the reviewer capability. We need to understand how is the flow from the very beginning when the user starting to order, pay the bill, and then write the review inside the app.
However, the understanding of business flow alone is not enough, we also have to make sure all of the things that we want to build are compliant with all of the business rules. Such as, we cannot just share or store any user sensitive data without any consent from the user, etc.
Idempotent & concurrency control
Idempotent control is a mechanism to make sure that one unique transactional message can only be processed once even if it’s received more than once. While concurrency control is a mechanism to make sure that the system can keep its data consistency and system behaviour as expected even when there are multiple invocations of API calls simultaneously for one unique or different transactional message that could happen due to network or system issue.
In the real world, there is no such guarantee that all the systems and networks will be running well 100% all the time. At some point, there must be some unexpected errors here and there. So, idempotent and concurrency control are extremely important, especially in payment ecosystem where one mistake could lead to huge fund losses. Also, this kind of mechanism will provide self-recovery capability in case any issue occurred, since the system could retry the process with ease.
Backward compatibility & extensibility
In my opinion, these qualities are really dependent on the circumstances that you’re facing when designing the API. There’s no silver bullet solution or advice that one can give to another unless the circumstances are exactly the same. But one thing that I would like to mention is, life will never be perfect. Under certain circumstances, there will be a time when we cannot achieve 100% backward compatibility or extensibility, then the last option we have is API migration.
For sure, we also need to design the migration with care and provide bulletproof migration steps to prevent any incident(s) happens during the migration. The migration usually will become a lot more complex when there’s zero downtime requirement. Even so, the key is to have some kind of monitoring, grayscale release, and rollback mechanism to mitigate the risk and impact.
Security best practices
As I’m not an expert in security then I’ll just mention a bit about this part. Nowadays, there are a lot of best practices that we can follow to make our API specification secure enough. For example, asymmetric digital signature to guarantee the authenticity and non-repudiation of the message, there is OAuth 2.0 to support three party data sharing with user consent, etc…
Standards and documentation
It’s very important to have certain standards for the API specification to make it more user friendly when someone wants to use it. The convention such as naming, path, HTTP method, header & body format, etc. are the keys. What’s more, all of that information needs to be documented properly so the person that wants to use the API can understand it clearly without much guidance from the API owner. This will be very helpful for the internal team as well. At the end of the day, it will reduce our wasted time to explain the API again and again.
Testing
When designing the API, we also need to consider the testing part as well (although this part could be done a bit later). We need to cover all of the possible test scenarios to ensure our API is 100% working and bulletproof from any kind of scenario, let it be positive, negative, idempotent, concurrency cases, etc. Performance testing such as load test and A/B test are also useful to track the performance of the API.
Monitoring & tuning
Monitoring dashboard and alarm are mandatory things to monitor the performance of the API regularly and it’s always better if we can have limiter or switch capability as a mitigation plan to limit or disable the API when there’s over capacity or critical issue occurred.
Last but not least, we need to find out about the number of TPS target. This magic number can help us to decide what kind of technology or technical improvement is needed to make the API performance matches with the target.
Those are the things that work for me and I believe there are many other things to improve the quality of the API. Feedback is welcome. Furthermore, one thing I would like to mark to end this article is, it’s quite a lot of things to consider, however…
“Big things always have small beginnings.”~Prometheus
So, in most of the case, what we need is only to create an MVP version of the API itself and we don’t need to wait until all the things have been perfectly set in place, although it is great if we can have that as early as possible.
