Difficult APIs, Tough Decisions
In the November 2018 APIOps Helsinki meetup we talked about Difficult APIs and tough decisions related to them for 2 hours including presentations by Matti Pisilä, Forenom and Tomas Heiskanen, Fourkind on Forenom Dynamic Pricing API.
The Dynamic Pricing API used for calculating prices for Forenom’s self-catering hotel apartments was deemed difficult because it used a machine learning algorithm learning on the fly with every purchase teaching it to be better. While the API itself has a very simple interface the difficulty lies under the hood. With all the AWS cloud components and backend integrations it was a formidable challenge to scale.
The API was first built and then came the bottleneck finding and scaling. There was a need for infrastructure changes, like introducing multiple instances of Redis and fiddling with gateways.
There were also totally new interface demands from a partner midway, which slowed the development and changed some plans. Caching and data lifecycle had to be considered carefully so not to slow the buying process, but also not to cause surprises with “too” dynamic price changes. Some business requirements also called for using 3rd party APIs, adding extra difficulty and performance requirements.
All of these issues are quite common when building complex APIs, but when building constantly updating machine learning APIs there is an extra layer of difficulty.
After presentation and snacks sponsored by Forenom, we went on to discuss deeper some of the challenges and solutions for them.
We discussed these issues that make the implementation difficult.
- Design
- Integrations to back-ends
- Business rules
- Performance vs. Caching strategies
- Network connections
- Security
- Office politics and culture
- Choosing the correct building blocks for the architecture
Some participants were familiar with APIOps Cycles method and we discussed how it can be used to anticipate issues faced by the Forenom team or other API designers and developers. For example by creating first an API Value proposition and business model with canvases, it is possible to get clearer idea of what is a broader view and all possible API consumers and their tasks. In addition, API Consumer interviews would have helped identify the unique requirements from a partner in early stages of design. Other things like planning the capacity, networking, information architecture and doing a quick risk analysis from business point of view are also useful when tackling a bit more difficult APIs. Or even to identify if an API is going to be “difficult”.
We also discussed shortly difficulties in consuming APIs, mostly documentation or lack of it and using uncommon authentication strategies. We looked at some example APIs and discussed how to solve specific problems like handling transactions with a stateless REST API. GraphQL was also mentioned a little, but since we had a meetup on that recently we mostly ended up discussing if Open API, at least generated from code was any good for documentation and if OpenID Connect was too complex to use or should be used for server-to-server flows.
In case you are interested to know more on those subjects, here is a link list:
See a video on APIOps Cycles from NordicAPIs Platform Summit 2018: https://www.osaango.com/blog/apiops-cycles-method-in-nordicapis-platform-summit-2018
Many of the design issues and how to avoid them or solve them are presented with research and case examples in the API Economy 101 book. It has been very popular in Finland and requested a lot to be also in English. Help us to get it translated by participating with 9 € or more in our crowdfunding campaign, deadline Dec 18th, 2018. In return you’ll get the book and a lot of students, teachers and professionals will get to use it.
If you’d like to host a meetup, speak in one or sponsor contact me in Meetup or LinkedIn. The meetups are free for participants.
