Managing internal and external developer experience in platform DX development
You might have stumbled upon the famous API mandate given by Jeff Bezos. One of the items in the mandate was “All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.”(CIO.com) That is still solid advice which will help you to minimise refactoring costs and prepares all APIs for future.
If you happen to be in position responsible for API Developer eXperience, it’s an intriguing yet challenging position! You are sitting in between the internal development team and customers (developers and their managers). Having this in mind, combining internal and external developer experience is not always a walk in the park. It’s more like Ying and Yang or two sides of a coin. It might occasionally require some compromises and accepting that not everything is optimal.
In this article I describe a few situation I was faced with during the first months in managing the build of Platform of Trust DX. My responsibilities were not limited to APIs only, but included management of developer portal, API documentation and sandbox as well. But let’s discuss three situations related to managing internal and external DX.
Microservice architecture vs exposed APIs
One of the first topics to discuss with acting architect was the amount and names of the exposed APIs. I was glad that development team had selected microservice style to build the platform. But soon I discovered that the exposed APIs reflect directly the internal structure of our platform. Bringing up this observation in the company Slack resulted to days long discussion with the architect. My point was that what makes sense internally — does not necessarily make sense for the external developers.
The capabilities of the platform are developed in small independed pieces (microservices). Each of them have API or more. Just exposing some of the APIs as is to 3rd party developers results to collection of hard to adopt and understand APIs. Understanding that while exposing your APIs, you need to step into the shoes of the consuming developer (external DX) — not the API developing developer (internal DX).
Just exposing some of the APIs as is to 3rd party developers results to collection of hard to adopt and understand APIs.
The clash between me and architect was due to thee fact that in his world internal developer experience is probably the only thing that matters to get things done smoothly. For me the external developer experience was and still is the most important thing. I want smooth and easy onboarding for our APIs to maximise customer base and satisfaction.
After several day long discussions in Slack we found an understanding that we are talking about the same subject (DX), but our approaches are just different and we have to find a compromise here and there. Some of the APIs are exposed as is, but the name has been tweaked. In some cases two or more internal APIs are smashed together in exposing layer while keeping them still separate in the Dark Side. Dark Side is the name our Chief Marketing Officer gave to all backend stuff.
RAML vs OpenAPI Spec
Another debate you might end up is which API description specification to use? Although OpenAPI Spec is the default answer for REST API description for most. The advantages of OpenAPI spec are:
- A large community and support-base
- High adoption rate, meaning lots of documentation
- Strong framework support
- Has the largest language support of any opensource framework
Our developers did not choose to use that but RAML instead. Why? OpenAPI Spec doesn’t allow for code reuse, includes, or extensions. With RAML developers were able to reuse definitions and pieces of the description again and again. The developers estimated the amount of redundancy process wise would be too much with OpenAPI Spec. As you know, good developers are lazy bastards and minimize the work load all the time. Having the internal process slick and enjoyable to work with was valued more than popularity of the specification format.
The advantages of RAML weighted more for us:
- Single specification to maintain
- Strong, visual-based IDE and online tooling with collaboration focus
- Allows for design patterns
- Easy to get started
Furthermore the poorer tool support for newer versions was not a problem for us. The selected approach did cause some extra work in getting the API documentation generation process done. I had defined that we go for the “Stripe style” 3-column model in which navigation is on the left and you can jump to wanted section in the documentation from there. In the middle you have the more detailed descriptions of options and endpoints. On the right you have code examples with various programming languages for the endpoints. Result looks like the below screenshot
How did we get the above done? Since we lean towards open source and hate building everything from scratch, I started to look for available open source implementations to use. Soon I found Slate which is ruby driven implementation of the wanted API Docs. The Slate is somewhat popular since it has been forked over 16 000 times and it has nearly 27 000 stars (in Github).
Running the Slate in dynamic service contains some security issues, but that was not a problem for us, since we use the build feature in Slate to generate HTML page from the documentation and serve it with Github pages under own domain.
The Slate is somewhat popular since it has been forked over 16 000 times and it has nearly 27 000 stars (in Github).
Slate uses markdown files to describe the content and I have raw RAML formatted API descriptions to start with. There was a little gap between what I have and what I want to see (the above picture). To get content from RAML files to Slate, I had to seek out tools to do some conversion. There was no tool to convert RAML files directly to Slate formatting. I had to convert RAML files to OpenAPI Spec first and from those to Slate content files. After that I had to solve how to inject code examples in the right spots in the documentation. All this has to happen automatically of course since managing the updates manually is a pain in the ass. Furthermore that would be against the selected CI/CD process we use in everything. As a result I had to write a small python program to do all needed operations: conversions, injecting code examples from files and generating the HTML content.
My job is to find the solutions how to enable maximum positive developer experience for the API consumers without killing the internal developer experience.
All that was added to Travis and now if new pull request is merged (after my review) to master in Github, the documentation is rebuilt and published automatically (if build was successful). The lesson learned here is that sometimes you need to do things somewhat not so directly, but you can find a way to satisfy pretty much everyone. I could have been an asshole and do the don Quijote fight against the windmills and require developers to use OpenAPI Spec to make my job easier. I decided not to. I want our developers to be productive and respect their well thought choices.
My job is to find the solutions how to enable maximum positive developer experience for the API consumers without killing the internal developer experience. If it requires more efforts for me and my team, then we’ll find a way. The result is good, it did require some more problem solving and a few more lines of code, but now everyone is happy. That counts more than a few extra hours of work.
Insomnia vs Postman
Postman is one of the most popular tool to dissect RESTful APIs made by others or test ones you have made yourself. Postman and alike tools can be used to get familar with APIs without writing any code. These tools are prettier GUI alternatives for testing APIs with cURL among other things. The point is that the same tool can be utilized by internal developers (building the APIs) and 3rd party developers consuming the APIs. One familar format to enable easy setup for 3rd party developers, is to export collections for the selected tool.
This is what Postman offers — collection with environment configurations. By importing the ready made collection of API calls the 3rd party can start exploring the API with own tools and own environment (desktop). As for GUI alternative is to offer browser based API console in which 3rd party developer can make requests to the APIs. Yet Postman and alike solutions are more versatile and often easier to use.
I asked our developers about why they chose Insomnia over Postman and reason is that again Insomnia fits better in the teams workflow and is considered more agile in development. The overall image of Postman inside the team was not that good:
“They [Insomnia] tend to address the bugs and feature requests in GitHub pretty fast”.
“whereas Postman refuses to provide a good fix for a critical issue causing issues to lots of people and has been reported years ago”
“compared to my last experience with Postman it’s [Insomnia] easier for both new users and experienced users, as well as just having more useful features, and most critically — less bugs.
I initially switched to it [Insomnia] simply because the `hosts` bug made Postman unusable”
The above reasons among others has turned our developers to use Insomnia instead of market leader Postman. Given the above situation my options as Developer eXperience lead are:
- Force the team to use Postman because that is most often expected
- Create / get a tool to convert Insomnia export to Postman collections
- Use Insomnia and offer needed configuration for that in public for 3rd party developers.
Option one is out of the question in short term. I will not force developers to use other tools than what they see fit. They have selected the tools for a reason and messing with that would cause more new problems than solve any existing ones. Second option is more likely to exist in some form (did not check yet) as a tool or service, but conversions contain always a risk to break something.
Thus I did select to go against the mainstream even if I know that some of the API consumers want to see Postman collections. We will provide Insomnia settings and package to fiddle with our exposed APIs. In the above picture is a simple “collection” of Product API endpoints and methods.
Of course we provide detailed guide how to work with the Insomnia in minutes and start learning the opportunities and options of our APIs.