Leveraging OpenAPI (Part 2) — Testing and automating your API
In the previous article of this series, we shared our experience of using the OpenAPI specifications to document a REST API, and the issues we encountered (fast growing number of endpoints, and unmaintainable schema) and the solutions we found (leveraging the specifications to split our schema in small and manageable files).
In the second article of this series, I will explain what other uses of OpenAPI we investigated and implemented.
Automate, you will
Our backend engineers thought we could get more out of our OpenAPI definition. There’s always been a concern in our codebase about the complexity and number of files generated by our clean-architecture based approach. Why not use the OpenAPI schema to generate controllers on the fly, to perform request validation and build responses without adding unnecessary code files every time (especially these POPOs/DTOs) ? That’s what’s been quickly prototyped and it was a game changer.
The way this works is two-fold: we added a hook in the Symfony compiler pass, we use openapi-psr7-validator, to cache some request validators and routes. Then we created a generic OpenApiController class that allows to map all routes in the schema to a use-case class, using a specification extension property, which we called “x-usecase” and contains the FQDN of the matching PHP use-case class, that’s for each route operation definition. We still have some legacy routes that exist and are not covered by OpenAPI, so we have a bit of plumbing to detect those.
Now, not only can we document our API, but this schema can be used at runtime too, and saves us from writing boilerplate code !
Beyond documentation: API Tester
At one point, we thought we could still get more from the API documentation. Why not auto-generate some tests from the OpenAPI schema to add a massive test suite to our already existing unit and integration tests ? Once again, a core team quickly prototyped what became Api Tester.
This PHP-based tool loads up our OpenAPI schema, and generates thousands of tests automatically to hit our API with nominal cases (based on the examples in the documentation) and random tests to foolproof our API behavior and check all validators and security. A game changer once again.
Hopefully we’ll open-source API Tester in the future, if you want to know more, some of our team members presented a talk about this at the AFUP Day Lille 2023, a french PHP event, and we also participated in the Product Crew webcast for an episode dedicated to clean APIs and testing (in French).
Toolchain
Our toolchain goes like this:
- php-openapi (which we might replace with an OpenAPI 3.1 supporting fork in a near future), we use it to lint and also at runtime
- spectral, a CLI utility we use for linting with an added custom ruleset
- redocly to generate HTML documentation
- swagger-cli which we use both for a first linting pass, then to bundle our doc back to a monolith before passing it to Spectral, which does not handle references across files too good, but would like to do other things good too (bonus point if you get the ref).
- openapi-psr7-validator another great package by thephpleague
We recently found that the brilliant people at JoliCode have released a PHP library that can help leverage your OpenAPI schema to scaffold an API for you: check out Janephp.
Thoughts
Along the way, we encountered many issues, but we often found out that we were not alone. Browsing endless GitHub threads we ended up having the impression that the OpenAPI specs were great, with a cool community open to requests, but the ecosystem was lagging a bit, especially with PHP. Three years after the release of OpenAPI 3.1 in April 2021, which brings a lot of cool features, third party tools supporting this version are not the majority. You will encounter the same engineer names across many different discussions (kudos to them), and it seems the development of several widely used repositories is stalled, with no maintenance, forks ending up being created. But it’s open-source, so you can contribute. Anyway, https://openapi.tools/ is a great place to find tools and check which specs version they support.
The road ahead : API-First design
This OpenAPI schema overhaul was worth the work, we can now have validation on API endpoint creation, component sharing, easier git history on component creation, mocking, frontend uses also… Future plans include more custom linting rules, and further CI steps regarding validation and detection of breaking changes in the API. We still work on our API Tester too.
Some updates may be required to simplify the folder architecture though, as some developers think that we went a bit too far, there’s a trade-off to be made between reusability and ease of use (browsing the schema without a dedicated plugin in your IDE can be a pain, adding multiple files for an endpoint may seem overkill to some).
Managing versioning may also be on the horizon, but it’s a tricky subject.
But the major change is adopting an API-First Design approach, which we kind of partially do already, but this needs to be formalized into standardized processes; which might be the subject of a future post.
TL;DR;
The OpenAPI specification and the libraries that support it can be leveraged at runtime to generate controllers, validators and view models on the fly. We also developed a custom tool, Api-Tester, to automatically generate tests from the documentation.
