Rev Your Engines: The SDK Generation Race is About to Begin!

Published in

APIMATIC

6 min readJan 18, 2024

Hey, speed demons and code maestros! 🚀 Buckle up for an adrenaline-fueled race through the SDK generation landscape featuring the contenders: APIMatic, Speakeasy, and OpenAPI Generator. As a seasoned SDK builder in the API realm, I’m taking these powerhouses for a spin to determine which emerges as the ultimate champion.

Today’s Race Schedule

So, fellow code warriors, fasten your seatbelts and join me. Let’s determine which contender will take home the coveted checkered flag in this epic battle of code prowess! 🏁

Disclaimer:
Some modifications were made to Lob’s OpenAPI definition to prepare for our race. We removed properties that are beta and premium and modified property values that are unique to a specific account, i.e., an ID for an existing object (template, address, etc.). This is advice we would provide a company to create runnable code snippets for all users. The modified version of Lob’s OpenAPI definition we used is available here.
APIMatic
We found two idempotency keys defined with similar names, Idempotency-Key and idempdency_key, resulting in a duplicate variable. We removed one of the idempotency_key parameters.
Speakeasy
Three arrays did not have the items attribute defined, so we corrected that and removed a script tag that blocked our test.
OpenAPI Generator
The merge_variables default value broke code generation, so we corrected this in our API definition.
You can put your API definition to the test with APIMatic.
Start your free APIMatic Trial Today

Pit Stop 1: Validation and Linting

In the world of SDK racing, precision is key. Before we hit the track, we’re making a pit stop for validation and linting. Which SDK ensures a clean and error-free journey, smoothly navigating Lob’s API definition with its oneOf, allOf, and anyOf twists and turns? Let’s kick off the race by inspecting their linting and validation prowess.

OpenAPI Generator

In testing OpenAPI Generator, I found the validate command wasn’t very useful in surfacing issues. I recommend running the generate command to receive more complete linting and validation. The warnings centered on correcting reserved words, ignoring maxLength, pattern validation, and examples on the requestBody while acknowledging limitations around allOf support. Sadly, the warnings lack line information or suggestions to address these warnings.

Speakeasy

Speakeasy also provides a CLI with a validate command that works better than OpenAPI Generator. Their validation results are more readable with line information. Unfortunately, no suggestions for fixing issues are provided. I also received a warning: “X is potentially unused or has been orphaned.” My investigation found these warnings to be false positives.

APIMatic

APIMatic’s validation provides actionable feedback for your API definition to improve code quality. Each warning includes details and links to documentation about the violation with suggestions for fixing it.

Building the Car: Code Generation

Comparing SDK code generators in every language can be time-consuming. I’ve found from experience that strongly typed languages like C# and Java are suitable for uncovering issues other languages may let slip by. For this reason, I chose Java to compare the three generators.

SDK generators will let you pass if your OpenAPI definition is syntactically correct. You can generate code, but you’ll also see warnings that some aspects of your API definition could be problematic. To assist you, code generators will remove or ignore things in your API definition that may interfere with successfully generating code.

Why do SDK code generators allow you to proceed? It’s a choice that allows you to experience the tool and examine the output without requiring you to spend hours resolving warnings of varying severity.

Will you want to optimize your API definition before publishing your SDKs? Yes, and APIMatic’s VSCode extension with 1200 rules specifically around code generation is an excellent tool for the job.

For those still in the evaluation phase, don’t spend too much time optimizing your API definition, as each code generator enforces slightly different rules; you’ll go crazy trying to please them all.

Using Lob’s API definition, I successfully generated Java SDKs using each tool. 🎉

Speakeasy did have the following warnings during code generation.

The component is potentially unused or has been orphaned.
Only enum types of string or integer are supported. Enum type won’t be generated and will be treated as base type
Enum is nullable but does not contain a null value
anyOf should only contain types which are compatible with each other, use oneOf if the response can only match one type at a time
anyOf will not be supported by Speakeasy, treating as oneOf
missing schema type for X component, but found properties treating as object

Starting Line: Code Compilation

APIMatic compiled successfully without any errors or warnings. Unfortunately, both Speakeasy and OpenAPI generators generated SDKs that failed to compile with 32 and 100+ errors, respectively.

To keep the race going, I got behind Speakeasy’s car and pushed it. From my previous experiment, I found inline headers in the Petstore API broke Speakeasy, so I removed all referenced headers in Lob. To my surprise, the generated Java code with Speakeasy compiled.

Note: removing API features is not recommended, but it was necessary to continue our race.

Sadly, I couldn’t get OpenAPI generator past the failed compilation.

Test Drive: Code Evaluation

For this comparison, I created a sample application for APIMatic and Speakeasy to retrieve a list of addresses and print the address’s street, city, and zip code.

APIMatic

Speakeasy

The address list response uses a generic object to hold the data. Because the data structure is unknown, I utilized a JSON tree structure to traverse and access address details. While the resulting code works, it is a suboptimal experience for Java developers.

OpenAPI Generator failed to compile and, therefore, was not included in the test drive.

Victory Lap: Documentation

What type of documentation do I include in my comparison? Any docs generated to support the SDK code library, including a getting started guide, SDK reference, code samples, and use case guides.

OpenAPI Generator

Parameters
Return type
Authorization
HTTP request headers
HTTP response details
Complete code sample

OpenAPI Generator’s code samples look runnable, but I noticed the create methods would fail as new objects don’t set any initial values.

Speakeasy

Parameters
Response
Code Sample

The code samples included in the SDK reference are complete files with import statements. Like OpenAPI Generator, the code samples look runnable, but when testing the listAddress method, the default parameters resulted in a 422 status code.

APIMatic

In addition to a ReadMe, APIMatic provides a hosted developer portal. The portal includes API reference docs and an SDK reference for each language. It’s easy to navigate and explore the methods, models, enums, and exceptions supported by the SDK.

The developer portal includes interactive documentation. Modifying property values in the docs update code samples in real-time. You can make live API calls from the documentation, similar to a Postman collection. Lastly, code samples default to a condensed display, but through the config settings, can show a complete runnable code sample. While testing the code sample address list, it ran as expected.