The Trials And Tribulations of Creating An SDK — Part 2
In the previous chapter I introduced you to the story of my adventures in creating an SDK. I talked at length about the definitions of APIs and SDKs and used a colorful, if not somewhat strange analogy of a retail transaction to explain the difference between the two. In this chapter I’d like to tell you some of the mistakes that I made along the way and what I learned in the process.
You might be wondering what exactly the SDK I created was for? It’s not particularly important for this story, but so that some of my examples below will make sense I will reveal that it is a Node SDK for the Oracle Cloud API. It’s not currently open source, though I do have plans to release it at some point (assuming I can navigate through my companies internal processes for doing so). Alright, let’s get into the story…
About a month ago, I was browsing through the Oracle Cloud documentation. I started working here as a Cloud Developer Evangelist about 6 months ago so I tend to spend a fair amount of time reading through docs to try and learn as much as I can about the products and services that we offer. On this particular day I noticed something a bit strange though.
I’ve never created an SDK before, but in theory they’re not that complex if the API is consistently organized and well documented. In this case, all calls to the API have to follow a very specific process to generate a request signature to be used as an
There were other caveats as well: certain headers to include depending on the HTTP request method, etc. Clearly, we’re starting to see why SDKs exist — to abstract this complexity away from your applications and make APIs easy to ingest and interact with. Imagine having to consult this document every time you wanted to make a REST call to this API and trying to keep all these rules and requirements straight in your head.
So, one concern of the SDK that I would create would be to handle making each HTTP signed request in a transparent way. Luckily, the documentation had an example of signing the request via NodeJS. This would be the basis for my
AbstractApiClient class that all individual clients would extend. To perform the signature, the SDK would need the user’s credentials, so the
AbstractApiClient would need an AuthProvider. I settled on two variants: a
SimpleAuthProvider and a
I decided that each individual service in the API should get it’s own API client, so an example of an
ObjectStorageClient might look like so:
ApiResponse object with a few properties that would represent the potential return values from every API call.
So, the basic “plumbing” of the SDK is in place. To summarize, so far I have created:
- Authentication Providers
- A base class for API clients that performs signed requests via the credential within the auth provider
- API response object
At this point I’m feeling pretty good about where I’m at. Next, I’d create each individual service client. I’d follow a pretty standard convention for the request methods where each client method would receive at most a single parameter representing the information required to make the request — a “request” object. The API method might have certain path parameters, a few query parameters and possibly some headers that it would accept, and some of these parameters might be optional, but that shouldn’t matter to the SDK consumer. The request object should have properties for all of them and hide how the SDK uses those properties behind the scenes. For example, a
ListObjects request for the Object Storage service might is defined as such:
ListObjects has some optional query string parameters:
And an optional header:
ObjectStorageClient.listObjects() method would end up looking like this:
ListObjectsRequest like so:
I was really happy — the SDK was coming together in a nice way and it seemed like it would be a really valuable implementation of the API. So I wrapped up the for the week on a Friday afternoon and headed into the weekend on a mental high note.
Monday’s don’t really bother me. I’ve worked from home for almost 8 years which has helped me to avoid some of the typical stresses of the work week like commuting and “talking to other humans”. The following Monday was not very different from most others, except I was excited to get back to work on the SDK. After a nice cup of coffee and skimming through the inbox I cracked open IntelliJ IDEA and my favorite web browser and went back to the API docs.
Then it hit me. The scale of this beast. Seventeen different services. Almost 700 unique API methods, each requiring a client method and at a minimum a request object (some of which required request details objects). What did I get myself into? Boilerplate or not, how could I possibly create all of the client methods and related classes to support this entire API? Maybe I could just create a single client — just the Object Storage client — that’s pretty popular. Then open source it and hope for community involvement. Nah, that ain’t gonna happen. Shit, what did I get myself into here? 😱.
I knew what had to be done. It was time to generate some code.
Now, you might be thinking this is where I wrap things up and tell you that I whipped up a quick code generator and pumped out all 17 clients, 700 request methods and related request and details classes in a matter of hours and celebrated with an ice cold beer while laughing with friends and stroking my beard. You might be thinking that, but you’d be wrong. This is where the true horror began.
You can read about that horror in the next chapter.