The Trials And Tribulations of Creating An SDK — Part 2

In the previous chapter I introduced you to the story of my adventures in creating an SDK. I talked at length about the definitions of APIs and SDKs and used a colorful, if not somewhat strange analogy of a retail transaction to explain the difference between the two. In this chapter I’d like to tell you some of the mistakes that I made along the way and what I learned in the process.

You might be wondering what exactly the SDK I created was for? It’s not particularly important for this story, but so that some of my examples below will make sense I will reveal that it is a Node SDK for the Oracle Cloud API. It’s not currently open source, though I do have plans to release it at some point (assuming I can navigate through my companies internal processes for doing so). Alright, let’s get into the story…

About a month ago, I was browsing through the Oracle Cloud documentation. I started working here as a Cloud Developer Evangelist about 6 months ago so I tend to spend a fair amount of time reading through docs to try and learn as much as I can about the products and services that we offer. On this particular day I noticed something a bit strange though.

Something is missing…

I couldn’t help but notice something was missing. Now, I’ll be fully transparent here — I’m mostly a Java guy. More accurately, I use Groovy almost exclusively and have written a number of applications using the Grails framework. I find Groovy to be very intuitive and easy to use and will often turn to it for everything from very simple scripting tasks to creating entire microservices with it (in conjunction with a framework such as Spark Java or Helidon). Still, I’ve been using JavaScript for much longer than I’ve been using Groovy. JS has been pretty much the only constant in my 15 years of web development, so I can appreciate and understand the growth and popularity of NodeJS. I’ve even written a few microservices using Node and ExpressJS. I wasn’t sure why we didn’t offer a Node SDK, but I figured I should create one and even if it never got released or used it would be a good coding exercise and would help me better understand our APIs.

I’ve never created an SDK before, but in theory they’re not that complex if the API is consistently organized and well documented. In this case, all calls to the API have to follow a very specific process to generate a request signature to be used as an Authorization header.

I may have added step 5. Yeah, I definitely added step 5…

There were other caveats as well: certain headers to include depending on the HTTP request method, etc. Clearly, we’re starting to see why SDKs exist — to abstract this complexity away from your applications and make APIs easy to ingest and interact with. Imagine having to consult this document every time you wanted to make a REST call to this API and trying to keep all these rules and requirements straight in your head.

So, one concern of the SDK that I would create would be to handle making each HTTP signed request in a transparent way. Luckily, the documentation had an example of signing the request via NodeJS. This would be the basis for my AbstractApiClient class that all individual clients would extend. To perform the signature, the SDK would need the user’s credentials, so the AbstractApiClient would need an AuthProvider. I settled on two variants: a SimpleAuthProvider and a ConfigFileAuthProvider:

SimpleAuthProvider and ConfigFileAuthProvider implementations.

I decided that each individual service in the API should get it’s own API client, so an example of an ObjectStorageClient might look like so:

ObjectStorageClient being created.

Each client request would need to return an object, and since JavaScript is a dynamic language I decided upon a single ApiResponse object with a few properties that would represent the potential return values from every API call.

ApiResponse object.

So, the basic “plumbing” of the SDK is in place. To summarize, so far I have created:

  • Authentication Providers
  • A base class for API clients that performs signed requests via the credential within the auth provider
  • API response object

At this point I’m feeling pretty good about where I’m at. Next, I’d create each individual service client. I’d follow a pretty standard convention for the request methods where each client method would receive at most a single parameter representing the information required to make the request — a “request” object. The API method might have certain path parameters, a few query parameters and possibly some headers that it would accept, and some of these parameters might be optional, but that shouldn’t matter to the SDK consumer. The request object should have properties for all of them and hide how the SDK uses those properties behind the scenes. For example, a ListObjects request for the Object Storage service might is defined as such:

ListObjects with path parameters.

ListObjects has some optional query string parameters:

ListObjects optional query string parameters.

And an optional header:

ListObjects optional header.

So the ObjectStorageClient.listObjects() method would end up looking like this:

And ListObjectsRequest like so:

A plain old class (can we call them POJOs in JavaScript too??) that accepts properties and hides how they’re used by the SDK. Manually creating this method was easy, and testing it out worked nicely.

I was really happy — the SDK was coming together in a nice way and it seemed like it would be a really valuable implementation of the API. So I wrapped up the for the week on a Friday afternoon and headed into the weekend on a mental high note.

Monday’s don’t really bother me. I’ve worked from home for almost 8 years which has helped me to avoid some of the typical stresses of the work week like commuting and “talking to other humans”. The following Monday was not very different from most others, except I was excited to get back to work on the SDK. After a nice cup of coffee and skimming through the inbox I cracked open IntelliJ IDEA and my favorite web browser and went back to the API docs.

Credit: Sticker Giant via Flickr

Then it hit me. The scale of this beast. Seventeen different services. Almost 700 unique API methods, each requiring a client method and at a minimum a request object (some of which required request details objects). What did I get myself into? Boilerplate or not, how could I possibly create all of the client methods and related classes to support this entire API? Maybe I could just create a single client — just the Object Storage client — that’s pretty popular. Then open source it and hope for community involvement. Nah, that ain’t gonna happen. Shit, what did I get myself into here? 😱.

And then…

Image by Free-Photos on Pixabay

I knew what had to be done. It was time to generate some code.

Now, you might be thinking this is where I wrap things up and tell you that I whipped up a quick code generator and pumped out all 17 clients, 700 request methods and related request and details classes in a matter of hours and celebrated with an ice cold beer while laughing with friends and stroking my beard. You might be thinking that, but you’d be wrong. This is where the true horror began.

You can read about that horror in the next chapter.