The Trials And Tribulations of Creating An SDK — Part 3
I’ve previously told you about my motivations in creating an SDK as well as walked you through the initial structure of the SDK that I created using NodeJS to interface with the Oracle Cloud REST API. My intention for the third entry in this series is to start walking you through how I generated the code for the SDK, but, before I do that I want to address one of the foundational architectural decisions that I made when I created the SDK.
Object and pass that along, it would end leading developers to refer to the original API documentation for the possible parameters and leaving it up to them to essentially build the entire request themselves. Meaning things like typos and nesting errors are really, really easy to be introduced.
getObject on an S3 bucket:
Now imagine you’re working in your application and you’ve created an instance of the S3 client. You type
s3.getObject( and wait for your IDE to provide intellisense to help you know what parameters
getObject accepts. Only you don’t get anything helpful at all because getObject accepts an Object that requires you to know the keys that it expects. So you head off to the docs, look them up, come back and paste them into your IDE. And then you do that again later on. And again. And all of the sudden you’re wondering why you’ve even plugged an SDK into your application because you’re spending almost as much time in the docs that you would have had you chosen to just call the REST API directly.
In the SDK I created, if you wanted to call that same
getObject method, you’d create the client and you’d see something similar to this in your IDE:
Ah, OK. It wants an instance of GetObjectRequest! So you create an instance:
And thanks to the magic of modern day IDEs we can see, even in a dynamic language, the property type, whether or not it is required and a description of the property from the jsDoc comments within the SDK. No TypeScript required.
I hope that clarifies why I made the decision to design the SDK in this way. I know it may feel a bit different than how you might normally do things in Node-land, but it’ll make sense after a short while. 😏
Now that I’ve got that out of the way, let’s get into my mistakes. As I said at the end of my last post, I’d arrived at the conclusion that the best course of action for creating the SDK was to generate the methods and classes via the REST API documentation. In my excitement, I made my first mistake. It’s a mistake we’ve all made when writing code — and a mistake I should have known better than to make after writing code for 15 years: the mistake of not thinking of the future consequences of my decision. See, I figured I could quickly and easily scaffold out the code by writing a throwaway generator using jQuery to scrape the documentation pages and generate the methods and classes. Now, don’t get me wrong — it worked. After spending a few hours handling a few edge cases, I ended up with a script that I could paste into the Chrome console that would scrape the current doc and generate the code. I ran it a few times and pasted the results into the client files. It felt good. In 5 minutes I’d have an entire client generated! I got tired of clicking, so I added some code to the generator to copy the code directly into my clipboard. Awesome! The next client, slightly more complex, took another 5 minutes (even with the time saving of setting the clipboard). Hmmm…what if I put a global click handler on all of the nav items so that it generated without me having to manually running it. Cool, that saves a bit more time. Still, there are 17 clients — 700+ methods to generate!! In my excitement, I decided to keep grinding — ignoring one of the golden rules of programming: if it feels wrong, it likely is wrong.
A day later, I had finished generating the entire SDK. I tested it, and it worked.
Then I tested a few different methods.
Shit. An edge case.
I spent the next few days on and off catching edge cases, modifying the generator and re-generating some code. Nothing required a complete re-generation of the entire SDK, but it was still starting to feel unmaintainable. I hadn’t even arrived at the inevitable change or addition to the API, but I knew a better process had to be found.
And that’s where I made my next mistake.
I’ll tell you about that mistake, and how I overcame it, in the next post.