How Not to Write an API

How Amazon got it wrong

Over the last year (2014), I’ve been deeply involved with implementing some really popular APIs. These include Facebook, eBay, Twitter, Stripe, PayPal, Quickbooks and Amazon.

SDKs VS APIs

One common pattern I see is that most APIs have a REST or NVP based API at it’s core. Then they generally have some method of signing a request. Following that is more than 1 way to send different types of data, between JSON, XML or a file stream.

But one pattern always puzzles me. The documentation on these core APIs are usually lacking. Instead, they develop SDKs for specific languages. Now, on the surface, that make sense. Let’s make it easy to use our API in your language of choice. But that idea is flawed.

The way you implement the API is not how I would do it. And you don’t update it to match current trends / changes.

Here is an example of what I’m talking about. Amazon MWS is their merchant services API. This is used for listing products on Amazon, updating prices, adding images, getting seller reports etc. Amazon MWS gives you a PHP SDK implementation of their API. There are 11 different API downloads, each consisting of 124 files at about 700kb per download. You can get them here http://bit.ly/1fyXYjn

That sounds pretty daunting doesn’t it? That’s a total of 1364 files for their SDK which, again, is there to abstract their core API to make it easier to use…

Let’s contrast that with their API playground (which is a fantastic idea, and everyone should have one). A playground is just a quick way to play with everything an API can do. Their playground is here http://bit.ly/1iEJVfc. It has just two drop downs, the first with nine options and the 2nd with a dynamic number. This playground of two drop downs can do everything that those 1364 files can do.

This playground of two drop downs can do everything that those 1364 files can do.

Let’s take a quick look at how Amazon’s developers think we should use their API. Our goal — List recent orders for my account.

The example file for this API call is in the Orders download > src/MarketplaceWebServiceOrders/Samples/ListOrdersSample.php

This file is 114 lines in 4,447 characters. First you need to see they include a config file, which you have to edit first. Note: the words they use in the config file do not match what they actually are. They also don’t tell you where to get these API keys. Hint: You need to provide AWS credentials.

Next we have two more config blocks. Now we have one of my favorite lines.

$request = new MarketplaceWebServiceOrders_Model_ListOrdersRequest();

Beautiful. The longest class name I’ve ever encountered.

Then they set some more information, invoke a function on this page and finally, print out an XML file of the results. Of course the data is not useable in PHP, you still have to parse the XML (not shown in their example). Isn’t that the point of the SDK?

I’ll just run through quickly how you would then take this and get a list of orders.

1. json_decode(json_encode($xml),true); to get a PHP array of the data.
2. Figure out which key you need to loop over for orders.
3. Handle the fact that because this was XML, if there is only one order it will not be a 0 index array, but an associative array.
4. Handle that Amazon only gives you 100 orders per request, so if you want more you need to make subsequent requests using listOrdersByNextToken.
5. Handle that Amazon will throttle your request if you make too many at once, so you need to read the documentation on how many calls you can make and program those sleep times into your code as you loop.
6. If you need to make other API calls, such as listOrderItems, you need to handle their throttling as well.
7. If you haven’t given up at this point, you’re probably being paid pretty well.

The moral of that list, is that I don’t trust how Amazon expects me to use their API. I’d rather just scrape the API playground, which has just 2 drop downs to do all of this work.

Amazon isn’t the only culprit of this.

Literal SDKs

The solution turns out to be pretty simple. It’s a bit more advanced coding at first, but your life becomes a pretty happy place.

Just implement the signature yourself. That’s it. That’s the key. Forget the SDK all together, just figure out how to make direct API calls in the language of your choice.

The result:

$AmazonAPI = new Amazon($config);
$orders = $AmazonAPI->call(“Orders/ListOrders”,[]);

The above two lines, do everything those 6 steps above do. Moreover, you could change the parameter in call() to match whatever the API playground has. And the data array passed in just matches literally what the API docs say.

This one function, $Amazon->call() does everything those entire 11 downloads in 1364 lines of code can do. And how big is the Amazon class I made? One file, 235 lines of code with comments.

How to make a literal SDK

This part varies depending on company. Ebay requires you have special X- headers that you then sign using sha256. Amazon requires that you sign a URIEncoded string in sha256. Quickbooks uses a sha1 hash.

To figure this out, you first need a working open source version. I couldn’t have built these literal SDKs without that. Basically the process is as follows:

1. Get 1 API call working in some open source SDK
2. Find the place where the cURL request is being made
3. Debug that request to learn what is actually being sent
4. Find the signature function and copy it
5. Try to make the call yourself now and start reading error messages
6. If you get stuck, ask developer support what you’re doing wrong

Amazon does some other super wonky things that I may detail in a later blog post. For now, maybe Amazon will see this and embrace developers. Stripe has done a fantastic job with this.

I make youtube videos teaching other programmers how to do my job. Check it out https://www.youtube.com/user/optikalefxx