Public API Design Factors

If you have developed a successful application/service on Google App Engine, chances are that you are thinking of exposing your application/data for use by the public. That is excellent news since it is likely to allow mashup developers to integrate your application into an overall larger scheme of things. This blog post goes into what factors you need to address before you begin on your journey to create a public API.

The web has moved from its image of hosting web applications to that of a platform. A platform is primarily, a set of building blocks that you can combine in innovative ways to build your own application. One of the key factors that have driven the web towards the “platform” avatar has been the emergence of a large number of Public APIs based on Open Standards. A Public API in simple terms is exposing key pieces of your application for the consumption of other applications.

A Public API is an important piece of any web application today. Over the last 5 years, public APIs have proliferated and everyone from the big companies to the newest startup bets big on a public API. According to the premier API tracking site, The Programmable Web (http://www.programmableweb.com), there are currently 2033 APIs available for applications today [dated 21st June 2010]

What can we do with the Public APIs? Here are some of them:

1. You can integrate a best of class functionality that is already being exposed by the Public API. For e.g. if you need to integrate Maps, simply integrate the Google Maps API. You do not need to reinvent the wheel.
2. You can provide a complete different user experience (interface) to an existing service.
3. You can utilize the Public API in a larger mashup that you are creating.
4. You can create applications running on devices that the current provider of the Public API does not address. For e.g. you could write mobile applications running on the iPhone, Android, etc — which the Public API provider does not even provide.

Now that we have discussed some of the points about what one can do with a Public API, here are some points about potential benefits that a public API brings to your existing application.

A Public API gives access to your platform to a huge number of programmers/applications.

1. Using open standards, they are able to build out applications using your API in ways that even the creators of the API could not have envisioned. And since the public APIs are usually built on Open Standards, they are integrated into a wide range of applications that run on hardware ranging from desktops to mobiles.
2. In the current market scenario, programmers do not want to build something that is already exposed by the publicly available API and if the API is backed by a scalable and high available environment, they will adopt it.

This article now builds on the premise that you are convinced that a public API will help your already available service. The article now discusses important points that you must consider before you embark on coding out the public API. Each of these points focuses on a theme rather than the implementation details and they are listed in no particular order of importance.

REST v/s Web Services (SOAP)

These are the two predominant models of invoking your API: REST and Web Service. REST is being preferred by most applications today, primarily because of its simplicity and implicit mapping to operations like GET, PUT, POST, etc. It is fair enough to say at this point that you should expose your services via the REST way, even if you have a Web Services (SOAP) mechanism of exposing your services.

Response Data Format (XML, JSON)

Every API call results in data being returned to the calling application. The data could be a result contains data records or in case of update APIs, simply a response text indicating “success” or “failure”. Two formats are predominant here: XML and JSON format. Your API should be flexible that it will allow the caller to specify which format they would like the data response to be sent back. Several APIs of late have been adopting only JSON as the supported data format. The best option over here at this point in time should be to support both if possible.

Service Contract Description

A Service Contract of your API is extremely important. It clearly defines what features your public API provides. This includes:

1. The different Services and their names
2. How they are accessible
3. Authentication mechanisms
4. Method signature for each method in a Service
5. Description of each parameter in a method

It is important that you take small steps in the initial release of your API. Do not expose all functionality at once. A preferred approach might be to expose a read-only API that first gives users access to your data and then introduce write-methods that allow users to save their information. Introduce newer methods/functionalities in later versions but release a fairly compact Service Contract in your initial versions. It will help you identify the chinks early enough and help to refactor the existing Service Contact and introduce newer ones.

API Authentication

One need not stress enough about how important it is to make your public API secure. You only want authenticated users to access your public API. There are various strategies and authentication/authorization standards emerging for you to begin with. The most common approach is to issue a API Key for any application that wants to use your public API. This API Key is provided on signing up and is to be provided with every request to your API by the calling application. You can then authenticate the key and also determine if the call is within acceptable Rate Limits (this is covered in a point later) that you have set per API Key.

An authentication standard OAuth is gaining widespread acceptance as a preferred way to allow users to first validate themselves with the Service Providers and then pass a token to the calling application that they have been validated. This removes the need for the calling application to store your users’ username/password with them. A large number of popular public APIs like Twitter have adopted this standard and you should have it on your radar.

Service Versioning

As your service evolves, the public API could also change to match the new features. You need to make sure that you do not break existing clients who have already binded to your existing public API. Enter versioning! Think about versioning your public API right from the first release. If you are using a REST like mechanism that inserting a version number like 1 or v1 might be a good start. With each version, specify clearly what is different, what the additional features are, etc.

Rate Limits

All public APIs should be free to exercise to increase their adoption in the initial phase. Unless you strongly feel that you need to have a paid public API right from Day 1, you should definitely make the public API free to use. However, it is important for it to make business sense at the end of the day. After all, computing resources do cost. An approach typically used by almost all companies is to build in Rate Limits (or Quota Limits) into their public API. Some examples include:

1. 10,000 calls per day
2. 1000 calls per hour
3. 1 GB Free total disk space (This is required in scenarios where you also persist the objects in your cloud as part of the public API)

The rate limits are typically reset at regular intervals like an hour, daily, etc. In certain cases, where you also provide storage space, the disk space quota is fixed at an upper limit and is not typically reset.

No matter what your functionality is, defining the rate limits is important because you will need to build checks in your public API infrastructure.

At the same time, there is a clear possibility that if your application and consequently your public API, is huge successful, then there will be applications which will easily go past your rate limits. That is a good problem to solve. In those scenarios (and you should expect them), you need to think about any of two approaches:

1. Selectively increase the rate limits for a particular application. You will first need to discuss the requirements with the original creators of the application.
2. Provide a paid option, where the application buys the increased limit either in tiers that you have set up or on a pay-as-you-go service. The pay-as-you-go service is generally preferred because you may not get the high spikes every single day.

Documentation

It is important to support your public API with strong documentation. Pay attention to every aspect of the documentation. At the minimum, you must document the following clearly:

1. End Points for different Architectural Styles (REST,WS) along with versioning.
2. Define all services, methods (actions) and their parameters
3. Provide sample request / response data formats for each method(action) in your service
4. Document the error scenarios clearly along with Server side status codes

The best way to get started in documentation is to look at the documentation of existing public APIs. Pick one that suits you and stick to it. Several public APIs even have a RSS Feed about their API documentation to inform about any changes.

Helper Libraries

It is important to envelop your REST/Web Service calls into easy to use helper libraries. Helper Libraries target a number of client programming languages/platforms like Java, Python, Ruby, Javascript, .NET, etc. A Helper library gives a quick starting point to get someone exercising your public API within minutes or hours. A Helper library should address the following (recommended):

1. Envelope the security calls (Authorization)
2. Envelope the REST/WS calls and Data Format (XML, JSON) parsing.
3. Optionally, return the results in Classes defined in the target high level language. For e.g. Java classes, etc.

Dedicated Public API Web page/site

It is strongly recommended to have a companion web site for your Public API. This companion website can serve as a one stop destination for all documentation related to your public API. It can also provide information like:

1. Signing up for the API and getting issued an API Key
2. Showcase applications / case studies of how people have used your public API
3. List official Helper libraries and contributed Helper libraries that make it easier to use your API.
4. An API forum where users discuss your API and which your API support engineers can regularly monitor
5. RSS Feeds that allow people to subscribe to API updates. Providing a “Sign up” for email updates to notify users is also recommended.

I hope this article gives a broad framework about the high level factors that you can consider your Public API release to support. Designing and Development can then commence much more smoothly. There are many other small details to consider, so please share your additional points in the comments for the benefit of all.

References

1. ProgrammableWeb (http://www.programmableweb.com)
2. Open APIs: State of the Market, May 2010. John Musser (http://www.slideshare.net/jmusser/pw-glue-conmay2010)
3. OAuth Protocol Site : http://oauth.net/