Jan 11, 2018
What Is an API and Why Should I Use One?
Have you ever been asked, “Can’t you just use an API for that?” and thought to yourself “What the hell is an API?”
Perhaps you started off by Googling “APIs” and quickly found yourself drowning in acronyms, jargon, and unhelpful exposition. Do I need to take a REST or should I be using SOAP to clean off my data? What is a WSDL, and why do I need to put one in my SOAP? Is an RPC like an RPG and how do I level up? Confusion is a common experience for the growing body of people who are suddenly being asked to know more about tools that have historically been restricted to the realm of the “software engineer”.
This article will help you understand what API’s are without the pervasive (and unhelpful) jargon surrounding them. I will also include a code sample that you can run, right here in your browser, to access one of my favorite publicly available APIs — the Reddit API. Despite the code examples, I am writing this article with non-software engineers in mind. This article is for the sales analysts, executives, HR specialists, and others with no programming background, but who have a growing need to interact with and understand the growing world of Web APIs.
What Problem Do APIs Solve?
As computer programs continue their bid become the de facto medium of business, people who have never had any formal programming training are increasingly expected to have a higher level of software literacy. From sales analysts to HR representatives we are all being asked to make “data driven” decisions; simultaneously more and more company data is being housed “in the cloud” by Customer Relationship Management (CRM) systems like Salesforce, Oracle On Demand, Sage CRM, and countless competitors. Business metrics which might once have been hosted in an in-house database (or in a series of spreadsheets) is being shipped to these third parties and locked behind web interfaces that are often opaque and notoriously frustrating to use. APIs are built as an attempt to free the data from the clutches of these web interfaces, but come with their own set of challenges.
CRM companies these days all seem to promise that they have an “easy to use API” and that their customers can use this API to fetch and process the data collected by the CRM. Unfortunately, while many of these API’s truly are “easy to use” for software engineers (it’s worth noting that some are impossible to use even for veteran engineers) they are often unusable by anyone in outside the software engineering department; sometimes because of a lack of programming knowledge and sometimes because of corporate bureaucracy. Regardless of the reason, this bottleneck is a severe hindrance for people who need the data most, such as data analytics teams. This bottleneck is such a big problem that many organizations are paying for another web service like Tableau to fetch their data from their CRM in order to effectively use the data to answer important business queries related to their customer data.
The fact that organizations are being forced to pay both a CRM to keep their data and a third party to view their data begs the question of the “ease of use” of these APIs; in many cases the convoluted path that data follows from the point-of-sale to the CRM to the online data-analytics platform further obfuscates the “source of truth” for business data which, ironically, makes it even harder to make “data driven” decisions. The people who need accurate data the most are being locked out of accessing it because of this complex web of data transmission.
What Is An API?
An Application Programming Interface (API), is an unfortunately overloaded term which might refer to several different things depending on the context. Wikipedia gives a broad definition:
“In computer programming, an application programming interface (API) is a set of subroutine definitions, protocols, and tools for building application software. In general terms, it is a set of clearly defined methods of communication between various software components” (Links and emphasis original to Wikipedia)
In general, APIs define the rules that programmers must follow in order to interact with a programming language, a software library, or any other software tool. Lately though, the term API is most often used to describe a particular kind of web interface. These Web APIs are a set of rules for interacting with a webserver (such as a Salesforce server), with the most common use case being data retrieval. API’s provide mechanisms for CRM customers to access and manipulate data stored by the API provider (Salesforce in this example). The user makes a “request” to a Salesforce webserver, that webserver accesses a Salesforce database (with the customers data), and returns it to the requester in a “response”.
This same request/response cycle is used when you access webpages in your browser. The major difference between an “API request” and a “webpage request” is what kind of data is provided in the response. A website returns HTML, CSS, and JavaScript which work together with your browser to render a webpage. Web APIs respond with data in a raw format, not intended to be rendered by a browser into a user experience. JSON and XML are the most common formats used for this raw data, and they are both flexible text formats for storing data. Nearly all programming languages have libraries that can “parse” JSON and XML, making them friendly choices for developers. Most modern APIs favor JSON over XML.
Because a web API uses an HTTP request you can query a public API (meaning one that does not require authentication/login) directly in your browser. Reddit has a public JSON API which delivers the “raw data” without any of the CSS, HTML, and JavaScript that is used to create a user interface. Try opening this link in a new tab: https://www.reddit.com/r/all/.json (you can add /.json to any Reddit url). You’ll see a specific kind of formatted text, this is JSON or JavaScript Object Notation, and it’s a more raw version of the information you can see at https://www.reddit.com/r/all/. In Chrome the JSON looks like this:
Firefox has added tools to make JSON more user friendly by making the data easier to read and adding a layer of interactivity to exploring the raw data. You should see something like this:
This raw data represents the posts on https://www.reddit.com/r/all and is meant to be parsed by a computer program rather than a person. The program doesn’t have to be complex, lets look at a simple example, even if you’re not a programmer:
async function getJson(url) { let r = await fetch(url);
let responseJSON = await r.json();
for(post of responseJSON.data.children) {
let d = post.data;
console.log(`Title: ${d.title}\nUrl:${d.url}`);
}
}getJson('https://www.reddit.com/r/all/.json');
This example defines a “function” which uses other built in JavaScript functions to “fetch” the same data that we were just examining from https://www.reddit.com/r/all/.json. Here, fetch is both an English verb, and a built in function that JavaScript provides to help developers make HTTP requests. We “await” the data, which takes some amount of time to be fetched from Reddit and sent to us. Then we loop over the “children” logging the title and url of each individual child to the console. In this case, each “child” is a post on Reddit — that fact is part of the definition of the Reddit API. Each API must define its own format for the data that it serves; developers typically read documentation provided by the API “maintainer” (Reddit in this case) in order to learn the data format and use it properly.
I encourage you to open the “developer console” in Chrome or Firefox and paste that code into the console. (hot keys to open the console are: ctrl+shift+j or cmd+shift+j in Chrome, ctrl+shift+k or cmd+shift+k in Firefox). Make sure you do this in a brand new tab without a website loaded in it, otherwise you might get an error. When you paste the code and hit enter you should see something like this:
While some programs that use a web API are quite complex, many are actually rather simple. You’ve just successfully run your first API Integration!
Why Use APIs?
For Reddit, the availability of the raw data has made it possible for 3rd party developers to release phone apps that display the same data with custom presentation. Many other API’s are built with the intentions to allow 3rd party developers to build interesting applications using company data. Spotify even showcases some such apps on their website. Apps that “consume” the API data are sometimes called API Integrations. For example, a product manager might ask a software engineer to “Write an API integration that consumes the Salesforce API and saves the data into our on-site analytics database.”
Some APIs, like the Reddit and Spotify APIs, are designed to expand the reach of the organization by making their data available to users, and enabling external developers to build products that are in some way reliant on the business, and so keep customers coming back. For example, Spotify featured the “artist explorer” in the hopes that users will find new artists, build new playlists, and therefore continue (or start) using Spotify.
Other APIs, like the Salesforce API, are part of a package that is sold to companies. Businesses that pay for Salesforce services see the existence of an API as an added value, because they can have their own software engineers build integrations that do two things:
- Send data from their in-house software programs (such as a webserver or point of sale system) to Salesforce directly, updating the data “in the cloud”
- Pull data “from the cloud” to their in house software systems (such as a reporting system or internal database)
Because the APIs simply provide data, there are no limits on how a company can then go on to use that data. Furthermore, these programs can be automated to run on a schedule reducing the need for someone to navigate the complex steps of exporting data manually via the Salesforce web interface. As businesses scale up, many find that the initial cost of building such an integration can save employees time and sanity by removing the need to interact regularly with a complex and sometimes frustrating web interface.
Another benefit of Web APIs is that, because they are built around the HTTP protocol, nearly any programming language can be used to access them. Python, R, Java, JavaScript, Ruby, and every other general purpose programming language has at least one HTTP library to make this process easier. However, more specialist languages like SQL do not have HTTP libraries.
As software continues to become ubiquitous CRM companies expect to see the number of API integrations built against their APIs to increase dramatically. Especially as basic programming literacy continues to rise, employees in many different departments outside of the software team may find great value in writing a few API integrations themselves. Often, the code needed for these integrations is short and relatively simple like the JavaScript example we saw above; with a little bit of learning and a little bit of perseverance you too can write short programs to query and interact with the APIs your business pays for!
