Three types of knowledge developers need when using APIs

Published in

Bits and Behavior

7 min readDec 11, 2020

As a programmer, I’ve used and struggled with the APIs of many code libraries and frameworks: from data visualization (d3.js, ggplot2) to statistics (many in R) to web development (Ruby on Rails, Bootstrap) and many more.

My two co-authors and I were interested in how to best categorize the knowledge needed to successfully work with APIs. Before I tell you our three categories though, a story of failure, learning, and then success:

A Story: Statistical Tests in R

Early in my PhD studies, I finally got my first dataset to run an experiment on. I wanted to use it to determine if people’s use of an online debugger varied with culture. To see if our dataset fit our predictions about culture and debugger use, I decided to use the R programming language to run some statistical tests.

I almost immediately ran into problems. I didn’t understand the statistical tests I wanted to use. I didn’t know how to use libraries in R to do almost any statistical tests. I gave up on this project after a few months of effort. A year later I realized I had learned enough to come back and run the statistical tests correctly and we eventually published our results.

So what did I have to learn to be able to come back and run the statistical test correctly? I learned things like:

Statistical concepts: Random effects in linear models. That the central limit theorem means that with a huge dataset, even if a variable isn’t normally distributed, you can still use linear models.
What R libraries and functions to use, and how they work: To make a linear model with a random effect of participant_id, you can’t use the lm function. Instead, use the lmer function in the package lmertest and add (1 | participant_id) to your independent variables.
How to use the various R commands and libraries together. I learned one of the basic outlines for doing statistical tests:

# create linear (or other) model
model <- lm(dependent_variable ~ independent_variables, data = mydata)# display summary of model to get the coefficients of the variables
summary(model)# run analysis of variance (anova) to find p values (statistical significance) including overall p values for categorical variables
anova(backStepModel)

After getting comfortable with this new knowledge (and much more I didn’t list), I was able to write the R script and interpret the results, getting me one step closer to a publication.

Now, with that story in mind, back to categorizing API knowledge:

Three Types of API Knowledge

After looking at prior research along with our own experiences and observations, we came up with three core types of API knowledge, and theorized how they worked and related to each other:

Note: We here imagine an individual reading or writing code by themselves. We ignore all the social dynamics involved in programming as well as issues like installation, running the code, and debugging.

Domain concepts and terminology

To use an API, a developer needs to know concepts from the domain of the API (e.g., statistics, visualization, web, etc.) which are used by the API. Developers also need to know what terminology the API uses to refer to those concepts.

For example, in my statistics story, I needed to understand what random effects in linear models are and that lmer is short for “linear mixed effects regression,” which is a type of regression that allows for random effects. In a visualization library, I’d need to know concepts like scales, axes, and bar charts (as well as what words the API uses for those).

When reading code, knowledge of domain concepts and terminology helps a developer understand the purpose of code (both overall and in its components). When writing code, knowledge of domain concepts helps a developer consider the theoretical solutions in the domain (which may or may not be possible with a specific API).

Execution facts

To use an API, a developer needs to know facts about how the API executes, such as what are possible inputs to an API call, and what are the outputs and side effects given those inputs.

For example, in my statistics story, I needed to know that if I pass (1 | participant_id) to the lmer function, it will generate a model with a random effect, but if I pass that to the lm function, it returns an error. In ggplot2, I’d need to know to use + to add features to my visualizations.

When reading code, knowledge of execution facts helps a developer predict what each API call will do. When writing code, knowledge of execution facts helps a developer consider the different ways they might call the API.

API usage patterns with rationale

To use an API, a developer needs to know the patterns of code used with the API, as well as the rationale for why those patterns work (in terms of domain concepts and execution facts). Code examples are a common and valuable source of API usage patterns.

For example, in my statistics story, I learned the pattern for doing statistical tests written above. In d3.js, a common pattern of code used to create an element for each data points is:

svg.selectAll("tagname") // Set rules for all elements "tagname":
   .data(dataset) // use the dataset "dataset"
   .enter() // any time there is a new data point
   .append("tagname") // add an element of type "tagname"

When reading code, knowledge of API usage patterns helps a developer understand the organization of code that uses an API, and understand how the pieces of that code relate to each other. When writing code, knowledge of API usage patterns helps a developer piece together potential programs that use the API.

How the three types of API knowledge relate

Now that I’ve defined the three types of knowledge, we can look at how they relate: execution facts are the details of how the API models domain concepts (e.g., d3.js can represent color as a string with a hexadecimal number that indicates red, green, and blue values); API usage patterns are organized by domain concepts (e.g., a domain workflow turned into code); and API usage patterns are built according to the rules of an API’s execution facts. Within domain concepts, terminology is the bridge between abstract concepts and the other knowledge types. Within API Usage patterns, rationale is the bridge between code patterns and the other two knowledge types, with rationale explaining the code pattern in terms of domain concepts and execution facts.

The content of the above paragraph visualized as three circles with arrows between them, along with labels. — Diagram of the relationships between the types of knowledge.

Since there are many domain concepts, execution facts, and API usage patterns, a developer will likely not know all of the relevant pieces of knowledge for what they are doing. Developers can search for more resources for learning, but they can also potentially infer missing domain concepts, execution facts, and API usage patterns:

A developer might infer a domain concept by looking at a term they don’t know in an execution fact or an API usage pattern and see what that term is used for.
A developer might guess an execution fact by knowing the domain concept behind a term in the API, or they might see the role of a piece of code in an API usage pattern, and guess how the code executes.
A developer might piece together a new API usage pattern by starting with abstract pseudocode based on domain concepts, and then use their knowledge of execution facts to fill in the pseudocode with concrete code.

Testing Our Theory

After categorizing these three core types of API knowledge and theorizing how they worked, we ran an experiment to see how these categories worked with actual API tasks. We gave 54 students a set of tasks and controlled which categories of API knowledge they had access to.

We found that each of the three knowledge types was sometimes useful and desired. We also found that having access to more information sometimes impeded our participant’s efforts. It appears that which knowledge component (and exactly which piece of information within that component) a developer needs is highly dependent on exactly which task they were doing. Just throwing more information at developers can slow them down as they try to wade through it.

Implications

While our categories and theory are new and definitely need more testing and study, there are a few implications from what we have so far:

People making API documentation and other API learning materials should make sure developers are able to learn and find relevant pieces of information from all three types of knowledge. Our study, though, showed that which exact piece of knowledge someone needed was highly task specific, so unless you already know their task (e.g., a tutorial), you’ll need to provide good ways to search for the right piece of information.
People working on automated documentation can make sure they are targeting all three types of knowledge in their efforts.
API learners aware of these categories can better understand what information they are lacking, and that awareness might aid them in searching out what they need.
I also hope that our categories of API knowledge and our theory about how they work provoke new questions and explanations about how people understand and use APIs.