API design 101 : build a nodejs REST microservice using GITHUB APIs

Introduction

APIs (Application Programming Interface) are a way to enable communication between different application components in one hand and between our application and other applications in the other hand.

A microservice is a small, single service that can be delivered through an API.

In this article, we will create a REST microservice that lists the languages used by the trending public Github repos in the last 30 days.

We’re gonna have two endpoints :

• (GET) /api/repos : returns the trending repos in the last 30 days from now.
• (GET)/api/repos/{date} : returns the trending repos in the last 30 days from the given date.

We have to calculate the total number of repos, and for each language we have to return :

• The number of repos using the language.
• The list of repos using the language.

Note that a trending repo is a most starred repo.

Tools used

• Nodejs : JavaScript runtime environment.
• Express.js : web framework built on the top of nodejs, we will use it to build our server.
• Axios : promise based HTTP client for the browser and node.js, we will use it to fetch data from the GITHUB API.
• VScode : modern code editor.
• Postman : HTTP client to test our API.

Let’s jump in to create our API.

Project structure

To build our API, we need three things :

• Routes : used to forward HTTP requests to the appropriate controller functions. We will have two routes in our API :
  - reposRouter : repos route without specifying the date.
  - reposWithDateRouter : repos route with a specific date.
• Controllers : Receives the requests and fetch data from the Github API, then returns the response.
• main file : This file will expose our routes to the user and starts the server.

Project architecture

The project structure is gonna be like this :

Project structure

Github APIs

Github APIs (or Github REST APIs) are APIs to interact with Github and fetch informations like users, repositories, and pull requests. In our case, we will use GitHub Search API to display informations about repositories.

To fetch the trending repos, we need to call the following endpoint :

https://api.github.com/search/repositories?q=created:>{date}&sort=stars&order=desc

And replace date with a YYYY-MM-DD date format.

The received data will be paginated, there’s no need to get the subsequent pages.

Now let’s move on to the realisation.

Getting started

Here are the steps to create our API :

1. Installing the Node packages.
2. Create our server using Express.js.
3. Declare our routes using Express.Router.
4. Create our controller function.
5. Testing our API using Postman.

Installing the Node packages

We will define the packages we need in package.json file :

package.json

What do these packages do ? axios is a promise based HTTP client used to fetch data from distant endpoints. dotenv is an npm module that loads environment variables from a .env file. express is the Node framework. moment is a javascript date library, we will use it to format our dates to the YYYY-MM-DD format. morgan is a middleware for logging incoming requests. nodemon is a command-line interface (CLI) that restarts our server on save.

Go ahead and create a package.json file with the given dependencies in the root directory, then run the following command :

npm install

We will be creating our app.json file to set up our server, since that’s our main file declared in package.json.

Create the server

Here’s our app.js file where we will start our server and import our routes :

app.js

• Base setup : In the beginning, we imported our needed packages : express, dotenv, morgan and our routes’ file ./Routes/repos where we will define our two routes. Then we set the port, define our app using express and call morgan with dev parameter, that will give us the simplest amount of information needed for our development lifecycle.
• Using our middleware function : use() function uses the specified middleware function, reposRoutes and reposWithDateRoutes

Start the server: we’ll have our express app listen to our predefined port.

When you run the server with npm start, you’ll get an error. We didn’t define our routes yet.

Let’s do this now.

Don’t forget to create a .env file in the root directory with a port value :

PORT = 5000

.env

Declare our routes

Instead of defining our routes in the main file app.js, we will use Express Router middleware. It’s good practice to declare your routes in a separate file. For our simple API, it’s not a big deal, but when you want to scale your app then you’ll have a problem in managing your routes. The Express Router gives us a great flexibility in defining our routes.

repos.js

• In the beginning, we imported express and our controller getRepos. Then, we got two instances for our two routes of the express Router.
• Then, we’ll use our routes to make a GET request to the given endpoints : http://localhost:5000/api/repos to get the trending languages in the last 30 days and http://localhost:5000/api/repos/{date} to get the trending languages in the last 30 days from the given date. We will define a controller function to handle the requests. It’s a good practice, you’d better get used to it.
• Finally, don’t forget to export the routes, so that you can use them, in our case, inside the app.js file.

Now let’s define our controller function.

Create our controller function

Although we have two routes, we will use only one controller function. We can separate them into two functions, but then we will be duplicating our implementation. You’ll see that we can create a controller function for both routes, depending on whether the date is passed in the URL or not.

getRepos.js

As always, in the beginning, we imported our needed packages : express and axios. The function getRepos() receive a request and return a response.

I used a lot of helper functions that I created to separate responsibilities and to make the code more cleaner. You can find them at root_folder/utils/utils.js.

I will add their implementations later in this blog.

getDate() returns the date object passed in parameter, if it’s the case, in the YYYY-MM-DD format. If the date is not passed in parameter (the user called the first endpoint) then it’s today’s date that’s formatted.

The result of the request to that Github endpoint (url) will be in the following format :

{
    total_count: "number",
    incomplete_results: "boolean",
    items: [
    {
      ...
        "name": "X",
      ...
        "language": "languageX",
    },
      ...
    {
      ...
        "name": "Y",
      ...
        "language": "null"},
    ]
}

For our API, we need the repos with non null languages.

After calling axios with the method axios.get(url), we store the repos with defined languages in the repos array using the function pushNonNullRepos.

Now, the repos array contains objects with two properties : name and language :

[
  {
    "name": "repo1",
    "language": "language1",
  },
  ...
  {
    "name": "repoN",
    "language": "languageN",
  }
]

We need to gather all the repos with the same language and count the number of repos for each language as well as the number of total repos. Each object will have three properties : language, number of repos and repos. The desired output will look like this :

{
  "total_count": xx,
  "result": [
     {
         "language": "languageX",
         "number of repos": x,
         "repos ": [
             "repo1",
                 ...
             "repoN"
         ]
     },
        ...
  ]
}

It’s the purpose of getTrendingRepos function, it takes as parameter an array of unique languages, that’s why we removed duplicates in languages, and the repos array. Inside the function, we will create an array that gathers for each language the list of repos using it. So that repos property will have that list as a value, and its length as a value of number of repos’ property. At the end, our function will return an object with the desired format, after adding the total_count property.

Our utils.js file will look like this :

utils.js

Testing the API

Now, as our API is ready, we can start the server using the following command in the root directory :

nodemon app.js

Let’s test our API in Postman :

• We’ll try with the default endoint, without specifying the date (the date of request : 2021–12–27) :

• Now we’ll try with a given date (for ex. 2021–07–29) :

So we need to call this endpoint :

http://localhost:5000/api/repos/2021-08-29

And we’ll get the following result :

Conclusion

This has been a quick look into creating a simple API with nodejs and Express 4. You can go further and create the frontend for this API, a beautiful UI with a list of repos and a search bar will be good.

You can find the source code of this app in the following repo :

github/Selharem/Express-Github-API

If you enjoyed this post, make sure to follow me on:

• Github: github/Selharem
• Medium : medium/@selharem
• linkedIn : linkedin/in/elharem-soufiane/
• Twitter : twitter/selharem

Thank you for reading. See you in another blog ✋.