Intro to Javascript on the Server, and APIs

Published in

thebit

19 min readSep 3, 2018

“woman and man sitting in front of monitor” by NESA by Makers on Unsplash

Before we begin, a disclaimer: Programming occurs at various levels of abstraction. Some people still write straight Assembly code, sometimes (albeit rarely) we need to change actual bits directly, but almost always we are using tools that do a lot for us behind the scenes. This is the nature of any library, framework, or even programming language. They automate a lot for us so we can focus on the good stuff, the meat of what we want to build. In this tutorial we will be using certain frameworks and libraries to help us, but it’s important to learn the language, not just the tooling. We will be discussing elements of the Javascript language as they arise, but they will be interwoven with the specific frameworks.

Javascript was originally written to provide basic functionality on the websites of the 90’s. Today, however, Javascript is run on everything from watches, to robots, to servers. For this tutorial specifically, we will be learning how to use Javascript on the backend of a web application to handle incoming user requests. We will also integrate a 3rd-party API to explore how engineers interface with external data sources. Let’s dig into it!

The Tooling

For this tutorial, we will focus on using Express, which is a micro-framework for server-side Javascript. It’s built on top of Node, which is just a JS runtime that runs outside of the browser (like on watches, robots, and servers, as previously mentioned).

Note: Before continuing, you’ll want to know the basics of setting up a new JS app with NPM or Yarn (don’t worry, for this project it will be easy). You’ll also need to make sure you have Node installed on your machine. If you don’t know how to do that, please see our tutorial on using NPM.

Create a new directory for this project, and make sure to npm init inside of it so we can start keeping track of our packages. After that, we can install express:

npm install --save express

For now, that’s all we’ll need, but it’s worth knowing that the server-side JS ecosystem is diverse and expansive. This means that there are thousands of packages you could add to your application in conjunction with express to handle authentication, middleware, file management, database connections, etc. A fully-featured Javascript backend app is likely to contain an assortment of packages all working cohesively. This is what I meant earlier when I called express a micro-framework: it only handles a small amount of functionality for us, the rest will come from other packages that aren’t affiliated with express at all! Neat.

Hello, Backend World!

Alright, let’s take a look at the official Hello World example that is provided in the express docs:

const express = require('express');
const app = express();app.get('/', (req, res) => res.send('Hello World!'));app.listen(3000, () => {
    console.log('Example app listening on port 3000!');
});

We can walk through this a little bit so we get a feel for the basics. Go ahead and create a file called index.js in your project directory and copy the above script into it. The first line is simple enough, as we’re just importing our dependency using node's require syntax. Then, we go ahead and create our app object by calling express().

Note: Because are calling express as a function on the second line, we know that the express package exports only a function. Packages need to export their critical parts, so users can actually take advantage of them. Packages can export a variety of data structures, so it’s important to read the docs of the package you’re importing to see how it should be used, or what you have access to.

After that, we call app.get() and pass it two arguments: a string which corresponds to a URL, and a function that determines what code should run when a user makes a request to that URL. In this case, our route is just / which is referred to as the root. The function will always accept two arguments, the request object, then the response object. The request is the HTTP Request that our app is receiving, and it contains a plethora of information, like data that the user is trying to upload, or information we might need to authenticate the user. This is the basic structure of a simple backend API: a user makes a request to a certain route of our app, and we run code to determine what to send back to the user. We will cover this in more detail later.

Note: The function that we pass to into request.get() is an anonymous function, meaning that we are defining it and passing it somewhere at the same time. We don’t need to give it a name because we don’t need to keep a reference to it anywhere since it won’t be reused by us. However, it’s generally a good rule of thumb to name all your functions, because names tell us what something does, which makes it self documenting. Also, if we name our functions, the name will be included in any stack trace if there is an error. This will make it easier for us to find problems.

Once we have defined a few routes, our script calls app.listen() which also takes two arguments: the port on which we should listen for requests, and a function to execute once this process is done. A function that is called at the end of a process is called a callback. In this case, we just want to tell the user that the app is ready to start handling requests, so our callback function just prints a message to the console.

Go ahead and start your app by running node index.js in the terminal. Make sure you run that in the directory that contains index.js otherwise you’ll get an error and nothing cool will happen. If it all works, you should see “Example app listening on port 3000!” logged into the console. If you don’t see that, try to see what error appears. You most likely aren’t in the correct directory, or you didn’t install the express dependency yet.

Now that the app is running, you can navigate to http://localhost:3000 in your browser, and you’ll see “Hello World!” on the page. Woo!

Note: An app is nothing without a web server, which is software that handles requests before our app gets them. The most popular web servers are Apache and nginx (pronounced “engine X”). If we didn’t run a web server, our app wouldn’t work on our computer, so express starts a little web server locally for us. Dealing with web servers is starting to get into dev ops territory, and less software engineering, but one needs to be aware of where our app fits into the greater ecosystem of running applications, both locally and in production environments.

Hello World, v2.0

Before we move on to extending the functionality of our app, let’s do a bit of refactoring to practice some Software Engineering principles, the most important of which will be ensuring that our code is readable and self-documenting.

Step one will be moving the callback function that we pass into our route, at app.get(). Let’s change it to look like this:

function handleIndexRequest(req, res) {
    res.send('Hello World!');
}app.get('/', handleIndexRequest);

This is a trivial example, but it will help us to get into the habit of ensuring that we name our functions, classes, and variables based on what they do. This way someone else, or even the original author, can look at the code and quickly get an idea of what precisely is happening. As they say, software engineering is just coding in a social context, so we always want to make sure our code is clear and readable for our coworkers who might later need to work on what we’ve written.

Right now, our only defined route is using the get method on our app object (Remember, a method is just a function that’s attached to an object or class). This means that this route will only respond to GET requests, and not POST, PUT or DELETE requests. A GET request is used to signify that a user wants to simply retrieve something from the server or application. When navigating to a URL in our browser’s address bar, such as when we had to visit http://localhost:3000, a GET request is made to that URL, and the browser will automatically display the response.

Note: We won’t be covering examples of how to handle forms and POST requests in this tutorial, but we will link some resources later on to explore that side of app development on your own. Allowing users to retrieve resources from you app is cool, but GET requests alone won’t allow you to build impressive, dynamic apps that allow users to upload and change information.

So now that we have a very basic app running, let’s extend it in two ways: adding a templating engine, and interfacing with a 3rd party API. This is going to require us to install some new packages, so let’s take these one at a time.

Adding a Templating Engine

Right now, our app will only return a basic response containing some text when a user makes a GET request to the root of our application. One of the more common response types, however, is with an HTML page. In order to send HTML pages from our app, we need to teach it how.

First, let’s install something called ejs, which allows us to build HTML files with Javascript on the server.

npm install ejs --save

Nifty. Now that we have our dependency installed, let’s tell our app to use it.

app.set('view engine', 'ejs');

That’s most of what we need to do in order to start using templates in our app! But keep in mind that this is just how express does it. If we were using a different Node framework and wanted to use a templating engine, it would be a different process. We truly do stand on the shoulders of giants, though, because this would be tough to do all by ourselves. Instead, we can rely on packages like express and ejs to help us build out what we want more quickly.

Note: Adding another package is not always the best thing to do, however. Any time we add another person’s code to our project, we are trusting them. More packages means more opportunities for security issues, or extra code we don’t actually need that may add bloat to our apps. In this case, templating can be very complicated, and ejs is very popular, so we shouldn’t hesitate to use it. No point in reinventing the wheel!

Now that our app can technically understand HTML templates, let’s create some new directories to store them. In the same directory as your index.js file, create a directory called views. Inside that, create a file calledhome.ejs. Notice the file extension, .ejs, which isn’t the normal .html extension we would expect to see. This is just an idiosyncrasy of ejs, so that the package knows which files it should look at when it comes times to create a new HTML file for a user.

This is what our file structure should look like at this point:

Let’s add some basic HTML to our home.ejs file, now:

<!DOCTYPE html>
<html>
    <head>
        <meta charset="UTF-8">
        <title>Is It Crypto</title>
    </head>
    <body>
        <h1>I think our template is working.</h1>
    </body>
</html>

That should do for now, we can make it cool and dynamic later. Let’s go ahead and test it by updating our index route function to send this HTML back instead of our “Hello World!” text. express has a variety of response types, such as JSON and HTML, which are probably the most common. We can use the render method on the res object (this is the response that we will send the user) by passing it the template to use. Update the handleIndexRequest function to look like the following snippet:

function handleIndexRequest(req, res) {
    res.render('home');
}

Note: The render method will automatically look in our views directory for the template we pass in as an argument (in this case home.ejs), so we don’t need to pass in the full relative path the template. This is something you would probably only know by reading the documentation for express. RTFM!

Now go back to your terminal, and kill the app if it’s still running by hitting Ctrl + C on the keyboard (this is the command to terminate a process). Then restart the app with node index.js again. When you refresh the browser, you should see some nice big text saying “I think our template is working” staring back at you! If you don’t see that, make sure all your files are saved, and that you installed the ejs dependency.

But ejs lets us do something even cooler, which is make our views dynamic. When you visit YouTube, you don’t always see the same videos on the Home page, as these things change. We don’t want to have to manually build out all of our HTML pages, we want to teach our app how to build pages based on our data. This is known as the MVC architecture, which stands for Model, View, Controller. Our model is our data, our view is our template, and our controller is our logic, like the function that we pass into our routes.

Now let’s make our template dynamic by showing something different depending on how where we navigate to in our app.

There are a fews way that our routes can contain dynamic information. We can store information in something called a query parameter. Query parameters are the things you see in a URL after the ?, if there is one. An example is something like ui/bento.html?reason=st where the parameter reason has a value of st. We can access these values in our route function as a part of the req object, with something like this:

const reason = req.query.reason; // this would equal 'st'

Easy! express manually reads the URL for us before we get it and organizes it into a neat package for us, so we can just check the req.query object to see what query parameters were sent. Other frameworks for building backend applications, like Django, will similarly prepackage request data to make it easy for us to access. Adding query parameters to a URL doesn’t make it a new route though, so if we were to make a GET request to http://localhost:3000?foo=bar then our route definition for the application root will still fire. So, let’s go ahead and update our route function to look for a query parameter!

function handleIndexRequest(req, res) {
    const name = req.query.name || 'Stranger';    res.render('home' { name: name });
}

Alright, we’ve got a couple new things in here, don’t we? We define a variable called name and set it to equal the “name” query parament or the string value of “Stranger”. This syntax just means that if the first part of the conditional is false, the second part will be used, so if we don’t have a query parameter called “name” in the URL, this name variable will be set to “Stranger” instead. Then below that, in the call to render you see we added a new object as a second argument. This is known as context, and it’s basically data we want to inject into our template. Right now we only need to worry about injecting the name variable, so we can pass just that in.

Note: When defining a Javascript object in which the key is the same as the variable name (like in { name: name }) we can simplify the syntax to something simpler like { name }. The JS runtime will know that this just means we want the key “name” to point to the variable also called “name”.

Since that information is being sent to our template, courtesy of the render method, let’s update our template to display it. Update the line with the <h1> to the following:

<h1>I think our template is working, <%= name %>!</h1>

This new syntax, the <%= %> stuff, is all specific to ejs, not Node or Javascript or anything. But now if you restart your app and refresh your browser tab, you’ll see this!

And if you add a query parameter called “name”…

How slick is that! Now our app can build our views based on our data. And this is just a trivial example, ejs can do a lot more if you want to read the docs and really start exploring. Now, what if we had more interesting data…

Using a 3rd Party API

Sometimes we will need or want to rely on external sources of data to extend the functionality of our app. To do this, we can use an API, or Application Programming Interface. The term “API” is used basically anywhere we’re talking about how we interact with a library, framework, or service. However, it most commonly refers to what endpoints are available on a backend application. Technically, we are building an API by creating routes with express, but our API isn’t meant for other developers to use.

But fortunately for us, there are plenty of free 3rd party APIs we can play with, so for this tutorial, we will use the nexchange.io API and display some basic cryptocurrency information. Usually, using a 3rd party service like this cryptocurrency API requires you to register and receive and API Key. This makes sure the company that maintains the API can prevent people from abusing their service. More on that in another tutorial, for now, we can just explore what data nexchange.io is allowing us to see.

Go ahead and visit https://api.nexchange.io/en/api/v1/currency/ and you’ll see the response we get from their servers. The long array of data contains some basic information on a wide range of currencies, and when we call this endpoint from our app, we will have direct access to the same information. So, how do we go ahead and make that request?

We will use something called fetch, which is built directly into the Javascript runtime you’ll find in browsers. Unfortunately, Node doesn’t ship with it by default, but we can practice our dependency management skills by adding it. Go ahead and install the node-fetch package. I won’t tell you how to do it, but you can refer to earlier sections of the tutorial if you can’t remember how! Oh, and don’t forget to import the dependency:

var fetch = require('node-fetch');

Once we have node-fetch installed, let’s update our root route function to look like the following snippet:

app.get('/', (req, res) => {
    const code = req.query.code;    fetch('https://api.nexchange.io/en/api/v1/currency/')
        .then(cryptoData => cryptoData.json())
        .then(cryptoData => {
            return code ?
                cryptoData.filter(crypto => crypto.code == code) :
                cryptoData;
        })
        .then(cryptoData => {
            res.render('home', { cryptoData: cryptoData });
        })
        .catch(err => console.log(err))
});

Whoa! Okay, this is different. The most noticeable difference seems to be the repeated call to then. This is because fetch returns a Promise. Javascript developers use promises to handle asynchronous code, which just means that it runs in the background, and we won’t know exactly when it’s finished running. Web requests are always asynchronous because sending data between servers always takes different amounts of time to execute.

Normally, code runs in the order it’s written. Line 1 is executed, then Line 2, then Line 3, etc. Asynchronous code throws a wrench into this pattern, though, and things can get confusing fast if we aren’t exactly sure of what happening. When dealing with async code, it could be the case that Line 1 is executed, then Line 2, then Line 4, then Line 3! For example, why wouldn’t this code work?

app.get('/', (req, res) => {
    let data;    fetch('https://api.nexchange.io/en/api/v1/currency/')
        .then(cryptoData => cryptoData.json())
        .then(cryptoData => {
            data = cryptoData;
        })
        .catch(err => console.log(err));    res.render('home', { cryptoData: data });
});

We declare the data variable, and when we get our crypto data back from the API, we set data to equal the crypto data, then we render our template with that data.

I don’t think so! fetch is asynchronous, so what would actually happen is:

We declare the data variable
We start the web request
We return our response
The web request completes, and we set data to equal the crypto data

If we were to structure it like the above snippet, our data would always be undefined in our template! We need to make sure we don’t send the response back until we have our data, and that means including the call to res.render() in the then method of the promise.

Promises can be chained using then to handle asynchronous actions easily. Each function that we pass into then isn’t executed until the previous one resolves. The catch method is executed if any of the then functions throw an error. We always want to make sure we handle errors, so the catch is required.

Note: The fetch API can do a lot of stuff involving web requests, but its default behavior is to make a simple GET request to the URL we pass in. That’s all we need for our example, but I wanted to explain clearly that all we are doing is making a GET request to the nexchange API, just like how a user will make a GET request to our app!

Before we move on, we should do another small round of refactoring to practice some Software Engineering principles. Functions should only really be concerned with one thing at a time, so if we find a function that is doing two totally different things, it means we should probably break it up into multiple pieces.

This is the idea of responsibility. Our route functions have the responsibility of sending back a response, but not fetching data from an API. The route function doesn’t care where our data is coming from, or how we get it, so we should break that responsibility out into it’s own function. Let’s refactor our route definition to look like this instead:

function getCryptos(code) {
    return fetch('https://api.nexchange.io/en/api/v1/currency/')
        .then(cryptoData => cryptoData.json())
        .then(cryptoData => {
            return cryptoData.filter(crypto => crypto.code == code);
        })        
        .catch(err => console.log(err));
}app.get('/', (req, res) => {
    const code = req.query.code;    getCryptos(code)
        .then(cryptoData => {
            res.render('home', { cryptoData: cryptoData });
        })
        .catch(err => console.log(err))
});

Now we have a getCryptos() function that returns a promise (fetch returns a promise, but so does then and catch. This is what allows you to chain them.) Now our route function can just call getCryptos()and it doesn’t need to worry about how we get the data. This way, if we decide to use a different API to get our crypto data later, we only have to change it in our getCryptos function.

Note: The main benefit of separating out code based on it’s responsibility is that it makes it more reusable. If we had another route that needed the crypto data, it too could just call getCryptos. In our small example, the benefit of the refactoring is negligible, but it will become necessary in larger apps. We just want to make sure you get some practice in sooner rather than later.

Oh, and one last note on promises. Whatever then returns is what the next then will get as an argument. This is what allows us to chain them. We usually use one call to then per transformation to the data. Let’s say we recieved some data from an API, then needed to sort them, filter them, and take the first piece, our promise code might look like this:

getAsyncData() // [3, 1, 2]
    .then(sortData) // returns [1, 2, 3]
    .then(filterData) // returns anything bigger than 1: [2, 3]
    .then(takeFirstDatum); // returns 2

We could wrap all that logic into one function, where we sort and filter the data, then take the first piece, but when our then functions start to become a little more complicated, it’s nice to define the functions somewhere else, give them a clear name, and just pass the function reference into then, like I did above. It helps with readability, and makes the code look sort of like a recipe of how to convert the API data into what we need.

Tying It All Together

In the above code snippet, you’ll see we updated our call to res.render to include cryptoData in the context object, which used to be name. Now we can go ahead and update out template to show the crypto data we get back. But actually, before we do that, let’s save everything and restart the app.

When you navigate to the root route…

… you’ll see the app is broken, and we get this error message and stack trace to help figure out what’s wrong. Can you figure out what happened? The first line is telling us its a ReferenceError, and below the code snippet you’ll see the message is complaining that name is not defined. This is just because we aren’t passing a variable called “name” into the template, so it isn’t sure what to do! This is an easy fix, but we wanted you to see a proper error and stack trace. You’re going to want to get familiar seeing errors and stack traces, because no matter how talented you become, you’ll still see them everyday.

Anyway, let’s update our template to look for cryptoData instead of name. Since cryptoData is going to be an array, we will have to learn how to iterate over data with the ejs template syntax. Here is our new home.ejs file:

<!DOCTYPE html>
<html>
    <head>
        <meta charset="UTF-8">
        <title>title</title>
    </head>
    <body>
        <ul>
            <% cryptoData.forEach(function(crypto) { %>
                <li><%= crypto.code %> - <%= crypto.name %>:                    <% if (crypto.is_crypto) { %>
                         ✅ 
                    <% } else { %>
                         ❌ 
                    <% } %>                </li>
            <% }); %>
        </ul>
    </body>
</html>

While the syntax is a little messy, this is really simple logic that you’ll get in no time. All we are doing is calling forEach on our array of crypto data, printing out the code and name of the each one into an li (list) element, then returning a ✅ if the currency is a cryptocurrency, and a ❌ if it isn’t. That’s just a basic if statement disguised in weird ejs syntax, though. Remember, when calling forEach or map, or something similar, you pass in a function as an argument, and that function accepts just one piece of data. You get to decide what happens for each piece. For example:

const animals = ['dog', 'cat', 'moose'];function printAnimal(animal) {
    console.log(animal);
}animals.forEach(printAnimal);// 'dog'
// 'cat'
// 'moose'

printAnimal gets run 3 times, because there are 3 items in animals. The first time, the animal argument is “dog”, the second time it’s “cat”, and finally it’s “moose”. We are doing the same thing with our crypto data, but instead of logging it to the console, we’re building HTML, which is way cooler!

Anyway, go ahead and save everything, then restart the app. Once you refresh the browser tab, you’ll see the final result!