Build Super Fast Apps in Node.js using Redis Cache
When trying to optimise our applications, one of the first things we look to is caching. Caching involves storing data in a high performance store (many times temporarily) so such data can be retrieved faster at a later time. Redis is an efficient key-value data store that has become very popular for caching. One of the things that makes Redis a good choice is the number of data structures it supports — such as strings, hashes, lists, sets, and so on. This gives us some flexibility!
In this tutorial we will build a simple application in Node.js and speed it up by using Redis to cache data. Our app will be used to retrieve historical exchange rate data from an API and display it to users. You can view the entire code for this tutorial on GitHub, and the demo here.
Prerequisites
To follow along properly, you will need an understanding of JavaScript (ES6 syntax). You will also need to have Node.js and NPM installed. You can find installation instructions for both here.
I used the RedisGreen service from Manifold, so you can also optionally create an account there and set up the Redis service!
Setting Up
We will keep our file structure very simple. To get started, create a project folder, and initialise a new project within it with this command:
npm init -yTip: The
-yflag helps to create apackage.jsonfile with default values.
We will be using the Express framework, so we will need to install and save it as a dependency. We will also be making use of the node.js redis client for communicating with Redis, and axios for making our API calls. To install and save all 3 as dependencies:
npm install express redis axiosTip: If you use npm 5, you don’t need to specify the
-Sor--saveflag to save as a dependency in yourpackage.jsonfile.
Next, we can create the files needed for our app. This is the file structure we will be using:
├── currency-converter-app
├── views
└── index.html
├── package.json
└── server.jsOur app view will be located in the ./views folder, while our server-side logic will reside in the ./server.js file.
Building The App Backend
For starters, in our server.js file, we will initialise an express app, and require all our needed modules:
// ./server.jsconst express = require('express');
const path = require('path');
const axios = require('axios');
const redis = require('redis');
const app = express();
Defining Routes and Responses
Next, we can define the basic routes for our application:
// ./server.js
...const API_URL = 'http://api.fixer.io';app.get('/', (req, res) => {
res.sendFile('index.html', {
root: path.join(__dirname, 'views')
});
});app.get('/rate/:date', (req, res) => {
const date = req.params.date;
const url = `${API_URL}/${date}?base=USD`; axios.get(url).then(response => {
return res.json({ rates: response.data.rates });
}).catch(error => {
console.log(error);
});
});
From the above, we can see that our application has 2 main routes:
/— the base route which serves our app’s main view. It displays theindex.htmlfile from theviewsfolder using sendFile./rate/:date— this retrieves the rate for a specified date, using the Fixer API. It returns the response from the API as JSON.
To start the app:
// ./server.js
...const port = process.env.port || 5000;
app.listen(port, () => {
console.log(`App listening on port ${port}!`)
});
Building The App View
Now, we can build a basic view for our app. We will be importing Bulma (my current choice of CSS framework) to take advantage of some preset styling and make our view look better:
<!-- ./views/index.html --><!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Currency Converter!</title>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.6.0/css/bulma.min.css">
</head><body>
<section class="section">
<div class="container">
<h1 class="title">Currency💰Converter!</h1>
<p class="subtitle">Get historical data about exchange rates quickly.</p> <div class="content">
<blockquote>
BASE RATE IS <strong>USD</strong>.
GET HISTORICAL RATES FOR ANY DAY SINCE 1999.
THANKS TO <a href="http://fixer.io/">FIXER.IO</a>
</blockquote>
</div> <div class="columns">
<form id="rateForm">
<div class="column">
<div class="field">
<div class="control">
<input id="rateDate" name="date" class="input" type="date" required>
</div>
</div> <div class="field">
<div class="control">
<button class="button is-primary is-stretched">
Get rates
</button>
</div>
</div>
</div>
</form>
</div> <div class="notification is-link is-hidden" id="visits-count">
</div> <!-- container for results -->
<div class="columns is-multiline is-mobile" id="results">
</div> </div>
</section>
</body>
</html>
The above code contains basic markup for our app view. Next, we will write the JavaScript code to retrieve the rates from our backend, and display the rates. We import axios once again for our API calls:
<!-- ./views/index.html -->
...<!-- importing axios for API calls -->
<script src="https://unpkg.com/axios/dist/axios.min.js"></script><script>
const datePicker = document.querySelector("#rateDate");
const form = document.querySelector("#rateForm");
const submitButton = document.querySelector("#rateForm button"); form.onsubmit = e => {
e.preventDefault()
submitButton.classList.add("is-loading");
const date = datePicker.value;
const url = `/rate/${date}`;
axios.get(url).then(response => {
submitButton.classList.remove("is-loading");
showRates(response.data.rates);
}).catch(error => {
console.log(error);
});
} function showRates(rates) {
let html = '';
for (rate in rates) {
html += '<div class="column is-one-quarter"><div class="card"><div class="card-content">';
html += `<p class="title">${rates[rate]}</p>
<p class="subtitle">${rate}</p>`;
html += '</div></div></div>';
};
document.querySelector("#results").innerHTML = html;
} // IIFE - Executes on page load
(function () {
// set today's date as default date
datePicker.valueAsDate = new Date();
})();
</script></body>
</html>
Now, we can test our app:
node server.jsYou should see this when you visit http://localhost:5000.
Caching Data Using Redis
Next, we will connect to a Redis store and start caching historical data, so we don’t need to retrieve data from the Fixer API everytime a new request is made for the same date.
So for example, if a request is made for the exchange rates as at the 1st of June 2000 once by Person A, when Person B requests the rates for the same date, our app fetches the data from the cache instead of hitting the API. This will make the response time for our app faster.
We will also store the number of requests that have been made for a particular date in our Redis store, and increment it every time a new request for the rates for that date.
Setting Up A Redis Service Using Manifold [Optional]
Manifold provides a RedisGreen service that allows you set up Redis and connect to it in less than 5 minutes. You can head over to manifold.co, sign up, create a project and provision your Redis service. It is very straight forward.
After setting up the Redis service, you can copy out the REDIS_URL credential as we will be using that in the next step.
You can also set up Redis locally by downloading it from the official downloads page.
Connecting To Redis
We will be communicating with Redis using the node_redis Node.js client. To connect to Redis:
// ./server.js...// connect to Redis
const REDIS_URL = process.env.REDIS_URL;
const client = redis.createClient(REDIS_URL);
client.on('connect', () => {
console.log(`connected to redis`);
});
client.on('error', err => {
console.log(`Error: ${err}`);
});...
We are setting the REDIS_URL gotten from the previous step as an environment variable and passing it to redis.createClient().
Note: By default,
redis.createClient()will use127.0.0.1as hostname and6379as port. If your setup is different, you can connect using the redis URL, or by supplying the hostname and port. For a list of connection options, you can check the official docs here.
We also use the connect and error events during development to check if a connection was made, and see any errors that exist. For a list of available events, visit here.
Saving and Retrieving Cache Data
Now, we can store and retrieve data from our Redis store. We will update the function that retrieves our rates so we can check for data in our cache first before making the API call, and also increment the count of requests for a the rates for a particular date.
Note: All Redis commands are exposed as functions on the
clientobject. Read more here.
Updating the function that retrieves our rates:
// ./server.js...
app.get('/rate/:date', (req, res) => {
const date = req.params.date;
const url = `${API_URL}/${date}?base=USD`; const countKey = `USD:${date}:count`;
const ratesKey = `USD:${date}:rates`;
client.incr(countKey, (err, count) => {
client.hgetall(ratesKey, function(err, rates) {
if (rates) {
return res.json({ rates, count });
} axios.get(url).then(response => {
// save the rates to the redis store
client.hmset(
ratesKey, response.data.rates, function(err, result) {
if (err) console.log(err);
});
return res.json({
count,
rates: response.data.rates
});
})
.catch(error => {
return res.json(error.response.data)
}));
});
});
});...
Note: The node redis API is entirely asynchronous. Callbacks need to be implemented to retrieve data.
In the code above, we specify two keys which we use for data storage:
countKey: Tracks the number of requests made for a particular date. It is incremented on every request to the same date.ratesKey: Stores the rates data for a particular date.
Some redis commands we make use of:
incr: Increments the count of requests for a particular date.hgetall: Retrieves the cached list of rates for a particular date.hmset: Sets a list of rates for a date.
Promisifying Our Redis Commands [Optional]
Optionally, to get out of nesting too many callbacks (avoiding callback hell), and handle all our errors more easily, we can promisify node_redis with bluebird.
First, we install the bluebird package:
npm i bluebirdNext:
// ./server.js...const bluebird = require("bluebird");// make node_redis promise compatible
bluebird.promisifyAll(redis.RedisClient.prototype);
bluebird.promisifyAll(redis.Multi.prototype);
This will add Async to all node_redis functions, so we can modify our rates route like this:
// ./server.js...app.get('/rate/:date', (req, res) => {
const date = req.params.date;
const url = `${API_URL}/${date}?base=USD`; const countKey = `USD:${date}:count`;
const ratesKey = `USD:${date}:rates`; let count;
client
.incrAsync(countKey)
.then(result => {
count = result;
return count;
})
.then(() => client.hgetallAsync(ratesKey))
.then(rates => {
if (rates) {
return res.json({ rates, count });
}
axios.get(url).then(response => {
client
.hmsetAsync(ratesKey, response.data.rates)
.catch(e => {
console.log(e)
}); return res.json({
count,
rates: response.data.rates
});
}).catch(error => res.json(error.response.data))
})
.catch(e => {
console.log(e)
});
});
Finally, we will modify the JavaScript for our view to show the count of the number of requests for each date:
<!-- ./views/index.html -->
...<script>
...
const visitsDiv = document.querySelector("#visits-count"); form.onsubmit = e => {
e.preventDefault()
submitButton.classList.add("is-loading");
visitsDiv.classList.add("is-hidden");
const date = datePicker.value;
const url = `/rate/${date}`;
axios.get(url).then(response => {
submitButton.classList.remove("is-loading");
// call function that shows the rates
showRates(response.data.rates);
// call function that shows count of requests for each date
showVisits(response.data.count, date);
}).catch(error => {
console.log(error);
});
} function showVisits(count, date) {
let html = `
The rates as at <strong>${date}</strong>
have been checked <strong>${count}</strong> time(s).
`;
visitsDiv.classList.remove("is-hidden");
visitsDiv.innerHTML = html
}
...
</script>
...
Running The App!
To run the app:
REDIS_URL=YOUR_REDIS_URL node server.jsIf you’re running Redis locally, you can simply do node server.js.
Running Your Application with the Manifold CLI [Optional]
It will be useful to note that Manifold also provides a CLI tool to help developers easily manage their services. It helps reduce the steps from provisioning to deployment to a minimum! You can get started on it here.
You can find a demo of the app we built in this tutorial here.
Conclusion
In this tutorial, we have learned how to speed up our Node.js applications using Redis. We have:
- Built a Node.js app from scratch,
- Provisioned a Redis server using the Manifold RedisGreen service,
- Connected to the Redis server, and
- Saved data to, and retrieved data from the Redis store using the node redis client.
If your next question is ‘so when do I cache?’ — remember when considering optimisations, if you are still thinking about if you need it or not… you probably don’t need it.
There are still a bunch of improvements we can make to our base app using Redis commands and other techniques. The entire code is hosted on GitHub, you can take a look and play around with it. Do you have any questions or comments about Redis, Node.js and Caching? Let’s discuss in the comments!
About Manifold
Manifold is the first independent marketplace for developer services and the easiest way to manage and share everything from JawsDB, a simple postgres database to Mailgun, an API that makes sending and receiving email dead simple.
For more info visit Manifold.co or follow us on twitter. This post is part of the Manifold Content program. Want to write for us?
