NODEJS SERIES

Redis Cache Implementation with NodeJs

In this article, we will look at the Redis cache and explore what makes it fast. In addition, we'll show how Redis is employed. Furthermore, we'll observe how to install Redis on a Windows computer and run some fundamental Redis commands. We will also discuss using Redis programmatically with NodeJs using code snippets and test results.

What is Redis?

Redis, which stands for Remote Dictionary Server, is an in-memory database. The working memory (RAM) of the computer is where Redis runs. As a result, it works more swiftly. Redis is a schemaless database that stores JSON objects with key-value pairs, much like a NoSQL database. It doesn't require time to build the schema and initialize the database. Therefore it can perform application testing faster. Rapid tests can boost development productivity.

Redis, on the other hand, is volatile. Considering that everything inside Redis will be lost if your system suddenly fails. However, we may replicate Redis to create data backups, ensuring that even if the Redis master instance goes down, the replicas will continue to function and contain the data. Redis is frequently used as a cache to enhance the performance of applications. Redis can store the data we access frequently or require a lot of computational time. In the future, we will have quick access to the data if needed.

What makes Redis so quick?

We wonder how Redis can be so speedy being single-threaded.

The fact that Redis is an in-memory database is the primary factor in its speed. Pure memory reading offers fast read/write speeds and quick response times, provided the data you store can't be larger than the memory.

The usage of IO multiplexing is the second justification. Redis primarily uses a single thread. However, the operating system permits a single thread to wait on numerous open socket connections simultaneously, thanks to IO multiplexing. It's possible that a single-threaded design won't use all the CPU cores in the current hardware. Therefore, it is standard practice for some workloads to operate many Redis instances on a single server to utilize more CPU cores.

The adoption of efficient low-level data structures is the third argument. Since Redis is an in-memory database, it can use many effective low-level data structures, such as LinkedList, SkipList, HashTable, ZipList, SDS, and IntSet, without having to worry about how to efficiently save them to disk.

Illustration showing IO Multiplexing, Single Threaded Execution & Redis Data Structure (Credits: Alex Xu)

How does Redis cache work?

When a client requests data, the server first searches the Redis cache for the appropriate key. If the key is present in the Redis cache, a cache hit has occurred, and the user will receive the cached data. If the key is missing from the Redis cache, it is a cache miss, and the server will retrieve the most recent information from the database or third-party resources via REST APIs.

Redis caching Illustration

Redis Installation on Windows

We can install Redis cache on Windows using Windows Subsystem for Linux. Follow the below steps.

1. Open the Command prompt in administrator mode
2. Run wsl — install command to install Windows Subsystem for Linux (WSL) in Windows.
3. Reboot your system to view the changes.
4. Set up a new username and password for your Linux system.
5. Launch the Linux terminal.
6. Install Redis using commands:sudo apt-get update followed by sudo apt-get install redis
7. Start the Redis server using theredis-server command.
8. Open the Redis CLI by running theredis-cli command

By default, the Redis server starts at 127.0.0.1:6379 and stores everything as strings.

Redis Keys Commands

Redis offers the following fundamental commands to carry out various operations on keys.

• SET key value , sets the key-value pair.
• GET key , get value for a given key.
• DEL key, delete a given key.
• EXISTS key, check whether a key exists or not.
• KEYS pattern, find all the keys that match a specific pattern.
• flushall, delete everything inside Redis.
• SETEX key seconds value, set key-value that expires after the given seconds.
• ttl key, returns the expiry time left for a key.

The usage and output of the commands are demonstrated in the below code snippet.

chikku@DESKTOP-21GGVDA:~$ redis-cli
127.0.0.1:6379> set firstName Luke
OK
127.0.0.1:6379> set lastName Antony
OK
127.0.0.1:6379> set age 32
OK
127.0.0.1:6379> set email luke@gmail.com
OK
127.0.0.1:6379> get email
"luke@gmail.com"
127.0.0.1:6379> keys *
"age"
"lastName"
"email"
"firstName"
127.0.0.1:6379> exists email
(integer)1
127.0.0.1:6379> del email
(integer)1
127.0.0.1:6379> exists email
(integer)0
127.0.0.1:6379> keys *
"age"
"lastName"
"firstName"
127.0.0.1:6379> setex city 10 Bangalore
OK
127.0.0.1:6379> keys *
"age"
"lastName"
"firstName"
"city"
127.0.0.1:6379> ttl city
(integer)4
127.0.0.1:6379> ttl city
(integer)3
127.0.0.1:6379> ttl city
(integer)2
127.0.0.1:6379> ttl city
(integer)1
127.0.0.1:6379> ttl city
(integer)-2
127.0.0.1:6379> keys *
"age"
"lastName"
"firstName"
127.0.0.1:6379> flushall
OK
127.0.0.1:6379> keys *
(empty list or set)

Redis List Commands

Redis Lists are simple string lists. We can append elements to the list's head or tail in Redis lists. The commands are given below.

• lpush key value , push the element to the leftmost end of the array.
• rpush key value , push the element to the rightmost end of the array.
• lrange key startIndex stopIndex, display the list of elements between the start and stop indexes
• lpop key , pop the leftmost element of the array.
• rpop key, pop the rightmost element of the array.

The code snippet below shows the use and output of the Redis list commands.

chikku@DESKTOP-21GGVDA:~$ redis-cli
127.0.0.1:6379> lpush colors red
(integer)1
127.0.0.1:6379> lrange colors 0 1
"red"
127.0.0.1:6379> lpush colors blue
(integer)2
127.0.0.1:6379> lrange colors 0 2
"blue"
"red"
127.0.0.1:6379> rpush colors green
(integer)3
127.0.0.1:6379> lrange colors 0 3
"blue"
"red"
"green"
127.0.0.1:6379> lpop colors
"blue"
127.0.0.1:6379> lrange colors 0 2
"red"
"green"
127.0.0.1:6379> rpop colors
"green"
127.0.0.1:6379> lrange colors 0 2
"red"
127.0.0.1:6379>

Redis Set Commands

Every element inside a Set is unique, unlike in an array. Elements in an array can be retrieved using an index, which is not allowed in a Set because it requires keys. In contrast to an array, which maintains its insertion order, a set is not ordered. The sequence in which the items in a Set appear cannot be predicted. The Set commands are as follows.

• sadd key member, add members to a set.
• smembers key, display the members of a given set.
• srem key member, remove a given member from a set.

The usage and outcomes of Redis Set commands are shown below.

chikku@DESKTOP-21GGVDA:~$ redis-cli
127.0.0.1:6379> sadd fruits Apple Orange Grapes Mango
(integer)4
127.0.0.1:6379> smembers fruits
"Mango"
"Grapes"
"Orange"
"Apple"
127.0.0.1:6379> srem fruits Grapes
(integer)1
127.0.0.1:6379> smembers fruits
"Mango"
"Orange"
"Apple"
127.0.0.1:6379>

Redis Hash Commands

Hashing allows you to store key-value pairs inside of a single key. The Hash commands are listed below.

• hset key field value, set a key-value pair to a hash.
• hget key field, get the value of a hash field.
• hgetall key, get all the key-value pairs of a hash.
• hdel key field , delete a given field from a hash.
• hexists key field , check whether the field exists in hash or not.

The following lists the usage and outcomes of Redis Hash commands.

chikku@DESKTOP-21GGVDA:~$ redis-cli
127.0.0.1:6379> hset address city Bangalore
(integer)1
127.0.0.1:6379> hset address state Karnataka
(integer)1
127.0.0.1:6379> hset address country India
(integer)1
127.0.0.1:6379> hget address city
"Bangalore"
127.0.0.1:6379>hgetall address
"city"
"Bangalore"
"state"
"Karnataka"
"country"
"India"
127.0.0.1:6379>hexists address city
(integer)1
127.0.0.1:6379>hdel address city
(integer)1
127.0.0.1:6379>hexists address city
(integer)0
127.0.0.1:6379>hgetall address
"state"
"Karnataka"
"country"
"India"
127.0.0.1:6379>

Redis caching with Nodejs

We are creating an Express application with GET and POST endpoints. We use external, fake REST APIs (jsonplaceholder.typicode.com/posts) to fetch a list of posts and to create new posts.

Follow the below steps to set up the application.

1. Create package.json with the npm init -y command.
2. Install all the required dependencies for the Express app as follows:

npm i express dotenv axios body-parser

2. Install Redis dependency by using npm i ioredis

3. Your package.json file should contain the following.

{
  "name": "redis-setup",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node index.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "axios": "^1.1.3",
    "body-parser": "^1.20.1",
    "dotenv": "^16.0.3",
    "express": "^4.18.2",
    "ioredis": "^5.2.3"
  }
}

4. Add the Redis host, port, ttl (Time-to-Live) and timeout to the .env file. The .env file also includes an optional REST API URL that serves the Redis demonstration's purpose.

The code snippet below lists the contents of the .env file.

REDIS_HOST = 127.0.0.1
REDIS_PORT = 6379
REDIS_TTL = 30
REDIS_TIMEOUT = 5000
BASE_URL = https://jsonplaceholder.typicode.com/posts

5. Start your server inside the index.js file as given below.

require('dotenv').config();
const express = require('express');
const bodyParser = require('body-parser');
const app = express();

app.use(bodyParser.json());

app.listen(8000, () => {
    console.log('server started!');
});

6. Create a caching.js file that contains the following:

• A Redis instance with host, port and commandTimeout. If a command doesn't return a reply within the specified milliseconds, a "command timed out" error will be thrown.
• set() method to set the cache key-value pair with an expiry. Before setting, the data should be stringified, as Redis values are always strings.
• get() method to retrieve key-value pairs.
• del() method to remove the cache key.

The code snippet for caching.js is added below for your reference.

const Redis = require("ioredis");
const { REDIS_HOST, REDIS_PORT, REDIS_TTL, REDIS_TIMEOUT } = process.env;

let redis;

// Create a Redis instance
(async () => {
    redis = new Redis({
        host: REDIS_HOST,
        port: REDIS_PORT,
        commandTimeout: REDIS_TIMEOUT
    });
    redis.on("error", (err) => {
        console.log(err);
    });
})();

// Get key data from Redis cache
async function getCache(key) {
    try {
        const cacheData = await redis.get(key);
        return cacheData;
    } catch (err) {
        return null;
    }
}

// Set Redis cache Key with a given expiry
function setCache(key, data, ttl = REDIS_TTL) {
    try {
        redis.set(key, JSON.stringify(data), "EX", ttl);
    } catch (err) {
        return null;
    }
}

// Remove given Redis cache key
function removeCache(key) {
    try {
        redis.del(key);
    } catch (err) {
        return null;
    }
}

module.exports = { getCache, setCache, removeCache };

GET Method with Caching

We are creating a GET endpoint /getAll to fetch all the posts from an external API. We need to choose a unique cache key to store the values. Furthermore, by invoking the getCache(key) method, we will determine if the cache key has already been assigned a value. The user will receive the cached data if the key is found in the cache, known as a cache hit. If the key is not found in the cache, it is considered a cache miss, and the data will be fetched from the REST API, and the setCache(key, data) method will be called to set the value. Refer to the below code snippet to create the GET endpoint.

require('dotenv').config();
const express = require('express');
const bodyParser = require('body-parser');
const axios = require('axios');
const { getCache, setCache } = require('./caching');
const app = express();

const { BASE_URL } = process.env;
const cacheKey = `getAll/posts`;

//Middleware
app.use(bodyParser.json());

//GET Posts
app.get('/getAll', async (req, res, next) => {
    try{
        const response = {};
        const cacheData = await getCache(cacheKey);
        if(cacheData) {
            response['message'] = 'cache hit';
            response['posts'] = JSON.parse(cacheData);
        }else {
            const result = await axios.get(BASE_URL);
            const { data } = result;
            response['message'] = 'cache miss';
            response['posts'] = data;
            setCache(cacheKey, data);
        }
        res.status(200).send(response);
    }catch(err) {
        res.status(400).send(err);
    }
})

app.listen(8000, () => {
    console.log('server started!');
});

Result of GET method before caching

To test our GET endpoint, we are using the Postman tool. Send the GET request to fetch the result. The below image shows the result of GET posts before caching.

GET Method Test Result before caching

A new key will be added to the Redis cache. We can identify the newly added key on Redis by inspecting the keys as follows.

127.0.0.1:6379> keys *
1)"getAll/posts"
127.0.0.1:6379

Result of GET method after caching

Send the GET request again to fetch the result once the cache key is set in Redis. The data will be fetched from the Redis cache. The below image shows the Postman result of GET posts after caching.

GET Method Test Result after caching

You can detect the performance difference by carefully observing the Postman test results. After using Redis caching, the response time is significantly slashed.

POST Method with Caching

We are creating a POST endpoint /create to add a new post to the external API. The data is passed inside the req.body. The Redis key already set for the post result should be erased when a new post is added. The removeCache() method will do that. By doing this, data inconsistency can be prevented. Refer to the below code snippet to create the POST endpoint.

require('dotenv').config();
const express = require('express');
const bodyParser = require('body-parser');
const axios = require('axios');
const { removeCache } = require('./caching');
const app = express();

const { BASE_URL } = process.env;
const cacheKey = `getAll/posts`;

//Middleware
app.use(bodyParser.json());

//Create new post
app.post('/create', async(req, res, next) => {
    try{
        const response = await axios.post(BASE_URL, req.body);
        if(response) {
            const { data: posts } = response;
            removeCache(cacheKey);
            res.status(201).send(posts);
        }
    }catch(err) {
        res.status(400).send(err);
    }
});

app.listen(8000, () => {
    console.log('server started!');
});

Result of the POST method

Send the POST request in the Postman tool. The body of the POST request is sent as JSON. The below image shows the result of creating post API.

POST Method Test Result in Postman

The Redis key is deleted upon the creation of a new post.

Summary

Redis cache utilization with NodeJs has been the topic of our discussion up to this point. Using Redis as a cache improves data retrieval performance by minimizing the need to access the slower underlying storage layer. Redis works in a single thread, yet it is still a fast in-memory database.