Getting started with Redis cluster

What is Redis?

Redis is an open-source, BSD licensed, advanced key-value cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps, and hyperloglogs.

What is Redis Cluster?

Redis Cluster is a set of Redis instances, designed for scaling a database by partitioning it, thus making it more resilient. Each member in the cluster, whether a primary or a secondary replica, manages a subset of the hash slot. If a master becomes unreachable, then its slave is promoted to master. In a minimal Redis Cluster made up of three master nodes, each with a single slave node (to allow minimal failover), each master node is assigned a hash slot range between 0 and 16,383. Node A contains hash slots from 0 to 5000, node B from 5001 to 10000, node C from 10001 to 16383. Communication inside the cluster is made via an internal bus, using a gossip protocol to propagate information about the cluster or to discover new nodes.

Source: https://rancher.com/blog/2019/deploying-redis-cluster/

Setup Redis cluster on windows

Prerequisites

To perform this demo, you need the following installed in your windows machine:

• VirtualBox (https://www.virtualbox.org/wiki/Downloads)
• Vagrant (https://www.vagrantup.com/)
• Git Bash (https://git-scm.com/)

I am going to use vagrant to creating a virtual machine inside VirtualBox

first, you need to create a folder for the virtual machine. I created a folder called Redis_Demo in my Desktop

after that, you need open git bash terminal and cd into that folder

next, I am going to create a virtual machine using the vagrant command

$ vagrant init joshfng/railsbox

then vagrant initialize vagrant file for our virtual machine

now we can start our virtual machine using below command

$ vagrant up

It will take a few minutes to create VM and boot VM

Now we can ssh into our VM and try to install Redis for that we can use below command

$ vagrant ssh

Next, I am going to install Redis in VM. for that enter the following commands:

$ wget http://download.redis.io/releases/redis-4.0.10.tar.gz
$ tar xzf redis-4.0.10.tar.gz
$ cd redis-4.0.10
$ make
$ cd ..
$ gem install redis

Now Redis installed in VM and we can create Redis cluster in VM for that I am using 6 configuration files

you can download these file from here:

xcodemaker/Redis_Demo

after download configuration files you can put these file into Redis_Demo folder

Now we can create six nodes using these configuration files for that open six terminals and enter the following command inside each of them:

terminal 1:

$ vagrant ssh
$ ./redis-4.0.10/src/redis-server ../../vagrant/node1.conf

terminal 2:

$ vagrant ssh
$ ./redis-4.0.10/src/redis-server ../../vagrant/node2.conf

terminal 3:

$ vagrant ssh
$ ./redis-4.0.10/src/redis-server ../../vagrant/node3.conf

terminal 4:

$ vagrant ssh
$ ./redis-4.0.10/src/redis-server ../../vagrant/node4.conf

terminal 5:

$ vagrant ssh
$ ./redis-4.0.10/src/redis-server ../../vagrant/node5.conf

terminal 6:

$ vagrant ssh
$ ./redis-4.0.10/src/redis-server ../../vagrant/node6.conf

first command :

second command :

Now six Redis nodes are running inside our VM

then we can create Redis cluster using these Redis nodes for that enter the following command:

$ ./redis-4.0.10/src/redis-trib.rb create --replicas 1 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006

then terminal ask “Can I set the above configuration?”

and type “yes”

now our Redis cluster was created, Our cluster contains three master nodes and three slave nodes. now we can test it. for that, enter the following command:

$ ./redis-4.0.10/src/redis-cli -c -p 7001

now you can store key-value pairs using the set command

now we can test how Redis cluster works. if I kill one node using command Ctrl+c When we killnode1, which was originally a master, we see that Redis cluster promotes node4 to master, and when node1 returns, it does so as a slave.

What nodes talk about?

Fault tolerance

• All nodes continuously ping other nodes…
• A node marks another node as possibly failing when there is a timeout longer than N seconds.
• Every PING and PONG packet contain a gossip section: information about other nodes idle times, from the point of view of the sending node.

Fault tolerance — failing nodes

• A guesses B is failing, as the latest PING request timed out. A will not take any action without any other hint.
• C sends a PONG to A, with the gossip section containing information about B: C also thinks B is failing.
• At this point A marks B as failed, and notifies the information to all the other nodes in the cluster, that will mark the node as failing.
• If B will ever return back, the first time he’ll ping any node of the cluster, it will be notified to shut down ASAP, as intermitting clients are not good for the clients.

How Redis manage its storage

Redis Cluster is the native sharding implementation available within Redis that allows you to automatically distribute your data across multiple nodes without having to rely on external tools and utilities

The entire keyspace in Redis Clusters is divided into 16384 slots (called hash slots) and these slots are assigned to multiple Redis nodes. A given key is mapped to one of these slots, and the hash slot for a key is computed as:

HASH_SLOT = CRC16(key) mod 16384