Distributed caching in Redis

Redis..

Redis is an open source in-memory data structure project implementing a distributed, in-memory key-value database with optional durability. Redis supports different kinds of abstract data structures, such as strings, lists, maps, sets, sorted sets, hyperloglogs, bitmaps, streams and spatial indexes.

Caching..

Memory Caching. A memory cache, sometimes called a cache store or RAM cache, is a portion of memory made of high-speed static RAM instead of the slower and cheaper dynamic RAM used for main memory. Memory caching is effective because most programs access the same data or instructions over and over.

The purpose of cache memory is to store program instructions and data that are used repeatedly in the operation of programs or information that the CPU is likely to need next.

Redis Architecture

Redis Data Types

Strings ::Binary-safe strings.

Hashes::maps composed of fields associated with values. Both the field and the value are strings. This is very similar to Ruby or Python hashes.

Lists ::collections of string elements sorted according to the order of insertion. They are basically linked lists.

Sets ::collections of unique, unsorted string elements.

Sorted Sets ::similar to Sets but where every string element is associated to a floating number value, called score. The elements are always taken sorted by their score, so unlike Sets it is possible to retrieve a range of elements..

Who uses Redis

Efficiency & Speed are the main of any data architecture. Largest apps responsible for processing massive amounts of data in real-time, Twitter decided to rely on Redis.

Related Commands..

AUTH , ECHO, PING , QUIT ,SELECT ,SWAPDB

PING [message]

Returns PONG if no argument is provided, otherwise return a copy of the argument as a bulk. This command is often used to test if a connection is still alive, or to measure latency.

If the client is subscribed to a channel or a pattern, it will instead return a multi-bulk with a “pong” in the first position and an empty bulk in the second position, unless an argument is provided in which case it returns a copy of the argument.

Return value

Simple string reply

Examples

redis> PING

"PONG"

redis> PING “hello world”

"hello world"

redis>

Redis failover manage

Examlpe

First up, you can certainly have more slaves than masters, which would be preferable to having a single slave per master so that you can have one fail and still have a backup slave for some redundancy post-failover.

If you have 3 masters w/ one slave each, then you can lose one master. The cluster needs to have a majority of the masters available for a failover to occur; after one master fails, there are still 2/3 left, so the failed master’s slave will fail itself over after its cluster-node-timeout has elapsed.

At that point you’ll have a fully functioning cluster w/ 3 masters, though only 2 of them will have a slave (at least until the failed master comes back online as a slave).

However, if you lose 2 of your 3 masters at the same time, no failover will occur because there will not be a majority of masters online. Your queries will get a ‘CLUSTERDOWN’ error until a majority of masters are back online. In the 3-master case that means one of the failed masters would need to come back online before a failover could occur to get the cluster out of failed state.

Definitely recommend you spin up a bunch of nodes on your local machine and give it a try. Issue ‘DEBUG SEGFAULT’ to a master and monitor the other nodes’ logs to see how they respond. Issue that command to two masters and watch the slaves continually try to contact the failed master without failing over.

Redis Cluster Architecture

Redis Cluster is an active-passive cluster implementation that consists of master and slave nodes. The cluster uses hash partitioning to split the key space into 16,384 key slots, with each master responsible for a subset of those slots. Each slave replicates a specific master and can be reassigned to replicate another master or be elected to a master node as needed. Replication is completely asynchronous and does not block the master or the slave. Masters receive all read and write requests for their slots; slaves never have communication with clients.

How to Create a Redis Cluster

Automatic Approach

Redis comes with a tool named create-cluster, located at installdir/scripts/create-cluster. This allows you to avoid the manual configuration described above.

By default, this utility will create 6 nodes with 1 replica and will start creating nodes on port 30000. In order to not modify the utility, is recommended to create a config.sh script in the same folder as create-cluster with the following

content:

PORT=STARTING-PORT-NUMBER
TIMEOUT=2000
NODES=6
REPLICAS=1

Start the node and create the cluster:

./create-cluster start
./create-cluster create

Stop the cluster:

./create-cluster stop

Clean up the folder:

./create-cluster clean