Redis: Master and Slave architecture with Kubernetes. Building Chat Applications with Redis Pub/Sub on Kubernetes

In this comprehensive guide, we’ll walk through the step-by-step process of setting up Redis master and Slave architecture on Kubernetes, building a real-time chat application with Redis Pub/Sub on Kubernetes.
In continuing to my last,

Part 1: https://medium.com/@harshaljethwa19/redis-deploying-redis-on-kubernetes-building-chat-applications-with-redis-pub-sub-on-kubernetes-f81a56ec0273

Part 3: https://medium.com/@harshaljethwa19/redis-mongodb-caching-with-redis-72dd92be9a1a

Part 2: Building Chat Applications with Redis Pub/Sub on Kubernetes:

In this setup, we’ll have one Redis master pod and one or more Redis slave pods. The master pod will handle write operations, while the slave pods will replicate data from the master and handle read operations.

Here’s how you can set it up:

Deploy Redis Master:

1. Deploy a Redis master pod using a Deployment or StatefulSet in Kubernetes. This pod will serve as the primary Redis instance for write operations.

Deploy Redis Slave:

1. Deploy one or more Redis slave pods using the same Deployment or StatefulSet. Configure each slave to replicate data from the master pod.

Configure Master-Slave Replication:

1. Update the Redis configuration in the slave pods to point to the master pod for replication. You can do this by setting the replicaof configuration option in the Redis configuration file or via runtime configuration.

Connect to Master and Slave Pods:

1. Use Kubectl exec to access the shell of both the master and slave pods to verify configurations and perform any necessary troubleshooting.

Test Replication:

1. Verify that replication is working correctly by writing data to the master pod and confirming that it’s replicated to the slave pod(s).

Save this below YAML to a file, for example `redis-master.yaml`, and apply it to your Kubernetes cluster.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-master
spec:
  replicas: 1
  selector:
    matchLabels:
      role: master
  template:
    metadata:
      labels:
        role: master
    spec:
      containers:
      - name: redis-master
        image: redis:latest
        ports:
        - containerPort: 6379

kubectl apply -f redis-master.yaml

Save this below YAML to a file, for example `redis-slave.yaml`, and apply it to your Kubernetes cluster.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-slave
spec:
  replicas: 2
  selector:
    matchLabels:
      role: slave
  template:
    metadata:
      labels:
        role: slave
    spec:
      containers:
      - name: redis-slave
        image: redis:latest
        ports:
        - containerPort: 6379
        command: ["redis-server", "--replicaof", "IP_ADDR", "6379", "--slave-read-only", "no"]

kubectl apply -f redis-master.yaml

Note: Replace the IP_ADDR with your master pod IP address.
You can get the master pod IP_ADDR with this command and same for slave node replace MASTER_POD_NAME and SLAVE_POD_ NAME

kubectl describe pod MASTER_POD_NAME

Now to Test the Master and Slave Write below command and test it.

kubectl exec -it REDIS_MASTER_POD_NAME -- redis-cli

kubectl exec -it REDIS_SLAVE1_POD_NAME -- redis-cli

kubectl exec -it REDIS_SLAVE2_POD_NAME -- redis-cli

Write below after executing the above command to test the master and slave:

SET key test1
GET key

In a Redis Pub/Sub setup, there are typically two roles involved: the publisher and the subscriber.

1. Publisher:

● The publisher is any Redis client that sends messages to a specific channel.

● Publishers can be any application or service that needs to send messages to other parts of the system.

● Publishers use the PUBLISH command to send messages to specific channels.

1. Subscriber:

● Subscribers are Redis clients that listen for messages on one or more channels.

● Subscribers can be any application or service that needs to receive messages from publishers.

● Subscribers use the SUBSCRIBE command to subscribe to one or more channels.

● Subscribers can also use the PSUBSCRIBE command to subscribe to patterns of channels.

Redis itself acts as a message broker, facilitating communication between publishers and subscribers. It maintains a list of subscribers for each channel and delivers messages to all subscribers of that channel.

Monitor Command:

● The MONITOR command is used to stream back every command processed by the Redis server in real time. This includes Pub/Sub commands such as PUBLISH and SUBSCRIBE.

Let’s implement a real-time chat application using Redis Pub/Sub. In this example, we’ll have multiple users sending messages to a chat channel, and subscribers (other users) will receive and display these messages in real time.

Here’s a step-by-step guide to implementing this:

Setup Redis:

1. Ensure you have Redis installed and running on your system.

Create Publisher and Subscriber Clients:

1. You can use any programming language with a Redis client library. Here, I’ll demonstrate using Python with the redis-py library.

Publisher Implementation:

1. The publisher will allow users to send messages to the chat channel.

Run Publisher and Subscribers:

1. Run the publisher script in one terminal to send messages, and run multiple instances of the subscriber script in other terminals to receive and display messages.

Test the Chat:

Enter messages in the publisher terminal, and you should see them being received and displayed in real time by the subscriber terminals.

This example demonstrates the basics of building a real-time chat application using Redis Pub/Sub.
Save the below file as publiser.py and execute it.

import redis

# Connect to Redis
r = redis.Redis(host='IP_ADDR_MASTER', port=6379, db=0)

# User input for message
while True:
    i = input("Enter the slave number to talk with him: ")
    if i == "1":
        message = input("Enter the message for slave-01: ")
        r.publish('chanel1', message)
    elif i == "2":
        message = input("Enter the message for slave-02: ")
        r.publish('chanel2', message)
    else:
        print("Kindly enter 1 or 2")

Save below file 1 with subscribe1.py and file 2 with subscriber2.py and execute it.

import redis

# Connect to Redis
r = redis.Redis(host= 'IP_ADDR_SLAVE1', port=6379, db=0)

# Subscribe to channel
p = r.pubsub()
p.subscribe('chanel1')

# Receive and display messages
for message in p.listen():
    if message['type'] == 'message':
        print(message['data'].decode('utf-8'))

import redis

# Connect to Redis
r = redis.Redis(host= 'IP_ADDR_SLAVE2', port=6379, db=0)

# Subscribe to channel
p = r.pubsub()
p.subscribe('chanel2')

# Receive and display messages
for message in p.listen():
    if message['type'] == 'message':
        print(message['data'].decode('utf-8'))

Note: Replace the IP_ADDR with your master pod IP address.
You can get the master pod IP_ADDR with this command and the same for the slave node replace MASTER_POD_NAME and SLAVE_POD_ NAME

Execute these files with the below commands as per order:

pip3 install redis
python3 publiser.py
python3 subscribe1.py
python3 subscribe2.py

Part 1: https://medium.com/@harshaljethwa19/redis-deploying-redis-on-kubernetes-building-chat-applications-with-redis-pub-sub-on-kubernetes-f81a56ec0273

Part 3: https://medium.com/@harshaljethwa19/redis-mongodb-caching-with-redis-72dd92be9a1a

We extend our Redis setup to build a real-time chat application using Redis Pub/Sub. We explain the concept of Publish/Subscribe messaging and demonstrate how Redis facilitates communication between publishers and subscribers. Step-by-step, we guide you through the process of creating publisher and subscriber clients in Python, running them within Kubernetes pods, and testing the chat application’s functionality.

To further enhance scalability and fault tolerance, we introduce Redis master-slave replication within the Kubernetes environment. By deploying a Redis master pod and one or more Redis slave pods, we distribute write and read operations, ensuring data consistency and availability. We provide detailed instructions on deploying and configuring both master and slave pods, as well as testing replication to validate its functionality.

Follow me :

Linkedin: https://www.linkedin.com/in/harshaljethwa/

GitHub: https://github.com/HARSHALJETHWA19/

Twitter: https://twitter.com/harshaljethwaa

Thank You!!!