Using Latest socket.io version with older library support

In this article, I’ll be discussing how to scale Socket.io across multiple services, using Go as the base programming language. Although I’ll be using Go as an example, the concepts discussed can be applied to any language.

The challenge I faced in my work was creating a service in Go that could communicate with clients using the latest version of Socket.io, which is not supported by Go. Additionally, our Socket.io servers needed to be distributed across multiple services so that any service could communicate with any client, regardless of where the connection is made.

To address this challenge, I’ll be walking you through how to scale Socket.io (taking it as a top-level library) and how to scale it across different versions that use different types of encoding and protocol. First, let’s define what scaling sockets means, how it works, and most importantly, why it matters. You can find this part of the article here.

So as you have already read, I used Redis to scale my sockets. So any message is now published to a channel(a room, or a queue or whatever you wanna call it) and is distributed to all the consumers. A simple pub/sub model.

So my Go support was only 1.4. I used this pub/sub model to create a parser so that I can publish messages to Redis directly instead of socket.io doing that for me. That message was in correspondence with the latest version of socket.io and hence picked up by the latest service. Here is the GO code on how you can implement the parser

The room that socket.io v3+ subscribes is in this format:

socket.io#YOUR_NAMESPACE#YOUR_ROOM#

The event emitted is in this form:

[
    ID,
    {
        Type: 2, 
        Data: {
     Event,
            YourDataAsKeyValueObject // Example name: "abc", age: 24
        },
        Namespace
    },
    {
        Rooms: [] // Leave empty if none
        Except: [] // Leave empty only
        Flags: 0x80 // flag 128 in decimal. Means a socket.io event flag.
    }
]

So the data for v3+ should be encoded in this format. An array with 3 elements. So I created my parser that creates the encoded format in this order.

Sample event:

[
    "uuidv",
    {
        type: 2, 
        data: {
            event: "chat",
            from: "whiterose",
            message: "You are just like a friend to me <_>"
        },
        namespace: "/chat"
    },
    {
        rooms: []
        except: []
        flags: 0x80
    }
]

So to emit a message in Redis you just need the room where you want to emit the message. I got both above and then just wrote a parser in GO.

package parser

import (
 "fmt"
 "github.com/vmihailenco/msgpack/v5"
 "matchmaking/constants"
 "matchmaking/pkg/handler/publisher/events"
 "matchmaking/pkg/handler/publisher/namespace"
 "time"
)

type SocketIOMessage struct {
 Id            string
 SocketMessage SocketIOMessageData
 SocketConfig  SocketIoMessageConfig
}

type SocketIoMessageConfig struct {
 Rooms  []string `msgpack:"rooms"`
 Except []string `msgpack:"except"`
 Flags  uint8    `msgpack:"flags"`
}

type SocketIOMessageData struct {
 Type      int                 `msgpack:"type"`
 Data      []interface{}       `msgpack:"data"`
 Namespace namespace.Namespace `msgpack:"nsp"`
}

type ClientMessage struct {
 Message       string                  `msgpack:"message"`
 Data         *interface{}             `msgpack:"data"`
}

type SocketIOParser struct {
 Namespace namespace.Namespace
}

func InitialiseSocketIOParser(namespace namespace.Namespace) *SocketIOParser {
 return &SocketIOParser{
  namespace,
 }
}

func (nsp *SocketIOParser) CreateSocketIOMessage(
 event string,
 clientData ClientMessage,
) []byte {
 now := uint64(time.Now().Unix())
 if clientData.Timestamp == nil {
  clientData.Timestamp = &now
 }
 socketMessage := SocketIOMessage{
  Id: fmt.Sprintf("%s", "abcdef"),
  SocketMessage: SocketIOMessageData{
   Type: 2,
   Data: []interface{}{
    event,
    clientData,
   },
   Namespace: nsp.Namespace,
  },
  SocketConfig: SocketIoMessageConfig{
   Rooms:  []string{event},
   Except: []string{},
   Flags:  0x80,
  },
 }

 socketFinalMessage := []interface{}{
  socketMessage.Id,
  socketMessage.SocketMessage,
  socketMessage.SocketConfig,
 }

 msgToSend, err := msgpack.Marshal(socketFinalMessage)
 if err != nil {
 }

 return msgToSend
}

func (nsp *SocketIOParser) CreateSocketIOEvent(room string) string {
 return fmt.Sprintf("socket.io#%s#%s#", nsp.Namespace, room)
}

package namespace

type Namespace string

const (
 DefaultNamespace Namespace = "/"
 ChatNamespace  Namespace = "/chat"
)

And when to send message I did something like this:

func (pub *Publisher) SendMessage(
 userIds []uint64,
) {
 for _, userId:= range playerIds {
  room := fmt.Sprintf("battle/profile/%d", userIds)
  pub.redis.Publish(
   context.Background(),
   pub.parser.CreateSocketIOEvent(fmt.Sprintf("battle/profile/%d", userId)), // This is the event in socket.io I want my message to be emitted as
   pub.parser.CreateSocketIOMessage(fmt.Sprintf("battle/profile/%d", userId), parser.ClientMessage{ // The first part is the room I want to emit my messag in and the second part is the data I want to pass in my event. If you don't have a specific room, don't pass anything. % is used interally by socket.io v3+
    Message:      "I am coconut",
    Data:         nil, // any JSON inside data field
   }),
  )
 }
}

So as I have both the room and the message, I am just emitting the message into the redis pub/sub manually.

To initialise for the above, we have publisher as:

func Initialise(redis *redis.Client, namespace namespace.Namespace) *Publisher {
 parserObj := parser.InitialiseSocketIOParser(namespace)
 return &Publisher{
  redis:  redis,
  parser: parserObj,
 }
}

So when socket.io was emitting message to Redis internally, let’s say version 1.4. It was emitting message formatted in a different format which is used by 1.4. Now I removed socket.io emitting messages and did it manually myself using the encoding used by socket.io v3+ and my message is send in socket.io v3.0+ format and is thus delivered without any issues.

So you can have a parser that emits messages in both forms so that all the services using any version of socket.io can listen and process the message.

So from the above approach, you can have your clients connect to whatever version of socket.io they are using and do a pub/sub model supported by socket.io to send messages seamlessly.

EXTRA: In order to get the format of the message, I read the implementation inside the NodeJS library for socket v4. After putting an adapter, socket.io broadcast messages to Redis for every emit. So I read the broadcasted messages using a CLI command

redis-cli monitor

With this, I was able to get the event that is emitted and I was able to create the correct parser that takes the message and parses it according to the latest socket.io v4 format.

This is a cleaver hack that works seamlessly. I wanted to work in Go and only other solution I had was either to use a message broker like RabbitMQ or a managed gRPC which is an overkill or create my own Go v4 socket.io library which is again a time taking process. Hope you liked this hack.