Using Coherence to manage HTTP sessions in Go

Introduction

A few months ago we announced the Coherence Go client, allowing native Go applications to access the Coherence data grid.

Today we would like to announce that your Go applications can now use Coherence as an HTTP session store via integration with the very popular “Express inspired web framework” Fiber. (https://github.com/gofiber/fiber)

In this article we will walk through creating a simple Fiber based web app which will store and retrieve HTTP sessions using a Coherence Community Edition (CE) cluster.

Why use Coherence for session storage?

Coherence is a scalable, fault-tolerant, cloud-ready, distributed platform for building grid-based applications and reliably storing data.

Using Coherence for session management provides a scalable and fault-tolerant session store allowing sessions to “fail-over” to surviving nodes if you have a server outage on the middle-tier or the Coherence storage-tier. It also allows you to easily scale your session storage-tier as your demand or load increases.

To demonstrate this best in a demo environment, we will create our environment using Docker Compose and configure a Traefik load balancer in front of a 2 node Coherence cluster to demonstrate the failover capabilities.

The Go Fiber session management uses Coherence as the store and connects to the Coherence cluster via the load balancer balancer using gRPC. Coherence automatically creates a backup of all objects on separate nodes to ensure fault-tolerance.

Pre-requisites

To get started, ensure you have the following:

1. Docker setup using either Docker or Rancher Desktop
2. Go 1.19 or above
3. Your favourite IDE

Create a new Go project

Open a terminal and create a directory for your example, and initialize using:

mkdir go-sessions
cd go-sessions
go mod init main

Download v2 of fiber:

go get github.com/gofiber/fiber/v2@latest

Download the Coherence storage driver for fiber:

go get github.com/gofiber/storage/coherence@latest

Note: This driver is an implementation using v1.0.3 of the Coherence Go Client.

Start a Coherence Cluster

You can use a commercial (14.1.1.2206.5+) or Community Edition (CE) 22.06.5+ Coherence cluster configured with a gRPC proxy server.

You can choose to create a single member cluster just to see this in action, or a load balanced cluster to show the best use case for Coherence sessions.

Start a load balanced cluster

Create a file called docker-compose.yaml in the go-sessions directory with the following content:

version: "3.5"
services:
  grpc-proxy:
    depends_on:
      - coherence1
      - coherence2
    image: traefik:latest
    command: --api.insecure=true --providers.docker --entrypoints.other.address=:1408  --log.level=DEBUG
    ports:
      - "8080:8080"
      - "1408:1408"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      coherence:
        aliases:
          - proxy

  coherence1:
    hostname: machine1
    networks:
      coherence:
        aliases:
          - machine1
    image: ghcr.io/oracle/coherence-ce:23.03.1
    environment:
      - coherence.cluster=cluster1
      - coherence.member=member1
      - coherence.machine=machine1
      - coherence.wka=machine1
      - coherence.health.http.port=6676
      - coherence.management.http=all
    ports:
      - 30000:30000
    labels:
      - "traefik.enable=true"
      - "traefik.TCP.Routers.coherence2.Rule=HostSNI(`*`)"
      - "traefik.TCP.Services.coherence2.LoadBalancer.server.Port=1408"
      - "traefik.http.services.coherence2.loadbalancer.healthcheck.path=/ready"
      - "traefik.http.services.coherence2.loadbalancer.healthcheck.port=6676"

  coherence2:
    hostname: machine2
    networks:
      coherence:
        aliases:
          - machine2
    image: ghcr.io/oracle/coherence-ce:23.03.1
    environment:
      - coherence.cluster=cluster1
      - coherence.member=member2
      - coherence.machine=machine1
      - coherence.health.http.port=6676
      - coherence.wka=machine1
      - coherence.management=all
    labels:
      - "traefik.enable=true"
      - "traefik.TCP.Routers.coherence2.Rule=HostSNI(`*`)"
      - "traefik.TCP.Services.coherence2.LoadBalancer.server.Port=1408"
      - "traefik.http.services.coherence2.loadbalancer.healthcheck.path=/ready"
      - "traefik.http.services.coherence2.loadbalancer.healthcheck.port=6676"

networks:
  coherence:

The above yaml creates three services.

• A grpc-proxy, — Traefik load balancer on port 1408 (the default Coherence gRPC port) load balancing to the cluster members
• coherence1 — Coherence member 1 with management enabled on port 30000 and gRPC listing on default port of 1408
• coherence2 — Coherence member1 with gRPC listening on default port of 1408

Start the environment by issuing the following command from the go-sessions directory:

docker compose up -d

To check that the Coherence cluster is ready using docker ps and ensure the STATUS of the coherence cluster members show healthy.

Create the code

In this simple session example we are providing 2 HTTP endpoints:

1. / — will create or load the existing session and store a count of how many times this endpoint has been called as well as first and last access time
2. /destroy — will simulate a “logout” function and destroy the current session and force a new one to be created on next access

We will add the code in sections to explain each of the parts. (The full code listing is below if you want to just copy/ paste)

With your IDE, create a new file main.go and add the following package and imports and global variables.

package main

import (
    "fmt"
    "github.com/gofiber/fiber/v2"
    "github.com/gofiber/fiber/v2/middleware/session"
    "github.com/gofiber/storage/coherence"
    "log"
    "strings"
    "time"
)

var store *session.Store  // gofiber session store

Continuing editing the same file, create the main function, which will:

1. Create a new Coherence store to be used by the Fiber session store
2. Intialize the fiber session store with the Coherence store and set the HTTP expiry to 120 seconds
3. Create a new fiber app

func main() {
    // create new coherence session store using defaults of localhost:1408
    storage, err := coherence.New()
    if err != nil {
         log.Fatal("unable to connect to Coherence ", err)
    }
    defer storage.Close()

    // initialize the gofiber session store using the Coherence storage driver
    store = session.New(session.Config{
        Storage:    storage,
        Expiration: time.Duration(120) * time.Second,
    })

    app := fiber.New()

Create a route for the/resource, which will:

1. Retrieve the session from the store, which under the hood checks Coherence
2. If the session is new (fresh) then set session attributes accessCount and firstAccess
3. If the session is an existing session, then increment the accessCount and update lastAccess
4. Display the current set of HTTP session values

 app.Get("/", func(c *fiber.Ctx) error {
     // retrieve the session
     sess, err1 := store.Get(c)
     if err1 != nil {
         return c.Status(500).SendString(err1.Error())
    }

    if sess.Fresh() {
        // new session
        sess.Set("accessCount", 1)
        sess.Set("firstAccess", time.Now().Format(time.ANSIC))
    } else {
        // increment the number of times we have hit this endpoint
       count := sess.Get("accessCount").(int)
       count++
       sess.Set("accessCount", count)
       sess.Set("lastAccess", time.Now().Format(time.ANSIC))
    }

    var sb strings.Builder

    sb.WriteString(fmt.Sprintf("Session: %s, new=%v\nSession values:\n", sess.ID(), sess.Fresh()))
    for _, k := range sess.Keys() {
        sb.WriteString(fmt.Sprintf("   %s=%v\n", k, sess.Get(k)))
    }

    if err1 = sess.Save(); err1 != nil {
        return c.Status(500).SendString(err1.Error())
    }
    return c.SendString(sb.String())
 })

Create a route for the /destroy endpoint, which will remove the session and finally, listen on port :2000.

    app.Get("/destroy", func(c *fiber.Ctx) error {
       // retrieve the session
       sess, err1 := store.Get(c)
       if err1 != nil {
           return c.Status(500).SendString(err1.Error())
       }
       id := sess.ID()

       // remove the session
       err1 = sess.Destroy()
       if err1 != nil {
           return c.Status(500).SendString(err1.Error())
        }
        return c.Status(200).SendString(fmt.Sprintf("session %v destroyed", id))
    })

    panic(app.Listen("127.0.0.1:2000"))
}

Run the Fiber application

Run the code example via executing go run main.go. You should see the following output indicating that fiber is up and running and the Coherence session is connected.

go run main.go 

2023/08/30 11:26:29 session: 29baf94f-efb5-4d65-b6f3-cb4c39f4d87a connected to address localhost:1408

 ┌───────────────────────────────────────────────────┐ 
 │                   Fiber v2.49.0                   │ 
 │               http://127.0.0.1:2000               │ 
 │                                                   │ 
 │ Handlers ............. 4  Processes ........... 1 │ 
 │ Prefork ....... Disabled  PID ............. 25016 │ 
 └───────────────────────────────────────────────────┘ 

Testing basic access

Open up a Web browser (Safari in my case), and go to the url http://127.0.0.1:2000. You should see something similar to the following indicating that a new session has been created.

Refresh the URL a couple of times and you will see that the session is no longer new and the accessCount has been updated.

Finally access the url http://127.0.0.1:2000/destroy which will destroy the session.

You can also wait for longer 120 seconds and then when you refresh the browser you will see that the session has expired and a new one has been created.

Testing failover

To show failover of sessions, we must first determine which Coherence node is storing the session so we can kill that node to simulate fail-over.

Normally we don’t care about knowing this as Coherence will create a primary and a backup on different nodes by default.

Making sure that you have refreshed the browser a few times to ensure your session is still valid for 120 seconds, issue the following two commands to query Coherence management to show you which node owns the data.

curl -s http://127.0.0.1:30000/management/coherence/cluster/caches/fiber%24default-store/members/1?tier=back | jq | egrep 'size'
curl -s http://127.0.0.1:30000/management/coherence/cluster/caches/fiber%24default-store/members/2?tier=back | jq | egrep 'size'

The output should be something similar to the following, which in our case means that member with Id=2 is holding the session.

curl -s http://127.0.0.1:30000/management/coherence/cluster/caches/fiber%24default-store/members/1?tier=back | jq | egrep 'size'
      "size": 0,
curl -s http://127.0.0.1:30000/management/coherence/cluster/caches/fiber%24default-store/members/2?tier=back | jq | egrep 'size'
      "size": 1,

We then need to determine which member is member Id=2, as it may be either of the coherence1 or coherence2 nodes. We issue the curl against member/2 path.

curl -s http://127.0.0.1:30000/management/coherence/cluster/members/2 | jq | grep machineName
  "machineName": "machine1",

Refresh your browser a few more times to ensure your session has not expired while carrying out the above.

In our case the container is machine1, so we issue a docker ps to find the container Id matching go-sessions-coherence1–1.

Issue the docker kill against the container you identified.

Refresh your browser again and you should see the accessCount has been incremented and the session is not new. This means that the session data failed over to the surviving Coherence node.

On the terminal where you started the main.go you may see the following message if the container you killed was the one that your gRPC session was connected to.

The fact that the gRPC connection goes to a particular container does not mean Coherence will store data there, Coherence balances the data across available storage-nodes.

2023/09/06 15:09:18 event stream recv failed: rpc error: code = Unavailable desc = error reading from server: EOF
2023/09/06 15:09:18 session: 9e4dd87e-1d7f-491d-a217-d2e536815987 disconnected from address localhost:1408
2023/09/06 15:09:19 session: 9e4dd87e-1d7f-491d-a217-d2e536815987 re-connected to address localhost:1408

Additional tests

To further explore the example, you can try the following:

1. After refreshing the URL a number of times, wait for longer than 120 seconds, the configured session timeout, and then access the URL again. You will see the the session is now new again (due to the session expiry), and the accessCount is back to one.
2. Open a second browser, say Firefox, and access the same URL. You will see that a different session is created due to the different browser. These sessions are independent from each other and stored accordingly in the Coherence cluster.

Conclusion

The above is only a simple example, but you can see how easy it is to store your HTTP session in Coherence to provide session failover and leverage the scalability, availability, reliability, and performance for in-memory session management and storage.

For more information see the following:

• Coherence storage driver for Fiber on GitHub
• Fiber on GitHub
• Coherence Go client on GitHub including extensive examples
• Learn about the Coherence data grid on the Coherence Community

Full code listing

For convenience, the full code listing is below.

Note: You can access the code here — https://github.com/tmiddlet2666/coherence-playground/tree/main/go/sessions

package main

import (
    "fmt"
    "github.com/gofiber/fiber/v2"
    "github.com/gofiber/fiber/v2/middleware/session"
    "github.com/gofiber/storage/coherence"
    "log"
    "strings"   
    "time"
)

var store *session.Store

func main() {
    // create new coherence session store using defaults of localhost:1408
    storage, err := coherence.New()
    if err != nil {
        log.Fatal("unable to connect to Coherence ", err)
    }
    defer storage.Close()

    // initialize the gofiber session store using the Coherence storage driver
    store = session.New(session.Config{
        Storage:    storage,
        Expiration: time.Duration(120) * time.Second,
    })

    app := fiber.New()

    app.Get("/", func(c *fiber.Ctx) error {
        // retrieve the session
        sess, err1 := store.Get(c)
        if err1 != nil {
            return c.Status(500).SendString(err1.Error())
         }

        if sess.Fresh() {
            // new session
            sess.Set("accessCount", 1)
            sess.Set("firstAccess", time.Now().Format(time.ANSIC))
        } else {
            // increment the number of times we have hit this endpoint
            count := sess.Get("accessCount").(int)
            count++
            sess.Set("accessCount", count)
            sess.Set("lastAccess", time.Now().Format(time.ANSIC))
        }

        var sb strings.Builder

        sb.WriteString(fmt.Sprintf("Session: %s, new=%v\nSession values:\n", sess.ID(), sess.Fresh()))
        for _, k := range sess.Keys() {
            sb.WriteString(fmt.Sprintf("   %s=%v\n", k, sess.Get(k)))
        }

        if err1 = sess.Save(); err1 != nil {
           return c.Status(500).SendString(err1.Error())
        }
        return c.SendString(sb.String()) 
   })

   app.Get("/destroy", func(c *fiber.Ctx) error {
       // retrieve the session
       sess, err1 := store.Get(c)
       if err1 != nil {
          return c.Status(500).SendString(err1.Error())
       }
       id := sess.ID()

       // remove the session
       err1 = sess.Destroy()
       if err1 != nil {
           return c.Status(500).SendString(err1.Error())
       }
       return c.Status(200).SendString(fmt.Sprintf("session %v destroyed", id))   
    })

    panic(app.Listen("127.0.0.1:2000"))
}