Automation QA Toolbox 101: Cache

This series aims to explore the various commonly used tools in the tech environment and infrastructure with a focus of its application and use in the context of an automation and from an automation QA point of view.

Cache

Cache can be seen as an alternate memory space to traditional databases that provides a quicker, faster and more efficient way of accessing stored data

The diagram above shows one of the many examples of using a cache in accessing stored data which not only reduces the processing load on the database, but also speeds up the overall performance of the service as information can be retrieved quicker and with higher stability with increasing “queries-per-second”. Many different variations can be branched out or adapted from using cache which can easily fit your needed use case.

Why is it important for Automation QA

With the many advantages of using a cache and proper management of its disadvantages, it is a popular option for many services and APIs which then also becomes a testing point for QAs downstream.

Knowing how to use a cache thus becomes an important skillset as an automation QA who has to ensure that he is writing the daily automated tests accurately and correctly.

As such we will be going through how to manipulate a redis cache which is also known for its data-type specificstorage as compared to other types of cache.

Simple Example of using Cache

To picture how cache works, just think of a normal key-value storage such as a has Dictionary or Hashmap. So we first start by defining how we will be connecting to a redis storage space with a specific port number for our specific cache via TCP.

import "github.com/go-redis/redis/v7"
func main() {
    redisOption := &redis.Options{
        Network: "tcp",
        Addr:    redisAddr,
        }
    redisCli := redis.NewClient(redisOption)

We do a simple verification that our connection to the cache is valid and working with a ping-pong test.

    pong, err := redisCli.Ping().Result()
    if err != nil || pong != "PONG" {
        panic(err)
    }

Now let’s paint the context of our example here. Imagine we have this cache which stores a simple diary log of your activities under the key date|hour|minutes . Let us store some diary logs in our cache.

    err = redisCli.Set("29062021|12|30", "Neko keeps meowing for food, feed.", 0).Err()
    if err != nil {
	panic(err)
    }
    err = redisCli.Set("29062021|15|0", "Neko shattered my glass cup", 0).Err()
    if err != nil {
	panic(err)
    }
    err = redisCli.Set("29062021|5|30", "Awoken by Neko jumping on me", 0).Err()
    if err != nil {
	panic(err)
    }

With that, we successfully stored 3 beautiful memories with the house cat Nekoon 29th June 2021:

• Feed pet cat Neko at 1230
• Neko knocked my favourite glass cup off the table and broke it at 1500
• Neko jumped on me while I was sleeping at 0530

Having stored them in the cache, we would also need a way to retrieve and relieve these wonderful moments with Neko. For example, the time when she broke your favourite cup 😇

activity, err := redisCli.Get("29062021|15|0").Result()
	if err != nil {
	    panic(err)
	}
	fmt.Println(activity)//"Neko shattered my glass cup"

Complete Code Example

import "github.com/go-redis/redis/v7"
func main() {
    redisOption := &redis.Options{
        Network: "tcp",
        Addr:    redisAddr,
        }
    redisCli := redis.NewClient(redisOption)
    // Store data in Cache
    err = redisCli.Set("29062021|12|30", "Neko keeps meowing for food, feed.", 0).Err()
    if err != nil {
	panic(err)
    }
    err = redisCli.Set("29062021|15|0", "Neko shattered my glass cup", 0).Err()
    if err != nil {
	panic(err)
    }
    err = redisCli.Set("29062021|5|30", "Awoken by Neko jumping on me", 0).Err()
    if err != nil {
	panic(err)
    }
    // Retrieve Data in Cache
    activity, err := redisCli.Get("29062021|15|0").Result()
	if err != nil {
	    panic(err)
	}
	fmt.Println(activity) //"Neko shattered my glass cup"

Versatility of using Cache

Now the dirry is all great except but as much as I wish to, life does not just revolve around cats right? What if there is a need to store more complex information that goes beyond just a simple string??

type DiaryLog struct {
    Event            string
    EventCategory    int32
    Location         string
    ActionTaken      string
    Cost             int32
    Troublesome      bool
}

Next let’s include some helper functions to better organise our code.

func GetKey(date, hour, minute int) string {
    return fmt.Sprintf("%v|%v|%v", date, hour, minute)
}

To store the struct in our cache, we first massage it into a byte array (instead of a string as above) through a marshaling step before storing them in our cache.

func main() {
    log := &DiaryLog{
        Event:          "Neko shattered my glass cup"
        EventCategory:  int32(19)
        Location:       "home"
        ActionTaken:    "Forgive Neko, clean broken pieces"
        Cost:           int32(56)
        Troublesome:    true
    }
    key := GetKey(29062021, 15, 0)
    
    logByte, err := proto.Marshal(log)
    if err != nil {
        panic(err) 
    }
    err = redisCli.Set(key, logByte, 0).Err()
    if err != nil {
	panic(err)
    }

To retrieve and read our data, we simply do the same 2 steps, but in reverse!

    memory := &DiaryLog{}
    dataArray, err := redisCli.Get(key).Bytes()
    if err != nil {
	panic(err)
    }
    err = proto.Unmarshal(dataArray, memory)
        if err != nil {
	panic(err)
    }
    fmt.Println(memory)

See the similarities in the process flow? Exactly. If you realise, we are only changing the type of data we store here from a string, to a byte array representing our given struct.

Complete Code Example

type DiaryLog struct {
    Event            string
    EventCategory    int32
    Location         string
    ActionTaken      string
    Cost             int32
    Troublesome      bool
}
func GetKey(date, hour, minute int) string {
    return fmt.Sprintf("%v|%v|%v", date, hour, minute)
}
func main() {
    log := &DiaryLog{
        Event:          "Neko shattered my glass cup"
        EventCategory:  int32(19)
        Location:       "home"
        ActionTaken:    "Forgive Neko, clean broken pieces"
        Cost:           int32(56)
        Troublesome:    true
    }
    key := GetKey(29062021, 15, 0)
    //Marshal data into a byte array
    logByte, err := proto.Marshal(log)
    if err != nil {
        panic(err) 
    }
    //Store in Cache
    err = redisCli.Set(key, logByte, 0).Err()
    if err != nil {
	panic(err)
    }
    memory := &DiaryLog{}.  
    //Get byte array data from cache
    dataArray, err := redisCli.Get(key).Bytes()
    if err != nil {
	panic(err)
    }
    //Unmarshal data to defined struct
    err = proto.Unmarshal(dataArray, memory)
        if err != nil {
	panic(err)
    }
    //Profit
    fmt.Println(memory)

Troubleshooting

One issue one might have with cache is accessibility. Sometimes, we just need an alternate way to do things aside from code. Fret not, for similarly, there are also many available tools to access cache. One of which that is most readily available is the redis-cli through our terminals. We simply access the cache by typing the command which specifies the host and port of our cache

redis-cli  -h <ipaddress> -p <port>

• if value is of type string → GET <key>
• if value is of type hash → HGETALL <key>
• if value is of type lists → LRANGE <key> <start> <end>
• if value is of type sets → SMEMBERS <key>
• if value is of type sorted sets →ZRANGEBYSCORE <key> <min> <max>

If you aren’t sure of your data type, simply use the command type <key> to find out.

Here we discussed the usage of a Redis Cache through Golang which can be easily adapted to your use case with the many other libraries available and also other types of cache, that follows a similar conceptual flow but with different context, and specifications and syntax.

Check out another popular tool, ETCd, which we often encounter during testing introduced by Luohua here!

~ Happy testing ~ Stay tuned to the next toolbox discussion~*