Secure Types | Memory Safety with Go

Published in

Trendyol Tech

13 min readAug 24, 2020

Have you ever considered about how anti-cheating systems work? How do hackers analyze and alter memory? We always store sensitive informations just behind primitive data types in the memory. But, we should not. If you are directly storing important informations (score, time, secret key, user informations, etc.) inside the primitive data types (int, string, etc.), then you do not do things right. Especially, if you are working on a mobile or production apps, we genuinely do not want to do this way.

Well, how do we secure that? How do we guarantee that primitive data in the memory will not change by hackers or hacking toys, never? How are we going to know if someone changes our living values somehow? What shall we do?

In this article, we will touch the in-memory primitive data type security, what is memory manipulation, and how memory data breaches happens. The main point of this writing this article is that to show how can be secure our primitive data types from attackers.

Problem

In our living software, we may have to store upmost important data values in memory. We can use and manipulate them during at run-time, as usual. What about security? Can’t someone who have any hacking toys, come and change those values? Of course, they can. Should we just let them change those? Of course, we don’t. Then, what?

OK, let’s take a look at what we going to up against to.

Reproducing the Memory Manipulation

This is the most important part. The purpose of writing this article. To show of how the memory violation happens. The important measure we should take in just before upcoming cyberpunk-ed years.

Step 1: To reproduce memory manipulation, first-thing-first we will create a bare-minimum app implementation from scratch. In order to achieve that, we are going to make a simple CLI application. The key point is that there should be at least critical-role and cheating-sensitive Integer variables. (I.E. Total Balance, Timer, Score, A secret value from config server, etc…)

In this step, we will do some gamification. In order to demonstrate how memory data security important is, we have to implement reproducible logic. Eventually, we will not have a hyper-casual game of course.

Game: Reach 100 score as fast as possible. Remember: Time is counting. Each after 10 score, you will gain 1 coin.
Rules: No rules included.
Keys: [i: increase the score, d: decrease the score, q: quit the game]

Step 2: How do we violate the memory? Simply, by analyzing it by hand with some crazy toys. You might say however, how?

In this step, I will continue with Cheat Engine. The one of the main reason I chose this is because it is free and open source. Moreover, it has advanced memory scanner and debugger. Furthermore, it is still being actively developed. That’s why I have been using this toy almost 5+ years.

Step 3: Time to manipulate a memory address of an integer or any data type. We will touch Score and Coin data values, which is already defined in our simple game. In order to get pointer’s address, we have to enter an Integer value at the Value field and click the New Scan button.

As you can see below, we have manipulated the value.

An example CLI app, `go run examples/games/secure/game_secure.go`

Introduction

Before deep dive into about to solution, we need to take a look at some of the basic things about memory and data storing.

Primitive Data Types

Since Go strongly statically-typed non-interpreted language, we can either define the variables with data types, as in mostly programming languages. That’s the key point. What if… If we give them brand-new features like in-memory-encrypted, and make them no more primitive?

The data types in Go can be classified as follows [0]:

Boolean: (true or false)
Numeric: (int: 8, 16, 32, 64 Bit (un)signed; float: 32 and 64 Bit signed; complex: 64 and 128 Bit)
String: (sequence of bytes)
Derived: (pointer, struct, array, union, function, slice, interface, map, ch)

So… Ain’t no problem that an XOR operator to data type value won’t fix.

Solution

To start with, let’s take a brief look at what an XOR (⊕) operator does exactly.

Exclusive or or exclusive disjunction is a logical operation that outputs true only when inputs differ (one is true, the other is false). [1]

What we are going to build by using XOR is that an XOR cipher. So, there are supposed to be two binary sets (one for the real value, one for the encryption key value) sitting on the a struct (our brand new non-primitive data type) when the a living program executes it. For example:

0000 1110: 14 (real value)
0100 0010: 66 (encryption key)
— — ⊕ — —
0100 1100: 76 (encrypted real value)

We set all of the binaries to 1 when inputs differ. And please note that, the encryption key (which is our Cryptographic Key) will be our decryption key at the same time. By doing this operation, we securely stored our real value 14 as decrypted value 76 in the memory. [?] Let’s get back the value from fake value as real value:

0100 1100: 76 (real value)
0100 0010: 66 (decryption key)
— — ⊕ — —
0000 1110: 14 (decrypted real value)

The math that’s being calculated in both cases is intentionally simple. Let’s start to implement this.

Implementing

What I want to mention that before start is that we will implement this for just Integer and String data types in order to keep the article short and neat. You can use the whatever language you want during the implementation. In this article, we are going to with go with Go.

Let’s head on over to how our brand-new Integer model will be.

0. Concept

At the end of Implementing chapter we want to be able to:

Obscuring (which means encrypting and decrypting) real value
Randomizing the XOR key
Setting, incrementing and decrementing the value
Make sure that as fast as primitive data type when doing operations
Make sure that the all functions works properly and covered by tests
Make sure that there will be no consistency problems

1. Struct Model

In the our custom brand-new SecureInt struct model, we have to store some important variables:

key: cryptographic key for data type
realValue: a data store for XOR-ed value for real value
fakeValue: a fake data store for detecting cheating attempts
initialized: before doing any operations on it, we have to check either if initialized

type SecureInt struct { 
  key         int 
  realValue   int 
  fakeValue   int 
  initialized bool 
}

Our function interface model will look like this:

type ISecureInt interface {
  Apply() ISecureInt
  SetKey(int)
  Inc() ISecureInt
  Dec() ISecureInt
  Set(int) ISecureInt
  Get() int
  GetSelf() *SecureInt
  Decrypt() int
  RandomizeKey()
  IsEquals(ISecureInt) bool
}

As you have already noticed, we have followed Fluent Interface design pattern. Additionally, to keep things single statement without requiring variables, we have applied Method Chaining to these functions: Apply(), Inc(), Dec(), Set()

In this way, we can able to build function chains:

int := secure.NewInt(7).Set(16).Inc().Dec().Dec().Get()

2. Initializing

It is simple to construct our SecureInt struct with default values and return the Interface model.

func NewInt(value int) ISecureInt {
  s := &SecureInt{
    key: KEY,
    RealValue: value,
    fakeValue: value,
    Initialized: false,
  }  s.Apply()  return s
}

The KEY will be the default initialization value: const KEY int = 7153234

The Apply() function will do some encryption things; we will touch this function below.

3. XORing

As we said at the beginning of the article, our goal is to make this piece of
nitty-gritty XOR function work and keep as simple as possible. It’s only job is to XOR value with the key and return the result:

func (i *SecureInt) XOR(value int, key int) int {
  return value ^ key
}

And, surprisingly enough, there’s nothing that speaks against this pragmatic solution. It’s actually quite elegant.

And, we may want to change our key to another new one.

func (i *SecureInt) SetKey(key int) {
  i.key = key
}

Even, we may have to randomize our key between state changes or an interval period, in case of the attacker know what the key is or if somehow found it, which makes this function critically important to use.

func (i *SecureInt) RandomizeKey() {
  rand.Seed(time.Now().UnixNano())  i.RealValue = i.Decrypt()
  i.key = rand.Intn(int(^uint(0) >> 1))
  i.RealValue = i.XOR(i.RealValue, i.key)
}

4. Encrypting & Decrypting

The compiler needs to encrypt and load the value as soon as initialized on runtime. That’s why we was used Apply() function at the NewInt() constructor.

Remember that the initialized of SecureInt has only one purpose. We do not want to encrypt the realValue again if it already encrypted. Therefore, we have to check whether initialized.

func (i *SecureInt) Apply() ISecureInt {
  if !i.Initialized {
    i.RealValue = i.XOR(i.RealValue, i.key)
    i.Initialized = true
  }
  return i
}

In order to get real realValue, we can just XOR realValue itself by the key.

func (i *SecureInt) Decrypt() int {
  if !i.Initialized {
    i.Initialized = false
    i.fakeValue = 0
    i.RealValue = i.XOR(0, 0)
    i.key = KEY
    return 0
  }
  return i.XOR(i.RealValue, i.key)
}

Let’s create a new function called Get() and call this function in it.

func (i *SecureInt) Get() int {
  return i.Decrypt()
}

And we may think of expose the SecureInt struct itself, in order to access unexported fields and functions, for testing and debugging purposes.

func (i *SecureInt) GetSelf() *SecureInt {
  return i
}

5. Core Functions

In order to increment and decrement the value, we will first have to decrypt the value, apply the required operations (inc, dec, etc.) and encrypt the value, which is piece of cake for us to do that.

func (i *SecureInt) Inc() ISecureInt {
  i.RealValue = i.XOR(i.Decrypt()+1, i.key)
  return i
}func (i *SecureInt) Dec() ISecureInt {
  i.RealValue = i.XOR(i.Decrypt()-1, i.key)
  return i
}

Since Go does not have implict operator overloading for canonical implementations to arithmetic operators like void operator ++() , we have to implement two functions: Inc() for ++ and Dec() for --.

And we should not forget about IsEquals() function. Remember, we can simply use foo == bar , If we do it this way, we just compare the memory addresses of two structs, not the values.

func (i *SecureInt) IsEquals(o ISecureInt) bool {
  if i.key != o.GetSelf().key {
    return i.XOR(i.RealValue, i.key) == i.XOR(o.GetSelf().RealValue, o.GetSelf().key)
  }  return i.RealValue == o.GetSelf().RealValue
}

Bonus: String Encryption & Decryption

So far, so good. We have learned how to obscure our Integers from attackers. Well, what about Strings? Maybe the most important data store type for our apps; we may be storing important data on them, just like: passwords, secret keys, etc. That’s why I want to touch this data type.

As we know String is, traditionally a sequence of characters. Accordingly, we will use the runes (a.k.a. int32) in order to store our Unicode string chars. Eventually, SecureString struct model will look like this:

type SecureString struct {
   Key         int
   RealValue   []rune
   fakeValue   string
   Initialized bool
}

Accessing a string one rune at a time is instead much simpler when iterating from the beginning of it, that’s why I do not prefer byte instead.

So, XORing will be as simple as SecureInt’s.

func (i *SecureString) XOR(value []rune, key int) []rune {
   res := make([]rune, len(value))

   for i, v := range value {
      res[i] = v ^ int32(key)
   }

   return res
}

That’s all. We XORed entire string value, char by char. In addition, we could encrypt each char with random key. Maybe like this: v ^ int32(key * i)

I think we did what we trying to achieve. We can write an example case to test all functions.

func ExampleInt()  {
   lhs := secure.NewInt(15)
   lhs.Inc()
   lhs.Inc()
   lhs.Inc()
   lhs.Dec()

   println("LHS: ", lhs.Get())

   rhs := secure.NewInt(99)
   rhs.Set(18)
   rhs.Dec()

   println("RHS: ", rhs.Get())

   println("LHS == RHS: ", lhs.IsEquals(rhs))
}

func ExampleString()  {
   lhs := secure.NewString("foo")
   lhs.Set("foo 2")

   println("LHS: ", lhs.Get())

   rhs := secure.NewString("bar")
   rhs.Set("foo 2")

   println("RHS: ", rhs.Get())

   println("LHS == RHS: ", lhs.IsEquals(rhs))
}

Output:

LHS: 17
RHS: 17
LHS == RHS: trueLHS: foo 2
RHS: foo 2
LHS == RHS: true

Bonus: Memory Hack Detecting

If you haven’t noticed yet by now, we kept the fakeValue’s value as 0 from the beginning at the article, for a special usage: The memory hacking attempts.

It is quite straightforward thing to understand how the detect memory hacking attempts: if we want to detect memory hack attempts, we have to keep both of encrypted (realValue) and non-encrypted (fakeValue) values, in order to compare fakeValue with a just-decrypted-value from inside of each function.

Well, how will we know that whether hack detect condition triggered? We can implement suitable design pattern for this. Either an Event Listener or Observer Pattern is eligible for us. Let’s get implemented for both of SecureInt and SecureString.

type Observer interface {
   Update(value string)
}

func CreateWatcher(name string) *Watcher {
   return &Watcher{Name: name}
}

type Watcher struct {
   Name string
}

func (c *Watcher) Update(value string) {
   println("An event occurred: ", value)
}

We will use Observer model in the Observable. Because, we will attach this (Observable) model to SecureInt and SecureString data types. In order to trigger whole observers, we have to call NotifyAll() inside our hack-detect logics.

type Observable struct {
   Observers []Observer
   Notifies   []time.Time
}

func (o *Observable) AddObserver(os Observer) {
   o.Observers = append(o.Observers, os)
}func (o *Observable) HasObservers() bool {
   return len(o.Observers) > 0
}func (o *Observable) NotifyAll(value string) {
   o.Notifies = append(o.Notifies, time.Now())
   
   for _, ob := range o.Observers {
      ob.Update(value)
   }
}

Let’s add this to our struct models. So, we need to know whether a watcher is attached, if so, HackDetecting should show us whether it is active or not.

type SecureInt struct {
   Observable
   ...
   HackDetecting bool
}

And we will need a function, in order to attach our watchers and set the HackDetecting bool field.

type ISecureInt interface {
   Apply() ISecureInt
   AddWatcher(obs obs.Observer)
   ...
}

Adding the AddWatcher(obs.Observer) function:

func (i *SecureInt) AddWatcher(obs obs.Observer) {
   i.AddObserver(obs)
   i.HackDetecting = true
}

And we have reached to the utmost critical part:

Detecting Hacking Attempts

What if we check equality between fakeValue and decrypted value in the Decrypt() function? Congrats, you have got a superior hack detecting logic.

func (i *SecureInt) Decrypt() int {
   // ...

   res := i.XOR(i.RealValue, i.Key)

   if i.HackDetecting && res != i.fakeValue {
      i.NotifyAll(fmt.Sprintf("hack attempt: %d", i.fakeValue))
   }

   return res
}

However, if you forget to set fakeValue to current real value just after new value assigns, which will cause the function to work with consistency issues. So, we do not want false-positives. We have to add set fakeValue to current new value at the Inc(), Dec(), Set() functions. Do not worry about that value being changed, we are still using the encrypted realValue, fakeValue is just a hoodwink thing. So, we have to refactor our functions a bit.

Inc()

func (i *SecureInt) Inc() ISecureInt {
   next := i.Decrypt() + 1

   i.RealValue = i.XOR(next, i.Key)

   if i.HackDetecting {
      i.fakeValue = next
   }

   return i
}

Dec()

func (i *SecureInt) Dec() ISecureInt {
   next := i.Decrypt() - 1

   i.RealValue = i.XOR(i.Decrypt()-1, i.Key)

   if i.HackDetecting {
      i.fakeValue = next
   }

   return i
}

Set()

func (i *SecureInt) Set(value int) ISecureInt {
   i.RealValue = i.XOR(value, i.Key)

   if i.HackDetecting {
      i.fakeValue = value
   }

   return i
}

Let’s test out our simple game with Cheat Engine again:

As you will see, we achieved something like:

It’s smooth sailing from here on out.

Recap

We learnt these:

What Memory Safety means
How attackers manipulating sensitive data values
How to create our custom primitive type
How to secure our data struct models
How to detect memory manipulation attempts from attackers

Conclusion

CRITIAL: If attacker, know about anything in Reverse Engineering, it’s slam, blam, sayonara cofferdam. Maybe, as a temporary solution, we can try out VMProtect or Themida like in-memory virtual machine tools. But, at the end of the day, a stubborn attacker will be remove or change all security barriers (a.k.a. instruction sets) by doing some Debugging and Decompiling things on your living whole codebase at the memory. However, doing some Obfuscation stuff on our compiled executable will make things even more difficult to reverse. Eventually, a great anti-tampering, million-dollar DRM software company Denuvo is cracking by hackers.
IMPORTANT: A very significant memory security violation can occur: Memory Dumping. In this case, if somehow an attacker find our secret encryption key, however, our memory security barrier will be in danger. An attacker who tries this technique, which means, already knows that we are using an encrypted memory. Actually, that’s why we have added a RandomizeKey() function. We have to randomize our key between in intervals, in order to prevent from memory dumping attacks.
If we want to use this technique in our productions servers, mobile apps and tools; we have to trust the consistencies, which means that there should not have any consistency issues, which is why we have added unit tests for each case.
Benchmark results are not bad according to primitive types. We do not make neither Memory Allocation nor heavy-math operations. If we run a benchmark which contains just Integer operations, you will see 1000000000 op/sec. That’s because we missed something. In order to get closer to real world values, we have to run our benchmarks without Inlining Optimizations.

If we run:go test ./… -v -bench=. -gcflags=”-N -l”

We will get these outputs:

Benchmark_PrimitiveInt    483072136 2.42 ns/op 0 B/op 0 allocs/op
Benchmark_SecureInt        48972958 24.1 ns/op 0 B/op 0 allocs/opBenchmark_PrimitiveString 457374873 2.66 ns/op 0 B/op 0 allocs/op
Benchmark_SecureString      5486892  218 ns/op 48B/op 3 allocs/op

Results (Approximately):