S4 - A modern open-source crypto library

At 4th-A Technologies we strongly believe in removing the barrier of trust — you shouldn’t need to trust us, you should be able verify things for yourself. It’s one of our founding principles.

It’s one thing to publicly state (and we have) there are no back doors or deliberate design weakness in our products, but it’s a whole other thing to back up this claim. I have found that transparency is a successful strategy. Let people openly know what you do and how you do it. With that in mind, we did the following:

1. We published a Storm4 Cryptography Overview white paper detailing exactly how we secure your data in Storm4.cloud and why we made the choices we did.
2. We made the S4 crypto library that we use in our products available for peer review inspection.
3. And since we found S4 so useful, we even released it open-source so it could be used by others.

Yes, we designed our own crypto library and used some of the best accepted algorithms in doing so. Let me be clear, we did not roll our own untested crypto algorithms. We only use widely accepted and tested algorithms. And, yes, this is not a task for amateurs. We recognize that, but amateurs we are not. I personally have been involved in building crypto libraries for use by third party developers for about 25 years and have seen some good, bad and ugly stuff.

In the end we wanted something that not only we could use, but that others could benefit from as well. So we built S4 on top of some great crypto code. A lot of the basic crypto comes from libtomcrypt, The same library Apple used in it’s crypto implementation. We even included some optimizations to take advantage of the hardware accelerated cryptographic hash functions on Apple’s platforms.

Additionally we added code from the SKEIN function library, which we found to be some of the fastest crypto out there, as well as the Bernstein/Lange Eliptical curves. We also added code to do Shamir’s Secret Sharing.

But that’s not all… The best part is we added an entire series of APIs to do management, operations on higher level key constructs and serialize and deserialize them in a JSON compatible form.

API Consistency and Simplicity

We designed the S4 programmer interface to be consistent and lead to no unpleasant surprises. Once you have invested the time to learn one set of operations, the learning curve for the others should be as flat as possible. I have seen so many cases of where a complicated API has led to vulnerable code, and we didn’t want to fall for the same mistake.

Our experience has also shown us that crypto algorithms can become obsolete and replaced as both computing power increase and attacks become known. So, S4 allows for the easy addition of new algorithms without having to modify a lot of the client code. It really bugs me when a library has different entry points for each algorithm. Something as simple as passing in an algorithm descriptor with a uniformed series of APIs goes a long way.

Portability

We wrote the S4 library in standard C11/C99 so that we could port it to a variety of platforms. We want to keep the crypto functionality modularized in our product so it’s consistent across all our software and we can standardize the unit tests.

The other cool thing is that our S4 API makes it easy to build wrappers for object oriented languages such as Rust, Go, Swift or node.js. In fact, some students recently ported it to JavaScript and Python. I have even done an experimental port to the Atmel Atmega platform used by Arduino environment. By the way, if you are so inclined to make a wrapper, feel free to contribute to the open source project.

Safety of Critical Security Parameters

The following is doing to get pretty geeky, so try and keep up. One of the things I learned from building crypto libraries in the past, is that even if you never intend to undergo the FIPS-140 validation process, you should still build them using the NIST Cryptographic Module Validation Program (CMVP) guidelines. Especially the ones that pertain to what NIST calls Critical Security Parameters (CSPs), or what most normal people call secret cryptographic keys, authentication data and passwords.

This means among other things that once key material is input into the library, it should never leave the library unencrypted. That all such key material be securely erased once they are no longer needed, else they are subject to attack. While this design criteria originates from early computer operating systems and a lot has been done in modern runtime systems to alleviate the problem, it’s still a valid way to mitigate many forms of attack. Sounds simple, doesn’t it? But you’d be surprised how many crypto libraries out there don’t employ defensive programming patterns.

Testability

Another thing that came of out the FIPS-140 experience was the need to have comprehensive unit and operational tests. NIST has specific requirements on how this should be done depending on the type of algorithm. Besides calling each API entry point, the operational test must include known-answer tests for cryptographic algorithms, random number generation tests in addition to pair-wise consistency test for public key algorithms. All of this to make sure that the code does what it is advertised to do. So we added that code to S4 too. It doubles as sample code demonstrating the API calls.

Modern algorithms

We chose to only support the algorithms and modes that we believe have been proven to be more resilient to attack. For example we picked up the ECC Curve41417. We did not however export the older RSA algorithms. If someone really needs an algorithm we don’t support, the library can easily be forked and they can submit a pull request.

So what can I do with S4?

We started off with a series of low level cryptographic functions. I call them “low level” because you are responsible for managing the key material. They work at the byte or block level — there is no protocol or packet specified. There are APIs to perform the following functions:

• Random number generation for both bytes and passphrases.
• Cryptographic Hash (SHA1, 224, 256, 384, 512, 512/256, SKEIN-256, 512, 1024)
• Symmetric Encryption Algorithms (AES-128,192,256 and TwoFish 256) in Electronic Codebook (ECB), Cipher Block Chaining (CBC) and CBC with padding mode.
• Message Authentication Code with above hashes and algorithms
• Tweekable block cipher (Threefish)
• Low Level Elliptic Curve (ECC-384 and Bernstien/Lange Curve41417)
• Password to key encoding (PBKDF-2)

I mentioned above that we also include a series of high level APIs to take cryptographic keys and convert them from/to JSON. We use these APIs extensively ourselves to do some of the cool things like how we bind properties and ownership to the keys. We talk about this in our Storm4 Cryptography Overview.

The high level APIs include the ability to:

• Create high level key objects (symmetric, tweekable and ECC)
• Securely free key objects
• Set, get, remove and list key properties
• Sign keys and verify signatures
• Serialize secret key objects to JSON, encrypted to pass-phrases, symmetric key or even to other public keys
• Serialize public keys and their properties to JSON
• Split keys into shares and recombine them

So how does this work?

Let’s take a simple example, a low level cryptographic hash of some data.

Before you attempt to use S4, you need to call the S4_Init entry point. Do this once to set things up depending on your platform and how you built S4.

Then you create a Hash object using the HASH_Init API. This encapsulates all the internal data and you get back an internal object pointer which you can pass to subsequent API calls. For example, the HASH_Update, which feeds bytes to the hash function and then the HASH_Final to get the result. When you are done with the function use the HASH_Free to securly free up all internal data .

err = S4_Init();

err = HASH_Init(kHASH_Algorithm_SHA256, &ctx);

err =HASH_Update(ctx, data, dataLen);

err = HASH_Final(ctx, hashBuf);

HASH_Free(ctx);

or another example using the tweakable block cipher (threeFish) functions.

TBC_Init(kCipher_Algorithm_3FISH512, key &ctx);

TBC_Encrypt(ctx, dataIn, dataOut);

TBC_Free(ctx);

You can see that the design pattern is consistent across the entire API whether you are doing hashing, message authentication code, encryption or password encoding (PDKDF2). This point of this design is that:

• Internal context is always initialized consistently
• Critical Security Parameters are always encapsulated by the library and can only be accessed by proper API calls
• Any internal data is securely zeroized and freed with a corresponding API call.

Why does any of this matter?

You might ask, why all the jibber jabber about API design? There are lots of crypto libraries out there and some are shipped with my favorite operating system? Good question. What we keep learning is that historically it’s not the crypto algorithms that are vulnerable to attack but rather the higher level code. For example, people haven't been very successful attacking the Bitcoin protocol, they attack the wallets. In the end we wanted to build a library that was as bullet proof as possible.

More to come.

S4 is a modern crypto library that is open source and cross-platform. We didn’t just build it for us, we built it for everyone. We will continue to develop S4 by adding more functionality, more platforms and more language wrappers.

See you on GitHub.