Benchmark FoundationDB with Go YCSB

FoundationDB is a very famous distributed database, after it is re-opened again, it quickly gains lots of attention. I also begin to study it at once, then I want to do a benchmark for it.

Although there is an official benchmarking , I don’t know the real benchmark cases. So I think we can use a standard way to do it by myself, and that is YCSB.

Why YCSB

YCSB is a benchmark tool developed by Yahoo. It provides a standard way to let the user evaluate the system on common workloads and focus on performance and scale out. Now, many systems have been benchmarked with YCSB, including our product TiDB and TiKV.

What you only need to do is to define your own workload like “99% read, 1% insert”, or “50% read, 50% update”, provide a request generation algorithm like “zipfian”, “latest”, or “uniform”, then YCSB will help you do the benchmark. And the core workload is suitable for the online transaction processing database.

Here is a simple YCSB workload:

recordcount=1000
operationcount=1000
workload=core

readproportion=0.5
updateproportion=0.5
scanproportion=0
insertproportion=0

requestdistribution=zipfian

Here we use the “core” workload, do 1000 operations (50% read, 50% update) We will use “zipfian” to choose the record needs to be operated in distributed.

FoundationDB Go YCSB

YCSB is easy to use, but it is written in Java and we have no Java API for TiKV, so I port YCSB to Go and build the go-ycsb. So here I will use go-ycsb to benchmark FoundationDB.

At first, we need to install Foundation C client, then we can use the FoundationDB Go client. This is not convenient and may be anti common Go way, because we can’t use go get github.com/apple/foundationdb/bindings/go/src/fdb directly. Hope the FoundationDB team can improve this later :-)

Then we can implement the YCSB DB interface for FoundationDB. For example, the Read function is:

Here we start a transaction with Transact and get a row from FoundationDB, if the row exists, decode the row and return to the data with a map of field->value. The whole source is here, you can see other operation implementations like Insert, Scan, Update and Delete.

Deployment

After I support FoundationDB in go-ycsb, the next thing is to deploy the FoundationDB and benchmark it. To my surprise, the deployment for FoundationDB is not easy, I even think this is a nightmare for me.

When I install FoundationDB, I find that the FoundationDB installs the data to the default directory (For linux, it is /var/lib/foundationdb/data), and the default data is stored on a HDD on my machine. So I have to:

• Stop the FoundationDB service.
• Move the data to a high-performance disk. In my machine, the disk is Optane, very very fast.
• Change the path in the config file /etc/foundationdb/foundationdb.conf.
• Start the service again.

As you can see, the operation is horrible, I have to complain it. :sob:

Benchmark

Here is the base workload configuration:

recordcount=1000000
operationcount=1000000
workload=core
fieldcount=1
fieldlength=16
requestdistribution=uniform
threads=10

We will use “uniform” to operate 1 million records. The concurrency is 10 by default, and below is the benchmark result with different workloads:

This is the benchmark result for one FoundationDB, I don’t do the benchmark for the FoundationDB cluster, because I’m tired of the deployment.

As you can see, the performance is cool, and what strikes me most is FoundationDB only uses very few CPU (<60% in my benchmark). But the result is different from the official benchmarking (the read performance is poor in YCSB), maybe I will investigate it later.

Epilogue

Now, benchmarking different systems becomes one of the most favorite thing in my life, I have been improving go-ycsb and let it support many systems like PostgreSQL, etc. If you have the common interest, welcome for contribution. you can email me (tl@pingcap.com) directly.