[PoC8] Performance tuning on Cassandra & EC2 to store 3 millions/minute requests by retrieving from a Netty-based REST web service requests

I’ve already implemented a working PoC for storing 2 millions requests a minute on Cassandra but the final graph that was crafted from result numbers showed me that configuration behaves badly.

So while I was looking around to find a workaround, I came across with this awesome blog stating that : The biggest performance gain for write is to put commit log in a separate disk drive.

So I was convinced to add a new SSD to each node in my Cassandra Cluster. This means that I need to add another volume on EC2. To handle the creation of another volume, I added an SSD with Provisioned IOPS (see image below for details) while launching my instance. The IOPS was set to 600, I think I’ll try the max value 3000 with an upcoming PoC.

To make this newly added Amazon EBS volume available to the Linux installation, I needed to format it first and then mount it. Just follow the steps detailed at here to handle it. They’re well defined and precise. Kudos to Amazon Developer Team…

After adding the new volume, it was time to change the configuration in cassandra.yaml for having the commitlogs stored in my new volume. I stated my mount_point as /commitlog for the new drive, you can do the same for sure.

commitlog_directory: /commitlog

The rest for getting the Cassandra cluster up and running was same as it had been done in the previous try-out, so I tested the same scenario with the same EC2 instance setup. Results were %50 better than the previous try-out with an average of 50k/sec.

The final point is: While working with Cassandra Cluster, SSD’s should be tailored with regarding configurations.