First, I would like to thank you for your amazing article.
Then, I was a bit in doubt what was your problem, exactly. Based on the article, you’re using 4 bare metals each one with 24 CPUs and 64GB of RAM to handle about 50K rpm which is a bit strange from my point of view.
Here in Betternet, using RoR over Heroku, we’re handling 15K rpm using only four 2X dynos (i.e. totally 4GB of RAM which cost 200$ per month). With a simple rule of thumb we can handle 50K rpm with much much less than the resources you’ve used in your system.
Did you try to optimize your web server (e.g. Puma, Passenger or .. )?
I should add that, about 70% of our requests are POST data and the other 30% are GET. We’re using PostgreSQL as our main storage.