How We Built A High-Performance Proxy In 1 Month

We are always excited to release a project and deliver it into production, to watch it grow and prosper. Sharing is caring — which is why we want to give you some insights into the projects we’re working on.

Today, we’d like to introduce you to FeedProxy.

A high-performance C++ proxy that occupies less than one CPU core and 6 MB of RAM to process 1500 HTTP requests per second. We built it in 1 month (1 man-month, to be more precise) and it’s been operating in production for 2 months, on 12 high powered bare metal servers in the Middle East.

The purpose of the proxy is to help an ad network integrate with others, when the latency of the partner ad networks is variable. The ad network using the proxy can then make performance guarantees, to clients injecting its ads, about its response time. Regardless of whether the other ad networks are down or having latency issues.

What was the process behind building such a feat of engineering in just 1 man-month? Here are a few key elements.

1. Our well designed development process. Based on a roadmap, weekly releases and estimates for each task, we’ve refined this process over the years and battle-tested it on teams with over 30 engineers working on the same project.
2. Our expertise in measuring and improving performance. Collectively, as a team, we’ve worked on websites totalling tens of millions of visitors per month, learning valuable lessons at scale. We’ve operated on clusters of upwards of 100 servers. In this particular case, we used a benchmark running on AWS (Amazon Web Services for Cloud Computing) that was generating synthetic traffic, in conjunction with well established profiling techniques — such as stack trace sampling.
3. Our constant communication with the customer. Always modifying the roadmap based on the feedback we received. In this particular case, we asked the customer to produce some production logs in order to understand the real production scale and the dynamics of the backend systems the feed proxy was being integrated with.

So, what did we learn from this project?

1. Always compare with the open source ecosystem. At first, we weren’t sure if we could deliver a proxy that operated within the performance constraints we had. So we took a stab at other open source projects, such as nginx, to see if they were able to process the amount of traffic we wanted, in the given resource constraints. That’s how we learnt it was possible.
2. Leverage open source to the maximum. We were afraid that there weren’t any good frameworks to build such a tool on. That’s until we started playing with Boost. Needless to say, we were very pleasantly impressed.
3. Start with a small deliverable. We do this in order to understand the bigger project better. The client approached us to build the proxy together, but we suggested to build the load testing tool first, to understand the problem and the traffic patterns better. For a fraction of the cost of the proxy, we were able to gain plenty of insight about what we had to do.
4. Start with load tests early on, when designing a high-performance system, to determine how it would perform under normal and peak usage conditions. Ideally, one should have a one-click way to benchmark the system. In our case, we wrote the custom load tester tool and always compared the performance metrics before and after each release.

And what’s our proxy doing now?

It’s been humming along nicely on the servers, processing a ton of traffic, and the client describes the whole operation as “boring”.

No problems, no alerts.

Moreover, we also implement persistent connections, so that for each partner ad network, we only connect to it once in a while. This saves the overhead of creating a TCP connection, every single time. We’re happy to say that this brought a 40% performance increase.

Where do we go from here?

1. Like always, when wrapping up a project this nicely, we high-five and get right back to work.
2. We move on to the next challenge, because we’re only just getting started. Stick around and we’ll keep you posted on further projects and developments.
3. And of course, to conquer the internet.