Robinhood Hosts its First Tech Talk
Robinhood recently hosted more than 50 engineers from around the Bay Area for the company’s first ever Engineering Tech Talk. In the sunny courtyard at Robinhood’s Menlo Park headquarters, attendees had the opportunity to hear from members of the company’s Engineering team about the unique challenges engineers at Robinhood tackle every day and network with peers.
Engineers at Robinhood work on projects where requirements are quite different than those found in other areas of technology given the focus of our business. Together, we’re working to increase participation in our financial system by democratizing it. To make that a reality, we have built a platform that serves millions of investors and facilitates hundreds of thousands of transactions a day. And, those transactions are incredibly important as they involve our customers’ money. An engineer at Robinhood is part of the team responsible for delivering a seamless and secure customer experience and ensuring the efficient, accurate execution and reporting of customers’ financial transactions.
Robinhood Software Engineer Nathan Ziebart kicked off the Tech Talk and shared one of the ways our eng team was able to improve and optimize the speed and efficiency of our batch processing. Every night, Robinhood needs to process all trades executed during the day for accounting and reporting purposes. This process runs against a single Postgres database, and historically it has taken many hours to finish. By identifying and addressing some key aspects of Postgres’ design that impose write bottlenecks — such as write amplification from indexes, and the need to checkpoint frequently to minimize the size of the write-ahead log, we were able to achieve a nearly 10x increase in processing speed with virtually no code changes. As a result, our engineers are now able to redirect their attention to new efforts that will allow Robinhood to better serve its customers.
Hui Lu, one of Robinhood’s Backend Engineers, followed with a discussion of how Robinhood has incorporated Kafka to synchronize data among our many microservices. A key challenge she highlighted was ensuring that independent services and databases stay consistent when transferring and reserving funds. Although our asynchronous, message-driven architecture makes it easy to plug in new microservices, Robinhood has to be certain that every type of failure scenario is handled correctly when it comes to customers’ money.
We were also joined by Gwen Shapira from Confluent, who discussed the challenges of providing Kafka as a managed service. She detailed some of the internal best practices used to keep Confluent Cloud reliable while serving a wide variety of clients with dynamic and unpredictable workload requirements. For the Robinhood perspective on Kafka, Infrastructure Engineer Jaren Glover shared some insight into how we were able to rapidly scale our own Kafka cluster with a very small team.
We look forward to hosting more Tech Talks down the line to continue to foster discussion among the eng community. If you’re as excited as we are about continuing to democratize the financial system, come join us at Robinhood!
