From the Drizzle of School Projects to an Industry-Sized Storm
A typical computer science degree consists of projects. A lot of them. My computer science classes at Rutgers University featured around 3–5 small projects spread throughout the semester. I could describe these small projects as a drizzle; they formed the backbone of my experience as a programmer, didn’t pose too much of a challenge or obstruction and there was always a constant stream of them.
Moving forward, projects started getting bigger. Some examples were a chess Android app, a rudimentary version control system, and a website for mock a travel agency with a database backend. As you can see, the projects were evolving to become bigger, longer, and with more people involved. They went from a steady drizzle to occasional showers.
All of those projects I did in school paled in comparison to the application I had the opportunity to contribute to at Xandr as part of the Clearinghouse Billing Team. The program is called Storminator, and if I described small projects as a drizzle and semester-long projects as showers, then it would be absolutely fitting to describe the size and scope of Storminator as a serious thunderstorm. Storminator handles a large part of Xandr’s billing and invoicing process; it aggregates monthly fees and feeds the information into the billing data pipeline. It’s a large project that contains about 200 files of source code, over 10,000 lines of code, and has been in development for six years so far. Furthermore, it connects to three large databases: a MySQL database full of contract information, a second MySQL database that logs data pipeline information, and a Vertica database that stores data of all the sales on Xandr’s platform. Why was a large billing application given a name as cool as “Storminator”? From what I can tell, the name is inspired by Apache Storm.
A brief overview of my project: The billing team supports tiered pricing, where the rate that Xandr charges clients for buying ads can go down once the client reaches specified spending thresholds. Below is an example:
Billing happens every month. When tiered pricing first came out, clients would be charged the full amount at the end of the month and then receive a refund for discounts according to their tiered pricing contract. This was not an ideal solution, so the billing team developed “Near Real-Time Tiered Pricing,” in which the Storminator program calculates spending every hour and updates the rate if they cross the threshold. It’s called “Near real-time” because the changes happen every hour as opposed to all adjustments happening at the end of the billing month. This is important because getting charged a lower rate allows that client to bid higher, win more auctions, and serve more ads on Xandr’s platform compared to clients being charged a higher rate. When this feature first came out, it only supported a monthly period, so clients would be reset to the highest rate at the beginning of each month. The limitation was that tiered pricing was still closely tied to monthly periods. My project was to extend functionality to allow for any kind of period, not just monthly. This would allow for contracts that would have quarterly and annual periods for near real-time tiered pricing. From a business standpoint, being able to enjoy the benefit of a lower rate for a longer period of time would encourage clients to spend more overall money on Xandr’s platform.
There are a few key takeaways from the contrast between my school projects and my project at Xandr. First, there’s the fact that Storminator is a huge project and I am adding to it. That required me to read other people’s code and understand it. In contrast, all of the projects I made in school were written from scratch by me (and my teammates for group projects). Previous engineers working on Storminator left a good amount of comments briefly describing some sections of code, but other sections were uncommented and cryptic. When I made my contributions to Storminator, I made sure to provide good comments for anyone needing to understand my code in the future.
Another thing to note is the complexity of the system that I worked on. That complexity justifies using advanced technologies, features, and techniques. We learn a lot of these things in computer science classes, but our school projects are too small to justify using them. I got to use a lot of cool things while working on Storminator. I used the JDBI library to pull data from the large databases and store them in custom objects. I manipulated those objects using Java 8 Streams. The contract database is designed with the Boyce-Codd Normal Form (BCNF) in mind in order to maintain data integrity. With the large scope of Storminator, Inheritance and Polymorphism are used all over the program to maintain good design. I learned to make use of the IntelliJ Debugger while testing my code and many IntelliJ navigation shortcuts to efficiently traverse the codebase. There is also a testing framework called Spock, using the Groovy language, which is used to write tests for all parts of Storminator’s code. These tests are run after making code changes to ensure that old parts did not break due to new changes. Finally, Storminator is maintained under version control using Git and Bitbucket. While I was working on my branch, there were also two other branches being used to develop separate billing features by other engineers on the billing team. All of these things are important but hard to build experience in small class projects.
One of the most insightful pieces of advice that I got at Xandr is to always be aware of how your technology is a part of the business. The software that we work on at a job does not exist in a vacuum, unlike the projects we make for classes. It is important to understand how your technology fits into the company’s business strategy and you have to be able to explain your technology to other people at the company who are not software engineers.
Getting experience contributing to a pre-existing project, using technologies that are only at their full potential when applied on a large application, and seeing how my work ties into business needs are all things that I experienced at Xandr that went beyond school. While small school projects gave me a constant drizzle of knowledge and foundation, my internship at Xandr was the perfect storm of real-world experience!
About the Author:
Brian Cheng is an Electrical Engineering and Computer Science double major at Rutgers University. As an undergraduate teaching assistant at Rutgers, he is especially interested in what makes a good learning experience. Besides software development, other interests include gaming and history.
