What it’s Like Coding in a Large Team
In 2003 I joined a small startup in San Diego called ProfitLine as a software engineer. I was part of an eight-person development team, comprising one development manager, three database administrators, and four software engineers. By Gusto standards, coding there was the Wild West.
Back then, the Capability Maturity Model, or CMM for short, was a measure of a development shop’s maturity. A score of 0 meant that progress was achieved mostly through heroics. A score of 5 meant you were working for NASA, with every piece of code diligently documented, tested, and endorsed by stakeholders (usually with an actual wet signature).
I think our shop was a 0. And this was by design, as it should be for most startups. Our business was still in the throes of proving its value to the market. And our mandate was to build and test new features quickly. In order to do this, we de-prioritized automated testing and instead leaned heavily on our human QA team for feature quality.
We were an ASP.NET shop and worked predominantly in C#. AJAX hadn’t been invented yet, and every click of a button produced a synchronous server-side request that made itself known to the user with an intermittent blank page as the browser loaded a fresh HTML document. The document came with a hidden encrypted string called “View State” representing server-side state for reconstitution in the next request.
There was no JQuery or AngularJS or React. We rolled our own Javascript, having to solve for all of the browser permutations (IE 6 and Safari mostly) on our own. But our use of javascript was limited, as instead we opted to use the magic of the ASP.NET framework which wanted server-side visits for nearly every UI permutation.
There was no GitHub. Instead, Visual Source Safe was touted as the preferred Microsoft source-control system. And it was prone to drop commit history due to algorithmic flaws that manifested during branching and merging.
Importantly, we didn’t do code reviews. GitHub’s pull request feature has introduced the code-review culture we used to see only with open source projects to companies. Today, almost every development shop in “Silicon Valley” does code reviews, where code changes are always scrutinized and approved by a fellow engineer before they’re committed to a mainline branch. In 2003, this culture wasn’t pervasive.
When I left ProfitLine in 2005, we were still operating at CMM level 0. I left to take a job with KPMG in Australia, where I would do only pockets of software development based on project needs. And when I returned to the States, I took a job on the management side of things with Active.com and didn’t write code professionally for about five years.
In 2012, I co-founded a San Francisco-based startup as the CTO, and took the habits I learned while coding at ProfitLine with me. We didn’t grow much. Five years later I’m a software engineer at Gusto. Writing code here is very different than writing code for a five-person company searching for survival.
Don’t merge. Rebase.
My first lesson was that merging the development branch into my feature branch to keep it up to date was bad.
Instead of merging, we rebase in order to maintain a linear commit history and reduce the noise in a given pull request.
Large change-sets are bad.
My second lesson is that code will ship faster when change is small and incremental. This requires a lot of up-front planning about how schema changes might affect our customers while code is deploying, and how to use feature flags to deploy sub-functionality in fragments until all fragments are present.
Test code structure matters, too.
Most of the code I write here at Gusto is code that tests code, as test coverage is critical for a fast-moving organization like ours. Indeed, much of what a code review focuses on is the structure of testing code, which felt foreign to me at first.
Test everything.
A lot of startups are ditching their QA teams in favor of building an engineering culture that takes responsibility for quality. And this doesn’t work without good test coverage.
Changes happen. A lot.
We have a lot of developers working on our codebase, and the mainline branch is constantly being updated with new commits. At first I went rebase crazy when working from a feature branch. Over time, I developed a cadence (rebase every few days) that kept my feature branch relatively current while also preserving sanity.
Production deployments happen several times per day.
When I worked at Active.com we deployed code every few weeks. And it was a big ordeal. IT gave us forms to fill out and a calendar to work with for scheduling. Releases could take no longer than the allotted time and needed a clear, documented rollback plan that would be implemented if we fell behind schedule. It was very, very stressful.
At Gusto, we run continuous testing automation through Buildkite, and developed a bot called “Sensei” that integrates with Slack, Jenkins, and Buildkite to release code to Production. Developers are empowered to trigger software releases using these tools at any hour of any day.
Depth is everywhere.
It would be hard to program without search engines or Stack Overflow. Indeed, questions like “Should I upgrade to React 15?” or “How can I achieve zero-downtime when deploying schema changes?” can be researched online. But there’s nothing like sitting down with a fellow engineer who is well informed on the topic and becoming informed through bi-directional communication.
Specialization emerges as development teams increase in size. Indeed, at Gusto we have specialists who “go deep” on narrow aspects of our stack, and who act both as thought leaders and advocates for their area of expertise. I find I can lean on folks to help resolve esoteric issues here more than I could anywhere else I’ve worked.
In conclusion, I’ve been working in the technology space for some 17 years, and I feel like 40% of what I know has materialized in the 9 months I’ve been working with the numerous and talented engineers here at Gusto. It’s intense and rewarding to say the least.
