What It’s Like to Be on Magoosh’s (Tiny) Engineering Team
In early April, I told my manager that I’d be leaving my company of 30,000 and team of ~50 engineers to relocate to California and join Magoosh — a company of ~30 with just two engineers. I was hit by a wave of anxiety and excitement. “Did I make the right choice? Totally! Wait…yeah? Definitely!”
I read through the Magoosh blog posts and asked plenty of questions during my interviews, so I had a pretty good idea of what it would be like. Now that a few months have passed, I thought it would be a perfect time to share my experience and the secrets of our high-impact but tiny engineering team.
Who is Magoosh and what do we build?
Magoosh is an edtech company in the Bay area with a mission to level the playing field by giving every student access to effective, affordable, and engaging test prep tools. We have served over five million users through a web platform for studying via video lessons, practice questions, etc. and a set of mobile apps to study on the go.
How does our team handle big architectural changes?
By “big architectural change” I mean reworking an existing system or module. Some reasons to do this: to support a new feature, to address performance issues, to improve maintainability, fix a major bug / improve stability, etc.
How often are big changes made?
We* are pretty good at avoiding big architectural changes altogether. Since I joined Magoosh, there haven’t been any notable big changes that have come up. When I asked Zack, who’s been on the Engineering team the longest out of us three, he spent a solid minute thinking back to the last time he was involved in one, which happened to be about two years ago!
There is a good reason big architectural changes are so infrequent.
Before we decide to start building a new app or feature, we run an experiment to determine whether the project is worth dedicating time to. For example, we recently validated our study schedules feature, screenshotted below, using data collected from a simple two-button survey. Since 70% of the responses indicated our users wanted the feature, we moved forward with implementing it.
This methodology is effective in limiting the number of architectural changes we need to perform while maximizing our impact as a company.
(*Note that the engineering team is a part of the product team here. When I say “we” I’m referring to both engineering and product people.)
What’s the process for making a big architectural change?
Before we make a big change, the engineering team follows these steps:
- Have team meetings to discuss the changes, alternative solutions, and any impact on user experience.
- Conduct code reviews to minimize risk of performance issues, security problems, edge cases, etc.
- Send out an internal PSA that a big change will be made, so the other teams, such as support, are aware in case they get questions from users.
How do we prioritize work?
New feature requests, bug fixes, and other ideas come in every day. With a team of three, we certainly don’t have the resources to take on all of them, so we have to choose which ones to work on while the rest accumulates into technical debt.
How do we decide what to prioritize? Read Zack’s post How 3 Magoosh Engineers Support 5 Million Users (and Still Sleep at Night) to find out.
How do we estimate effort?
Estimates of effort can play an important role in how we decide to prioritize our work.
For example, suppose you had to decide between building a new app (estimated effort: four months) and reusing an existing platform (estimated effort: two weeks). The amount of effort might sway your decision!
For most requests, we make a first-pass estimate which usually ends up on the aggressive side (meaning the project will often — but not always — take longer than the estimate). If it’s clear that the project will be a big effort (in the order of weeks), we’ll likely add a buffer for unforeseen obstacles, and that estimate will still be slightly aggressive.
It can be challenging to provide an accurate estimate. If you’ve ever had to provide one, you already know this. When we start to anticipate a project will take longer than the original estimate, we’re likely to pause, reassess, and reprioritize as necessary. This happens occasionally, and fortunately it’s not perceived as a fault.
(Notice that in the above example, the estimates are in units of time. Some teams score effort in points or by difficulty. As far as I know, we haven’t tried that. It hasn’t been an issue for us, so I anticipate we’ll stick with time estimates.)
What are the engineering team’s bottlenecks or pain points when adding new features?
Every engineering team hits its own bottlenecks when building new features. Since adding new features is a straightforward way to improve the product and differentiate from competition, it’s important to be able to build them as quickly as is appropriate. Therefore, it’s good to know what the bottlenecks are and how they affect our workflow in the long term.
Here are the biggest bottlenecks we’ve identified:
- Our web platform is built on Rails, and we use some Javascript/Coffeescript with jQuery. As we add new features that introduce more client-side interactivity, this is starting to get messy. We also don’t have APIs for everything, so we’re not in a position to convert to single-page-application immediately.
- Keeping dependencies and libraries up-to-date. For some of our mobile apps, we haven’t updated our video player because it depends on a newer version of React Native. Since we weren’t aren’t actively keeping up with React Native releases, the upgrade effort is growing due to breaking changes with each release.
- When we make backend changes, we often have to go through multiple frontend interactions to get to the right state before we can confirm a change behaves as intended. For example, when Albert, our associate software engineer, was working on practice sessions on our website, he often had to navigate through different pages and forms to test his work, which slowed his iterations. With more backend tests or more APIs, we may be able to iterate faster.
We recognize these bottlenecks, but we don’t necessarily prioritize them just because they are bottlenecks. We think about the larger impact and prioritize them according to the process detailed above.
What processes do we have in place?
By “process’’ I mean the guidelines, instructions, and checklists that ensure frequently occurring tasks are performed consistently, regardless of who does them. One example of process is a checklist we follow to restart a system. Another example is instructions for how to deploy code to production.
On our team, the processes we have in place are primarily related to maintenance. We don’t have a process laid out for everything we do. However, we do have recurring tasks set up at daily, monthly, and quarterly intervals so we reduce the chances of important maintenance falling through the cracks.
As we continue to grow and build out new features, I anticipate we’ll continue to document more processes. For now, we have the important stuff covered.
What are we experimenting with?
Experimenting is a common thing we do at Magoosh. We run small experiments and validate them before increasing scale and resourcing.
Recently, we tried a tool called Runnable to see if it would make testing easier. We also experimented with outsourcing our Rails minor upgrade (and later outsourced a major upgrade). Now we’re trying out CodeClimate, and we’ll validate through internal feedback.
Additionally, we have company-wide hack days, so this is another chance to experiment.
What are the current skill gaps?
The biggest skill gaps we’ve identified on our team are web security and devops.
For devops, we’ve outsourced to a company called Reinteractive. They handle scaling servers up and down, network issues, database admin, etc. This has the added benefit of letting us focus on features instead of worrying about pager alerts and security patches!
For web security, we run automated security scans on a quarterly basis. We also occasionally have experts audit our system and we keep security in mind for code reviews.
How does the engineering team get feedback?
Internally:
We get internal feedback through standard channels such as code reviews, one-on-ones, and quarterly reviews. One thing also worth noting is all Magooshers fill out personal “Guide to getting the best out of me” questionnaires, which describe how they prefer to receive feedback.
External:
Most of our external feedback comes distilled through product managers or the student help team in the form of NPS score (user ratings), noteable reviews, surveys, or conversations with students.
Last month, we tried an [optional] company wide experiment where we read and discussed feedback from many other sources such as forums and YouTube comments.
What are our learning opportunities?
Learning is highly encouraged across Magoosh. With approval, Magoosh covers educational books, conferences, and workshops that are work-related.
In addition, everyone has a monthly class stipend that can be used towards learning anything (e.g., a new language, how to cook, etc.).
Within the engineering team, we also share knowledge about new libraries, technologies, best practices, etc. to learn from each other, but one thing we’re currently looking for is mentorship from a senior engineer.
What’s the bus factor?
With just three engineers, our bus factor is understandably low — if even one of us were unexpectedly unavailable, we would see an impact on ongoing and future projects. They wouldn’t be stalled forever, but they would definitely be delayed or reprioritized.
In terms of keeping critical systems up, we’d probably be okay. Zack’s been out on vacation for three weeks at a time, and Albert’s comfortable working with the backend. We have two ongoing themes to mitigate this risk. The first is to increase redundancy for the core app: Albert is taking over tasks and projects that usually go to Zack. The second is to hire another Rails engineer who can start learning the system.
Who knows? Maybe you’ll be the next to make our engineering team a lot more impactful and a little less tiny.
Interested in working at Magoosh? Check out our open positions.
If you don’t see an open position that’s perfect for you, please submit your resume via the general application.
Learn more about our philosophy and culture here.
