For Eventbrite’s Engineering Team, Communication Matters As Much As Code
To help tell this story, we partnered with Job Portraits, a creative studio that tells stories about fast-growth companies. For the interview below, their team spoke with Simon Willison (Director of Architecture), Natalie Downe (Director of Frontend Engineering), Galen Krumel (Sr. Director of Engineering), and Allison Lacker (Senior Software Engineer) at our San Francisco office at 5th and Mission. Check out open positions with Eventbrite engineering here.
What is Eventbrite and how do you make people’s lives better?
Simon: Eventbrite is the world’s largest self-service ticketing platform. Event organizers around the world use us to run their events, both free and paid, and event attendees use us to discover events ranging from hackathons to festivals.
Allison: We’re especially focused on the important functions that make an event experience seamless — things like allowing people to pull up tickets on their phone. We also focus on developing tools that help organizers get the word out about their events, which, in turn, helps people find fun events to attend. It’s both sides of the coin.
Simon: Every startup wants to make the world a better place — where you run into problems is finding a business model that aligns with that goal. We’ve found that when our customers are successful, we’re successful. There are hundreds of thousands of people around the world who make a living through Eventbrite. I think it’s a noble thing to build tools to help them do that.
A lot of engineering teams are solving problems that only happen in the digital world, but you guys are building something that brings people together in the real world. How does that affect your work?
Galen: Our work with Maker Faire is a great example of this. We’ve worked with them for a good while and they tend to jump at using our new products. With the most recent Maker Faire in New York, some of our team went out and worked on the ground with them. We helped check in attendees, tested some of our newer technology, including an RFID solution, and just hung out with the event organizers. I think that’s a rare opportunity for engineers, to see how the work they do is powering real-world events.
Simon: Lots of people who work here are attending Eventbrite-enabled events that have nothing to do with work. For instance, we recently went on a tour of the sewage system in San Francisco. We had been watching a usability test and that event came up, and we sat there going, “You can tour the sewage of the city? That’s awesome!”
Galen: It’s also our job to interact with customers. We’re meeting and talking with organizers and hearing everything they love or hate about using the product, which is totally unnerving and frightening, but also kind of cool. A few months ago I was visiting our Nashville office and we went out for dinner. Coincidentally, there was a speed dating event organized with Eventbrite happening at the restaurant that night. I went and talked to the organizer about how she uses our product, what she likes and doesn’t like. It’s cool to be going about your life and then you bump into people, who are like, “Eventbrite! I love you guys.”
Allison: I remember I called my parents when I got a job here, and I was expecting them to know nothing about Eventbrite. Because, you know, nobody outside the Bay Area can keep track of all the startups. But my mom’s a teacher and said, “Oh, I just got a ticket to a conference with them.” And my dad added, “Yeah, I’m going on a century bike ride through Eventbrite.” Even my parents, who live in the Midwest, know and use Eventbrite.
How is the engineering team organized?
Allison: We split people into teams of five or six, organized around customer needs. I work on the discovery team and our goal is to help people discover cool events. We have our own metrics, and I have ownership of a specific part of the product. It’s nice because I can go in-depth and I’m not jumping around all the time.
Galen: We used to do it along the lines of our tech stack, like a mobile group, a data group, but that was inhibiting focus, so we reorganized.
Simon: I should also mention we have two other offices with engineering teams. Our main team is here in S.F., another is in Mendoza, Argentina, where we acquired a company two years ago, and we have a small but growing group in Nashville. Because of the time zone differences, the teams operate independently on a day-to-day basis, but at the project level we collaborate very closely.
What are a few of the most interesting problems someone might get to work on as part of this team?
Simon: One of the most interesting technical challenges is managing giant traffic spikes. We have a baseline level of traffic all the time, but when tickets for some giant event go on sale, we’ll have 10,000 people hitting “buy” in the first two minutes. Our needs for uptime and reliability are extremely high, because if our site goes down we’re damaging people’s businesses. And if our site goes down during the check-ins at a big music festival, that can cause real-world havoc. So we maintain a very responsible engineering environment to ensure we won’t break things for our organizers.
“Our needs for uptime and reliability are extremely high, because if our site goes down, we’re damaging people’s businesses.” –Simon
Allison: We have some fascinating problems within discovery and recommendations. Say you’ve only been to one event on our platform. How can we extrapolate out to show you other events you might enjoy? What questions should we ask you? There’s a lot of machine learning and human connection needed to figure those problems out.
Natalie: From a Frontend perspective, there are interesting challenges around our code base, which is 9-years old. We give a lot of attention to building code that’s reusable and maintainable, while at the same time experimenting with exciting new technologies that can help accelerate growth. The challenge comes in integrating the new technologies — like WebGL, which we’re using for reserved seating — while not sacrificing the robustness of the rest of our stack.
Allison: The seat designer in our reserved seating product is actually really interesting, since it’s essentially a web-based design program. People often ask me, “Aren’t you done? Is there anything left to create?” My response is, “Did you know we have a reserved seating feature? Did you know we have team registration and bib check-ins for races? Did you know we’re accepting payments in South America?” I don’t think people realize how diverse our event organizers are, and how different their needs are.
We know that there are a lot of things probably 90% of engineering teams do across all companies. Are there any of those standard things Eventbrite doesn’t do, or that you have strong opinions about?
Simon: Yes, actually. All good software engineering teams use automated tests, but a lot of companies enforce testing through a shaming mechanism. If you break the build, a siren goes off and there’s a rubber chicken that gets put on your desk. It can be a very negative experience. So last year we built a system we call Proposed. It’s a way of shipping code where, instead of pushing code straight into our main code base, you push it to a special “proposed” branch. The tests then run on your code, and if your code passes, it gets merged in. But if you made a mistake and you’ve broken the build, you get a message from a Slack bot saying your code failed and the merge was canceled — and nobody else knows.
When we introduced this, our build went from being broken about half the time to being broken less than 5% of the time. Because there’s no shame anymore, our testing culture has really shifted. Testing used to be something people were a little bit afraid of, and now it’s a comfortable, everyday part of how we work.
“You’re not going to get called out for something you’ve broken here.” –Natalie
Galen: I think this feeds into our culture of being accessible to programmers and engineers who are newer to this line of work. It’s good for managers, too, because we used to spend so much time prompting people to write tests.
Natalie: This helped a lot with shared code ownership and knowing everyone’s in it together. You’re not going to get called out for something you’ve broken here. Nothing is perfect and things still manage to break, but now when you break the build you just make a sacrifice to the altar table, which is viewed as a more positive thing.
Wait, you have an altar?
Allison: Haha, yeah. It’s a really nerdy pun. The SQL “alter table” command is used to alter the code in an existing table. I don’t think everyone gets that joke. It actually started as an altar where people, when they’d get back from vacation, would make a “sacrifice” to the team of food or whatever, from wherever they’d been. But we also have a tradition where, if you get through the Proposed system and something still breaks in some spectacular way, you bring a box of donuts for everyone in the morning. So now when someone breaks the build, we get to celebrate.
So that’s how you deal with mistakes. How do you celebrate good things?
Simon: We should talk about the Gnomes. Nearly two years ago now, Steve French decided that the site wasn’t performing fast enough, so he bought a gnome that looked like a ninja. He said, this is the Ninja Performance Gnome, and if you make the site faster, you get it on your desk. We thought, “Okay that’s interesting, but there are other things we want to improve as well.” So people started buying gnomes with a specific theme in mind, like there’s a gnome reading a book on a toilet — that’s the documentation gnome.
“I got the gnome with a machine gun. That’s for deleting code.” –Allison
Gnomes evolved into this completely informal peer recognition system with no management input at all. Basically, if you care about something, buy a gnome and give it to someone who’s doing the thing you care about. The only rule is when you give someone a gnome, you have to take a photograph of them with it and upload it to the gnome Slack channel, and include a note on what the gnome is for. Well, the other sort-of rule is you’re not allowed to count the gnomes or try and index them in any way. There is no gnome census. I can’t remember why, but that’s very important.
Allison: We don’t want gnomes to become part of an official recognition process, because then it might become political.
Have you gotten a gnome before?
Allison: Yeah, I recently got the gnome with a machine gun. That’s for deleting code. There are also the Frontend gnomes, which are all superheroes: Supergnome, Batgnome, and Captain Gnomerica.
Simon: My favorite gnome is this Tyrannosaurus Rex that’s grabbing other gnomes.
Allison: That’s the recruiting gnome!
Natalie: Alli even made a gnome.
Allison: I was searching for a gnome for accessibility but couldn’t find anything online. Natalie and I were actually taking a ceramics class together, so I made a little guide dog gnome with a hat and beard.
Check out open positions at Eventbrite.
Why is this an exciting time to join the Eventbrite engineering team?
Simon: Right now we’re in the process of transitioning from a single, core codebase to a group of independent micro services that communicate with each other. One of our interesting challenges is, how do you rewrite and replace the oldest parts of the code without freezing the world and shutting down development on new features? Some companies try to do complete rewrites, which is incredibly risky — and miserable work. Instead we’re doing it incrementally as we break pieces off into those micro services. When we’re building a new feature, often we’ll use that as an excuse to take the underlying code and rebuild just that.
Does this have something to do with the re-orgs you mentioned?
Galen: From a management perspective, re-orgs allow us to continue to perform as a nimble startup. On a day-to-day basis, we know what we’re doing. On a month to month basis, we know what we’re doing. When we start looking quarterly and annually though, the economic climate is changing and the competitive climate changes quite a bit. So on a regular basis, the executive team looks at the type of challenges we want to take on, and, if that strategy and our engineering organization don’t line up, re-orgs have been one solution.
“’Re-org’ is this terrifying word at other companies, but I think re-orgs are exciting.” –Allison
Allison: I know “re-org” is this terrifying word at other companies, but I think re-orgs are exciting. It makes it much easier for us to change roles or focus. I actually used to be a Frontend engineer, but since our last re-org I’ve been working in data land and writing search algorithms. Because of the support around learning, re-orgs have given me the opportunity to acquire more skills and knowledge. Our upper management has done a great job of using re-orgs to both solve work problems and make things more interesting for individuals.
Galen: We’ve done two re-orgs in the last two years, and the feedback is that that’s a good tempo. I love hearing that Alli thinks of them as an exciting opportunity. I think a big reason they work is, when we want to re-org the team, everybody is invited to share their personal interests. We’re dedicated to making sure people here feel challenged and excited about what they’re working on. If someone feels like a change, I want to know about it. Using Alli as an example, we want to make sure whatever she’s working on, it’s something she’s excited about. We consider ourselves very people-focused; our goal is to make sure the company is successful, but we also want Alli to be successful in her career.
Are there people who might not be the right fit for this team?
Natalie: I personally care a lot about work-life balance. Not many startups are good at this…you know those people you meet in a Lyft Line at 11pm, and they’re just now on their way home or still emailing in the back seat? That’s not healthy. We work hard, we work decent hours, and there’s definitely huge amounts of dedication, but when we’re on our weekend, we’re on our weekend.
Galen: I identify as somebody who shows up and, while I’m in the office, I work as hard as I possibly can. Then I firmly put on the breaks and go pick up my son from pre-school. There’s a limited amount of time on this planet I get to spend with my family, so I carve it out and I’m pretty protective of it. And I expect that everybody else on the team respects where others stand with their time.
“There’s a limited amount of time on this planet I get to spend with my family, so I carve it out and I’m pretty protective of it.” –Galen
If you need everyone on your team to be available all the time, this probably isn’t the place for you. If you want to be constantly plugged in and connected to work, that’s your decision, but nobody here tries to force that on others. Also, people who want to just show up, work on a concrete technical problem, and they don’t really ask why, they also might not enjoy working here. I wouldn’t say that they’re not welcome, but we do the work that needs to be done and sometimes that specific technical problem you had your head wrapped around, maybe that’s not the most important thing.
Natalie: Also, someone who thrives in an argumentative environment and likes bashing ideas out, they’re not going to do as well as someone who can be collaborative.
Simon: We filter out assholes very effectively.
Allison: An important thing to note is that, when you have informal rules like our “No Asshole Rule,” in the wrong hands that can translate into “we don’t hire people who are different from us.” I think we do a good job of the opposite, of saying, hey, people are different and that’s great. Even if not everyone wants to go buy gnomes, that’s OK.
It sounds like training and mentorship are an important part of the culture here.
Natalie: Absolutely. We actually run code labs, which draw on the huge diversity of skills and experience on this team. They’re a safe space for people who don’t have a traditional engineering background to come and ask questions. Although we encourage people to take classes and learn new skills, we hope they can learn most of what they want to from the Eventbrite engineering team. We work hard to mentor people who are just beginning their careers, as well as providing an environment of learning for engineers at every level.
For example, a number of teams have office hours. There’s security office hours, Python office hours, Frontend office hours and design office hours. We also have Frontend Fridays, where people give presentations and lead discussions. Then every two weeks is the Engineering Hoe-Down, where we do code reviews and training on a wider basis.
“We work very hard to share our skills and mentor people who are just beginning their engineering career.” –Natalie
Simon: Once you’re at the senior engineering level and beyond, listening and empathy become super important. If you can produce amazing code but you don’t have good people skills, then it’s something you’re asked to improve — it’s necessary for getting promoted as an engineer. We have a huge diversity of experience levels and backgrounds, and I really like that about our engineering culture. In the same day I might be working with someone who has 6 months of engineering experience, and then someone with 20 years.
Galen: The senior engineers lead most of the education sessions. To be an effective mentor, you need to be able to make a programmer with little to no experience feel comfortable asking what they may feel are stupid questions. Especially in code labs, it doesn’t work if anyone is brusque because we don’t want people feeling bad and doubting their abilities, thinking this an inaccessible trade.
Allison: It also helps us build a more diverse team. If you focus on being a supportive and welcoming community, you’re going to attract a lot of people who may have felt ostracized from other communities.
Galen: A lot of engineering organizations are homogeneous, especially in terms of people’s backgrounds, which can have a weird effect on conflicts. At most of the other jobs I’ve had, interpersonal conflict was either avoided or not dealt with effectively and ended up hurting morale. On this team, people are very different from each other, so it’s built in that we have to work through conflict, not avoid it. But you know, we can say this because we hire reasonable people who have good listening and empathy skills. When you have that, things get solved pretty easily.
A lot of engineering teams like the idea of mentorship, but it’s hard for them to actually prioritize it for senior devs when they could be coding. How do you guys think about that?
Simon: Some of our most productive engineers have only been engineers for a few years. Engineering productivity does not always relate directly to experience, or how clever you are. It’s often about being able to dive into problems and just crank until you get to a solution. We have some really amazing engineers with non-traditional backgrounds. They’ve learned most of what they know here at Eventbrite.
Galen: We’re not hiring people to just write code as fast as possible. We want to build a lasting engineering organization. And we’re doing that by hiring smart people with a desire to learn. And also hiring people who like to teach. You get much further investing in your people than you do by hiring for a specific expertise or specialty. That’s how we’re building a sustainable business.