Beyond the Traditional Job Description: Engineering at Civis

by Jonathan Cobian

What we do at Civis is complicated. Data science is already a highly technical topic, and staying at the forefront of the industry requires everyone involved — from operations to engineering — to understand basic data science themes. The result is a highly collaborative atmosphere that is one of the best aspects of working at Civis. As a member of our cross-functional Research and Social Science team, I’ve not only learned about data science, I have gained new, invaluable skills beyond what you might expect from a traditional software engineering job.

Collaboration at work

I meet frequently with members of our Applied Data Science (consulting) team to conduct user interviews and prototype new features. I hear about the challenges they are solving (for example, how to scientifically evaluate ad performance), and suggest tools to improve their data science work. In return, their first-hand feedback allows me to iterate quickly on new software. Having a large group of power users sitting nearby that consistently push the limits of our software tightens the feedback loop and provides engineers with great user context.

I also work closely with our Data Science and Social Science Research & Development teams to productize their scientific concepts. Most recently, our teams collaborated on implementation for our proprietary survey weighting methodology, which has helped us accurately predict several key election results.

But it’s not just data scientists that I work with here. I’ve met with our Sales and Marketing teams so they understand what we’ve built, and likewise, they can explain how Civis is going to share it with the world. I’ve worked with the Finance team to build automated cost reporting that comes directly from our software, and with our Design team to ensure a consistent user experience across products.

Working with so many teams can be overwhelming, but throughout all these interactions, I have a Product Manager right by my side, who serves as the glue to make the whole process work. They ensure the software we build meets user expectations, explain software capabilities to stakeholders and users, and help set the roadmap of features to build and bugs to fix.

Learning data science

Civis has smart data scientists with widely varying backgrounds, from physics to economics to linguistics. That diversity fosters constant learning about data science concepts.

Throughout my time on this team, I’ve learned more about machine learning and survey science than I ever thought possible, from how a random forest classifier works, to quota sampling (how to make sure the right combination of people take a survey), to survey weighting (how to make sure your respondents represent the larger population). I’ve used more whiteboards than I can remember, and despite the complexity and general feeling of being outside of my comfort zone, the experience has been rewarding, fun, and, most importantly, has allowed us to build excellent software that does complicated work. The team is helpful and eager to teach, ensuring we as engineers can understand and effectively contribute.

In general, Civis has a great culture of teaching and learning. Apart from formal work interactions, many Civis employees host brown bag lunch sessions to explain topics such as Redshift query planning, how to interpret EXPLAINs, clustering and unsupervised learning, and new changes to Angular.

Providing engineering guidance

On the flip side, while we learn from others, engineers are also empowered to share and advise on our skills across the company. I’ve taught others concepts like poker planning estimation, the importance of testing, containerization, and tips and tricks for building stable production-grade software. We recently built out additional tooling that allowed our data scientists to run Jupyter notebooks in our Kubernetes cluster, so they could access our survey data without worrying about security control, environment set up, or resource limitation. While not all ideas require engineering support (e.g., when R&D needs to quickly prototype and evaluate a new idea), engineering provides mentorship and guidance on turning novel ideas into a productionized pipeline.

Furthermore, we join early conversations with existing and potential users, so we can better understand our clients’ needs, and set realistic expectations around feasibility and timeline at the offset. When something doesn’t seem possible at first, we put on our research hats and explore what technologies or tools might help us get the job done.

Software engineers at Civis work on real-world data science problems with people from different departments, which allows us to build software that makes accurate, useful insights for our clients. In the process, we are constantly learning and teaching others, which makes coming to work every day more rewarding.

Interested in joining Civis? Check out our open positions here: https://civisanalytics.com/careers.