The unique role of Assessment Design Engineers at CodeSignal
By Dmitry Filippov, Engineering Manager
If you’ve taken any of CodeSignal’s Certified Coding Assessments, you’ve interacted with the work of our team of Assessment Design Engineers. The same is true if you’ve used one of the questions in CodeSignal’s task library during an interview or test. Assessment Design Engineers are responsible for coming up with interesting programming challenges to assess a wide range of developer skills.
This is a pretty special role, requiring creativity, a broad knowledge of software, and an eagerness to learn and explore. Assessment Design Engineers only exist at a few companies in the world, so we decided to shed some light on the work our team does.
What Assessment Design Engineers do
Our team creates practical tasks that challenge engineers in specific skills-based areas like backend, frontend, DevOps, analytics, data science, and mobile development. Our work covers two areas: our tasks library that customers can use in CodeSignal’s tests and interviews, and our Certified Coding Assessments that anyone can take.
1. Continuously improve our task library for CodeSignal customers
Assessment Design Engineers are responsible for maintaining and growing a rich library of tasks (over 4,000 total) that customers can pull from for our CodeSignal Test and Interview products. While customers can design their own coding questions, it’s our team’s job to make sure that they can also choose from a wide variety of high-quality questoins across many different topics. We focus on the kinds of practical, job-related tasks that help our customers reduce bias in their interview process.
When we’re designing tasks for the library, we have two end-users: the hiring manager or interviewer who is going to use this question, and the candidate who is going to be asked to solve it. To create a good experience for both kinds of users, we have a bit of a Goldilocks problem on our hands. The task has to be hard, but not too hard. It should be interesting, but not a reading comprehension test. It can’t be something that the candidate has seen a million times, but it shouldn’t be coming from left field. And it has to take just the right amount of time to complete. To read more about how we think through some of these design questions, check out this article.
2. Develop our Certified Coding Assessments
One of the most challenging parts of our work is building and maintaining our Certified Coding Assessments, which test a range of core programming skills. This is one of the most challenging parts of our work. As our assessments are one kind of industry standard for programmers, we have to be very careful to ensure that our questions are balanced, unbiased, and protected from plagiarism. We are continually improving our existing certifications, and plan to add more of them to standardize other job areas as well.
Designing an assessment is not a fast process. We are constantly brainstorming, monitoring, and testing assessments within our team and externally. Throughout, we collaborate closely with our team of researchers to get an academic perspective on the types of questions we should be asking. We also work with CodeSignal’s product team to make sure that our IDE and platform can support these questions.
By serving as a bridge between the research and the product, Assessment Design Engineers have a key impact on CodeSignal’s roadmap. For example, the idea for supporting project-based tasks that resemble a real-life coding project originated during our assessment design work, and is now an essential part of the CodeSignal platform.
3. Help implement product features
Assessment Design Engineers also contribute to the CodeSignal platform and product by sometimes implementing features related to building our assessments and tasks. We also implement support for new languages and tools, and work on improving the internal content-related tools we use every day.
One product feature we worked on was developing a fast way to import solutions for our task library questions. Now, the customer can click a button to auto-generate the answer for their custom test. And recently we’ve been working on improving the current way of running and managing libraries for JVM languages like Java, Kotlin, Groovy, and Scala.
The skillset needed to be an Assessment Design Engineer
The role of an Assessment Design Engineer is quite unique. One of the key skills is the ability to understand a wide variety of software engineering roles. For example, what are the skills you need to have as a frontend engineer? How would we design a fair, high-quality assessment for those skills?
Assessment Design Engineers also need to be able to quickly get up to speed on new technologies. The world of software engineering changes constantly, and there are so many languages and frameworks that it’s impossible to know them all. Every day, we’re creating content for new technologies that no one on the team is an expert in. Even when you’ve never worked with the tool, framework, or technology you’re assessing, you have to be able to design great content for it. Assessment Design Engineers have to love learning new things in a short amount of time. A strong foundation in software best practices is a big help, too.
Think you might be interested in the Assessment Design Engineer role at CodeSignal? We’re growing our team and looking for creative and kind team-players to join us in building the #1 tech assessment platform. Take a look at our Careers page for more information.
Dmitry Filippov manages CodeSignal’s Assessment Design Engineering team. His main focus is planning, communication, and collaboration across teams. Everyone on the team contributes to this type of work, though! We have a horizontal structure, so your manager doesn’t feel like your boss.
Originally published at https://codesignal.com on May 10, 2021.
