A Summer at Gradescope

What I learned as a design and front-end engineering intern.

Intro

Gradescope is a Berkeley ed-tech startup founded in 2013, and its goal is to improve the grading experience for professors and students at large universities. Here’s an explainer video made by my awesome coworker Kevin.

Praise the panda.

Gradescope began as a solution to the tedious task of grading, and was created for use by Berkeley’s CS188 class. Over the past few years, Gradescope has spread to over 50 schools, and it’s been used to grade millions of questions. It’s saved a lot of professors tons of headaches.

The Workflow

Here’s a quick rundown of a typical Gradescope workflow:

1. Instructor Bob gives out an exam to his or her class, like normal.
2. Bob visits Gradescope.com, tells Gradescope that he’s given a new assignment, uploads his assignment PDF, and starts creating an outline. In the outline, he tells Gradescope exactly where each question is located on the test PDF.
3. Instructor Bob and TAs scan in completed student exams and upload to Gradescope.
4. Instructor Bob assigns specific questions for each TA to grade, and each TA will grade all the answers for his or her assigned question. So TA Anna will grade every student’s Q1, and TA Daniel will grade every student’s Q2.
5. Once grading is done, Instructor Bob reviews the grades, releases the scores, and sends an email to each student with the announcement. For the next couple of days, Bob opens regrade requests — students who believe their exam was graded incorrectly can submit a regrade request.
6. Bob and TAs marvel at Gradescope’s awesomeness.

A screenshot of the grading page. TAs can easily see which question they’re reviewing, and assign points based on a clear rubric (right side).

Here’s the grading page in action. Instructor Bob and TAs each see a different part of the test, depending on which question they were assigned to grade. The rubric is really cool (its located on the right sidebar). Each point deduction or addition has a clear and descriptive comment to help the TAs grade accurately. In addition, the TA can add or change up the rubric as he likes — Gradescope will updated the changes to all submissions with that rubric item applied. So if Bob applies rubric #2 to fifty tests, but realizes that he should deduct 1 point instead of 2, he just has to change the rubric once.

If all this still sounds very foreign to you — don’t worry. One of the first things I discovered as a design intern was that Gradescope is a very different product. Very few people grade exams for their job, and even if you do, Gradescope changes up a whole lot compared to the “traditional” workflow. Usually, people have to try out the product before truly understanding how it works.

But that’s also why working as a design intern was so rewarding — I couldn’t assume that the users understood how to use the product. A lot of applications today are a variation of some other well-understood workflow. Gradescope isn’t similar to any other piece of software, so I had to do a lot of user research and testing. Designing for Gradescope has been a pretty cool experience, and I’m excited to show you some of my work below.

Project 1: Test Authoring

I’m not a professor, but I bet that a lot of them still use Microsoft Word to write exams (totally out-of-the-blue guess, so don’t quote me). One of my first projects at Gradescope was to design a custom interface for instructors to write exams.

Research

I’ve taken about a million tests since first grade, but I’ve never really sat down and considered how they’re really structured. So to start off the summer, I examined released tests from universities across the country and noted their similarities. I grouped questions into 4 different categories:

1. Choice questions: think multiple choice, choose one of the above, choose all answers that apply — those types of questions.
2. Input questions: these could be a simple as “fill in the blank below” or as complex as a paragraph proof of linear independence.
3. “Order” questions: these require the student to put steps in order, like “choose the correct order of traversal in the graph using dijkstra’s algorithm”
4. List questions: stuff like “list all of the necessary enzymes required for x reaction”.

Initial sketches

Prototyping

After the research, it was time to start sketching. Through this process, I drew inspiration from a couple of sources:

1. I really like how edX designed their question-creation interface. On edX, the professor writes questions in plain text, and the website parses and converts it to HTML elements. There was no need to choose your question input types from a complex menu, and the simple “just type” interface allows a smooth transition from previous programs an instructor might be familiar with (like Microsoft Word).
2. I type in markdown often, and one of my favorite websites is StackEdit. A feature that I love is the scroll sync — as you scroll either pane, the other stays in sync, so you can always see the raw markdown and the converted HTML side-by-side.

My initial sketches drew inspiration from these sources, and I sought to create a hybrid. I wanted to design an interface that was easy to pick up, so I didn’t settle for something that was completely reliant on raw text input (as was edX). edX’s interface, in my opinion, was also hard to navigate, because you could only edit (in a modal) one question at a time.

Test authoring

This is the “home” view of test authoring. It uses a two-pane layout, with each side showing the same content, but formatted differently. The left side contains raw text and a basic structure, designed to make it really easy to add and organize questions. The text on the left is parsed and converted to the final exam forman on the right.

Adding new questions

Adding new questions is as easy as clicking the “new question” section and choosing one of four choices. These choices correspond to the 4 question categories listed above: choice, input, order, and list. Once the instructor clicks on one of the categories, a popover menu appears to specify the type of question. They can see a preview of the final layout on the right hand side.

I designed the preview pane because during initial testing, it was difficult for many to get used to the raw text layout. Because this page was already sectioned off into “draft” and “final” layouts, I wanted to reinforce that idea, so the popover is also split into two parts, “draft” and “final preview”.

Project 2: Code Grading

My second design project at Gradescope involved one of our most highly requested features — programming project management.

Design and Prototyping

After meeting with a couple of Berkeley CS professors, I learned that there were actually two parts to this design:

1. Manual code review: this include assessment of code quality and style, and required TAs to go over every student’s code
2. Autograder: this is pretty standard in most CS classes

So from the start, I knew that the user workflow for code grading would be different from the existing Gradescope workflow. Whereas the existing workflow was very unidirectional, the workflow for grading programming projects wasn’t. There was no way to define the “order” of grading. Instructors could autograde first, then review manually. Or they could review manually as submissions rolled in, then autograde. You get the point.

Therefore, it was critical to design a dashboard to integrate this process and make it easy for instructors to keep track of their process in either workflow:

The instructor dashboard for programming projects.

I’m going to rewind a bit here and explain how an instructor actually sets up a programming project. Once again, meeting with professors revealed a lot about the general workflow:

• Most of the time, there’s some kind of template or starter code, from which the student begins the project.
• There’s also projects that are completely freeform, where the students decide the code organization and files to include.
• Professors like to comment in specific parts of the project, whether it’s to point out an error or comment on coding style.
• Many of them really enjoy Github’s code-diffing interface (I’m a big fan of it too)

So the first step in creating a project is determining the template. The professor uploads the starter code for the project (if there is any), and marks which files they’ve asked the students to modify.

After that, there’s a few more setting boxes to tick off — then the project is ready!

The actual code viewer is based off the existing Gradescope design — we wanted to deviate from what’s expected as little as possible. Instructors and TA can easily add or subtract points based on coding style, and students can clearly see the points they’ve missed.

Student view

Some other stuff

• Reworking and mobilizing the team page
• Working on a global help and FAQ modal
• Making the application sidebar faster and more useful
• Daily lunches in downtown Berkeley with the team

Conclusion

It was a blast working with Gradescope this summer! It was my formal introduction to working in a team-setting and iterating through designs.

Before Gradescope, I thought design was a simple, direct process. But after this summer, I learned that it really is a cyclical workflow, and that nothing is more important than iteration, iteration, and more iteration. (and user testing in between!) There were so many times where I would be proud of some design I created, just to watch it break during usability testings.

I just want to say thanks to the great Gradescope team, because they challenged me the entire summer during all of our critiques and standups. Thank you Sergey and Arjun for being great mentors!

Finally, I’m just really excited to start using Gradescope in my own classes. After a summer of developing, it’s quite fun to test the changes I’ve made.