Creating a Condition Evaluation Algorithm

At the time, the Pathways program at the Carnegie Foundation was using Qualtrics to administer surveys to students. Qualtrics was expensive and a little too feature-heavy for what we were using it for. So I was asked to write a survey application. Having our own survey application meant we had control over how the data was formed, stored, and exported. It meant we could implement our own API and support LTI, a protocol used often in education to support having external assignments that can pass back grades or outcomes.

Fast-forward to two years later. Other programs at the Carnegie Foundation had started to use the survey application for their own surveys, and soon I was hit with a survey that had a unique feature request: some of the questions needed to be filtered based on a respondent’s attributes or based on their response to a previous question. For example, respondents who belonged to XYZ School District should not see the questions on page 7 of the survey. Or respondents who answered “Yes” to a question on page 5 should not see the questions on page 12.

In our survey application, surveys are crafted by first writing multiple YAML files, collectively called a survey package. There’s one file for the general configuration of the survey, and one file for each section of questions. This gets uploaded to the application and stored in our cloud storage, along with corresponding records inserted into the database. Whenever a new feature is added to our survey application, changes need to be made in three places: first, new YAML code needed to be devised for use in the survey package; then, this new code needs to be translated into new columns in our database tables or tacked onto existing columns as a chunk of JSON; third, the application itself needed to read this new information and enforce it.

Why did I use YAML for writing our survey packages? I chose YAML because the language is very English-like, easy to understand, and comes without all the extra punctuation that something like JSON needs. When our analytics team, or other folks within the programs, need to know how the survey questions and the response options are coded, I simply share the YAML file with them and they’re able to get the information they need without having to learn new jargon or feel like they’re reading code.

Writing the YAML code

You begin by setting a type for your condition evaluation. This can be all, any, or none, and correspond with the python functions of the same name (none() being implemented as not any()). Then you make a list of conditions. For each condition, specify whether it will be based on a survey item response (from the same survey) or based on an attribute of the respondent. Set the label of the survey question or the respondent field to check. Set whether you want it to equal the values below or not equal the values below. Then you write a list of values for the condition to check for.

Here, we are using the all type. This means we evaluate the conditions like we’re using the python function, all(conditions). Our conditions list above contains two conditions. First, we are checking if the respondent’s response to the question bg_role is equal to the value management. Second, we check if the respondent’s custom_school_district field is equal to the value xyz_school_district. We loop through each condition and evaluate it, eventually ending up with the results in a list. This list is what’s fed into the python function we chose as type at the top. We evaluate that and end up with a single boolean, determining whether we show or hide something.

{'type': 'item_response', 'label': 'bg_role', 'operation': 'equals', 'values': ['management']},
{'type': 'respondent_field', 'label': 'custom_school_district', 'operation': 'equals', 'values': ['xyz_school_district']}
all([True, True])

The conditions evaluated to True, so the respondent would be able to see whatever follows the conditional statement (the next page of questions, the next question, or the next possible item response).

Say we used any for our type. The conditions would be evaluated like so:

{'type': 'item_response', 'label': 'bg_role', 'operation': 'equals', 'values': ['management']},
{'type': 'respondent_field', 'label': 'custom_school_district', 'operation': 'equals', 'values': ['xyz_school_district']}
any([True, True])

Still evaluates to True, so the respondent will be able to see what follows.

Using it in a survey package

How do we use this in a survey package? We include a conditional statement at the beginning of whatever we want to gate behind the conditional. Here’s an example of using it on a page of survey questions:

When the respondent reaches page 6 of the survey, the conditional statement will be evaluated. If it evaluates to True, then it presents the page like normal. If it evaluates to False, it kicks the respondent to the next page, page 7. If there is a conditional statement for that page, it gets evaluated, and so on.

What about using this for a survey question?

Now our conditional statement is for the question sec2_q9. If it evaluates to True, it shows the question to the respondent. If it evaluates to False, it does not show the question.

How to use it for a specific item response:

The conditional statement is used for the “Strongly Agree” option in question sec2_q9. This isn’t a real example. You wouldn’t hide the “Strongly Agree” option for a question like this one. But that’s what is happening here: If the conditional statement evaluates to True, show the option to the user. If not, hide the option.

At what point does the conditional statement get evaluated? Client-side? Server-side?

Conditional statements are evaluated server-side. The whole process is invisible to the user. That way, the user is only presented with the content they should see and they are unaware of the conditional mechanism.

Source Code

View a simplified implementation of my condition evaluation algorithm here: