How to develop great programming problems for your classroom
We write a lot of coding problems here at Grok. Mostly recently, we created, updated or refreshed 165 problems for the NCSS Challenge, which had 14,024 participants — that’s a lot of problem authoring! The big question is: how do you write effective and engaging programming problems?
Developing coding questions for students can be intimidating, especially if you’re not confident with programming yourself. We’re going to look at the process Grok uses to write problems for students to solve, and how you can bring those strategies into your classroom.
Step 1: What are you trying to teach?
When writing problems, we focus on a single new programming concept per problem. We introduce the new information in the notes leading up to the problem, and then have one or two problems that test that concept.
For example, if we’re trying to teach how to change a string to all-caps, we would introduce the upper function in the notes, give some examples, and then write a problem that uses upper. We might then add a second problem that uses it in a slightly harder or less intuitive way, for extra practice and to reinforce the idea.
For the classroom
Identify which programming concepts you want to teach your students, and the order in which you want to teach them. Start with easy concepts and build up to harder ones.
Step 2: Design the solution to the problem
When writing problems, we first look at the skill or strategy this particular problem is supposed to teach. What’s the simplest version of this concept we can test with the knowledge the students already have?
Tip: you can use our Python Playground to experiment with your own programs from scratch.
We know what they already know: we wrote all the problems leading up to this one. We know what they might not know so well: the problems just preceding this one, and harder concepts from the previous week.
Our problems often use techniques learnt in previous weeks, but we try hard to make it a gradual process of building up skills; if each new concept has its own problem, we can sprinkle in familiar ideas from earlier weeks and build a good foundation for harder problems later.
At this point you’ve got a core concept and a few possible revision concepts. Start by writing a program that uses the core concept. It can include any concepts they already know — we teach input very early and use it in almost every problem. For example, using our upper() example from earlier:
You can leave it as-is, or you can make it more complex by adding some of your revision concepts. If they’d recently learned for loops, for example, you could add that in:
If you want to make it harder, you can add as much complexity as you want. For example, assuming they already know if statements, adding conditions for when to use upper():
Simple is usually better. It’s hard enough to learn a new programming concept without having to remember three other new concepts at the same time. The problem should focus on the specific skill being learned — don’t make them use split() inside a nested loop while also reading from a file if they’re all new ideas! Pare down the problem until it’s focusing on the specific skills you want your students to be practicing.
For the classroom
Write a program that uses the core concept you want to teach, and optionally some revision concepts.
Step 3: Add some flavour
Now there’s a solution, but what about the problem text itself? At this point we’re trying to describe how to develop the solution we just wrote (without giving too much away!)
It’s important that the text is clear, engaging and age appropriate. For the Newbies stream of the Challenge, for example, our benchmark for the reading level is years 5 and 6 — and it’s tricky to get right!
Our problems usually have a reason for the program (as flippant as that reason may be), a description of the intended behaviour, any caveats (often called edge cases in the computer science world), some examples, and an optional hint.
A reason helps develop strong problem solving skills; students are given a context and a (simplified) real-world problem, and learn to use those two things to work towards a solution. This is a totally different (and more useful) skill than being able to solve a problem that says “do 10 for loops like this, then print something, then use join to join up this list, …” The reason should try to engage with a theme students understand. We don’t always get this right (we’re not teens anymore!) but we try to make our problems topical and relevant.
The examples should try to highlight any caveats. In the examples here, we show what happens both when the word is already all caps and when it isn’t, as well as using an example with punctuation.
Here’s some problem text for the solution we wrote above:
You always use CAPS when you get EXCITED about things, so you want it to be easier to YELL WITH EXCITEMENT over instant messenger. (Reason)
Write a program that takes an input, and if it’s not already in all caps, prints out one capital letter per line. If it is already in all caps, it should print “You yelled that already!”. (Behaviour)
Any punctuation in the input should stay exactly as it is. (Caveat)
Here’s how it should work: (Example 1)
Here’s an example where you’ve already yelled the word: (Example 2)
Remember: To check whether something is upper case, you can use isupper(). (Hint)
For the classroom
Use this Reason/Behaviour/Caveats/Examples/Hints framework to develop the problem text for your solution.
Step 4: Explain your solution
The key to solving any programming problem is breaking it down into smaller steps. We aim for solution notes that explain not just what the solution does, but how we got there. Sometimes it even means writing some code that won’t be included in the final solution, to test parts of the code along the way. Programs are rarely written straight from start to end!
For the problem we’ve written here, we didn’t write input + if + print + else + print + for + print in that order, and we don’t expect students to either. They need to read the expected behaviour, and identify the individual ideas hidden there.
Our explanation would go something like this:
First up, we need to ask the user for input. We already know how to do that, so let’s print it out to see what it says:
But we don’t just want to print it out! We need to work out whether it’s uppercase or not, and if it is, print something out:
We’re halfway there, but we still need to do something to the word when it’s not uppercase. We know we eventually need to print out one letter per line, but for now, let’s just focus on making the word uppercase:
So close! The last thing to do is to print out one letter per line. We can do that using a for loop:
For the classroom
Think through the steps you took to write your solution, and build up your explanation one step at a time. If you’re stuck, think about how you can break down your solution to add one concept per step.
Step 5: Develop testing skills
One of the most important skills in programming is how to test code effectively. For the Challenge we have fancy automated markers to do the heavy lifting for them, but how can you encourage good testing techniques for your students in the classroom?
The first thing we always test is the examples in the problem text. Does the output of the program exactly match the examples? This is asking the eternal question: have you read the question properly? Get your students to run their program with the example output and make sure it’s all correct.
Next up, we test a few varieties of the main cases. In our uppercase example, we would test a few uppercase and non-uppercase words, and maybe a mixed case word.
Then, test the edge cases (our caveats in the problem text). Try a few words with punctuation marks. Try a word that’s only punctuation marks! Try to guess what might trip up the program, and give them a go.
Last, we test any unusual inputs we can think of. What happens when there’s no word at all? If the problem involves counting, what if there are 0 things to count, or a negative amount of things? This helps to make sure they’ve really understood the question and their program is able to generalise.
For the classroom
Try to develop in your students a habit of thoroughly testing any code they write. There can be multiple different solutions to one problem, so it’s important to know that the solution does what they think it will!
Writing good programming problems is tricky, but with a framework and a plan it gets a bit easier. How do you write problems for your students? Have you got any tips for other teachers? Let us know in the comments!
