Improving intro CS by explicitly teaching programming skills
While intro CS courses teach concepts, they could/should also teach SKILLS!
You’re in your first day of your first computer science (CS1) course. The instructor wants to “dive right in.” They want you to open up your computer and to the “IDE” you were supposed to download before class and copy some code projected on the screen. You copy the code. They tell you “don’t try to understand [the code] too much.” That’s funny to you because you don’t understand any of the magic that just occurred! Some code was written in English words but not typical sentence structure. After pressing a button to run the code, “Hello world!” appeared in a funny font. What did you just write and why did you get “Hello world” and your neighbor get many lines of scary red text? How did the code your copied make “Hello world!” appear? And was that it, or was there something more? So much to infer, so little instruction on how to do so!
I intended for the above example to demonstrate that programming requires many skills which we don’t necessarily teach. We may be tracing code to check for errors, comprehending what code does, or writing correct syntax, perhaps to fulfill some goal. But in many CS1 courses, instruction often does not acknowledge, let alone teach, these skills explicitly. We may instead teach these skills simultaneously, in a conglomerate. We may expect learners to infer understanding through repeated exposure to a diverse spectrum of examples. But attempting to implicitly teach skills invites misinterpretation, which can lead to greater errors as learners do more complex programming tasks.
To ensure learners develop mastery of different skills related to programming, we must sequence and structure these skills in our CS1 instruction. Educational psychologist Jerome Bruner argued that instruction requires specifying “the ways in which a body of knowledge should be structured so that it can be most readily grasped by the learner,” as well as “effective sequences in which to present the materials to be learned.” So taking this, I argue that we can improve introductory programming instruction by explicitly structuring and sequencing programming skills.
For the rest of this post, I’ll describe a theory of instruction which considers four different introductory programming skills, describe example instruction, and share some initial results which suggest that explicitly teaching different skills can improve learning.
Our theory: read before write, knowing before doing
Prior computing education and learning sciences research have identified trends relating to programming skills:
- Learning to read/trace code may be a precursory skill to learning to write code.
- Tracing and explaining what code does at a high level are different skills.
- Knowing how code works is different than learning how to use it to problem solve.
Given this, we can propose 4 distinct skills that we can teach sequentially:
- Read semantics: trace code line by line and predict state changes as the code executes.
- Write semantics: write correct syntax
- Read templates: Recognize the goals and parts of a template (common code pattern)
- Write templates: Use a known template to write code to solve a problem.
These skills follow a pattern of learning to read/comprehend before writing and learning program semantics (how code works) before learning templates (how to use code to do interesting things). The table below summarizes these skills across the read/write and semantics/templates dimensions.
Example Instruction: Exercises to observe skill mastery
To see a tangible application of our theory, we translated this work into an introductory Python curriculum. The learning materials assumed no prior programming knowledge and covered data types, variables, operators, print statements, and conditionals.
For the learning materials to reflect our theory, we provided specific instruction and practice for each of the four skills described previously. The instruction focused on teaching each specific skill. The practice focused on evaluating demonstrated knowledge of each skill. The figure below summarizes how a learner demonstrates knowledge of each skill using the classic example of a variable swap.
Let’s walk through example learning materials which teach each skill incrementally. We’ll describe the instruction, exercises, and challenges we observed in this study and follow-up studies relating to the exercises we designed.
S1. Reading semantics (tracing)
To teach S1 (reading semantics), the instruction provides examples framed around real-world contexts to motivate the purpose of a new construct (e.g. conditionals) before defining the programming construct. Because S1 focuses on teaching the new programing construct and does not focus on how to use the construct to problem solve, we rely on the real-world examples to motivate the construct. S1 instruction uses examples to motivate a new construct and explains the components of the construct (e.g. the if condition, optional else condition).
Practice for S1 involves tracing code line by line and ensuring learners can predict intermediate and final program states. This practice is consistent with tracing exercises that are common in many current CS1 curriculums. The figure below demonstrates an example S1 error.
To capture intermediate program states, practice may involve filling out a memory table which has learners externalize changes to program state (variable updates).
S2. Writing semantics (writing correct syntax)
Teaching S2 is about focusing on writing correct syntax. The previous skill (S1) taught learners to understand what a program construct does; this skill focuses on having learners be able to write code which uses the construct correctly. Our instruction of S2 focused on emphasizing parts of the syntax that could be confusing or overlooked. For conditionals in Python, the instruction emphasized the colon after a condition, proper indentation, and “elif” as the term for else if conditions.
To practice S2, we had learners translate “unambiguous” natural language to correct syntax and comment each line of code. Because S2 is about writing correct syntax and not about problem solving, demonstrated knowledge of the skill only involved correct “translation.” To differentiate between errors in understanding program behavior (S1) and correct syntax (S2), we had learners comment their code. If the program behavior deviates from the commented description, then this indicates a S1 error. If the program behavior aligns with the commented description but is still incorrect, this indicates a S2 error. An example of a S2 error is shown below.
This “translation task” is a new form of practice that we are still working on developing. In our initial evaluation and more recent think-alouds my colleagues and I have recently conducted, we found challenges to creating these exercises. One challenge is in making the natural language description as unambiguous as possible. Ambiguity may still exist in our word choice and grammar. Furthermore, how to indicate scope is somewhat unclear (use used bullet points because they reflected the indentation in Python). Another challenge we had was that learners found commenting each line cumbersome. How to write unambiguous descriptions of code and how to evaluate learners’ comments are open questions.
S3. Reading templates (recognizing templates’ objectives, components)
While prior skills (S1, S2) focused on learning program semantics (how code works), we now transition into how to use code to problem solve. As mentioned previously, our theory approached this with the use of templates, or reusable code abstractions. Examples of templates include variable swap, checking float equality, and using conditionals to find a maximum or minimum value.
Learning to read templates is fundamentally about learning patterns of computation that solve a class of problems. We framed learning a template as understanding its objective/goal as well as the components/steps that comprise it. Before introducing a template, the instruction provided an example or visualization that attempted to make the objective and steps of a template more relatable and concrete. An example of S3 instruction on finding the max/min value is shown below.
Demonstrating understanding of S3 involves looking at code and determining if it correctly implements a template. In the example below, we asked learners to select the answer that best describes the error relating to the digit processing template (using modulus to extract each digit from an integer). The memory table on the right of the figure demonstrates that the learner was able to trace the code correctly, indicating their mistake was related to reading templates (S3) and not reading semantics (S1).
We designed these exercises to measure S3 knowledge, so improvements could be made. In our evaluation of these questions, we found that learners often found the wording of the multiple choice options confusing. Some learners reported knowing the template was incorrect, but not being able to select between two multiple choice explanations. For these exercises to be effective, answer choices must be clearly written and non-overlapping with other options.
S4. Writing templates (writing code w/ templates)
Learners now know how code works (S1, S2) and we know a few templates (S3); now it’s all about putting everything together!
Writing templates is about writing code that applies a template to fulfill an objective. The instruction for S4 focused on providing rules to address errors that could arise when translating a template into code. Whereas writing semantics (S2) instruction specified rules to prevent syntactic errors, S4 instruction instead focuses on rules to prevent logic errors when implementing a template.
Exercises to demonstrate S4 knowledge involve providing a learner with a problem description, asking them to 1) write a “step-by-step” plan for solving the problem, 2) writing code to solve the problem, and 3) commenting the code they wrote. A correct plan suggests strong S3 knowledge. Code that follows the proposed plan suggests strong knowledge of S4.
Asking learners to plan their code without giving them training to do so resulted in a diversity of responses. Because we are trying to differentiate between errors in S1-S4, we asked learners to plan before writing code and comment code afterwards. While these additional steps may help learners think more deeply and learn more from the exercises, many study participants found these tasks quite laborious. So because we did not teach them how to plan or comment code and because many felt it was laborious, the quality and quantity of plans and comments varied quite a bit.
Evaluation: Skill instruction improved learning!
So by now I’ve described a theory of instruction which considers 4 skills and described learning materials where we can teach each skill and provide practice to observe understanding of each skill. But does this actually improve learning?
Our initial evaluation compared learning materials which taught each skill to equivalent learning materials which did not emphasize teaching different skills (a control condition we felt was representative to typical CS1 curriculum). Our evaluation was very small (4-5 participants in each condition), but we did find evidence to suggest that learning materials which provided instruction and practice for specific skills resulted in better learning.
More specifically, we observed the following benefits from teaching skills incrementally:
- Learners completed more practice exercises. We hypothesized that teaching skills incrementally would be less overwhelming than learning multiple skills simultaneously. We found that learners in the experimental condition completed more practice exercises, supporting this hypothesis.
- Learners made fewer errors, especially on more advanced skills. Because knowledge of later skills relating to templates) are built upon knowledge of earlier skills (relating to semantics), we hypothesized that weak knowledge of earlier skills would result in compounding errors in later skills. We found evidence to support this hypothesis when we observed that learners in the experimental group made fewer errors, especially in later skills.
- Learners had a greater depth of understanding. We hypothesized that learners would demonstrate a greater depth of understanding because they practiced each skill incrementally. When qualitatively coded the plans and comments on post-test questions according to the SOLO taxonomy. We found that learners in the experimental group produced plans and comments which were more complete and demonstrated more higher-level understanding than learners in the control group.
So while the evaluation was small in scale, it did indicate that this incremental approach to teaching programming skills could actually improve learning outcomes. We theorize this is because incremental instruction is less overwhelming to a learner, enabling them to develop understanding on a particular skill before adding complexity in the next skill. Neat!
Conclusion: Towards teaching skills novices need
While we believe explicit instruction of programming skills can improve how we teach computing, we’re just getting started! We have only just proposed this theory, thought up of one way (of many?!) to design learning materials according to this theory, and evaluated it with a handful of students. There are so many interesting questions on how to build off of this work for research and translate this work into learning materials that can benefit novice programmers. Open questions that we are aware of (there’s more, I’m sure) include better motivating tracing (S1) and writing correct syntax (S2) instruction because instruction and practice can be quite boring to learners because it is (by design) agnostic of what code can do. We are also thinking about how to improve the design of the practice questions we designed to observe mastery of specific skills (S2–S4). Finally, we wonder how to incorporate instruction on problem solving or designing templates (rather than having them be taught to you) into our theory.
We are currently working to improve the practice questions we designed and are working with Code.org to inform curriculum changes into their 2020 release.
For now, all I can believe the promise of this work is this:
More targeted instruction and practice of programming skills can result in learners feeling less overwhelmed, receiving more targeted feedback, and improving their learning experience!
This blog post accompanies a Computer Science Education Journal article I wrote with my (very thoughtful and hardworking) collaborators (link to CSE journal). For more on this theory, example instruction, and initial evaluation: read the pre-print, see the study materials on our GitHub archive, and reach out to me (via Twitter: @benjixie, other ways)!
To cite this article:
Benjamin Xie, Dastyni Loksa, Greg L. Nelson, Matthew J. Davidson, Dongsheng Dong, Harrison Kwik, Alex Hui Tan, Leanne Hwa, Min Li & Amy J. Ko (2019): A theory of instruction for introductory programming skills, Computer Science Education, DOI: 10.1080/08993408.2019.1565235
