The invisibility of failure in computing education

Over the past few years I’ve pivoted from research on developer tools to a new focus on computing education research (CER). I was tired of seeing learners fail, drop out, or worse yet, self-select out of computing altogether because they viewed it as too hard, too boring, too irrelevant.

Four years in, I’m still surprised by how rare this sentiment is in academia, particularly in Computer Science departments. In fact, most faculty in CS departments I know view CER as “just teaching”, “not computer science”, or “not hard”. In fact, I used to have this opinion of CER before I jumped into it.

Where do these negative and dismissive opinions about computing education research come from? I’ve been compiling a list:

• Most CS researchers are only familiar with the SIGCSE conference, and if they know anything about it, they know that its attended primarily by instructors, publishes short 6-page papers, and in its history has mostly published anecdotal observations about teaching innovations. If this is all someone knew about the field, they’d be right: most of the work at SIGCSE is not research, or at least not rigorous research. This has changed slightly over the past five years, but not much.
• Many CS faculty subscribe to the “geek gene” theory, believing that some people are born with the ability to code, and others not. If this is true (which it’s obviously not), there’s really nothing to be done to improve computing education, since learning doesn’t depend on the quality of instruction. That short circuits any interest in investigating better ways to teach and learn computing.
• CS researchers value computational things that haven’t existed before, that expand the power of computing. Contributions in CER generally aren’t new computational things, and even when they are, their power is in shaping how learners think, reason, and problem solve, not in creating new computational possibilities.
• The high-performing students that survive CS programs mask the failures of CS programs. Students get jobs, they create powerful software, they get paid more than anyone else, and they become productive members — and sometimes leaders — of the software industry. This survivorship bias makes faculty forget about the 50% of students who dropped out of CS1, the students who graduated without the ability to write a program from scratch, and the tens of thousands of students in our universities that would never even consider CS because of the racial and gender homogeneity or the unwelcoming culture.
• When students fail to learn, we often don’t see these failures because we don’t have good measures of learning. Most exams in CS classes test for declarative knowledge about syntax, semantics, and algorithms, and for program execution tracing skills. They seldom test for the ability to do the things that CS faculty actually value: elegant, modular design; efficiency; algorithmic problem solving; task decomposition. And they rarely test for the things that the software industry cares about: clear communication, planning skills, decision-making, self-awareness, reliability, and so on.
• Students successfully create software. That means they’re learning, right? Not necessarily. It’s very hard to see what went into creating a program. Most students make it through CS programs by leveraging TAs, classmates, and StackOverflow, and sometimes by cheating. If the goal is to educate students who can independently solve computational problems, that students have created things is no evidence of this ability. (That’s not to say that students should work alone — they shouldn’t — its just that teamwork and the Internet tend to confound our measurements of learning, and make us thing we’re succeeding when we’re not).
• There aren’t many examples of tenure-track CER faculty, creating a chicken and egg problem. Why would a CS department hire tenure-track CER faculty when there aren’t many Ph.D. students in CER doing amazing research? But why would there be any students if there aren’t any tenure-track faculty? Even if CS departments did value CER — and some do — there aren’t yet many researchers to hire.

Despite all of these problems, I’m optimistic. And there are concrete things we can all do to eliminate all of the biases above:

• Read the CRA white paper I helped write on the importance of CER and share it with your CS chairs, deans, and colleagues. We wrote it to make the case.
• Make sure your colleagues know about ICER (the ACM International Computing Education Research conference). This, and the TOCE and CSE journals, is where the most rigorous research is being published.
• Invite computing education researchers to speak at seminars, so departments can get to know what great research looks like.

Slowly but surely, we’ll bootstrap this thing into existence.