Bits and Behavior
Published in

Bits and Behavior

A mountain top view of the northwest end of Lake Lugano and the city of Lugano tightly nested around it, with many larger mountain ranges in the hazy distance.
My view from Monte San Salvatore, south of Lugano, Switzerland (a 2.5 hour hike).

ICER 2022 trip report: Together again, as bits and atoms

A crowd of thirty attendees with Matthias at the center explaining something.
Matthias shares the local history.


Matthias in front of a slide that says “Hybrid Fun”, virtual participants, #building instructions.
Matthias describes our pans for hybrid.


Julie in front of the slide “Historical Computing as Pedagogy” with pictures of computing history such as Ada Lovelace.
Julie smith gives her lightning talk.
Miranda in front of a slide “Assessment of Computing for Elementary Students” ACES.
Miranda explains an acronym.
  • Miranda Parker (San Diego State University) presented “A Pair of ACES: An Analysis of Isomorphic Questions on an Elementary Computing Assessment”. She talked about the careful redesign of an assessment for primary students about loops and sequences, creating additional isomorphic questions. Interestingly, and importantly, changing the assessment’s illustrations did not result in differences in scores, but changing agent movement direction did create different difficulties, possibly because of interactions between spatial reasoning and language literacy, but I’d imagine also cultural knowledge.
  • Xinying Hou (University of Michigan) presented Using Adaptive Parsons Problems to Scaffold Write-Code Problems”. Across two studies, she explored opportunities to use inter- and intra-problem scaffolding to support the development of program writing skills. She found that students found scaffolding to be helpful for getting started, for finding hints to translate their ideas into code, to help localize faults. A controlled experiment demonstrated that the scaffolding reduced practice completion time, but differences couldn’t be observed due to a ceiling effect in the pre-post assessment.
  • Juho Leinonen (Aalto University) presented Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models”. He discussed a method of creating items automatically, using a combination of templates, Codex, and single shot learning to generate program explanations. The team found through an expert analysis of generated items that most items (~70%) were sensible, novel, and well aligned with the provided themes, domains, and concepts, suggesting their utility as an aid in a more elaborate item design process, but certainly not a replacement for expert item design.
Jayce in front of a methods slide, describing sample, dependent variable, and nesting.
Jayce Warner presents an empirical analysis of Texas public school data.
  • Sarah Narvaiz (University of Tennessee) and Lexi Lishinski (University of Tennessee) presented Self-efficacy, Interest, and Belongingness — URM Students’ Momentary Experiences in CS1. Using an experience sampling method, they specifically investigated how students from groups underrepresented in CS have shifts in their momentary sense of belonging over time. They found a variety of differences and interactions, but I found the interactions between self-efficacy and final grades particularly interesting, suggesting that students experience failure differently depending on how marginalized they might feel. I wondered how much this was mediated by stereotype threat.
  • Jayce Warner (UT Austin) presented Gender, Race, and Economic Status along the Computing Education Pipeline: Examining Disparities in Course Enrollment and Wage Earnings. They quantitatively examined multiple dimensions of disparity as predictors of course enrollment majoring, and wage earnings, while account for multi-marginalization. With their large sample of students from the Texas public data system, they found that the biggest reduced likelihood factors were girls of all racial identities, but especially so for Black and Latina girls.
  • Spencer Yoder (North Carolina State University) presented Gender, Self-Assessment, and Persistence in Computing: How gender differences in self-assessed ability reduce women’s persistence in computer science. Through survey and interview methods, they examined an introductory programming course. They found that women at the institutions studied self-assess lower despite having comparable ability but come with higher rates of persistence (likely survivorship bias). Many of the reasons women reported for these trends were because of experiences of disrespectful treatment with presumed incompetence, but also held higher standards of learning than than their male peers.
Maria in front of a slide that says “Searching for Truth ? Rhetoric.”
Maria opening her talk on argumentation.
  • Elijah Rivera (Brown University) presented Plan Composition Using Higher-Order Functions. He talked about the importance of representation in planning using the domain of floor plans, then suggested that higher order functions are one helpful set of primitives for planning, especially in data science. To assess the viability of this, they explored how much students understand the language of higher order functions and how much they could plan and implement with them. They found that students generally understood the logic of higher order functions, that they could form correct non-executable plans.
  • Jamie Benario (Google) presented Using Electrodermal Activity Measurements to Understand Student Emotions While Programming. They examined students’ self-reported emotions and the events they attributed to them. They found 21 distinct events that triggered emotions, including things like halted progress, finding a resource, observing a failure, fixing a defect, or simply not knowing something. There were notable shifts from planning to implementation phases, associated with a major reduction in stress response.
  • Maria Kallia (University of Glasgow) presented When Rhetorical Logic Meets Programming: Collective Argumentative Reasoning in Problem-Solving in Programming. She gave background on the philosophy of persuasion (Aristotle’s pathos, ethos, logos), on Toulmin’s deconstruction of human reasoning and formal logic, and on Walton’s argumentative reasoning patterns. Maria and her collaborators applied this to examining how programmers of varying expertise engaged in argumentation during group programming: experts used collective and altruistic arguments, graduate students collective but egoistic arguments, and novices with collective “monologues.”
A group of ICER attendees in a selfie overlooking Lugano in front of the cathedral.
A group selifie at the Cathedral of Saint Lawrence.
Dustin in front of a slide that says “Misaligned Systems”
Dustin talks about undergraduate research misalignments.


Al in front of their slide “Teaching & Learning Inclusive Design Skills”
My student Alannah gives their lightning talk.
Kai in front of a “Research questions” slide.
Kai presents on teamwork struggles.
John finishing his point while Neil stands beside him in front of a slide titled “Time for Table Talk”!
John wraps up his talk on notional machines about call stacks.
Manu at the podium with a title slide that says “PRODUCTIVE FAILURE”
Manu starts his talk.
A slide that reads “Research question, RQ1: What teaching practices do teachers frequently use in their programming classroom”?
Xiaohua describes her research questions.
  • Xiaohua Jia (Leiden University) presented Teaching Quality in Programming Education: the Effect of Teachers’ Background Characteristics and Self-efficacy. She examined 164 teachers use of teaching methods in programming courses in Chinese schools and how these practices were mediated by self-efficacy and knowledge. They found that direct instruction and classroom management were the most common teaching skills enacted, and that different degrees of quality in these were related to teacher‘s self-efficacy.
  • I presented my Ph.D. student Jayne Everson’s paper “I would be afraid to be a bad CS teacher”: Factors Influencing Participation in Pre-Service Secondary Computing Teacher Education. We explored the factors that shape teacher candidate’s willingness to pursue a CS pre-service pathway and found that many surprising factors deterred CS teaching, including fear of extra teaching preparation, a fear of lack of respect, and a fear of perpetuating poor quality CS instruction they had experienced in their own education. But we also found motivating factors, such as correcting this historical injustices in their CS learning, empowering youth with what they viewed as a critical literacy, and connecting CS with other disciplines.
Amy, Megumi, Jean, Matt, Mara, Benji, and Alannah in front of the alps vista.
My lab does a group selfie.
A worksheet with a student’s many scribbles and circles showing a programming plan for an open ended project.
Diana shows an example of a student plan.


  • Diana Franklin (University of Chicago) presented Investigating the Use of Planning Sheets in Young Learners’ Open-Ended Scratch Projects. She talked about the tensions of 11–14 year olds’ ambitions in open-ended project-based learning, and the merits of planning documents to try to bound ambition to learning objectives. In a comparison of classes and school years with 103 students, Diana’s team found that when youth are expected to make plans, they do make them and do implement most of them, but that many students don’t implement interactivity, but that requiring plans was associated with meeting more project requirements. (The study didn’t reveal reasons for these trends).
  • Jane Waite (Raspberry Pi) virtually presented What do We Know about Computing Education for K-12 in Non-formal Settings? A Systematic Literature Review of Recent Research. They used the UNESCO definition of “non-formal”, as institutionalized, intentional, and planned education by a provider that supplements or is an alternative to formal education (and so they were not talking about informal learning). Their analysis of 6 years of studies found that most studies focused on interest, perception, engagement, and self-efficacy effects with the goal of broadening participation and a strong (likely publication) bias toward positive effects.
  • David Gonzalez-Maldonado (University of Chicago) unexpectedly virtually presented Comparison of CS Middle-School Instruction during Pre-Pandemic, Early-Pandemic and Mid-Pandemic School Years. David’s team examined changes in the delivery of the Scratch Encore curriculum in pre, early, and mid-pandemic school years by examining data from Scratch projects and partner school districts. They found that after lockdown, Scratch activity essentially disappeared, but later, it recovered to baseline levels mid-pandemic. Pre- and mid-pandemic auto-grading performance also returned to baseline.
Benji and Alannah on the right and the audience on the left.
Alannah and Benji presenting online after a positive COVID test forced many virtual.
  • Monica McGill (, Knox College) Joey Reyes (Knox College) presented Surfacing Inequities and Their Broader Implications in the CS Education Research Community. They explored equity barriers that different researchers face conducting computing education research through a systematic literature review and survey. Most of the factors they found were capacity issues, including lack of funding, collaborators, publication venues, participants, tools, instruments, job prospects, time, and a sense of belonging, recognition of and respect for the discipline.
  • Neil Brown (King’s College London) presented Launching Registered Report Replications in Computer Science Education Research. He described a shared concern that our research would not replicate and described questionable research practices such as practices that lead to publication bias, manipulating data to “p-hack”, discarding hypotheses in favor of post-hoc significant results. He then described and advocated for pre-registration as a way of preventing questionable practices. I publicly committed to implementing pre-registration for ACM TOCE.
  • Alannah Oleson and Benji Xie (University of Washington) presented our second ICER paper, A Decade of Demographics in Computing Education Research: A Critical Review of Trends in Collection, Reporting, and Use. We examined how researchers are categorizing people demographically and how that reinforces power, hierarchy, and hegemonic norms. The paper examined how data was gathered, reported, and used, finding that papers generally are quite exclusionary and ambiguous in their reporting and use of terminology, especially with aggregate terms like “at risk”, “underrepresented”, and “non-STEM”.
Noelle in front of a slide that reads “The shortest path to ethics in AI”
Noelle introduces her title slide and topic.
  • Noelle Brown (University of Utah) presented The Shortest Path to Ethics in AI: An Integrated Assignment Where Human Concerns Guide Technical Decisions. She described an approach to tying ethical considerations to the specific technical details of algorithms, particularly in graph search algorithms and examined how assignments might simultaneously assess technical and sociotechnical concepts. Their (very clever) assignment design challenged shortest path algorithms with a different cost function around someone traveling a path with a chronic condition. The core idea was situating technical decisions in a specific social context, and assessing the extent to which solutions considered both technical and sociotechnical dimensions of the algorithm and social context.
  • Amreeta Chatterjee (Oregon State University) presented Inclusivity Bugs in Online Courseware: A Field Study. They examined CS courseware for inclusion flaws using a novel automated tool based on the GenderMag method. They demonstrated that the tool was able to find many of the inclusion problems that an expert could, and that faculty using the tool independently were able to use the tool to find some of these problems, but that faculty sometimes disagreed with the tool’s analyses.
Jan at the podium and the four student volunteers in front of a thank you slide.
Jan thanks the student volunteers for running a great hybrid event.




This is the blog for the Code & Cognition lab, directed by professor Amy J. Ko, Ph.D. at the University of Washington. Here we reflect on our individual and collective struggle to understand computing and harness it for justice. See our work at

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Amy J. Ko

Professor of programming + learning + design + justice at the University of Washington Information School. Trans; she/her. #BlackLivesMatter.