Four years of studying exploratory programming

Developers explore, but rarely have the right tools to do it.

When people write code, they explore. They try to implement something, see how it might work, and use that prototype to decide whether to build more, or back off, and try it a different way. This activity of exploratory programming is an inherent part of designing software, but it’s rarely supported by tools, nor is it a skill that’s taught in school. The result is that programmers either don’t explore, or if they do, they end up with a big mess of alternatives and no good way to manage them. Unfortunately, version control systems are heavyweight solution to this exploration, since they require developers to commit code to repositories when they’re not even sure if the code is worthwhile to commit yet.

This problem was the focus of a big National Science Foundation grant that was led by Brad Myers, Margaret Burnett, Martin Erwig, Gregg Rothermel, Anita Sarma, and myself, which we titled Variations to Support Exploratory Programming. We spent the past four years inventing tools, theorizing about variation in code, and studying and evaluating our ideas.

Here’s a list of just some of the discoveries we made:

• We explored how data scientists track variations in their code.
• We built tools for Jupyter notebooks that support a novel versioning model.
• We studied how web applications change over time and how those changes affect test cases.
• We invented ways of repairing broken web application tests.
• We built theories of how people forage for variations that they’ve archived.
• We built a new method called GenderMag for designing software that ensure gender inclusiveness.
• We invented a new theory of variations called the Choice Calculus, which has been the basis of many programming language and software engineering projects that support the management of variations, such as software product lines.
• We invented tools that support selective undo of edit history.
• We investigated problem solving strategies that novice programmers use and how they interfere with their ability to explore variations.
• We built loggers that support the visualization and exploration of edit histories.
• We built a time-travel debugger called Timelapse that allows web developers to capture lightweight traces of program behavior for later inspection.
• We built problem solving tutors that give developers of all experience better strategies for managing variation.
• We studied the role of mentorship in learning to code.
• We investigated attributes of software engineering expertise and how they support exploration in programming.

These are just some of the many discoveries we made in four years.

These projects resulted in more than sixty publications, several publicly released prototypes, and a lot of other impact. For example, one of my former students, Brian Burg, has begun to upstream his dissertation work on Timelapse to WebKit, promising a future of time travel debugging in our web inspectors (he now works at Apple). Other projects like GenderMag have begun to be adopted by many HCI faculty in how they teach design. I’m sure that the project’s numerous technical and empirical discoveries will have impact for decades to come, as our ideas wriggle their way out of academia into products, classrooms, and minds.

Other impact was on education. For example, the grant supported 26 Ph.D. students and 30 undergraduate research assistants, all of whom supported or led the research above. We also taught several summer camps, reaching hundreds of high school students learning to code. Many of those undergraduates have gone off to get PhDs. In some ways, all of this learning is the immediate impact that large NSF grants have, preparing students not only for graduate school, but also for careers in industry in which the open-ended problem solving they learned in research support the hard problems they face at work.

It’s always bittersweet when an NSF-funded project ends. Teams dissolve, students graduate, and the ideas spread into the world, silently. But there’s always another project. I have four other NSF grants I’m working on now, with new collaborators, new students, and new ideas, all shaped by the past four years of work with my excellent collaborators at Carnegie Mellon, Oregon State, and University of Nebraska, Lincoln. On to the next big idea!