This shall be CS50 2016
David J. Malan
For some time has enrollment in CS50 at Harvard been trending upward, as have been CS enrollments nationally. Whereas the course had 282 students on campus in 2007, the course had 750 students on campus in 2015, per the below.
To accommodate the growth in enrollment have we grown the course’s support structure proportionally, nearly each year recruiting more teaching fellows (TFs) and course assistants (CAs) to hold sections and office hours, approximately maintaining a TF:student ratio of 1:12. But the course’s demographics have been changing. Whereas in 2008 only 34% of CS50's students described themselves as “less comfortable” (with computers, the Internet, the mere idea of being in CS50, etc.), in 2015 as many as 58% described themselves as such, per the below.
At the same time has attendance at office hours been rising, the result, perhaps, not only of changing demographics but of our own popularization thereof. Whereas in 2011, attendance at office hours peaked at 120 students per night (20% of 607 students), attendance in 2014 (as best I recall) topped 350 (42% of 825 students), more than might attend a week’s lectures. To be fair, not necessarily a bad problem to have, since working alongside classmates and staff on actual problems is arguably time better spent than merely listening to CS50 together in lecture. But even with as many as 30 TFs and CAs on duty at once, students might still need to wait upwards of an hour for help, since even if we spend (unrealistically) just 5 minutes per student, each of us can only help 12 students per hour.
Moreover, even with comfort levels in equilibrium in recent years, the course’s weekly workload has (unintentionally) been rising, per the below.
But they weren’t identical. Indeed, nearly each year have we added something more to the course, particularly its problem sets, whether a new problem or resource or video. And it’s occurred to me in this, my tenth year, that we’ve never really removed anything from the course! In 2012 alone we integrated into some problem sets a series of shorts as well as check50, a command-line tool via which students can now check the correctness of their code prior to submission. Not only do those shorts take some time to watch, check50 may very well be driving students to spend more time on problem sets, since they now know (before their TF finds out!) whether bugs remain. As before, not necessarily bad problems to have, since students do find the shorts helpful and check50 useful. But, after ten years, the problem sets have put on some weight. Our own freshman 15 … or more hours per week, perhaps?
And so our new (school) year’s resolution has been to get back into shape. Indeed, this past spring, we went back to first principles, rethinking every aspect of the course’s design, taking no feature for granted, ultimately preserving what (we think!) works and changing what hasn’t. And we also focused all the more this summer on R&D, creating and experimenting with new curricula, new technologies, new experiences, and new workflows, thanks to our biggest summer team yet!
Here’s a (non-exhaustive) look of what’s new for Fall 2016.
What’s New and Why
It’s via lectures that the course introduces each week’s material. But sitting in Sanders Theatre, beautiful though it may be, has never been a particularly effective way to learn complex material. Indeed, insofar as the material within a lecture tends to be cumulative, whereby example i assumes an understanding of example i-1, it’s all too easy for a student to miss or misunderstand some detail, the result of which is a suboptimal experience in that lecture thereafter. Much as we encourage questions in realtime, only so many students (out of hundreds) can avail themselves of such, and only so many are comfortable doing as much. And sometimes we all just need a moment for something to sink in.
If only there were a way to pause or rewind! Simple though those features may be, I daresay we’ve been nearing the point for some time whereby it’s a better educational experience to watch CS50's lectures online than attend them in person. Indeed, with pause and rewind buttons accompanied by searchable full-text transcripts of lectures, variable playback speed, and hyperlinks to related resources, all of which lend themselves to more active engagement with the course’s lectures, it’s hard to argue that sitting and listening live in a theater is better. With that said, I do think there’s something to be said for the emotional engagement, if not excitement, that a shared experience affords. After all, consider the difference between attending a concert and listening to a̶ ̶C̶D̶ Spotify. (That we’ve played music in past years’ lectures is just a coincidence.) But after that initial engagement, the downsides start to kick in. Indeed, many students just want to get down to work, which, online, they can do at their own pace.
Moreover, CS50 has long been a “MW” course, meeting most every Monday and Wednesday for roughly twelve weeks. Yet those Wednesday lectures have always been awkwardly timed. Each necessarily introduces material for the following week’s pset, even though students are focused on the current week’s still. And, so, many students, rationally, simply don’t attend them. Indeed, a favorite chart from Fall 2010, thanks to Sam Moulton, is that term’s self-reported attendance, below. Notice the spikes. And how dare we hold class that one Friday!
To be fair, we could become an “M” course, with twice-as-long lectures on Mondays, but odds are that would only exacerbate the effect of missing or misunderstanding some detail mid-lecture. In fact, for that same reason did we actually start to shorten in Fall 2013 CS50's twice-weekly lectures from 90 to 60 minutes each, factoring out 102 coding examples for which Sanders never felt like an optimal venue for mastery. But a side effect is that lectures since have been perceived by students as lighter, with that material no longer part of a contiguous narrative.
It’s worth noting that most students who don’t attend Wednesdays in person do watch them online. (And some do both.) And increasingly do students watch Mondays online too. Contrast the first 18 lectures (for which we have self-reported data) of Fall 2014 and Fall 2015, per the below, which captures that trend. As before, attending in person and watching (again) online are not mutually exclusive.
Incidentally, it’s worth noting that we appear to have data to suggest that 4% of Harvard students overstate their attendance, seeing as Fall 2015’s second lecture (week0f) took place in New Haven. Ahem. Fall 2015’s eighth lecture (week3w) was shot in a studio at Harvard (because of a scheduling change), though, so that lecture’s 0% is as expected.
The fall of blue and rise of red between 2014 and 2015 is the result, we suspect, of multiple factors (one of which could certainly be me). In filming that second lecture at Yale (so as to connect the two campuses at term’s start), we introduced more students more quickly to the course’s own videos, explicitly messaging that watching online is okay (if not superior), and more students than usual latched on to that medium. And CS50's official allowance of “simultaneous enrollment” was by then in its second year, with all the more students aware that watching lectures online is not only allowed but encouraged. Much as I missed Fall 2014’s taller blue bars, though, there continues to be little evidence to suggest that attending CS50's lectures in person improves outcomes (as we’ve long reassured in supporting simultaneous enrollment), at least as per quiz scores. Indeed, Fall 2015’s distributions of quiz scores at both Harvard and Yale were essentially the same, and only one of those campuses was local to Sanders. This finding appears to corroborate an independent finding by HILT, who looked at CS50's Fall 2010 data on simultaneous enrollment in 2012.
This is all to say that there are quite a few reasons to do things differently and, we think, better in Fall 2016.
And so in Fall 2016 will students only be asked to attend in person the course’s first lecture and last lecture, aka Week 0 and Week 11, respectively, for the shared experiences thereof. (Though students with conflicts can still watch those lectures online.) Each of Weeks 1 through 10 will be filmed a few days earlier than usual, still in Sanders, with students welcome but not expected to attend. (The onus will be on me to keep things interesting, but the team and I have practiced!) For lectures that lend themselves to audience participation will students be invited to sign up in advance if they’d like to partake. Each of CS50's twelve weeks will now comprise just one (longer, sometimes denser but pausable) lecture, the video of which will be released on a Friday to coincide perfectly with a pset’s release.
Wednesdays, meanwhile, we will reclaim, repurposing them for walkthroughs of the current week’s pset, which I’ll lead still in Sanders but down in the orchestra section. (At Yale will be similar walkthroughs on Thursdays.) The effect, we hope, will be more intimate quality time than Sanders’ stage traditionally allows. For students who are not able (or ready) to attend will we continue to embed (new and improved versions of!) Zamyla’s own walkthroughs in the psets themselves.
Problem sets will now be due on Mondays at noon, rather than Fridays, thereby allowing students a 10-day window for each instead of just 7-. By design will pset i’s window overlap by a weekend with pset i+1’s so as to allow students to start some psets early if they so choose, per the below.
Much of the weight that psets have packed on in recent years will also be shed, with psets rewritten to be no less rigorous but still trimmer, with less front matter like shorts and less tl;dr. And instead of a “standard edition” and “Hacker Edition,” problem sets will now offer a mix of “less comfortable” and “more comfortable” problems from which students can choose, depending on their own comfort therewith. In years past the editions were mutually exclusive, whereby starting and potentially not finishing a Hacker Edition was an unnecessary gamble.
In years past, sections were offered on Sundays, Mondays, and Tuesdays. Sundays, though, were awkwardly before Mondays’ lectures. And Tuesdays were always a bit close to psets’ deadlines on Thursdays or Fridays.
In Fall 2016 will sections now be offered on Mondays, Tuesdays, and Wednesdays, thereby allowing students at least a full weekend to watch the week’s lecture, with office hours additionally available thereafter.
In years past were course-wide office hours offered Monday through Thursday at night, with Thursdays especially stressful (for students and staff!) as midnight approached, with psets due Fridays.
In Fall 2016 will course-wide office hours be offered on Wednesdays and Thursdays at night (after all sections have wrapped) and on Sundays during the day, thereby concentrating more staff at once on fewer days and nudging end-of-week opportunities for help toward less stressful afternoon hours. And the course will now offer lower-key office hours nearly every day atop HSA in Burke-McCoy Hall, where students will be able to drop in for help with a pset (a la the Math Question Center) from an on-duty CS50 TF or CA, particularly between classes, when all they might need is an answer or two in order to unblock an otherwise productive afternoon or evening of psetting.
The course’s web-based programming environment, CS50 IDE, has been updated for Fall 2016 to be all the more friendly, particularly for those less comfortable, with more advanced features still available for those more comfortable. Of particular note is the IDE’s new-and-improved graphical debugger, debug50, built by CS50's own Dan Armendariz atop gdb/mi, as well as help50, a command-line tool that will translate arcane error messages from clang and the like (using regexes) into friendlier, TF- and CA-like advice.
CS50 Coding Contest
For some years now has the course been bookended by CS50 Puzzle Day at term’s start followed by the CS50 Hackathon and CS50 Fair at term’s end, with term’s middle seemingly a missed opportunity for a course-wide event that brings classmates together, if competitively so, just for fun. And so will Fall 2016 inaugurate the first-ever CS50 Coding Contest, a mid-semester competition in C (for those students who’d like to opt in) inspired by Problem Set 5’s Big Board, prototyped this summer with students online via the first-ever CS50x Coding Contest.
In Fall 2016, we’ll shoot CS50's lectures not only in the usual 4K way (okay, I suppose that’s not all that usual) but also in virtual reality (VR), whereby students online (or even in Mather) will be able to experience lectures vicariously using Google Cardboard, Samsung Gear, Oculus Rift, HTC Vive, or the like, a la the teaser below. For privacy’s sake, portions of Sanders will be digitally obscured. (Dehance!) And toward term’s end will CS50 Explained be re-shot in VR as well to allow teachers beyond Cambridge to feel, if virtually, all the more part of pedagogical discussions.
All this and more. This shall be CS50 2016.
Special thanks to CS50's own Andi Peng, Annie Chen, Anthony Bau, Arturo Real, Brian Yu, Cheng Gong, Christian Saravia, Colton Ogden, Connor Doyle, Dan Armendariz, Dan Coffey, Davey Proctor, Doug Lloyd, Erin Carvalho, Glenn Holloway, Hari Anbarasu, Holly Knights, Ian Sexton, Jack Deschler, Jacob Lurye, Jason Hirschhorn, Jordan Hayashi, Kareem Zidane, Lauren Scully, Luke Jackson, Maria Zlatkova, Marinda Horan, Melinda Li, Miles Fertel, Noe Miniel, Patrick Rebeschini, Ramon Galvan, Rob Bowden, Stelios Rousoglou, Summer Wu, Tobias Büschel, Walter Martin, and Zamyla Chan for their contributions this summer to all this and more.