From Bootcamp to Mastery: A Five Year Journey

As I look across the learn-to-code industry — with the proliferation of bootcamps, MOOCs, and alternative learning options — I often wonder why we (Launch School) are the only program that’s 100% mastery-based. There aren’t a lot of viable pedagogical options from which to choose, especially if the focus is on skills and results rather than credentialism. Yet, no one teaches in a mastery-based way except us. As I thought more about this, I realized that we, too, started teaching programming in a typical “bootcamp” fashion, and it was due to a unique confluence of personal and business factors that led us to focus on mastery-based learning.

This is a story about how we built Launch School over the last 5 years and how our opinions around programming, teaching, and business led us to a Mastery-based pedagogy.

The Backstory

I’ve known Kevin since 2002, when we were both software engineers at IBM. We had always talked about working on something together, but the opportunity never came up. Finally around 2012, we had a window of time where both of us were looking to do something new. We only knew that we wanted to work on something together, but didn’t have any concrete ideas. After months of careful deliberation, we decided to focus our thirties on Education.

Of course as programmers, the first thing we set out to do was to build a revolutionary Learning Management System (LMS) that would end all LMSes. As we worked through the specifications and design, one thing became painfully obvious: we had no idea what we were doing because neither of us had any deep experience with teaching or education. So naturally, before we could build a LMS, we had to get some experience teaching real students. Now, I’d like to think that we’re both pretty well-rounded people with a lot of interests, but we both really only had one skill that could attract students: programming. Towards the end of 2012, we decided to give up our (extremely) high paying jobs and try teaching people programming so we could better understand the problems around education and teaching (…so we could build an LMS to end all LMSes).

I share this backstory because this origin story will come back to influence many of our later decisions. It’s important to remember: we didn’t see an opportunity to make money and came into teaching programming as an exercise in learning about how to educate people.

Side Note: we quickly dropped the LMS idea because we found out students don’t buy LMSes, and selling a new LMS to large organizations requires a skill that we weren’t interested in developing.

2012–2013: Bootcamps

Unbeknownst to us at the time, this was the golden era of learning to code. In an odd case of multiple-discovery, we started our teach-people-to-code exploration at nearly the same time as many other companies, who later collectively came to be known as “coding bootcamps”. It was during this period that a few intrepid companies were starting to prove that you could get graduates a high paying salary after training for only a few months. That short duration caught everyone’s eye. Dev Bootcamp, in particular, nearly single-handedly created the “coding bootcamp” industry; to this day, it’s called “coding bootcamps” mostly because of Dev Bootcamp.

I happened to be based out San Francisco at the time and met with Shereef Bishay, founder of Dev Bootcamp, in their Chinatown office. Shereef became interested in what Kevin and I were doing and offered a partnership: we could roll our courses under the Dev Bootcamp brand and become their “preparatory” program. Because of their initial success, Dev Bootcamp started attracting a larger variety of students and many of their applicants lacked readiness. Not being interested in working for someone else, we declined. Besides meeting Shereef, I also grabbed beers with other local bootcamp founders, like Roshan Choxi and Dave Paola, founders of Bloc.io. It felt like something big was about to happen in the industry and San Francisco was the epicenter.

Meanwhile, Kevin and I continued executing our cohort-based courses. Our courses during this period were similar to ones you’d find in college: daily live lectures with a cohort of about 20–30 students with courses that lasted about a month. I had recently attended an online GMAT prep course offered by Knewton (they no longer do this) and was inspired by the format of their live lectures combined with ad-hoc quizzes. It forced participants to pay attention and follow along, and it felt like a much better experience than a typical college lecture, where you could sulk in the back of a large classroom and not ever engage with the instructor. The idea seemed promising: using innovative online tools, we could teach small live cohorts and ensure that everyone engaged with the material.

In order to figure out what topics to teach, we asked students what they would be interested in learning. Not surprisingly, they mentioned all the advanced topics that employers demanded: TDD, APIs, Rails and Angular (this was before React was popular), testing, algorithms, data structures, design patterns, best practices, etc. By this point, Kevin and I each had over 10 years of software engineering experience, so the list of topics seemed straight-forward enough and we set out to teach them.

The problems we encountered were immediate and obvious.

• Student readiness levels run the gamut. It’s impossible to teach TDD when someone doesn’t know basic programming principles. We can’t talk about APIs when students didn’t know HTTP. We can’t walk through algorithms when students can’t control nested loops.
• Related to the first issue, students didn’t keep pace with the lectures. About half the students stopped attending the live lectures after the first week. Though all lectures were recorded, few made an effort to make up for lost time and instead elected to go at their own pace. By the end of the month-long course, only a few students were still attending the live lectures.

The above two problems forced us early on to decide if we cared about students’ comprehension at the end of courses. If we didn’t, the solution would be easier: we could just sell recorded videos and content for a fixed price and focus our energies on marketing the content. On the other hand, if we did care about comprehension afterwards, we’d have to find another teaching format because while the idea of live lectures with quizzes seemed good in theory, in practice, most people don’t have the discipline to finish a rigorous course. And without the threat of withholding a credential, we couldn’t do anything to force people to show up.

These problems also forced us to think hard about who our audience was. If companies like Dev Bootcamp were able to train people for high paying jobs, why couldn’t we do the same, if only we selected the right students? My previous experience as an Engineering Manager told me that companies are willing to pay $15,000 to $30,000+ as a referral fee for qualified candidates. Couldn’t we monetize that end if we could find and train good students? This line of thinking only made things more confusing, because if we keep pushing on that logic, wouldn’t it be easier to just become a recruiting company? Why bother doing all the hard work of trying to train unprepared people when we can just filter for the best? That seemed like a more viable business, especially since all the startup literature says to charge businesses instead of individual users wherever you can.

Our initial stab at teaching people programming yielded some stars who landed great jobs, but that was, as is true for most education institutions, a result of selection bias as opposed to our amazing training methods. The choices in front of us were either to 1) figure out a way to make money and give up on making sure students actually understood the material, or 2) figure out a way to better train people for comprehension and not worry about optimizing for revenue.

We made a few critical decisions then that we still adhere to today:

• Students are our customers, not employers. By eliminating employers as a possible revenue source, it brought clarity to what we were supposed to do. One of the things we wanted to do was to help people, not only to make money for ourselves. After all, we had just quit high paying jobs to do something meaningful together. Helping employers didn’t seem very meaningful to us personally and while we were ok with that being a side effect of producing great programmers, we didn’t want to incentivize ourselves to become a recruiting company.
• We decided to not take venture funding. Though it may have been a bit early in our lifecycle to make that decision, we felt that training companies do not have a significant viral first-mover advantage. Instead, the advantage was in long-term reputation. Sure, it’d be possible to over-promise and over-hype the marketing in the short-term, but our hypothesis was that over time the lack of results will catch up with the hype. We had decided to dedicate our entire thirties to this experiment, and we felt that this long-term mindset could be an advantage in the education space. It’s interesting that Shereef, Roshan, and Dave opted for the opposite route with their companies and took on venture funding.

The consequences of those decisions significantly focused our energy.

By identifying students as our customers, we aligned ourselves with students and started to focus on pedagogy and comprehension, rather than throughput and conversion. It also meant we’re a B2C company and not a B2B company. This had implications to our processes. For example, we stopped doing sales calls to employers to try to get them to purchase licenses in bulk. Instead, we took time to have calls with every prospective student.

By going the bootstrapped route, we decided on a low-burn long-term financial plan, which usually meant sacrificing marketing for curriculum development. In our hypothesis, there’s no rush to get to market, and it’s more important to protect Launch School’s reputation by always doing “the right thing for the student”. Venture-backed companies have a “fail fast, fail often” mentality where growth rules above all. But in education, “failing” means negatively affecting students’ lives. We weren’t comfortable with purposefully hurting even a small group of students as part of the business plan.

2013–2014: Tealeaf Academy

We continued running our synchronous cohorts and the problematic patterns kept repeating cohort after cohort. We took everything we learned and decided to change our curriculum in a couple of important ways:

• From synchronous to asynchronous (aka, self-paced). Instead of relying on live lectures that were sparsely attended, we’d move to recordings that students could watch at any time.
• From one 1-month long course, we moved to 3 courses that would take roughly 4 months in total. The courses would start from the ground up, teaching basic programming principles to start, then building up to web development basics, and finally to all the advanced concepts employers wanted.

These two changes made a huge difference and students understood this sequence of courses much better. Instead of feeling overwhelmed in the first week, students could complete lectures and assignments on their own schedule. We didn’t give too much thought to the pricing structure and continued to sell the courses at a fixed price per course.

Even with the new self-paced 3-course sequence, results still varied widely. Some graduates got jobs that paid over $100k, and others who finished all 3 courses said they didn’t learn a thing. We posted the $100k student on our testimonials, but it felt like selection bias and not real education for all. It felt that despite our efforts to avoid becoming a recruiting company, we just ended up creating a recruiting company with a 3-course filter.

The whole point of charging students and forgoing funding was so we can align ourselves with students and do the right thing for students. So how can someone pay over $2,000 and spend over 4 months, and then say they didn’t learn anything? Even if it was a small number of students, that was still a crushing result for us. We couldn’t let it go and write it off as people being unprepared.

We decided to zoom in on the problem and try to understand the core of the issue. We participated in countless 1on1 sessions with students who were struggling and began noticing patterns. We would pair with students who were struggling in course 3 and see that what they were struggling with was not the advanced topic, but fundamentals. They couldn’t build an API not because they couldn’t intellectually understand the concept of an API, but because they didn’t know how HTTP worked. It had nothing to do with intellectual ability, but everything to do with understanding of prerequisite knowledge. When we asked “don’t you remember HTTP from course 1?”, they’d say something to the effect of “sure, kind of, but I went through that part pretty fast, and to be honest, it’s still a little fuzzy”. After seeing this over and over, we realized that we were missing a critical component in our courses: assessments.

After teaching people for 2 years, we learned what teachers across the world have known for centuries: you must have some test of mastery to demonstrate comprehension.

Upton Sinclair once said, “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” We fell into this trap by not thinking carefully about how our pricing fit with our pedagogy. We never seriously thought about adding rigorous assessments because it meant that less students would enroll in and pay for subsequent courses. We were financially incentivizing ourselves to usher students to subsequent courses without regard to mastery, which is in direct conflict with our mastery-based values. We charged per course, so adding assessments would have resulted in less revenue. The key lesson we took away from this observation was: be aware of how pricing introduces natural blindspots to your company or product.

2015: Lessons Learned

Having taught people for over 2 years at this point, we had enough information to go back to the lab and build a curriculum from the ground up anew. We spent the next year studying, researching and debating about what a great training program looked like. Over and over, we found ourselves constantly trapped by incompatible goals. For example, we wanted a democratic learning program that could cater to all, but how do you reconcile that goal with the desire to drive people to high paying jobs? You either have to give up the high paying jobs or you have to filter based on experience. If you only have a 4 or 6 month timeframe, what topics do you cover and how do you make sure people are following along? Is it ok if only the top 10% or 20% understand the material at the end?

To address these incompatible learning goals, we started from our own first principles by thinking about how we’re different, what our core beliefs were, and our personal stance on learning and comprehension. One idea that came up over and over in our research and discussions was operating for the *long-term*.

If we take a long-term perspective in our business operations, then it’d be possible to also take a long-term perspective on our pedagogical approach for the curriculum. We can’t have a company that’s focused on chasing quarterly revenue results and reconcile that with a long-term curriculum. The company’s vision and the pedagogy must be aligned. After realizing that, we made an important decision: we decided to not only spend our thirties on this, but to spend the rest of our careers on this project. That seems dramatic and conjures up images of a sworn blood oath under a full moon, but it wasn’t a hard decision at all and we made it fairly quickly and unceremoniously. That’s because 1) we didn’t have any other good ideas in the pipeline, 2) we believe that working on this problem will positively affect the world, 3) we believe in each other and don’t want to work on separate things, and 4) teaching online allows us to engage with a worldwide community of students, which brings a certain joy to the project. We didn’t have any reason to stop, and we thought that by focusing on decades in the future, we could use that perspective to our advantage.

Suddenly after that shift in perspective, we could see how a willingness to think about 10, 20 years into the future allowed us to unlock long-term value, both for us as a business and our students. While there were a lot of short-term incompatibilities between learning goals and business goals, these issues melted away when considered in the span of years and decades. Suddenly we could focus on skills to last a career, rather than chase short-term fads. We finally found a way to align personal, business, and student goals.

Just like how a long-simmering programming puzzle may come into more focus as one spends more time digesting it, the education puzzle began to unfold for us as we shifted into long-term thinking. With the long-term perspective as our north star, we came up with the following values for our business and learning pedagogy:

• Mastery of fundamentals first.
• No time limit for each course.
• Assessments to test mastery.
• Pedagogy-led pricing.
• Don’t focus on short-term revenue.

All these ideas taken together formed the foundation for our Mastery-based Learning pedagogy at Launch School.

2016: Launch School

It took us a year to build the new curriculum, and at the end of 2015, we launched Launch School. We didn’t have proof that this new curriculum would be good; it seemed right based on our experience and values, but since we just started, we didn’t have any concrete results to show. We asked prospective students to trust the process and asked if learning fundamentals to mastery made intuitive sense. We didn’t do any market research and built the new curriculum based off of our own standards of excellence, so we weren’t sure how people would react. Would they look at our proposal of learning indefinitely and then compare it with a 3-month bootcamp and laugh at us? Would they agree with us that the issue with learning advanced topics and frameworks was all about understanding fundamentals? The current marketplace was full of hype about turning around a six-figure job after a few months. How would people receive the idea of potentially learning for a few years?

Fortunately for us, some people chose to trust the process and started learning with us, from fundamentals with mastery.

2017: Capstone

By focusing on fundamentals, we felt we were setting up students for long-term success. But we still had the “last mile” problem to solve to demonstrate that there’s a quantitative difference between those who took time to learn fundamentals vs those who didn’t. After all, if the results between learning fundamentals for 2 years and cramming frameworks for 2 months are the same, why bother with the fundamentals?

Towards the end of 2016, we were able to take some of our Launch School students and put them into an intense instructor-led program to see if we could address the “last mile” problem. We created Capstone, a finishing program where students could apply their already-mastered fundamentals to more complex engineering problems. We wanted to show the world what’s possible when you take years to really learn something well by putting our Capstone graduates into the marketplace. We spent most of 2017 running Capstone cohorts and observing their performance in the most competitive markets in the United States.

2018: Results and Outcomes

Finally in 2018, we were able to showcase the results so far. Because it took a few years for us to wrap our head around the problem, and then a few more years for students to complete our curriculum, we are only seeing quantitative results now in 2018. Of course, we had many small victories along the way with many of our students saying our courses changed their lives, but teaching fundamentals for years also meant taking us farther away from concrete results. Now that we have them, the results are astounding; see for yourself.

Why doesn’t anyone else do Mastery-based Learning?

To address the question that initially triggered this article, I think we were the only ones who arrived at Mastery-based Learning because of the following.

We’re bootstrapped.

Other programs focus on financing and pricing innovation, partnerships, scholarships, marketing, government sponsorships, accreditation/credentialism, business process innovation, niche audience segmentation, but none seem interested in pedagogical innovation. I believe that we were able to focus squarely on pedagogy because we kept expanding our time horizon, which wouldn’t have been possible with venture funding. Had we taken investors’ money, we’d have been pressured to find a path to hyper-growth before the money ran out. This is why so many funded coding bootcamps are under stress and can’t innovate on one of the most important attributes for educational companies: their pedagogy.

Quality over data.

I like to think I’m a data-driven person, but many operators act larger than they are. Most small education companies are not operating at the scale of Amazon (the archetype for the soul-less numbers driven company), and yet they use numbers to override values. Numbers and data are important, but you must have some opinions on quality regarding your craft that you can’t compromise on regardless of what the numbers say. Had we followed the hard logic of numbers from our first year of teaching, we would’ve ended up a recruiting company because that’s what the data says employers wanted. There are also things we won’t do, no matter what the data says. For example, we just plainly refuse to “fail fast, and fail often” because it hurts people (also, we make enough honest mistakes that we don’t need a company philosophy to push for more). I remember first hearing about this concept and thought “that’s a great hack for startup founders”. But when you’re on the receiving end of this ideology as a customer, you think “what a bunch of amateurs and assholes”. In order to do the right thing, you have to have an opinion around quality. If you don’t yet have one, it’s important to move slowly and figure it out until you do. Following a 100% numbers driven analysis, no one would arrive at Mastery-based Learning.

Have core values.

A lot of people treat starting a business as a treasure hunt for revenue. In the course of running a business, many decisions come down to this choice: make money or improve quality. It seems counterintuitive, shouldn’t the higher quality product make more money? In industries where results are not obvious or delayed by months and years, it’s very possible to over-promise and lead with marketing. In such industries, it’s much easier to first make money and then figure it out later (another venture-backed mantra: “fake it until you make it”). One major lesson I learned starting Launch School was in learning more about myself. For example, I learned that there wasn’t one or two lines, but lots of lines I wasn’t willing to cross to make money. I learned about who I was, and who I wanted to become and it’s not a great entrepreneur or the founder of a multi-million dollar company. For me, it’s about trying to build something worthwhile that lasts as long as possible. It’s about enjoying the daily process of work and doing something positive for the world and working with people I enjoy being around. Just as high salaries are actually not the end goal for students at Launch School (they are a side effect of learning to mastery), revenue is not the end goal of the business side of Launch School — it is a side effect of becoming a meaningful long-term organization. I believe that this perspective is what helped us to unlock the long-term value behind Mastery-based Learning.