Class 6 Notes Essay — Reid Hoffman, John Lilly, Chris Yeh, and Allen Blue’s CS183C Technology-enabled Blitzscaling course

Here is an essay version of my class notes from Class 6 of Stanford University’s CS183C — Technology-enabled Blitzscaling — taught by Reid Hoffman, John Lilly, Chris Yeh, and Allen Blue. Errors and omissions are my own. Credit for good stuff is Reid, John, Chris Yeh, and Allen’s entirely.

This class was an interview by John Lilly, of Jennifer Pahlka — the founder and director of Code for America (CFA) — on the founding and scaling of the CFA program. The below is my paraphrasing of their conversation. To recap from last class, we are now on the Tribe Stage of scale — where organizations with product market fit begin to scale in operations.

Video of the class, notes are below:

I. Generalizations vs. Specific Examples

Before Jennifer’s interview there was a short discussion on how to process information from this class.

A common piece of feedback from students in the class is they wanted to hear more specific stories about specific situations. The next couple of lectures will all be from operators — not investors — and operators are in the guts and details of their own organizations all of the time.

The tough part is these specific examples are not the only way to do things, but rather one example. In the class we are going to try to explain the frameworks on how to think about things and then give specific examples within these frameworks.

For example, community was heavily talked about during the Mozilla story and in this class, but community is not the only way to grow a company. Keep that in mind while reading about the specific examples.

II. The Genesis of Code for America

John Lilly: What is Code for America (CFA)?

Jennifer Pahlka: CFA started as “teach for america” but instead of teachers it was for coders. The long term vision was to create an environment where the government can work for the people, by the people, in the 21st century.

The program started off as a service year program — where senior engineers would take a year long break from their career, not be paid full market rate, and get to work on meaningful problems our country faced.

Their first year they started off with 20 fellows and signed up 3 city governments — Boston, Philadelphia, and Seattle — to solve their civic problems through better software. They help governments take a data driven, iterative, and user centric look at the problems they faced.

They mostly worked with local governments because this is where all of the service delivery takes place — where the services the government provides is actually given to the people whom which those services are for.

John Lilly: How was the first year of CFA, what’s the story of how you started the organization?

Jennifer Pahlka: Before CFA, I was running the Web 2.0 series events with Tim O’Reilly and I had this idea to start this program. I was working on it part time until I could leave my position and work on it full time.

In 2010 we had our first opening call for cities to apply to the CFA program. We spent the first full year getting our city partners signed up and securing the initial funding to start the program. Our goal initially was to find the cities who wanted “a little bit of Silicon Valley” within their city governments.

Our tactic in the beginning was to create a nice looking website — which made us appear to look like we had a real organization behind the program. When we first started CFA, we didn’t have any city partners on board, no funding, no fellows, nothing besides the idea.

During our first year, our goal was to go out and find customers (city governments) then we would work to get the fellows. Our staff, fellows, and funding all came together right at the start of the first fellowship year. The whole first year felt like we were laying down our tracks as the train was coming down the tracks.

John Lilly: Why did you start CFA? What was the founding story behind the program?

Jennifer Pahlka: Through the Web2.0 Conference I was also working on a conference called Gov2.0. This was back in 2007–2008 and the big push for government technology was to get the government on social media.

Tim O’Reilly, who I worked with, had this notion that Gov2.0 was much more than social media — that the disruptive nature of technology also applied to government as well — instead how could you think of the government as a platform?

In the course of this work I was going to DC often and talking to people who were doing technology within government. What I saw was so bad — so much money was spent and wasted building products for government. It was an eye opening experience and I wanted to do something about it.

I’m not technical myself so I was first looking for someone to start this program but I realized no one else was going to do it. Even though I wasn’t technical I had spent a long time building a network of people who would be interested in CFA, so I could help solve this problem in my own way.

III. Initial Customers and Pricing

Question from the audience: How did you convince your initial customers to want to be a part of the program?

Jennifer Pahlka: One tactic we used — and still continue to use — is we made it a competition. We weren’t trying to sell our city partners anything, we were giving them a reward of bringing coders to their city.

All our initial customers were our early adopters and either had the departments or the political capital to afford to be able to try something new.

We still made mistakes though, in fact we initially had 4 cities. We were going to work with the city government of Washington DC. We had the right partner and a signed contract but right after the program was started mayor Adrian Fenty — our supporter for the program — lost his re-election and a new mayor came in. The first order of business for the mayor was to fire all of the fellows during lunch on their first day of work.

They were able to do this because CFA has the government pay the salary of their fellows + a mix of philanthropy.

John Lilly: This part is generalizable in the fact that most startups find a very special class of initial customers in the beginning — the early adopters. You don’t want mainstream customers as your first customers because you’ll be making a lot of mistakes in the beginning.

Question from the audience: How did you decide how to price your packages to the initial customers?

Jennifer Pahlka: It was simple. We took the amount we were going to pay the fellows — added a little bit — and asked for that amount. All of the customers except one paid the full price.

John Lilly: This is another generalizable lesson from the fact that many startups either as for:

• Cost + a little big more
• Just pick a number and ask for that much.

This sounds crazy in the beginning but its how you discover the optimum price. One thing to know is if your customer says yes to the price, than you know you priced too low. Next time ask for more.

III. The Product of Code for America

John Lilly: Moving further along, when did it feel like you had product market fit for CFA?

Jennifer Pahlka: First some context, in each city we would typically bring in 3–4 fellows per city. Each team typically consisted of one designer + two engineers, and together the team had all of the skills to build a basic web app.

There were two times when I felt like we had product market fit.

The first was in Boston. At the time, Boston had recently changed the rules about what schools you were assigned to based on the address of your home. They thought it was a good idea to tell people about this change through a 28 page brochure which explains all of the changes but doesn’t tell you what schools your kids could go to.

The parents were so frustrated by this change — there were all these news reports and videos of parents crying because they were so frustrated by the new policy. In addition the mayors office was getting inundated with angry letters and letters asking which schools their kids could attend.

The fellows in Boston started working on an app which would let parents type in their address + their kids age — and it would tell you what public school your child was allowed to attend.

The full app to 8 weeks from idea to shipped product. Nigel Jacob who was one of their partners in Boston — and now on the board of CFA — told them that if they had gone through the normal government channels — this app would have taken 2 years and $2M to completion. This is when Jennifer realized the cost and timeline of CFA vs. the normal process was just dramatically different.

The second time they felt product market fit was during CFA’s second year in Detroit. At the end of the year their partner in Detroit said — “I have spent 25 years in public service and I gave up on serving the public. After the tools CFA has created I feel like I can serve the public again”.

The big realization was the “product” of CFA was not necessarily the fellows or the products they delivered — but rather it’s the ability to government to see how technology products can be build in dramatically different ways than they are used to.

IV. From Founding to Scaling

John Lilly: How big is CFA currently?

Jennifer Pahlka: We now have 45 full time staff, 24 current fellows (many more alum), and 3,400 people who regularly go to CFA Brigade meetings.

Brigade meetings are these weekly meetings in City Halls around the country, filled with technologist. Think about is as the fire department for technology, they come in and help assist with technical issues their cities are facing.

What’s interesting is a lot of people consider themselves “part of Code for America” not just the fellows themselves. We have a big alumni network and we now have 6 startups who have spun out of CFA.

Another interesting observation is many of our fellows have graduated and are now going into city government and hiring us — and city government officials are joining CFA as fellows. The whole network is getting much more intermixed.

John Lilly: CFA also started an accelerator to bring in outside startups, how did that come about?

Jennifer Pahlka: After the end of the first year we saw that we were creating products which had the potential to become sustainable companies.

If we were going for our mission — to create a government for the people by the people — we needed to also change the companies which served the government.

We met many founders who were building products but not understanding the value of the government as a customer. Therefore we started incubating and bringing in companies to a government focused accelerator program.

John Lilly: How big is the government technology market?

Jennifer Pahlka: The market is around $200B domestically, and this is just for the technology. If you count service delivery, this market is dramatically bigger.

We spend about $1 Trillion dollars a year on government safety net programs, and many of these programs are delivered through very poor technology.

John Lilly: Let's talk about product lines, CFA is doing brigades, fellows, and accelerators — what products are you thinking about now?

Jennifer Pahlka: After our initial growth we have since started to focus on four main areas:

• Health & Human Services
• Safety & Justice
• Economic Development
• Communication & Engagement

We now think about these products lines not just in terms of the fellowship but through products we can create and companies we can create and sell to government in these areas.

John Lilly: Why did you decide to focus these areas?

Jennifer Pahlka: Over the last 4 years we have been asked to do projects in each of these areas. When we first started we used to be driven mainly by mayors:

• The mayor of the city would ask CFA to come to their city
• The mayor would get a good PR boost — “The Mayor is bringing in tech talent to our city!”
• Then we would do the projects the mayor wanted us to. For example the Boston school project was something which was driving the mayor crazy and wanted us to fix.

Over time instead of just doing cool demo projects, we wanted to build things that got real outcomes therefore we had to pick a couple areas to focus in on and go deeper. At the same time these 4 areas were the areas which had the “hair on fire” problems which were important to solve from both the government and citizen perspective.

Programs such as the delivery service of food stamps, delivery of child welfare, etc are huge problems for both the government and the citizens who are receiving these services.

Question from the audience: Why didn’t you do CFA as a for-profit company instead of a nonprofit?

Jennifer Pahlka: My reasoning at the time was I was asking people for a year of service, not being paid at market rate. People might not have joined if we weren’t a non-profit.

Today we now have 6 companies which have spun out of CFA and some aspects of CFA might be more for-profit driven. What’s more important is we want to play a role in the ecosystem of making government more efficient.

John Lilly: This was similar to Mozilla. We realize we couldn’t compete symmetrically with Microsoft (Internet Explorer) but we could compete in a new way. For us it was being open sourced and community driven but our quid pro quo was the Mozilla team was not going to become super rich from this effort. We had a community to our community and the nonprofit status was part of it.

Question from the audience: Where did the early funding for CFA come from and how long did it take to raise the money?

Jennifer Pahlka: In the beginning we raised $30K to get started in 3 checks of $10K each: 1 came from a friend, 1 came from the Steve and Jean Case Foundation, and the last came from the Sunlight Foundation. This was enough to get started.

Our first larger check came from Stacy Donohue of the Omidyar Network, this was the first Foundation to give a check to pay for salaries. We weren’t getting paid until that point.

Question from the audience: How difficult was it to close the first big deal?

Jennifer Pahlka: The first couple of deals with city governments were around $250K. It was difficult.

One of our tactics I mentioned before is we made it a contest. If the city won they could put out a press release. Another tactic which was important was they were only able to get the press once they had the contract signed.

One of the things they learned by working with the government is if they don’t have a deadline, it takes forever. It’s much better to have a forcing function to get something done.

John Lilly: Forcing functions are good in all contexts — not just with government or non-profits. Another tactic that works is FOMO — the fear of missing out — aka if they don’t sign the contract by a certain date, another city will get the prize.

V. Leaving and Managing

John Lilly: Ok let’s fast forward to 2013 — it’s the 3rd year of the fellowship, you are in 3 cities, you are hitting product market fit — and then you decide to leave and become the US Deputy Chief Technology Officer for the White House. Why did you do this?

Jennifer Pahlka: Todd Park who was the CTO of the Whitehouse, created a program modeled after CFA called the Presidential Innovation Program and he wanted me to come and help him run it.

I told him that I couldn’t move to DC and I have a startup, but I really liked Todd and I liked the idea of working within the Federal Government. I told him I would help him find a person, but I wanted him to think beyond the Presidential Innovation Program and to create a core service within the Federal Government to do good technology.

At the time I was visiting the Government Digital Service in the UK and I thought we should do something similar here in the US. I used recruiting opportunity to pitch him on doing something. I didn’t have any intention to run this program, in addition to my startup I have a daughter which I couldn’t move to DC.

However Todd Park is the best recruiter in the world. Todd is very persistent and will do anything to get you to join — I basically didn’t have a choice. Todd created this program within the Federal Government (This became the USDS) and said I should come to help run it — I was not intending to do this but it was an opportunity I couldn’t pass up.

The other big factor was I kept hearing from our fellows and customers that if I didn’t personally work within government, I wouldn’t really understand the problems from the inside. If I was going to spend the rest of my life asking people to work for the government, I would need to go and do this for myself. Now I can say I’ve been in your shoes and I know how it feels.

John Lilly: Once you stepped away, the team was built out at CFA. What was it like to come back? How did your perspective change?

Jennifer Pahlka: It was a strange experience. I had lived through a year in the White House which included the failure and rebirth of Healthcare.gov and the creation of the USDS. Looking back on my experience I felt like I didn’t succeed, it was a good experience but hard to tell when you are in the middle of it.

When I came back to CFA I was expecting a breath of fresh air. I was now the CEO, I had full control, I didn’t need to convince other people to do things.

However when I came back I realized the organization had changed a lot. Everything looked the same, so it took me awhile to diagnose what had changed. Looking back on it I should have done things differently.

The first thing when I came back was I met with every single person individually. 50% of the people who were there now were new since I had left. The problem was I wasn’t asking the right questions, the language was the same but the strategy had shifted.

When I left, the team had great intentions but went in a different direction than I had expected. It took some time and this was a big wake up call for me on how organizations worked.
Question from the audience: What would have the right questions been?

When I came back, the recruiting cycle was already set for the next set of partners and fellows. I didn’t ask questions about our sales offering and how we were framing our product because I assumed it was the same as before.

Instead I should have looked at things from a clean perspective starting from the ground up. Another thing I noticed were all of these zombie projects had great intentions but no metrics, no goals, and no progress. It was difficult to kill these projects but also to demonstrate that I trusted the team and I wasn’t a micro-manager.

John Lilly: When you came back, the staff doubled since you were gone. How did the CEO job change and how did you manage the company different?

Jennifer Pahlka: We needed much more operational strength in terms of managing people.

John Lilly: What does this mean “operational strength”

Jennifer Pahlka: We needed more help day to day. For example right now I am doing this talk and I am due to get the OKR’s (Objectives and Key Results Report) back from all of our focus areas lead.

Instead of being in the office I am here speaking, there is a lot of blocking and tackling which is needed. Someone has to do one-on-ones with everyone, make sure everyone is clear on goals, etc.

Right now we are hiring for a COO, we need someone for consistency. We need someone who is going to read through every single OKR — actually read them — and make sure we are on the same page and everyone else is on the same page.

Another thing that I have noticed is the words you say do not always mean the same thing to you as they do to other people. I go through this experience a lot. The things you intend vs. what is expected don’t always match up.

I need to remind myself daily that — I know I said these words, but this might not be what they heard — I need to double check. It’s hard to do this when I am on a plane to go fundraise, or speak, or work with partners, etc.

One famous quote which I am always reminded of is:

The single biggest problem in communication is the illusion that it has taken place.

John Lilly: As the organization gets bigger the CEO starts to say things like: “alignment,” “communication,” and all of these other buzz words.

The reason CEO’s say these things is they are trying to get the organization to make the same set of decisions, whether they are in the same room or not. The is very difficult, especially when the company is growing quickly and each new employee hasn’t had a lot of time to work with you directly.

Once an organization brings on more and more people, the CEO will say words and some of the staff will hear it perfectly clear and some of the staff will think you are speaking greek.

The other problem is the CEO learns new things everyday and slightly alters the old stuff, little by little over time. While the organization is getting bigger and bigger, you can’t get enough time with everyone to notice.

The rule I developed for myself at Mozilla was to say:

• Only a simple number of messages
• Repeat these messages over and over again
• If I need to make a change to the message — make sure its a big change — and make sure this change is noticeable.
• Repeat this over and over again.

Question from the audience: What was your strategy to penetrate culturally less accepting cities? For example cities who aren’t progressive and aren’t open to change from tech people?

Jennifer Pahlka: This is a common early adopter to mainstream customer question.

Our strategy has always been to make CFA seem normal. The more any city can look at another city that looks like them (similar size, similar political views, similar demographics, etc) and see how they are doing it — the more normal it becomes.

The trick is to just show up, make it work, and people get used to it. Over time the things that seem crazy, now just seem normal.

John Lilly: How do you tell whether CFA is winning or not? It doesn’t seem like there are clear user or financial metrics to look at.

Jennifer Pahlka: The official answer is we have clear metrics which we can report back to our funders such as — # of applications build, redeployments to cities, etc.

The unofficial answer is to be able to influence governments to make smarter technology choices. For example the State of California is about to put out an RFP for a system that will take $1B and take 5 years to build. In my experience things that take 5 years and cost $1B have less than a 2% success rate, especially within government.

Instead, it’s better to build iteratively, in small steps, at a far lower budget, in much faster cycles. The ability to be able to influence the decision to stop the existing process — because we have done something that has worked for them in the past — that’s when we know we are winning.

John Lilly: Project forward 10–15 years for CFA — what is the government like in the ideal case?

Jennifer Pahlka: The ideal is to make interacting with the government, feel more like a consumer app experience — similar to Uber or Facebook. This doesn’t just include us in the room but everyone the Government is working for — people on food stamps, people on welfare, people on unemployment, etc. This should be for everyone.

The ideal case would be that people have more faith and trust in government. Right now it's hard to support government and hard to support in programs such as social safety nets because our experience with them makes the government look incompetent.

If the citizens believe in government, then we can invest in government. I believe that if we are going to spend $1B on a program, this money should to helping people, not building a computer system.

Question from audience: One of the biggest hurdles with working with government, is it seems hard and seems it would include lots of red tape. What are your thoughts on this?

Jennifer Pahlka: There are many hurdles with working with Government, the biggest one is the procurement process.

Procurement is the way government buys things and we buy software the same way we buy pencils or trucks. We spec them out to extreme detail, we build it all at once, we add many extra steps in the process, etc. Procurement has good intentions — it’s supposed to reduce fraud, waste, and risk — you don’t want officials handing out contracts to their cousins.

This means if you are a startup you might experience 9 months of procurement just to get the bid. Also the paperwork is intense, for example one requirement is you need to certify that your office bathrooms are labeled appropriately.

9 months is also just to get the contract. These projects typically take 3–5 years to build. Within this time frame, the technology might have changed, the environment might have changed, etc — big monsterous products just don’t work as well.

Our belief is you should start small and build with your users. The good news is this process is slowly starting to change. Many organizations use CFA as a procurement hack — instead of using software, CFA has gone through the pain of procuring people who can build things.

Things are changing for the better, it might not be fast, but it is changing.

Looking forward to the next class!