How We Do Product
Our small but mighty™ product & data team is responsible for implementing and analyzing the features for donors and teachers who visit DonorsChoose.org. Basically, we solve our users’ problems through technology (with tons of help from our colleagues on the engineering team, of course).
Where ideas come from
The work our team takes on can come from many different places: feedback from our teachers and donors, changes in the competitive landscape, or suggestions from our colleagues around the org. Wherever they come from, we work to make sure that these ideas are framed as problems that need to be solved. This is important because it keeps the it-would-be-cool-if ideas from distracting us from our mission:
“DonorsChoose.org engages the public in public schools by giving people a simple, accountable, and personal way to address educational inequity. We envision a nation where children in every community have the tools and experiences needed for an excellent education.”
How we prioritize our ideas
Prioritization is our chance to make sure we’re solving the right problems. Our product roadmap is a list of all of the potential projects we could be working on. We meet weekly to prioritize this list with input from Charles (CEO & founder), Oliver (CTO & head of product), and other leaders around the org. Keeping these meetings brief, frequent, and casual has worked really well for us to keep our prioritization just-in-time so that we can react to the changing needs of the business.
We use a shared Google sheet since it’s easily editable and universally accessible to everyone within the org (transparency!). Although the roadmap maintains a rough prioritization of ~100 ideas, only the top 20 or so are real, meaning that we’ll get to them in the near future. Everything else is pretty much in limbo until someone lobbies for it to get closer to the top.
For the past three years, our rallying cry during prioritization has been to focus on the core experience and pay down UX debt that had accrued over our first fifteen years of doing business. For our donors, anything related to the donation flow falls into “core experience” — understanding our model, finding a classroom project to support, completing a donation. For teachers, it means everything from signing up, to getting their classroom request successfully funded. So what do we focus on once we’re done paying down all the debt? Well, I’ll save that for a future post.
Assembling the project team
Once a prioritized project makes it to the top of the roadmap and is ready to kick off, we put together a team made up of a PM, a designer, a UX researcher, an engineer, a subject matter expert (who has a deeper understanding of the specific problem we’re trying to solve), and a representative from the customer experience team.
We’ve found that the ideal size for our project teams is about five people for us. We make sure those five people commit to attending weekly check-ins, staying involved in the project Slack channel, watching all related user research, contributing timely feedback, and keeping their larger team updated about the status of the project. It is asking a lot of our non-ProdEng colleagues, so we try to make being part of these teams fun and the check-ins worthwhile.
We have silly names for each project team (the “mates” work on checkout, “hubbies” work on the teacher hub, etc) which sometimes include a team cheer that we use to close out our meetings. My colleague Tea also had the idea of awarding each team member with a team lapel pin so they can earn badges as we ship stuff. It’s been a big hit and resulted in folks feeling more invested and following through on the project team responsibilities.
The process
Once the team is assembled, we run through five major steps in the product development cycle:
Define
We define the main problems and the scope. What are we trying to solve? What are we not trying to solve? What are the key questions we need answered to move forward? This is a research-heavy phase that includes studying our users to find out what we don’t know and looking inward at our data to find out what we do know. We also define the key metrics that we plan to measure at this stage: what does success for this project look like?
Ideate
This word always feels jargon-y when I say it out loud, but it is the best way to explain this phase. This is where we kick around potential solutions to the problem at hand and explore some possible directions for the next phase. Often this means putting lots of sticky notes around a room, white boarding, or developing user personas. The goal is to narrow down the list of possible ideas to something we can take action on.
Prototype
Once we have clarity on some possible solutions, we put them down on paper. It could be anything from loose pencil sketches, to “dumb” mock ups created in Sketch, to partially functional code mockups created in Codepen. Since this is an iterative process, we often do all three in the same project. For example, we might validate a pencil sketch within the project team, then create a realistic-looking mock-up, wire them together with inVision, and put it through a few rounds of user testing to make sure it is solving the problems defined in the earlier stages. Once we’re getting close, we might build a code mock so it feels real and really put it through the paces with users.
We put such rigor into the prototype step because we want confidence before our engineers start writing code. As a lean non-profit, we are especially protective of our engineering resources and don’t want to waste their time.
Build
The build stage can last for anything from days to months, depending on the size of the project. During this time, the engineer sitting on the project team will turn the prototypes that we’ve refined into reality, though they generally have already done a great deal of work already. This can sometimes feel like the “black box” portion of the project, so we make sure to continue the team check-ins to stay up-to-date on progress and other team members shift their focus into preparing for launch. Once it’s functionally sound, the designer will dig into the code to make sure everything looks and feels right.
Analyze
Once things are built, we test some more. Usability testing with the “final” product can lead to more tweaks, but if things are looking good we move it to QA testing. We call this a “testing party” 🎉 and supply pizza in a desperate attempt to get more colleagues around the org involved. In addition to really helping to kick the tires of our work, these sessions help bring awareness about the changes we’ve made to people outside of the original project team. And pizza.
Once that’s done, we usually try to AB test the feature/changes using Optimizely. If a project doesn’t lend itself to AB testing, we might start with a pilot group and roll it out slowly as we build confidence in our solution. However we release it, we measure everything very carefully so we can adjust as needed.
Post-launch, we keep analyzing the results, which may include checking in with our users again. We also commit to sticking with the project for a couple of weeks to deal with any fallout from the initial launch. There are always follow-up cases with small fixes and refinements, but those are often prioritized separately. Since “perfect is the enemy of good,” we try to be intentional about calling something done when we feel like it’s solving the problem we identified in a meaningful way.
Why we’re writing this
This general introduction to who we are and what we do is just a start. With this blog, we aim to spread knowledge (it’s the DonorsChoose way) that could be applicable to teams facing similar challenges at non-profits and other companies working in the crowdfunding or educational space. Follow us if you’re curious to hear more about our process, the decisions behind any of the changes we make, or just about product, user research, data, and design in general.
