Product development in a time of crisis

Over the past weeks, I’ve been approached by several colleagues who were disappointed that the momentum we at ResearchGate had made towards implementing better processes for product development had been disrupted by the COVID-19 crisis. In various ways, they all asked the same question: how long until we can get back to where we left off? How long do we need to “survive” this?

I realized that we as product leaders haven’t done a good enough job at communicating the fact that there is no “correct” way to build products — there is just the right way for the given time and opportunity. And that strengthening the competency as an organization and as individuals to react to changing times and opportunities is one of the most important processes of all.

In this sense, the companies that do “survive” this time will be the ones that treat this period not as a pause in normal processes but as an extension of their continual work to react quicker, smarter, and more collaboratively to the changing landscape.

Times are difficult and stressful, and “survive” is indeed an appropriate word given the gravity of the situation. But that’s all the more reason to give people an understanding of how to approach every day not just as a day to endure, but a day to build lasting change for the future.

Let’s get rid of our old crutches and build new muscles to stand on

Now is not the time to “pause” momentum made towards better product development; it is a time to adapt existing learnings to a new reality. So what does it mean to adhere to the principles of unbiased and scientific product development in a time like this? When data is too noisy, and user behavior no longer reflects “reality,” does data-informed decision making simply get replaced by gut feelings?

In short, what will great product organizations now change in their processes in order to survive this time and emerge stronger on the other side?

First, let’s talk about what won’t work.

We cannot A/B test our way to the future. For one thing, the behavior we see now does not reflect the future. Even at ResearchGate, a platform for scientists — who some might assume wouldn’t be as affected as other industries — we see dramatic and rapidly changing local spikes in behavior. Scientists, like everyone, are living in an entirely different world from the one we used to know. They are at home, communicating online, and dealing with a crisis that affects their families and livelihoods.

As an organization, we need to build products that solve our users’ current problems but also connect to what we believe the future will look like. There is no dashboard or A/B test that can point us to that place. We will have to make decisions where the quantitative impact cannot be easily determined (and may even be negative), but because we deeply understand the users and customers we are solving problems for, we’re willing to take a leap.

We also simply don’t have time to let things progress at the typical pace; the safety of data comes with a major time cost.

The good news is that blind iteration is not science in the first place. Relying entirely on dashboards and A/B tests can create a false sense of safety that leads to low-quality products. Leaders in the product community have been working hard to reduce how much weight we put on these crutches. Now the crutches have been ripped out from under us, and we have no choice but to learn how to stand more quickly than we ever thought. But if we work hard, these muscles will serve us even when this crisis is over.

So what does this mean from a principles perspective? As companies push the pace, what can they use to ground work in a common mantra, and make it clear that this is not about enduring but also about building new core competencies and product principles?

At ResearchGate, we’re focused on using this time to double down on our processes in a few areas:

Get closer to the user or customer

The companies that react best to the current crisis will be the ones that return most quickly to first principles and rebuild their sense of intuition and empathy. I would expect every PM, designer, user researcher, and many engineers to be on a call with a user or customer multiple times per week. Setting up a company goal of “user exposure,” or how frequently ICs are exposed to insights from user interviews, can also make sense. And importantly, senior leaders should join these calls as well; the processes of renewing intuition about the right problems to solve should extend to every level.

Take a scientific approach to gut decisions

Matching the current time means making some big decisions based on intuition, but that does not mean we should throw out the scientific approach. In fact, the scientific approach is all the more necessary when making gut decisions, because the relative power and speed of the intuitive mind can be a powerful weapon only if its biases are constantly honed by rational processes.

Here are a few specific examples of how many of our teams are tackling bias at the moment:

Record hypotheses upfront, and set up the right structures to store knowledge. Whether it’s what you think you’ll find out in an interview you’re about to conduct, or the response to an email you think you’ll get, record your hypotheses before you test. These can live directly in tickets or tasks, or better in a decision journal or experiment-tracking database where it can be stored permanently and reviewed as a whole. This also means storing failed experiments as well as successful ones, because the former can be equally or more helpful to refining intuition. And just as quantitative dashboards are invaluable for establishing alignment around a shared metric target, this effort to store and find trends in qualitative data should result in something that is as useful for alignment as that graph you watch weekly as a team.

Work to reduce bias in data collection. One of the most pervasive biases we will find is self-confirmation. If possible after conducting the interview or test, have a colleague who is not familiar with the hypothesis analyse the results (in addition to doing it yourself). Then compare both of your results with your data to see which one is right. Science calls this “blinding” which is why everyone is hearing so much about “proper double-blind clinical trials” during this crisis.

These are just a few examples of some processes that I think many companies should revisit in times like these because they will be crucial for the work ahead. The greater point, however, is that we as leaders have the opportunity to give people the feeling that they are building something that is not simply plugging a temporary hole but meant to last for the future.

Many companies are now building products to hit new and fast-moving targets, and this often means throwing out old plans that were built methodically from data stretching back months or even years. These quick pivots can easily lead to the feeling that “good” work has been lost, or that processes are moving “backwards.” But few companies now have much time to set new directions that match a changed and changing world. Creating excellent processes which are grounded in a scientific approach to product development, and which can also hit new and fast-moving targets, is exactly what transformational product companies have done in the past, and will continue to do in the future.

This crisis is a forcing function to do these things well, and so it should be a call to action to see where weaknesses exist. But to suggest that the goal is simply to wait until old processes can be restored would miss a huge opportunity to grow — and to give much needed purpose to the people and teams involved in this effort.