Leveling up your Ops and Research — a strategic look at scaling research and Ops
TL:DR: we have a new matrix you can use to create a strategy for scaling research and ResearchOps. It’s tricky, so you might just need to read the whole thing. The matrix can be found at the bottom of this page.
It can feel weird to talk about leveling up your research, about scale when we’re all collectively, around the planet, bringing ourselves into our family groups and staying at home. The truth of the matter though, is that there’s never been a better time for a clear vision than now.
Way back in 2004, I started my undergrad where a big part of it was learning about strategy. From 2017 on, alongside my work, I started a Ph.D. where the focus is on how we create change. What are the small levers that can create great change? In working in an Ops role for the past 3 years, I’ve seen some of those levers.
In the ResearchOps Community, we’ve done/are doing 3 global projects — we’re gluttons for punishment, or just super passionate about both the field of research and operations. Probably both.
The What of Ops
Our first global project was ‘What Is ResearchOps?’ — completing 33 workshops in 17 countries and a survey.
We pulled all the data together using a taxonomy we developed from the survey, creating a network map of the data. The question ‘what is ResearchOps’ was obviously set up to tell us the what. The map of that looks like this:
This framework has been a guiding hand to many as they set about creating a ResearchOps function within their teams (you can find the Mural version of it here).
The 8 Pillars: Towards a conceptual framework
Towards the end of 2018, a group of us on the board of the community (Emma Boulton, Holly Cole, Tomomi Sasaki and myself) realised that the taxonomy, or the conceptual framework we’d applied to understand the project could be used to understand the relationship between research and operations. Emma Boulton took this forward with the 8 Pillars (read more about it here). This rang a bell with so many people from both disciplines (research and research operations), it remains our most viewed article in the ResearchOps publication ever. If you haven’t read about it before, I’d encourage you to do so!
New understanding: Pace Layers
The Research Skills Framework, led by Dave Hora and Tomomi Sasaki was our second global project. This set about understanding the skills that are important to researchers as they advance in their careers. It is a granular level view that aims to create a dynamic understanding of technical craft skills and interpersonal skills, with tools for visually mapping progression. This deepened the community’s understanding of the factors that influence the kinds of skills that are necessary across a researcher’s career.
The Research Repositories/Libraries project (ResearchRepos for short), our third global project, has been ongoing since July of 2019. The project has 120 researchers working on various aspects of the project, one of which has been interviewing people leading libraries or leading ops from all over the world. We’ve interviewed people running the small research repositories to the biggest and longest-running ones.
I’ve been lucky enough to do about 30 of the interviews myself. As I did that, and also worked in my own context, where we have a mature level of engagement with research, something struck me. And it was about Pace Layers.
“Fast learns, slow remembers. Fast proposes, slow disposes. Fast is discontinuous, slow is continuous. Fast and small instructs slow and big by accrued innovation and by occasional revolution. Slow and big controls small and fast by constraint and constancy. Fast gets all our attention, slow has all the power.” -Brand, S., 1999, The Clock of the Long Now, p.34
In 1999, Stewart Brand wrote The Clock of the Long Now: Time and Responsibility. In it, he suggested Pace Layers could be used to understand complex systems.
This is deeply analogous to research, because Pace Layers are all about time, speed and depth. in design, this is a known entity- it was actually Mark Boulton who introduced me to Pace Layers, it may be completely familiar to you all. I believe so. An important point about the above quote is that ‘fast learns, slow remembers’, also, noting that ‘fast gets all our attention and slow has all the power’.
In many ways, organisations with sufficient research to have need of an operations function can also be understood as complex systems. They tend to have researchers who use varying methodologies, have people who do research (PWDRs), and people who need research as well as full-time researchers. All these people influence how research gets done, why and when.
This is a visual picture of Brand’s Pace Layers. The layers represent depth and pace. The further you go down the layers, the slower it moves, but the more the layer serves as a foundation. Using Pace Layers to understand different research methods can help with strategy development as it takes a holistic approach to research and Ops.
Rather than developing a strategy for the research team and a strategy for the Ops team, and a strategy for any other teams within the ecosystem, it is a whole of system view.
To Brand, the relationships between layers are key to the health of the system. More specifically, as both Brand and his co-creator in the clock of the long now project, Paul Saffo pointed out, conflicts caused by layers moving at different speeds actually keep things balanced and resilient. Paul called this “constructive turbulence.” Managing this constructive turbulence is the key to understanding what is constraining you and how you might scale. Understanding the layers and the 8 pillars as a matrix provides you with a helpful tool for diagnosing your strengths and weaknesses, and gives you a path to scaling and deepening your capacity across the layers.
Let’s look at the layers as they pertain to research methods:
A consistent complaint of researchers who use slower and more in-depth research methods (those closer to the practice of ethnography and anthropology- closer to ‘people’ and further away from researching people in the context of ‘things’), is that they struggle with constant pressure to reduce the cadence of their research. To deliver according to the cadence of business, rather than deliver within the traditional research methodology (lots of observation and research at the start, during which almost nothing is ‘delivered’). Understanding methods using Pace Layers helps to communicate the value of the slower, deeper layers, and also to see the friction between those layers.
On the other hand, the research that gets done at the top is the one that makes all the noise. It is quick, there’s high demand and researchers tend to be working within agile sprint cycles. Researchers up here, struggle with people saying what they’re doing isn’t really research, that it has no value. Sometimes, they have research leads or execs wanting to be more strategic or to get more from the research than is possible. The noise and speed can make it hard to drill down through the layers and get support for those slower types of research.
How about ResearchOps across the layers? Does the focus of Ops vary?
In short, yes. But the longer answer is that all research methods require every aspect of the 8 pillars to be in place in order for research to occur, but the focus is different depending on the dominant method used.
Pace layers and 8 Pillars as a matrix: a path to scale
If we bring the 8 Pillars and Pace Layers together in the one space, we start to see how the matrix might be used to understand how to scale. We begin to gain a picture of research and operations within the context of an individual organisation.
Case study 1:
The first case study is a research team of around 50, doing generative longitudinal, generative, causal and descriptive research. It’s a big company, there are PWDRs doing the evaluative stuff, but separately. The evaluative stuff is valued as a part of design and delivery, but not valued so much for strategy and policy. You could say this org has a high level of research maturity.
What we can see, is the research team in this org will tend to be focused on managing participants, consent and ethics and I’d suggest they’ll be running a research library, or wanting to. Their environmental challenges will be about continuing to show the value of what they’re doing, though they won’t need to evangelise research as such. Instead, the push back will be the time it takes, the cost, it will be managing the melody of long and slow with the needs of business. They will do that through a rigorously managed panel, good participant experience, and by building their base of research to a level that others can dip into it as needed. Their research is very manual, so there’s not a lot of focus on tools. All that contextual data will generate some hefty needs around consent and data maintenance. They are likely to employ a digital librarian and have a good relationship with their legal team.
How might they scale?
The team needs to see their strength is in their skills, in the research library, which they can use to support the PWDRs to level up their research. They can be the foundation on which everything grows. If they want to move into faster layers, jumping straight to the top layer isn’t going to be effective, because they don’t have the tools and tech in place to do so.
They’ve got two options- move up the layers from the bottom (moving more heavily into descriptive research), or join forces with the PWDRs doing the evaluative research, and develop their skills downwards until the whole ecosystem has strengths across the layers.
Case study 2:
What about a tech company with a focus on evaluative research? What does their ops look like? They are much more likely to have PWDRs (people who do research) or decentralised researchers working across design, development and product teams, much more likely to be wrestling with time, with getting insights to the right people at the cadence they need it. They’ll be wanting to concentrate on research as a team sport. They might have descriptive and some generative research, but it will all be in support of the top, noisy layer.
You can see there will be a heavy focus on the tools and infrastructure needed to support things like moderated and unmoderated usability testing. There will be a need for resources in the form of tools, templates, and guides for the PWDRs. A community of practice might be on the radar if the org wants to move further down the layers. Their consent is lightweight, most of the research is de-identified right from the start. They won’t be thinking about a library, and if they do, it will be likely held in whatever system the developers use- these are the confluence libraries, the libraries in Jira.
How might they scale?
They could move further down through the layers by putting on some more experienced researchers to stretch and develop their PWDRs. That crucial research hire (or hires) will allow them to do some of the lower research. They probably have researchers in their org skilled in statistics due to the more quantitative nature of the faster research. They could quite easily move down towards descriptive and causal research without too much investment in tech or a significant increase in Ops functions.
They’ll need to be extra careful as they move down to communicate about the slow, deep layers and how they fit in the research lifecycle. If they scale too quickly, panel management and data management (in terms of ethics and consent) will become a problem. Those are the bits Ops people can focus their attention on, working with the researchers brought on to extend their research skills. If they finally make it down to the layer of wanting a research library, for use of research assets, rather than using research as evidence (as people tend to do with evaluative research), then this will be their flag that they’re on their way to the very bottom of the layers, and well on the way to a fully scaled research practice.
Another piece of the puzzle, a tool in your toolbox
We now have a significant power- to create a strategy. The 8 Pillars framework acts as a lightweight strategy model, just like a traditional PESTLE model — it includes the political, economic, social technological, legal and environmental factors involved in making research happen.
If we add a few extra ingredients — a vision, a mission, we’re getting towards what we need. But to really create a robust strategy, we need to know all the moving parts. And we need to see how they contribute to the whole. Luckily, it is the function of ResearchOps to do exactly this.
As you can see from the above matrix, you now have the capacity to see your research within the whole system, and all the moving parts that make it up. You get an understanding of the ways in which research is, and isn’t, at the centre of your organisation. The Pace Layers/8 Pillars matrix is a crucial piece of the puzzle, allowing you to see all the moving parts. To visualise where your existing strengths and weaknesses are, and just what you need to achieve a fully scaled research practice.
We in the ResearchOps Community hope you find this matrix useful in evaluating where your organisation’s focus lies, how your research fits into that focus, and how this influences your research operations. Even more, we hope this matrix can serve as a map for scaling, helping you anticipate where you might need to adjust your focus, how you can best use the skills and resources you already have.
To access the full Advancing Research conference content (session recordings, tripnotes, sketchnotes, and speaker decks, please contact info@RosenfeldMedia.com.
A detailed description of the Pace Layers aspect of the matrix is featured in an essay in the forthcoming Research Practice book, edited by Gregg Bernstein. You can receive a notification once this book is published via Gregg’s newsletter.
This matrix is only the next iteration of thinking from the ResearchOps Community, and like all things from the community, is licensed under CC BY-SA 4.0 by the ResearchOps Community. When in doubt, please ask. If you’d like to join the conversation and help us evolve and develop the field of ResearchOps, you can join the community by completing the join form on our website. See you there.
