Researching the Research Cycle: Part 1
by Mark McElhaw and Dana Chrisfield.
Researching researchers is sooooo meta. We rarely get the chance to reflect on our own processes, let alone research each other. This article takes a look at the research process and its relationship with the tools we use, by documenting our evolution and asking where next.
Last year the ResearchOps community embarked on another global project investigating research repositories. It’s taken longer than we anticipated, both for reasons of scope and 2020 — the year that time forgot. For example, a global series of workshops planned and ready to go, then when Covid-19 hit, we were going to make them virtual. However zoom meetings quickly became the opposite of exotic and extra time was reserved for meaningful encounters or just having some downtime.
Instead we decided to focus on some of the value propositions that had surfaced either intentionally or serendipitously. One of joys though, with elongating this project, is that it’s given us more time to distil our thinking. And one of the unexpected areas of interest has been the research process. Is it discrete or ongoing? And how does the way we record, store and share affect the value and impact of the research? But first an analogy.
Computing power and speed seem to be on an ever increasing trajectory, with quantum supremacy now on the horizon. However it wasn’t always that way. The 19th century had a smattering of engines as they were called then, the Difference Engine being among the most well known. A proper working model only saw daylight in the 1980’s and is a delightful clickety-clack of a contraption by modern standards. If you look at it whirring away, you can just about make sense of its machinations. Viewed through a sociotechnical lens, we can see that research is strongly affected by the tools we use. Research repositories are at a similar point of evolution as the difference engine; they still have some way to go before they are frictionless. So we need a research cycle that allows us to think about the way we conduct, store and share our research in order to get to that point.
Implicit bias
A quick search on “research process” will reveal a myriad of diagrams, each refracted through the researcher’s own area of interest, and focussed on either discrete and ongoing processes.
A good place to start is with The researcher’s journey by Dave Hora, which has been used as a basis for another ResearchOps project, the Research Skills Framework. This is a classic example of a discrete piece of research.
(Fig. 1. The research process model)
A good example of ongoing research is Blooms Taxonomy, which describes the learning process. Each stage is required to build the successive one. It’s often described as a pyramid in that each subsequent stage requires an increasing level of skill. While this perspective is not without its own controversy, it potentially sheds light on the distinction between discrete and ongoing research. The research process model (Fig. 1) stands in contrast to this evolution, where difficulty fans out from the center, with more advanced skill/experience needed for the start and finish.
(Fig 2 Bloom’s revised taxonomy by Anderson and Krathwol — 2001)
The two previous examples are all about the research process but what about the way that research is stored and most importantly reused? We don’t really think of tools as affecting our research because everything seems so easily available. This flies counter to Activity Theory, which postulates that all activity is mediated by tools, rules, community and division of labour amongst other things. This oversight has huge implications for the way research is conducted and the value of research insights.
We needed a research process that would:
- Reflect the knowledge management context,
- Accommodate all of the repository touchpoints in the research space,
- Cater to a variety of research perspectives that the people interacting with these systems might have.
The first area of interest was the relationship between primary and secondary research and so the curly Q was born. As researchers researching researchers, our own bias led us to focus on primary research — the collection of data first-hand in the course of a research effort — as the default, as the most common type. But we soon realised that secondary research is the prime use case for repositories and most people’s work.
(Fig. 3. The Curly Q)
With primary research, we are creating new datasets. But secondary research (aka desk research) encompasses all of the efforts in searching for, finding, and using data and insights that were previously collected or generated and making sense of them in a new context. Secondary research is ironically the primary type of research that most people conduct. We all do it. We just aren’t necessarily conscious that we are doing it. Secondary research is the reason that we need a thoughtful approach to storing research data. Secondary research is the reason we need repositories.
The Research and Repository perspective
We landed on a diagram that we have been referring to as the Tube Map as that is what it (sort of) looks like. The inspiration is the Glasgow Tube map if you have to know.
(Fig. 4. The Tube Map)
Here we are able to make a clear distinction between primary and secondary research. The right side illustrates the (very recognizable) flow of primary research. The left side captures the separate cycle of consuming, reuse, and sensemaking that occurs outside of a typical start-to-finish research project, and speaks to the way that primary research is taken, edited, redacted, reformulated, and represented by product owners, clients, and different departments.
Primary and secondary is dead! Long live gathering and sensemaking!
While it’s been good to make a distinction between primary and secondary research it also highlights an overlap. Many researchers in academia conduct primary research to create a dataset that they, and others, conduct secondary research on. Often this dataset is closely guarded, as it can also spawn it’s own field of work. Academia is all about papers and citations. If you look at the primary and secondary cycles you’ll notice that analysis and synthesis are really subsets of the broader category which is sensemaking. Additionally we found from our interviews that people spend a considerable time sourcing relevant material from disparate sources in libraries and online in general. We don’t realise how much time we spend conducting secondary research. A lot of the sensemaking for primary research is based on insights and connections made during secondary research. This leads to the conclusion that primary and secondary research don’t really fit modern commercial research. In reality people are either gathering or sensemaking. And they are doing this in repeating iterations more akin to an infinite loop.
(Fig. 5. The Infinite loop)
Q, tube or loop?
We’ve found that people will have a favourite diagram. Again it seems to depend on what people are using the diagram for and what mindset they have. And this brings us back to the original anecdote at the start. By the end of this adventure, we think we will be talking in terms of an infinite loop (Fig. 5) with big data gathering data and systems being able to sensemake in realtime. But it doesn’t really help researchers now nor the current evolution of research repositories. And that’s the point. From a maturity perspective research repositories still have a long way to go.
For now, we need something clunkier. Something we can get our heads around that specifies where the issues are in the current service and product design, so that we can improve both. The tube map helps us do this by highlighting a variety of things, such as:
- The efficiency of research process,
- The level of governance,
- The quality of the research deliverable,
- The capability of the organisation,
- The maturity of a research repository.
We’ve started to map all the operations that are involved in the research process. This has given rise to a new area, namely KnowledgeOps. It applies to the field of knowledge and information management. KnowledgeOps captures the breadth of storage systems from a document library (information management) to an insights repository (knowledge management). This also helps resolve a dichotomy drawn out in interviews, between the storing of research-specific and project-wide information. Think of this as a stake in the ground, as it’s impossible to do justice to these two fields in a single paragraph.
(Fig. 6. The Research+Ops Cycle)
Food for thought
Of course we expect this to change and split into additional layers, for example governance or templates. Each step can also be a cycle in its own right. For example, tagging has its own management process for grooming items in a taxonomy.
We’ve also added licensing as a major step. The emergence of research repositories also fundamentally changes how we think about consent in research. This shifts consent from belonging to a single point in time (ie the primary research), to something more fluid where the reuse of data depends on the context and purpose. Think of consent as a distribution licence, where people will have varying levels of access depending on their reason for using it. Most importantly it is the participant who decides what is shared, not the organisation.
Lastly, further consideration should be given to the way that primary and secondary research vary. Blooms taxonomy seems like a great place to start for secondary research. After all isn’t research, learning by another word? The research process model (Fig 1.) is also a great primer for primary research. Maybe they will combine, maybe they won’t. But that is for another article. This is just the sum of our thinking, at the end of this phase of the research repositories project.
Back to the metaness… while we’re still researchers, researching the researchers’ research process. Do you have a preference? How would you like to see this evolve? To infinity and beyond? Or a clickety-clack map that reflects the delays, bottlenecks, points failures and audit trails in your daily work? Is KnowledgeOps a thing? Is it even a cycle? And what is the relationship with ResearchOps? We’re obviously biased, but would love to hear your thoughts!
Special thanks to Brigette Metzler (aka the knowledge), as well as, Benson Brigitta Norton, Dorthe-Maj Jacobsen, Holly Cole, Ian Hamilton, Tomomi Sasaki and Kim Porter for noodling the tubes with us.
