Keeping Context in Context
How to see the big picture when performing literature reviews.
Context is key to the knowledge synthesis process when writing a literature review for an academic project. Forgetting to capture the page number, paper, or author of a quote, for example, is a minor mistake that can be very costly, potentially requiring you to skim hundreds of pages of text. We here report on an observational study of how researchers capture context and how these practices vary based on the tooling used: some tools make contextualizing information easier than others. At the end of the article we will recommend some tools for improving your literature review skills.
Literature review is the process of searching the (mostly academic) literature for prior art relevant to your current project. This search for information is almost never confined to a single source. By definition, you are reviewing literature to find the boundaries of human knowledge. This synthesis process can leave behind a mess of intractable information spread across physical and digital documents and note-taking applications. Ironically, in this very study on the challenges of context capture and reuse, I struggled to conduct my own literature review. I experimented with a combination of tools including Google Docs, Notion, Lucidchart, and Miro boards. I was trying to optimize my literature reviewing process, but each tool had its tradeoffs and combining them made keeping track of observations and citations even more challenging.
The last tool I used in my literature review, Roam Research, was invaluable for helping me synthesize ideas and results for this study; however, it wasn’t without its drawbacks. The red circled areas in the image taken from my Roam database below show where referenced information was deleted or moved only to be replaced with an unusable id code. Losing this information and simply not knowing what was lost resulted in unnecessary backtracking and lost time.
The example above illustrates how context —such as history of changes, metadata, and more — can be extremely important for reusing information in a literature review. A special challenge around this kind of information is that it’s not always obvious what contextual information will be needed later on, and adding it by hand can often be very tedious and distracting from the main task.
To understand how to design literature reviewing systems that better support context capture and management, we studied how researchers capture contextual information in their notes and annotations, and how this varies across generic vs. specialized systems for synthesis. A special feature of our study was that we observed researchers working on their own problems in their own settings, to better understand the nuances and details of current realistic practice. We observed three recurring patterns of context capture across our four participants, two of whom were using tools with special affordances for literature reviewing (NVivo and LiquidText) and the other two who were using non specialized tools (Google Docs, OneNote, and physical pen and paper).
In this post, we highlight two particularly interesting patterns of context capture that we saw in our participants’ specialized tools, which we think have high potential for supporting more effective literature reviewing.
Supporting context with transclusion
Both NVivo and LiquidText support some form of transclusion, where information can be disembedded from one place, embedded in another, and able to be directly referenced back to its source from the new location rather than requiring manual lookup with metadata. This feature is particularly useful in situations where the researcher needs to revisit the original source of a piece of information to gather more context. In our study we observed multiple instances of important contextual information being located at the source location of a disembedded note or highlight. Transclusion-like features can make this sourcing process easy and automatic.
Supporting context with flexible contextualizability
We observed participants separating and interacting with units of information which had higher levels of contextualizablity in the specialized group. The participant who manually transferred the margin notes from a physical document also needed to manually transfer any metadata they might need in the future. Both LiquidText and NVivo attached metadata such as page number and filename to highlights extracted from PDFs. Additionally, in NVivo, one feature enables a progressive revelation of context surrounding a disembedded note or highlight. Combined with transclusive properties such as hyperlinks back to the source, all of these features can make recontextualizing information much easier.
Closing thoughts
In an exploratory analysis conducted at the end of our study, we observed that the participants using specialized tools captured more contextual information than the participants using non specialized tools. However, this reveals several questions which can be the focus of future work:
- Do the qualitative differences in context capture mechanics lead to more context being captured?
- Do variations in context capture patterns correlate with ease/quality of downstream synthesis?
The context capture mechanics of specialized tools appear to make synthesis easier by reducing the amount of backtracking needed to find mismanaged or uncaptured information, but it remains to be seen whether or not this extra capturing of and support for context leads to gains in downstream synthesis.
Tooling recommendations
The following recommendations are for tools which support the literature reviewing process with some of the aforementioned features:
- Citavi: Reference management and knowledge organization. Features transclusion/ability to trace annotations back to original source
- Non-Linear note taking tools, such as Athens Research, Roam Research, and Obsidian, all support transclusion and flexible contextualizability.
