Interactive marginalia
For this year’s annual Interactive Fiction Competition, I entered a hypertext story called Harmonia. (It came in third place, hooray.) The story, in part, pertains to the ways in which narratives are re-read and re-interpreted over time, as expressed by annotations and scholarly commentary. I’ve always been fascinated by the scribbles found in old or scanned books; I once wrote a program to deface books for me. Harmonia is a story about the dialogue that annotation represents between readers separated by years, decades, or even centuries.
Competition authors often write postmortems about their works, and because reviewers expressed interest in the annotation interface in Harmonia, I’ve written about my design and implementation process. There’s some technical detail near the end, but this article should be broadly accessible to anyone with an interest in the visual and experiential design of digital narrative.
(You don’t have to have read the story to follow along here, though it would take about an hour if you did. If you’re pressed for time, Sam Kabo Ashwell wrote a review that is thorough in its engagement with the themes and astute in pointing out some of the story’s weaknesses.)
A very brief history of annotations on the web
The Golden Notebook Project, a collaborative reading of the Doris Lessing novel by seven selected readers, is my go-to ur-text for digitally annotated fiction. The site is from 2008 but I still draw inspiration from it:
The main text is book-like but still quite readable on a screen, and—appropriately for this project—the commentary shares equal weight. Because the discussion is high level, the annotations aren’t granular at all; they’re tied to the level of the chapter, but not word or even paragraph.
In a more modern example, The Washington Post uses Genius (originally the site Rap Genius) to annotate particularly knotty political stories:
These annotations are highly granular. The Post is providing a gloss on exact phrases or sentences, some of which carry specific legal meaning. The use of yellow as a highlight color plays to the sense that we’re engaging in Serious Business.
Both The Golden Notebook and the Post implement annotations as marginalia, where the otherwise blank space beside the text is filled by the annotation. (There’s a lively community around web marginalia and I recommend Hypothes.is when implementing annotations in non-fictional contexts.)
Another interface option is to overlay the annotation as a modal or pop-up, as Wikipedia does for citations and references:
On the whole, though, there isn’t a single unified interface for web-based commentary or annotation, the way there is for, say, filling out forms or navigating using menus. Every site or project re-invents the wheel.
Marginalia versus modals
Presenting an annotation as a pop-up has some decided advantages over rendering it in the margin of the page:
- A pop-up is a very compact representation, not requiring one or more wide margins to accommodate the annotation. It works well on narrow mobile screens.
- It is a native digital affordance, so potentially a good fit for a digital story.
- The position of a pop-up makes it crystal-clear which text is being annotated.
Popups and modals are familiar UI components and I could assume that they’d feel natural in a web-based presentation, but I decided popups were not appropriate for this particular story:
- I wanted the design for Harmonia to feel like a loose digital interpretation of paper, not a “website,” and popups aren’t very book-like.
- Popups are awkward to use: they require one click to open and one click to close, and closing them usually means clicking on some kind of button — also not very book-like.
- Popups overlay the text until they are closed. It was important to me that the reader be able to have the narrative and the annotations fully readable at all times.
Asides and links in hypertext
In the narrative of Harmonia, characters are frequently commenting on other texts. In some cases, the commentary is a modern scholar writing notes in the margins about a work from a hundred years ago. In the bulk of the story, though, the commenter is also the narrator, as the protagonist Abby reflects back on her experiences from an unknown time in the future.
Abby’s annotations act like parenthetical asides. A common method to handle asides or digressions in Twine is to simply link to a new page. When done well, this cleanly focuses the reader’s attention on the text at hand. From Cactus Blue Motel by Astrid Dalmady:
Selecting BROCHURES takes you to a new page with a generous description:
There are two reasons why the new-page UI works well here:
- Balance: Each section of Motel is roughly the same length (about a screenful) and each digression is about the same weight. The reader doesn’t click a link only to be disappointed by a perfunctory description.
- Consistency: The back arrow is clearly marked, close to the text, and always in the same place. When the reader clicks “back,” the main text loads quickly. The experience is smooth, stable, and predictable.
Otherwise, I don’t think that linking to a new page is usually the best approach in hypertext. The reader loses their place in the story, the new text may not fully justify the effort, and it runs contrary to reader expectations both on the web and in print. It’s an effect that can be used well in some stories, like Motel, but it’s not a great default.
Hyperlink-centric annotations
The fundamental unit of hypertext is the link—specifically, the exact words that are linked. Consider this sequence of text from Stephen Granade’s moving hypertext story about the dissolution of memory, Will Not Let Me Go:
“You told him you and he were through.”
Wait, that can’t — You wouldn’t — He was one of your best friends.
[user clicks]
He is one of your best friends. You can’t believe you’d want him gone.
The sequence reinforces the protagonist’s confusion about time by drawing your attention to the tense of the verb. The presence or absence or part-of-speech of every word under a hyperlink matters in hypertext fiction:
“The friendly wolf” suggests that this particular wolf’s demeanor is notable.
“The friendly wolf” is just the wolf who happens to be in front of you.
So my last design constraint was to retain a visual hint of which words triggered the annotation even after the click had already occurred. In the Genius example, the annotated text is in yellow highlight; in Wikipedia, it’s at the tip of the arrow. I wanted a design that accomplished the same goal but evoked handwritten marginalia.
The final design
Harmonia’s annotations are initially presented as normal hyperlinks in the narrative:
When the user clicks, the solid underline changes to dotted, the annotation appears in the margin, and a curved line is traced between them:
The annotation will appear on the margin closest to the link, to minimize the length of the connecting line and the amount of text the line has to obscure. The dotted underline retains the sense that this was once a link (and is precise about the words that were selected), but is distinguishable from an active, clickable link.
Hyperlinks in Harmonia don’t always trigger annotations: sometimes they just advance the story, or offer the reader a choice among options. I considered whether to apply a different style to each link type, but opted for simplicity when my testers didn’t feel it was necessary. (Some reviewers disagreed!)
In some places in the story there is commentary-on-commentary—annotations which then link to other annotations. In this case the new marginal note appears beneath the other, connected by a vertical line:
Each character in the story is assigned a font and a color. My goal was that after a few repetitions, readers should be able to tell who was the author of any given note just by context and the visual cues. I favored fonts that were narrow and legible at a small size, particularly for Abby, who writes the majority of the annotations.
(I’m acutely aware that the story cannot really work for non-sighted readers, but I wasn’t able to settle on a non-visual representation that would adequately express the relationship between the hyperlinks and the annotations. I do think that the majority of digital fiction ought to be accessible to screenreaders—as authors we should seek to accommodate this in our designs.)
Line items
The connecting lines are implemented using the jsPlumb library, which can draw SVG bezier curves between arbitrary HTML elements. The annotations are absolutely positioned in the margin, and as a consequence I needed to manually handle collision—the annotation code will try to drop new notes below existing ones. This can be buggy, especially if the reader clicks links out of the linear reading order, and if I had to do it over again I’d use a different positioning strategy and let the browser manage all that for me.
I wanted the lines to look hand-drawn, so they have an SVG filter applied to them that introduces noise. I like the effect, but at least one reviewer thought it looked pixelated. In testing I found that Safari has a buggy implementation of SVG filters and as a result the effect is only applied in Chrome where I knew it to work.
Animations
As seen above, the lines animate when drawn. I knew that the overall design of Harmonia was quite busy, with lots of fonts, text colors, line drawings, and embedded documents, so I hoped that a subtle animation would help orient readers without annoying the hell out of them.
Animation in hypertext should be used very sparingly; most animations are too slow, and an animation that looks cool the first time becomes tedious with repetition. I made the animation quick and ensured that it only happens once per annotation — if you undo a choice by hitting back the line simply disappears; if you resize or refresh the browser, the lines reappear but don’t animate again. These may seem like edge cases, but when asking readers to spend an hour or more on a single web page, it’s well worth considering the whole lifecycle of the experience. Nobody complained about the animations, so I consider that a win.
Mobile last
Initially I thought I’d have to prepare an alternate design for phone-sized screens. When I tested with my phone in portrait mode, it was indeed too narrow to accommodate the UI, but to my surprise, landscape worked fine and the text was still readable even with my middle-aged farsightedness.
I used BrowserStack to determine that recent Android phones also worked in landscape. I added an unsubtle message in portrait mode directing users to rotate their device. One of the very first reviewers completed the story using an iPhone, so I’m happy that worked out, though I don’t recommend waiting until the end to see if your UI concept works on very small screens.
Future interfaces
Many of the entries in this year’s Interactive Fiction Competition featured novel or custom web interfaces for their stories. I’d love to see more discussion about UI and UX best practices for browser-based hypertext fiction—topics like:
- The role of the browser itself in story UI: Can you move forward and back through browser history to navigate story choices? Does the browser remember where you left off? When is it appropriate to deliberately break browser conventions?
- Accessibility: should hypertext fiction be usable by mobile browsers? By screenreaders? How can we facilitate translations? How can non-technical authors get help making their stories more accessible? (There’s already funded activity in this space, which is terrific.)
- Best practices in use of multimedia: when are sound, video, and animation additive rather than distracting? How can authors augment their stories in creative and meaningful ways without alienating readers?
Hypertext fiction is not new, predating the web by close to a decade, but the medium has transitioned into a period of relative maturity thanks to popular frameworks like Twine. I’m reminded of the parser IF period of the late 90s, when authors, freed from low-level programming by expressive domain-specific languages, could come to a consensus around design concepts like “fairness” and “cruelty.”
This is a fertile period for any emerging creative medium. Readers benefit from some amount of convention and predictability—stories are more approachable, with fewer unpleasant roadblocks that distract from the storytelling. Paradoxically, authors benefit from convention too—they become empowered to break those conventions in narratively meaningful ways. I’m excited about where this community is going, and hope you are too.
References
- Read Harmonia online.
- Story source code. The code for the annotations.
- Another author’s thoughts on visual design in their entry.
- The Interactive Fiction Competition.
