An Immersive Reading Experience

Team Members: Saumya Kharbanda, Lisa Otto, Kaylee White

Our teamed explored how AR could enhance the personal reading experience. By linking your home’s heating/cooling, lighting and sound system to a book’s narrative ebb and flow, we could create a subtle immersive experience.

To demo the experience we created an AE video with the lighting changes, hid the projector in the ceiling and matched our video to page changes that our actor performed.

THE PROCESS

Exploration

We discussed ideas from each of the team members’ previous projects:

  • An informational sign that lets you (and only you) know about the progress being done on issues in your neighborhood that you care about (from Lisa’s ‘311’ project)
  • A way to draw on no surface at all (from Kaylee’s ‘any surface becomes a screen’ project)
  • A way to read without the burden of holding the book or other device (from Saumya’s ‘book annotation’ project)

We decided to explore a way to ease casual reading. We brainstormed gestures that could be done as simply as possible — for example, having eye movement recognition so you could just look off of the page to turn to the next one. And hand gestures for more complicated actions — like flipping to another chapter.

We also started brainstorming how we could create the Pepper’s Ghost demo using a pillow, iPad and glass from a picture frame.

Refining the Concept

After discussing in class, we began to consider that having a holographic book may not be the best use of AR. We realized that people often fall into one of two categories when reading a book: collaborative (wanting to share and discuss ideas) and personal (wanting to become highly immersed in the book). We brainstormed on the characteristics of those experiences and considered how AR could enhance them.

The Collaborative Reading Experience:

A physical book becomes a conversation. As you read, you find quotes that are creating discussion and read what other people are saying without having to shift mediums. The book becomes a place for deep conversations (i.e. “this point seems to contradict the last” rather than “the book was contradictory”).

The Personal Reading Experience:

Create an immersive experience with the book. The book is linked to your home’s lighting and cooling system. Markers in the narrative create subtle shifts in the atmosphere. This enhances the experience for the reader and invites family members to share what is typically a private experience.

Focusing

Just as Gandalf’s emotions control the “mood” of the room — lighting (maybe heating/cooling) — so could the narrative of a book control the mood your living room.

After receiving feedback from Austin and Dhairya, we decided to focus on the immersive experience of reading so we could explore the controls and gestures further. Austin pushed us to explore a more natural way to control the intensity of the “mood.” We thought about objects that you normally bring with you when you read. If we’re arguing that reading isn’t just about you and the book but the whole environment, then perhaps more of the environment should become part of the controls.

We particularly liked the idea of enchanted objects from this video (#3):

After discussing many objects associated with reading (classes, coffee, the pen), we settled on a blanket. The more you snuggle into your blanket, the more intense the experience gets. Indeed, the more you snuggle into a blanket, the more you are becoming cocooned into another world.

Here is an example of how fabric can become an interactive material:

Dhairya also helped us think of the idea of ‘saving a moment’ — not just bookmarking the text but saving the mood of that text. So perhaps as the story got dark and gloomy, you could be able to look back on the rosy to parts to remind yourself of how the narrative has transpired.

Dhairya also pushed us to think of the demo as a narrative in and of itself so we started to map out our demo in terms of narrative arc:

Show your support

Clapping shows how much you appreciated Lisa Otto’s story.