Enhancing traditional books with Augmented Reality

How the creative process changes when designing for analogue and digital media

Our first book, The Adventures of the Time Tub Twins, can be read from start to finish as a normal book, or enhanced with AR scenarios via our app.

Augmented Reality (AR) is hard. This industry has a history of companies who have tried, and failed, to enrich their content through AR. It was very important to us that if we were to use AR, we would focus on what we believe to be its key quality.

AR offers a natural way to engage with the content by allowing the user to view it from where it feels most comfortable.

Technology has revolutionised the way we tell stories, both by capturing the real world (pictures and videos) and by creating imaginary worlds (animated movies and games). AR makes possible the creation of a powerful new medium for interactive storytelling that combines the best of physical and digital worlds. And who better to enjoy it than children? As digital natives, a tablet can oftentimes be a more natural user experience for them, and AR can help them discover the joy of reading a physical book.

The Time Tub Twins began life as a traditional lift-the-flap book about a gifted child able to build brilliant inventions; but it was missing something. The themes the book focuses around — creativity, invention, science — made it perfect for innovation through AR. What if we could make the reader decide what invention to make? What if they could decide what characters they meet? Thinking about producing content for AR suddenly opened up a whole new world of storytelling possibilities.

The Time Tub Twins.

Designing for both analogue and digital worlds and bridging that divide between them can be challenging. In a normal book, identification with the characters and their feelings is supported by the illustrations and the text. How do we take a book, a naturally static medium, and add to it an element of interactivity?

We focused on adding value to the storytelling experience, allowing our young readers to stop being passive spectators. The AR layer we add to the pages of the book offers an open window to the story: children can decide how it unfolds through the interaction and personalisation of 3D characters and objects, thus becoming active storytellers. This adds to the book a virtual “replay“ feature that allows them to participate in a different experience every time they read it.

Not only do these features make the book more engaging and stimulating, they also make the creation process much more rewarding for us. Content displayed in AR can be viewed from any angle, similar to viewing the world through a camera lens, so all content we create must be 3D. And most importantly, AR makes possible the introduction of temporally dynamic elements, which involves designing characters that are animated and that can interact with both each other and the user.

One of our 3D characters. The grey skeleton enables it to become animated, deforming its 3D mesh similar to how our skin changes shape when our bones move.
Early character design sketches.

We are creating a new medium: not quite a book, not quite a video, not quite a game. Contemplating a painting is not the same as interacting with an art installation: they both tell a story, but convey different emotions and meanings, and stimulate the senses in different ways. The same can be said about a traditional book and our books.

Follow us on Medium to be part of our creative journey. And help us to make it happen by supporting us on Indiegogo, where you can also learn more about our project.


Originally published at medium.com.