Project 1: Way-Finding Through Speech

Faith Kaufman
Sep 10, 2016 · 7 min read

A collaborative Javascript project by Faith Kaufman and Chris Perry. Our goal was to make a system for finding books in CMU’s Hunt Library using a speech-responsive interface that we named “Book Hunt.”

09.06.16 | Project Begins

We created a google doc for brainstorming:

After much deliberation, four ideas really stood out to us:

  1. food tour (either restaurants or free food)
  2. Scavenger hunt (incorp of cultural probe ideas from last year)
  3. Whats going on (events through fb)
  4. Interactive guide to Phipps

After thinking through the four ideas, we had a moment of insight…

Our idea could be like a scavenger hunt, but within one space on campus. We settled on Hunt Library, thinking that it would be really fun for people to go on a Book Hunt adventure in moments like when they are bored while studying.

For our conversation with the user, we were inspired by the question of

“How are you feeling today?”

We were thinking a series of questions like this could give the interface a good idea of who the user is and what they are looking for in a book. We will then use a combination of absolute and relative directions to take them to the book.

Other ideas for questions were:

How much time do you have?

How much energy do you have?

What kind of books do you like?

We feel really excited about the direction this project is already going in.

09.08.16 | Building a Storyboard

We also started to see problems like — “Can you use a speech-operated interface in an environment like the library where you’re not allowed to talk?” We resolved this issue by making it so that most talking is done on the first floor, which at Hunt, is not a quiet floor.

Our story is as follows:

Our First Storyboard
  1. They open the application and go to the first floor.
  2. They have a conversation with the app with headphones in.
  3. It instructs them to go to the fourth floor. (absolute)
  4. It instructs them to walk north. (absolute)
  5. It gives relative directions until they locate the book.
  6. It gives absolute directions of where the book is on the shelf

After meeting with our instructor, Q, he suggested that we reduce the conversation and stick with one question:

How are you feeling?

And he also recommended using the P5.js Rita speech library so that we can really showcase the power of speech as an interaction. Based off of their response to the question — ex) good, stressed, adventurous — we can locate a book that matches their mood.

We also thought it would be a good idea for the interface to use call numbers as a way of making sure that the user is on the right track to locating the book. And if they are lost, it can use the information of the call-number to figure out the location of the user. This is a really cool application of the class objectives of our class, “Information and Space.” We are literally locating the user using information that they provide.

We really want to use an android instead of LED display kiosks because we want the device to follow the user on their Book Hunt journey.

09.10.16 | Group Meeting

We created a quick list of books and matched them with corresponding feelings:

  • Julio Cortazar — Letters from a lady in paris — Anxious, excited, weird
  • David Foster Wallace — Infinite Jest — bored, adventurous
  • Franz Kafka — The Metamorphosis — angsty, mad, adventurous,
  • Maya Angelou — I know why the caged bird sings — stuck, trapped, stressed, alone/lonely, frusturated
  • Virginia Woolf — The Lady in the Looking Glass — Stuck, pent-up, anxious, happy, okay, fine
  • Douglas Adams — The Hitchhiker’s Guide to the Galaxy — stressed, sad, happy
  • Mitch Albom — The Magic Strings of Frankie Presto — curious, bored, happy
  • Mary Oliver — New and Selected Poems — tired, frustrated, okay,
  • George Orwell — 1984 — adventurous
  • Aldous Huxley — Brave New World — adventurous
  • Haruki Murakami — 1Q84 — bored
  • Ray Bradbury — The Veldt — tired, bored, adventurous
  • Don Norman — Design of Everyday Things — curious, disillusioned
  • Antoine de Saint-Exupéry — The Little Prince — Tired, scared, confused, sad, old, okay, fine
  • Roald Dahl — revolting rhymes — stressed, sad, alone/lonely, weird, happy, okay, fine
  • Madeleine L’Engle — A Wrinkle in Time — Happy, adventurous, tired

New Storyboard

We talked through our narrative and how to make it more compelling. We fleshed out a beginning and end to our story, and also made the scenario more specific.

To Do

  • start coding tomorrow — applying Rita library

09.21.16 | Starting Visuals

Primarily, we discussed whether we should do a compass or a scavenger hunt map. We decided on a scavenger hunt map, but later wondered if the compass could be an intro to the application.

09.22.16 | Code-Planning

We began by mapping out all of the functions:


The red color is a more in-depth view of the code we will be using.

More in-depth view of code

Meeting With Q

He wants us to take more of a focus on visuals


09.24.2016–09.25.2016 | Visuals


Planning Out the Visuals

Starting to Code the Visuals

Process Video

We sourced the colors from one of our book covers. This gives the user a hint of the book that they will get to.

The ring that moves will become a compass with a line moving around, also constructed of dots. After the line spins, it will land on north, and the dots will fade out.

09/28/2016 | Visiting the Library

We used a kiask to locate our book, The Metamorphosis.


It then creates a map. Ours follows this method but is more abstracted. We will use floor, row, and call number to navigate to the book.

Map with floor, row, and call number.

We practiced finding our book.

10/1/2016 | An Update on Visuals

It also draws itself in a more complex and randomized route based off of the following math:

Drawing out how to get the path to work

This is how it ended up working out.

Screenshot of the path

Here is a screengrab of how the interface will look:

A glimpse at our visuals

10.02.2016 | Speech, Sound, & Visuals

Also, after asking Q for feedback, he gave us a lot of suggestions on visuals which we have tried to fix in our code.

10.12.2016 | Final Works

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store