The Recipe Helper

Carrie Mah
Jun 2, 2017 · 7 min read
The Recipe Helper uses a tablet and leap


UX Design, Visual Design, Sketching, Prototyping, Usability Testing

Note: the following is image heavy. If you’re on mobile, please watch your data limit!


Optimizing the recipe flow for novice chefs by augmenting a mobile device using the Leap Motion API.

A project in November 2015.

The Problem

The challenge was to create a physical augmentation of a mobile device by using hardware to extend the phone/tablet and approach an application-specific problem by leveraging a particular set of tasks.

Learning to cook can be cumbersome when you’re following a recipe on a mobile device or tablet. Both the interaction and recipe design create a frustrating experience.

Skip to prototype explanation and click here

Skip to the outcomes and click here

The Journey

Understanding the cooking experience

I surveyed people with diverse cooking experiences and asked them about all stage of cooking. One big pain point is that devices get messy when interacted with. Based off people’s comments, recipe formats should:

  • Be easy to scan, as cooking food is a timely process
  • Be flexible, reducing work when converting measurement units and seeking ingredient substitutions
  • Provide concise instructions in different mediums (video, images, text) to accommodate different learning styles
  • Condense recipes to enhance readability and reduce the amount of scrolling

The initial decisions

I identified two problems to solve:

  1. What does the recipe format design look like, and
  2. How to interact with the device

As my recipe-following experience was limited to formats where ingredients and directions are separated, I searched for different recipe formats.

I found a popular Gantt chatt alternative by Cooking for Engineers. It summarizes steps required for each recipe. Although easy to scan, it’d be difficult including additional information like images, videos, ingredient substitutions, and recipe notes.

Other notable designs included Clear Honest Data for visualizing ingredients; Modernist Cuisine & Rouxbe for combining both the ingredients and corresponding steps in the same row; and Visualized Recipe & PFoodi for visualizing ingredients, steps, and time duration to complete.

Recipe format sketches

I sketched ideas based on existing interactions. Speech control was not common or well-received yet, but based on my initial interviews, hands-free gestures streamline the cooking experience.

Interaction sketches

I wanted to keep things simple and needed an easy-to-use gadget for developers and users. I found that The Leap Motion was easy to install and recognizes several gestures.

Interacting with the device should feel effortless. When users move to a preferred direction, Apple Mac’s mouse interaction makes sense. To avoid accidental interactions, activating information should be deliberate like keeping the finger in the same location.

Finalized interactions

Refining the recipe layout

After receiving feedback for the initial designs, I sketched a detailed recipe layout. The timer indicates how long a recipe takes, separated into different colors for preparation (blue) and cooking (orange).

Refined recipe format sketches

I had enough time to do a quick prototype for desktop use and test with potential users. The design elements were well-received, but requires a lot of scrolling on tablet devices.

Overview of the new recipe format
The equivalent measurement unit tooltip
The ingredient substitution tooltip

Integrating interactions and testing the design

Based on previous user feedback, I separated the layout into pages for a recipe overview and its steps. To accurately test the new layout and interactions, I created a physical sketch.

To minimize clutter, I merged the progress bar and timer section. Tapping on the timer allows users to keep track of their current time progress.

Progress and timer sketches
Timer and progress flow

A user’s reading experience should be streamlined. Additional information like equivalent measurement units and ingredient substitutions are activated deliberately. Ingredient amounts are visualized in quartered circles and color values, where lighter values represent small measurements.

Additional information sketches
Additional information flow

Videos enhance the recipe’s instructions, and should be integrated in the user’s reading experience.

Video sketches
Video flow

In this round of testing, I simulated the experience for users by informing them the gestures they can use and adding paper elements based on their interaction responses. Feedback was overall positive and layout modifications were made.

Testing Leap Motion placements & stands

Success with The Leap Motion was inconsistent because pointing gestures are not optimized. I initially attempted to position the device on top of the tablet, but it did not balance perfectly and resulted in buggy interactions.

Initial Leap Motion placement

I 3D printed a Leap Motion case and attached it to the tablet like a clip. This reduced balancing issues, but was not ideal for interactions.

Refined Leap Motion placement

Using the existing equipment I made, I positioned the Leap Motion on the table. Although it takes more space, gestures are now optimally registered.

Final Leap Motion placement

The Solution

I refined the design layout to include instructions on using the system. The modified layout optimizes screen real estate and improves readability.

The final prototype provides an overview of the interactions, a recipe, and separates each step in its own page. Steps include ingredients, instructional videos, and images.

The Recipe Helper screens
Visual narrative for the prototype gestures and interaction
Visual narrative for the prototype screens
Visual narrative for the prototype visualizations
Recipe overview
Ingredients and tools
Step 1 — cooking
Equivalent measurement unit and ingredient substitution tooltips
Step 2 — preparation
Step 3 — cooking
Step 4 — finishing

The final interactions use simple gestures with one hand. Swiping left or right moves to an instructional step and swiping up or down reveals more ingredient information. Moving a finger in a circle starts or pauses the time progress bar, and holding a finger down starts or pauses a video.

The Recipe Helper interaction demo

The Outcomes

I received the Most Useful Award at the YYCJS Annual Nerd Off competition.

I also presented at the Canadian Undergraduate Technology Conference’s Maker Fair and encouraged students to learn web development.

The Learnings

I really enjoyed working on this project because I had a chance to do a lot of cool things:

  • Understand users and refine the problem
  • Sketch and prototype in different mediums
  • Test multiple times
  • Iterate the design based on feedback and usability tests
  • Create a video demo
  • Present and demo The Recipe Helper

To improve the system, The Recipe Helper could be built using the device’s camera. Leap Motion was a bit cumbersome to work with and gestures didn’t always work.

The Tools

Software: Sublime Text, Adobe Illustrator

Technologies: HTML, CSS, Javascript, LeapJS, LeapStrap, reveal.js

Frameworks: Stanford Design School’s Design Thinking Process

Thanks to Dr. Sonny Chan and David Ledo for providing feedback and suggestions.

Recipes, photos & icons from Allan Reyes, Max, Dawn T, Charmie777, and Dmitry Baranovskiy. CSS animations & effects thanks to Captain Anonymous, Andréas Lundgren, Junxi, Chris Coyier, and the creator for jQuery progress timer bar.

Have any suggestions or constructive criticism? Please comment below! I am always looking for ways to do better.

If you liked what you read or learned something new, support with 👏, spread the 💙 or follow me!

If my background, experience, and skills interest you, please get in touch!

Carrie Mah’s Portfolio

Portfolio of @misscarriemah with case studies & processes in UX, UI, and front-end development. Inspired by curiosity, creativity, and chuckles.

Carrie Mah

Written by

UX Design Consultant || Dedicated to improving lives || #ux #cx #design #web #stem #diversity #womenintech 💻

Carrie Mah’s Portfolio

Portfolio of @misscarriemah with case studies & processes in UX, UI, and front-end development. Inspired by curiosity, creativity, and chuckles.