Setting up a Wizard of Oz prototype in Figma

We tested a prototype while editing its content live. We learned a few tricks to make that edit less stressing and improve your Wizard of Oz experiment set up.

In the field of human–computer interaction, a Wizard of Oz experiment is a research experiment in which subjects interact with a computer system that subjects believe to be autonomous, but which is actually being operated or partially operated by an unseen human being. — Wikipedia

A prototype is a medium to learn. By bringing a lower-cost version of an ideal product to the real world, teams can learn faster and make sure they are building the thing right without incurring on big costs.

Recently we felt the need to help validate a product idea at Babylon Health. As we often do, we planed a lab exercise to test our prototype with people. But, it became clear to us that it would only make sense to test the prototype if it had good content.

The lab setting is really artificial when testing with people, and whilst you can never fully solve the biases that go with that we wanted to provide users with an experience of using the product as true to life as possible.

So we did a Wizard of Oz experiment. It consists of testing a product with people and making them believe the product is autonomous. In reality, someone is secretly operating behind it.

Let’s imagine a voice interface that asks people what they want to eat and then shows recipes on a screen for the food people chose. A Wizard of Oz experiment can be ideal to test such a product. While the participant is talking about pasta, someone can be filling the interface with recipes for pasta. From the perspective of the participant, it will look like the interface is actually listening and responding appropriately. As if it was a real product.

How we did it and what we learned

We used Figma to run our prototype. We wanted to fill it with content that was personal to the people we talked to. Me and Jerram White, our User Researcher, collaborated to design the prototype and plan the lab sessions.

We divided the sessions into 2 parts. During part 1, Jerram would ask questions about the things we wanted to fill in the prototype. While he and the participant discussed, I would fill the prototype with the respective answers. Then, during part 2, the participant would test the prototype.

We ran the experiment and learned a lot with it. But to do this experiment and to edit a prototype live can be stressful. We learned a lot of things that can help us run a Wizard of Oz exercise smoothly next time in Figma. Here are some of them.

Take advantage of Components

We used Components for all the content we decided to fill in based on answers from people. This means that if we had a UI Card that we would fill with what people told us they ate for dinner, we would make that UI Card a Component. And then we would use that Component across the many screens it would appear.

This became useful during the sessions because while I was editing it live, I had one central place to focus on. There was no need to go around artboards and search for all the UI Cards I’d have to edit. I could make the edit in one single Component and see the change across all artboards. ✨

Make use of the auto-layout feature on Figma

Figma recently released the Auto-layout feature. Sho Kuwamoto describes what you can do with Auto-layout in Figma:

Buttons can resize with their text. Lists can rearrange themselves when items are moved around. And elements can be nested to create complex interfaces which respond to their content.

This is important since it’s hard to predict how much content this session will generate. For example, for one person I might write down one dish they had for dinner last night in the UI Card. But if they went to a wedding, I will need to write down more dishes. Because I am in a live session, I do not have the time to resize the card so more content can fit. Auto-layout lets me focus on writing content while the UI Card scales automatically.

Use arrows to point where you need to edit content

Even though we had Components all gathered in one place, many of these Components required multiple content edits. For example, the UI Card that we would fill with what people told us they ate for dinner needed more than that. It could also require us to change the category of food to match what people said they ate.

When editing live, it can be stressful to know if you’ve made all the correct content edits. For that, I used arrows to signal where in these Components required edits. Then when editing live, I followed these arrows and knew that as long as I went through them all, it would be fine.

Let your team know you are testing

In fact, let everyone who can edit your file know that you are testing a prototype on Figma. You don’t want to end up with sudden changes during a lab session. Let everyone know when the session will take place and what file will be used for the prototype.

You can never be too careful 😅

You may even signal it in your file which screens they shouldn’t change because they are being tested live.

Be careful where you click…

We used the Figma Mirror app to set up our Figma prototype on a phone. Once they are in sync, if I click on a frame on Figma, the phone will mirror the frame I clicked on. This can be dangerous when running a live experiment like this.

During part 2 of this experiment — when people were interacting with the prototype on the phone — I had to be careful not to click on any frame on my Figma file. I did click once on another frame and that resulted in the prototype switching to another screen. In a matter of a second, I clicked back on the frame the person was interacting with. Immediately the person was back where they were before my miss click. That solved the problem quickly but taught me a lesson.

Note on clicking on frames

While you mustn’t click on other frames, you can still edit content while people are using the prototype. If you have frames inside of the main frame, you can also click on those and edit them freely. That will not switch the prototype view on the phone.

Test your session beforehand

You should practice this session with the person who will help you run it. It helps:

• Understand if the questions you designed help edit the prototype.
• Understand how the Figma Mirror app works
• Make sure that you created the necessary Components
• Make sure that the arrows are all there and pointing in the right direction

When is it useful to run a Wizard of Oz experiment?

It might not be a good idea to run a Wizard of Oz experiment for every prototype. It depends on what you and your team want to learn.

• Use it to test how people respond to personalised content. For example, if participants answer questions on an onboarding flow, you can personalise the content of the product based on what they answer. It requires good coordination from whoever is operating the prototype, but it puts personalised content in front of people.
• Use it to build advanced logic. A chatbot prototype is a good example. You can set up a conversation with a participant and be the bot. While you are testing conversation flows by writing them up live, the participant believes it is a bot. The result one gets is very similar to as if one was testing a real complex chatbot.

Have you run any Wizard of Oz experiments with Figma or other tools? What did you learn? Share your thought below 👇

Thank you to Adrián López Hernández and Jerram White for reviewing this post!