AR Interaction Design Toolkit
Designing AR interactions is not easy, especially for interaction designers switching mindsets from traditional 2D GUI design to a field of immersive computing. I have been doing researches around AR and affordances for my thesis project, and to solve some of the difficulties that I encountered during my research as well as to help easy the AR interaction design process, I here propose a toolkit called “AR Interaction Design Toolkit” to facilitate the ideation process of interaction design for augmented reality, and help designers to conduct generative research and brainstorming new design concepts.
Where does it come from?
As I mentioned earlier, the concept of the AR Design Toolkit comes from my initial thesis research topic, which is “Managing Perceived Affordances in Augmented Reality”, and I attempt to research how we might enhance perceived affordances to create intuitive user interfaces for augmented reality. The goal of my research is to propose design recommendations for AR interaction and interface design for AR designers, creators and developers, therefore it is a process that research through design, but also a research for design.
However, during the research process, I realized there aren’t many research frameworks for augmented reality design, and how and where to get started is quite challenging for me. As a result, besides of the literature review, I initialized the research by looking up other AR design examples either from some interesting AR/VR products, or small demos and prototypes that people made and posted on Twitter or Medium. I gathered many of those examples and edit them together as a short video. Then I tried to synthesis the findings from the research, I found there’re 5 different aspects that could potentially affect user’s perceived affordances, which are Lighting, Shape, Acoustic & Haptic, Motion, and Color & Shading. These five aspects mainly define how we perceive the affordances, in other word, we can possibly manipulate these 5 aspects to enhance the affordances or make them clear and easy-to-understand. In this sense, more or less, the main questions here is what kind of affordances we want to reinforce and also how we can manage those 5 aspects and creatively come up with a good combination of these variables.
However, what kind of interactions we want to focus on? Or what kind of affordances or interactions that are most important, or we say most frequently used. To answer that, I conducted a photo diary study to invite people to take photos of their everyday products which are either intuitive to use or confusing to use, and I especially prompt them to concentrate on the physical products, because I think there’re close connections between the affordances among physical products and environment to that in AR, since AR user interface is 3D and immersive. I believe by understanding people’s mental models of physical product would be very beneficial for the research in AR.
Here’re a few insights I got from the photo diary study:
- Ergonomic shape reveals the affordances to users.
2. Signifier (text, icon e.g.) can amplify the perceived affordances, especially indicating products’ functionalities.
3. Positive and negative shapes that match each other disclose their connections, making the affordances perceived to be clear.
4. Lights can indicate the status of the system, also provide directional guidance.
Combining with the research into industry examples and the photo diary study, I found that Rotate, Press & Select, Pull, and Slider are among the most frequently used interactions. Therefore, I firstly focus on those four major interactions for my research, and them propose the four interactions cards.
The AR interaction toolkit is useful for me to conduct brainstorming sessions with other AR designers, ideating new design concepts, or run a generative workshop. With some pilot use test, I also realized that there’re still a lot of space for improvement, such as expanding the interactions, tweaking the prompts on element cards, but I believe the toolkit is a starting point to figure out frameworks that can help designers think structurally. But still more and in-depth user testing is still needed in order to see better whether the concept works.
What is “AR Interaction Design Toolkit”
AR Interaction Design Toolkit is a set of cards that help AR creators to conduct generative research and brainstorming design concepts by combining two card together (one interaction card, and one element card), to ideate various possibilities for AR interaction design patterns.
Currently, there’re two kinds of cards, the first one I name as “Interaction Cards”, which are a set of cards with some basic interactions on it, such as rotate, press, pull etc. Those interactions represent some of the most frequently used interactions in Head Mounted Display (HMD) AR interactions, offering as a prompt for designers to think about. It could also serve as a prompt for research participants, for instance, designers could recruit people to do participatory design workshop and brainstorm ideas together. I plan to expand the interaction cards later on, and add with more complex and nuance interaction patterns so that designers can get the depth of them.
The other type of card is the “Element Card”, which has five kinds of major elements that I believe would affect the perceived affordances, and they are “lighting”, “Color and Shading”, “Motion”, “Acoustic and Haptic” and “Shape”. The five element cards serve as a framework to help designer to structure their thoughts and offer a starting point to generate interesting concepts.
Each Element Card has a short instruction that prompts and instructs designers to think about how this element can be used or affect the affordances, thus to facilitate them to better think about how to make the interactions more intuitive.
The way I would imagine designer to use it is that they can select one from the interaction card, one from the element card, and combining those two cards together to think about how tweaking the variables from the Element Card would affect the interaction in the Interaction Card. Therefore, designers could begin to think creatively and brainstorm various ideas to enhance the perceived affordances.
Next Steps
For next steps, I plan to use the toolkit to help me brainstorm more concepts, and also open source the cards to public so that I could gather feedbacks directly from other designers who’d like to give a try. I think the toolkit is useful when tweaking and testing the interactions by manipulating all those different parameters, but how it can expand to broader use case is still un-explored.
In addition, I also want to add another set of status card, indicating the different status that could take place during the interaction, such as Hover Status, Selected Status, and Normal Status. In this way, it provides more depth and takes a more practical approach to solve the problems, since when I prototyped the AR interactions, there is no way to skip the different status and in fact they become quite important to convey the system information, for instance wether the system correctly recognised user’s hands. Moreover, designers would be able to dive deeper and think throughout the whole interaction process, rather than only focusing on one status of the interactions.
Last but not least, the AR interaction toolkit is not meant to be a perfect design toolkit for AR interactions, and it’s still very much in progress. The original goal to propose this toolkit is to help me conduct research and better help other generative research participants to better understand the concepts. I hope it’s useful for some scenarios, and I am also looking forward to seeing more interesting design toolkit for AR comes out in the future.
