Build Your Own AR Prototype — SXSW 2019
Create an Interactive AR Cocktail Maker
This tutorial is the companion for the SXSW 2019 Workshop of the same name, co-delivered by leading digital agency Huge and AR design and prototype platform wiARframe.
In this workshop, we will explore an application of AR, that we call Hyper Competence. We define this as, the ability for us to do something that we previously didn’t know how to do and to appear almost instinctive in our execution.
Since this is for SXSW, we have to prototype a relevant hyper-competence, something that encompasses the immediate needs of visitors… Let’s make an interactive AR experience that guides us through making the perfect Margarita :-)
Our prototype will display a holographic mixing glass alongside step by step instructions that the user can follow to make their drink.
Prerequisites
Currently, wiARframe is invite-only. If you don’t have an account, then you’ll need to request an invite first.
If you’re new to wiARframe, we recommend reading Getting Started with AR Prototyping in wiARframe before you start this tutorial.
Create a New wiARframe
From the Home Page, click the New wiARframe button to get started.
In the pop-up form, name it mARgarita Mastery, and set the Scene Type to table top. You can leave the visibility setting as open — this means that anyone can view your wiARframe and remix it.
Importing our Cocktail Mixing Glass
We start with a blank canvas, a top-down view of our scene, let’s add our Cocktail Mixing Glass to it. Click anywhere on the canvas to add a new object.
In the pop-up form, set the Object Type to model and name it Mixing Glass.
In the inspector on the right of the screen, either click on the placeholder white cube or expand the Properties > 3D Model property drawer and click the upload icon.
This will open up the asset manager, where we can select a 3D model to upload. For this tutorial, we’ve prepared you a custom 3D model and some 2D graphics, download them here (http://bit.ly/mARgarita_assets).
Once you’ve downloaded the assets, extract them, then drag and drop the mixing_glass.fbx onto the asset manager upload box. Once uploaded it will open up in edit mode.
Adjusting the colours
When you upload a model, we pull out the main colours for you, so that you can easily swap them. If you expand the default style drawer, you’ll see that the first detected colour is grey. I prefer the blue. Click on the grey square and change the colour to light blue from the colour picker that appears.
Close the default style drawer, and then click the select button in the bottom left corner of the window. This will assign the 3D model to the Object in our scene.
Adding Properties to Our Mixing Glass
We want our virtual glass to overlap the user’s real glass in the real world. Accordingly, we want them to be able to reposition, rotate and scale the virtual glass to get a perfect fit. We can do this by adding 3 additional properties to our object.
Movability, Rotatability and Scalability
Click on the + button at the bottom of the Properties inspector to add additional properties, then select Movability from the menu that appears. Repeat this for Rotatability and Scalability.
In each of the three new property drawers, tick the boxes to ensure the user can move, can rotate and can scale the object.
Adjusting the visibility settings
To give our virtual mixing glass a more holographic feel, we can reduce the opacity and change the render mode i.e. the way our mobile app will display the 3D model on screen.
Click on the + button at the bottom of the properties inspector to add one more additional property, then select Visibility from the menu that appears. Drag the opacity slider to 50%, and change the Render Mode to shaded + wireframe. This is going to colour in the model as normal but also draw the outline of the 3D shape on top of it. Finally, set the wireframe colour to a medium shade of blue.
Increasing the Size
Lastly, I think the glass is initially a little too small in relation to our table top. Select it on the grid and then drag the pink boxes on the corners to resize — — you can also reposition it by dragging.
Previewing in the app
At any point, we can preview our creation in the mobile app, now seems like a good time. Install or update the iOS or Android app if you haven’t already. Then log in — you can skip the wiARframe intro if it’s your first time and it auto loads.
From the home screen select your mARgarita Mastery wiARframe and open it. Follow the instructions to detect a tabletop and then tap to select it. You should now see your holographic mixing glass displayed and you should be able to reposition, resize and rescale it.
Let’s get back to the editor and add the instructions.
Adding some UI
We’ve added the Mixing glass that the user is able to move, rotate and scale so that it matches the position of their own mixing glass. Next, we want to display step by step instructions of what to do. We’ll do this using a Panel object (a 2D floating rectangle) that displays a sequence of images with instructions and buttons.
Click anywhere on the canvas to add a new object. In the pop-up form, set the Object Type to Panel and name it Instructions.
From the inspector, either click on the placeholder white square or open the Properties > 3D Image drawer and click the upload icon.
This will open up the asset manager, where we can upload a 2D image. From the sample collection that we provided, drag and drop the step_0.png file onto the asset manager upload box. Once uploaded it will open up in edit mode, we have nothing to edit so just click select to assign it to our panel.
Reposition & Resize Our Objects
We now have two objects in our scene, let’s space them out a bit and resize them to use more of the available space. We’ve found the following layout to look good but feel free to experiment.
Adding Properties to our Panel
We want the Instructions to be in 3D space but we want it to be easily readable by the user, a trick we can do here is something called billboarding. By binding the rotation of a Panel object to the camera, we can ensure that it always faces the user wherever they stand.
With the Instructions object selected, Click on the + button at the bottom of the properties inspector to add an additional property, then select Rotation Binding from the menu that appears. In the new drawer, set the Target to Camera and the Bind Type to look at.
Optionally, you may also want to add the Moveability property to the Instructions so that the user can drag to reposition it.
Adding Interactive Hotspots
With our Instructions panel selected, we can click on the Interaction Layout tab to see a preview of our image, we can now click and drag to create a hotspot over the begin button. This will create an interactive area, that by default responds to the user’s touch.
In the inspector, expand the Interactions drawer, and select the first interaction, rename it Begin Instructions.
We can now add one or more effects to this interaction, such as changing the properties of any object in our scene. In wiARframe to allow multiple object property changes at once, we group these property changes into groupings we call States.
Click the + Add an Effect button at the bottom of the effect list, and in the form, create an effect that changes the state of the Instructions to a new state, called Show Step 1.
This will automatically create a new state and select it for us. In this new state, we then add the properties that we want to change. We’re going to change two things, we’re going to swap the image being displayed and we’re going to activate some new interactions.
Show Step 1
With the Instructions object selected and the Show Step 1 state selected, click on the + button in the properties inspector and select Image.
This will open up the asset manager so that we can select a new image. Drag and drop the step_1.png onto the upload zone. Once upload click the select button in the bottom left corner.
When we change the image, the editor will automatically recalculate the scale to match the aspect ratio. As we know our image size is identical for all images, we can delete the Change Scale property added, by clicking the X next to its name.
Change the Interactions
With the Instructions object selected and the Show Step 1 state selected, expand the Interactions drawer and where it says no change in purple, click the triple dots and change this to change — This tells our app that this state is going to change the interactions that are active on this object.
Open the Interaction Layout tab, then click and drag to create a new hotspot over the next button. In the property inspector, we can then name this interaction Go To Next Step.
Click the + Add an Effect button at the bottom of the effect list, and in the form create an effect that changes the state of the Instructions to the Next State — this is a special instruction that will apply whatever the next state is on our object, very useful in our scenario.
Repeat the above process to create a new interaction for the back button called Go To Previous Step. Create an effect for this new interaction that changes the state of the Instructions to the Previous State.
We now have an active Go To Next Step and Go To Previous Step interaction, we should also be able to see the disabled Begin Instructions interaction.
Show Step 2
We can duplicate the Show Step 1 state by clicking the duplicate icon in the properties drawer.
By default, it will be called Show Step 1 copy. Click the edit button (pencil) in the properties drawer, and rename it Show Step 2.
Then click the upload icon in the Properties > Image drawer, and replace the image with the step_2.png. Again we can remove the Change Scale property, by clicking the X next to its name.
Show Steps 3 to 8
Repeat the above process for steps 3 to 8.
Show Step 9
For the last step we can again create a duplicate the previous state but we’ll want to also disable the next interaction as there are no more steps.
Duplicate Show Step 8, rename it Show Step 9, and replace the image with the step_9.png.
Expand the interactions drawer and simply toggle-off the Go to Next Step interaction.
Previewing in the app
This is a good point to load this in the app and test our interactive instructions — if you already have the mARgarita Mastery wiARframe open, you can tap the wiARframe logo in the top right corner, to open the menu and hit the reload icon.
We should now see both our moveable glass, as well as our billboard instructions. If we tap on the begin, next or previous buttons, it should navigate us through the instructions.
This is already pretty useful but let’s enrich it by triggering some 3D animations on our mixing glass.
Adding Animations
If an uploaded 3D model includes animations then we can play a different one whenever an object changes state. Our sample model includes some animations that illustrate the various steps, and we want these to play alongside the instructions.
To achieve this, select the Mixing Glass from the Scene Layout, and add a new state by clicking on the triple-dot icon next to the state name and chose + New State.
In the new state, click the edit button in the properties drawer and rename the state Animate Step 1.
Click on the + button at the bottom of the properties inspector to add an additional property, then select Animation from the menu that appears. In the new drawer, set the Animation to Static Mixing Glass, and leave the looping toggle off.
Animating Steps 2 to 9
Duplicate the Animate Step 1 state by clicking the duplicate icon in the properties drawer. Click the edit button on the newly created state, and rename it Animate Step 2. Then expand the Animation drawer and change the animation to Fill Mixing Glass to 35ml.
Repeat this process for steps 3 to 9 using the following animations pairings:
- Animate Step 3 > Fill Mixing Glass to 55ml
- Animate Step 4 > Fill Mixing Glass to 75ml
- Animate Step 5 > Fill Mixing Glass with ice
- Animate Step 6 > Shake Mixing Glass
- Animate Step 7 > Pour into Cocktail Glass
- Animate Step 8 > Garnish Cocktail Glass with a slice
- Animate Step 9 > Static Cocktail Glass
Triggering Animations
We’ve created animated states on our glass but currently, these are never triggered. We want our glass to change state at the same time as our instructions. Fortunately, we don’t need to add new interactions to do this, we just need to add extra effects to the existing ones.
Select the Instructions object in its default state. Expand the Begin Instructions drawer and click the add additional effect button.
In the form that appears create an effect that changes the state of the Mixing Glass to Animate Step 1.
Next, change the Instructions object to the Show Step 1 state. In this state, expand the Go to Next Step interaction and add an additional effect that changes the state of the Mixing Glass to Next State. Now expand the Go to Previous Step interaction and add an additional effect that changes the state of the Mixing Glass to Previous State.
Previewing in the app
Our mARgarita Mastery wiARframe is now complete, open up the app and test it out. You should now be able to cycle back and forth through the instructions and have them accompanied by animations.
There’s only one thing missing… some REAL Tequila!
What Next?
There are a number of ways you can extend this project or reuse the ideas we’ve covered. Below are just a few suggestions of what you can do next.
Share it
You created something cool that anyone can experience, so you might want to share it.
When no objects are selected, the inspector will show information about your wiARframe. You can add a thumbnail, a description and even a link to a blog post about the project. You can then get the share link, by clicking on the share icon below the thumbnail. You can either share a link to the playable version in the app or the remixable version in the editor.
Change the interactions to respond to the user’s gaze
It might be hard to hold your phone and make a cocktail at the same time, so change all of the interactions to use stare gestures instead of taps. This way a user can prop up their device and then stare at the buttons to trigger the next steps in the process.
Add an Info Panel
We’ve created some extra information graphics for you that displays the history of the Margarita, the ingredients required and a list of suggested variations. Using these images create an additional panel that allows the user to toggle between the 3 bits of information.
Your favourite Cocktail
Redo this tutorial (or Remix it) and replace the Margarita with your favourite cocktail.
Other Competencies
Taking the lessons we’ve learned here, what other applications of Hyper Competence can you prototype, such as cooking, furniture construction, drawing.
