Images can be a beautiful starting point for any webAR experience, but now you can design webAR that goes beyond the Image (see below). Your images can be part of your brand, your logo, your product packaging or advertising, a retail display, poster, brochure or business card. In fact almost any image can be used to blend the real world and the digital world using webAR. But that’s just the start.
Using images to unlock digital content
In our last post we looked at all the different ways people can access your webAR experience. Once your webAR experience has loaded, the web browser can then start processing the video stream from your camera.
Each frame from your camera is analyzed to search for a predefined image called a Target Image. If this Target Image is recognized then the frame is further analyzed to understand how it is positioned in the real world. This is called Estimating the Target Image Pose.
This process of recognizing a predefined Target Image and estimating it’s Pose is repeated roughly 30 times per second and is commonly referred to as Image Tracking.
Once a Target Image is identified and it’s Pose is estimated then this information can be used to overlay your digital content on top of your “view through the camera”, to create the illusion that this content is attached to the real world Target Image. This process is called Registration.
All of this has to happen in real time with as little delay as possible, and it must also take into account responding to any interactions from your user.
You can create your own Target Image by uploading any image file to a webAR creation platform like awe.media. This image can be a photo or image file (e.g. png, jpg, tif or gif). To ensure your Target Image works well it’s best to pick an image that has a reasonable amount of detail and also has a reasonable amount of variety in it’s contrast.
Photos of “Polar bears in snow storm” or “White benches in a white kitchen” usually don’t work very well, making it hard for your Target Image to be recognized, or leading to “wobbly” and unstable digital content once it’s displayed.
However, the great thing about a webAR content creation platform like awe.media is that you can easily upload any number of Target Images to quickly test how well they work.
Using images to link to other digital content
Of course you don’t always want to overlay your digital content on top of your Target Image. In some experiences it makes sense for your user to scan a Target Image and then be immediately linked off to another web page. In this case your Target Image can act like a more attractive QR code.
In some webAR experiences you may have quite a few different Target Images, for example if you were augmenting a magazine or art exhibition. In these cases you may want to offer your users a “Scan” interface that lets them quickly identify which image they’re currently pointing at.
This “Scan” interface presents the same type of “view through the camera” you normally expect of webAR. However, in this case there is a “Scan” button overlaid on top of it, and when the user taps this button the current frame is captured as if you were taking a photo. This image is then compared against all the Target Images in your project to see if it matches any of them. If it does then the relevant content is automatically loaded and the Image Tracking process can start.
Of course you can customize this “Scan” user interface to match your brand or design.
In our last post we looked at how QR codes can optionally be used to launch your webAR experience using the built-in camera app on iOS or Android. Once your webAR experience has loaded you may want to let you users scan another QR code. With a small amount of custom code you can easily enable this option so your users don’t have to leave your webAR experience to scan another QR code.
If you do choose to use QR codes in this way, you can also choose to use them as the Target Images too. QR codes are not as beautiful as other more natural looking Target Images, but if this makes sense in your interaction design then this is a perfectly valid choice.
Skipping the scan
If you only have one, or a small number of Target Images then you may want to skip the “Scan” interface altogether. In this case you may assume the user has your Target Image in front of them, for example if they are looking at a poster or product packaging. This makes for a streamlined “First Use” experience where the user is launched right into your webAR content as soon as the browser has loaded.
Using multiple images
As we discussed above, you can use more than one Target Image to weave together different content and create more sophisticated webAR experiences.
However, tracking each Target Image requires quite a lot of computational power and in order to ensure you get a good experience on all devices (including lower end devices) the awe.media platform focuses on tracking one Target Image at a time.
To give you more design freedom you can also choose to pre-load multiple Target Images so any of them can be immediately recognized, without any other action by the user.
In this case, all the user needs to do is move from one image to another and the relevant content is automatically displayed over the currently visible real world Target Image. And if more than one Target Image is visible then the one that takes up the most of the viewport wins.
The other benefit of this approach is that it makes it easy for you to layout your content over each individual Target Image, one at a time.
Designing beyond the Target Image
Normally, if the user moves away from your Target Image then the digital content overlaid onto it will disappear. But you don’t have to be limited to this type of interaction design. One of the options provided by the awe.media platform is the ability to design an experience that’s much broader than your Target Image.
With one switch you can choose for your content to keep displaying once the user moves away from your Target Image. In this case your webAR experience moves from “‘tracking your Target Image” to “tracking the movement of your mobile device”. There are a wide range of different types of tracking options available and in future posts we’ll dive into each in more detail. But for this post, the key point is that even in Image based webAR you don’t need to be limited to just when the user is pointing at your Target Image.
Not only does this let you design beyond your Target Image, it also offers a new range of user interactions you can create. For example you may want to trigger a specific event when the user moves away from your Target Image, and trigger another event when they move back to your Target Image. Or you may just want them to start with your Target Image and then move onto exploring the space around them.
This means you could play a specific animation when the Target Image goes away (e.g. if you had a butterfly model this could start it flying) and then you could play a different animation when the Target Image comes back (e.g. they can use the Target Image to catch the butterfly).
This also means you can layout content around the user, behind them and away from the Target Image. Then lead them to look around the scene. In this way you really can use the Target Image as a launching point for a whole range of experiences.
The design options are endless.
You can use a wide range of user interactions to make your webAR experiences more engaging. You can unlock content just by pointing in a specific image or direction. You can tap or click on an object to interact with it. You can swipe, pinch, zoom, or scroll. You can use the orientation of the device to change the layout or behavior of your content. You can even use more obscure things like date, time, language and location to change your content or how your interactions work.
Using a webAR creation platform like awe.media you can layout and design animations that move your digital content around the screen through an easy to use web based interface. You can scale, rotate and move any object in your 3D scene in response to any of the interactions listed above. Or you can import pre-made 3D animation clips (e.g. a dancing character) and play these clips in response to any of the interactions listed above, allowing you to weave together a richly animated webAR experience.
This should give you a good introduction to the new User Experience issues you need to consider with Image based webAR and how this is much more than just the Augmented Reality content itself. This includes a whole new way of planning and designing content. New media formats and workflows. Plus a range of different webAR modes & technologies including Image tracking (covered in this post), Face tracking, Geolocation, World tracking, 360°/VR experiences and much more.
The key message we hope you take away from this is that webAR is here right now and it provides an amazing opportunity for creative people like you to weave new and exciting immersive experiences.
Looking for help getting started?
Here’s a guide to setting up your free trial and getting started creating webAR. And here’s a guide to creating your first Image based webAR experience.
If you have any questions or need any help please contact support [at] awe.media — we love to help!