Using Unity and Vuforia to Make a Basic AR Application
Meet Jessica. Hi Jessica. 💁
Jessica wants to learn Japanese, so that she can better understand her favorite Japanese anime, obviously “My Neighbor Totoro”. She simply just wants to be able to have a conversation with someone in Japanese. Really basic stuff, like talking about the words around her and the things she’s been doing. That way, it could be easier for her to understand what her favorite characters are talking about without subtitles.
She goes to take an in person Japanese class at her local library and meets a bunch of other people just like her! She tries to practice speaking with them in Japanese, even easy sentences like, “I want to sit in that chair.”
But, she’s run into a little dilemma: she doesn’t remember the word for chair! Now, of course this is a rather common thing, vocabulary isn’t something you can just memorize in a day.
But, how will she be able to practice her speaking if every time she wants to have a conversation she gets stuck on a vocabulary word?
One of the most important things about learning a language is learning new vocabulary. Sometimes it can be inconvenient to have a dictionary on you all the time and to look up the words you need on the spot. But what if you could find the name of an object in whatever language through a simple app?
With Augmented Reality, this is possible.
I attempted a trial of this using Vuforia and Unity 3D to detect a picture of a chair, and display an AR label that calls it “いす” or “Chair” in Japanese.
There are 10 basic steps to creating a simple AR application:
1. Download Unity and Vuforia
You may download both on their websites: Vuforia, and Unity. You can’t do anything without the platform to do it!
2. Import Vuforia into Unity 3D
Vuforia is an extension to Unity. You can begin by creating a project in Unity 3D, and importing Vuforia into the project. This will allow you to do VR/AR specific actions within Unity.
3. Add AR Camera and Image Target to your Hierarchy
We will be creating a Target-based Augmented Reality application. This means that an AR projection is triggered by a target image. So you will need a target image outside of the application that you can display to the camera so that the application can detect it and project your image of choice.
4. Create a license in Vuforia
This license will link your Vuforia project to your Unity one.
5. Paste License Key to your application
Select the project license you have just created and paste it into the program through the “AR Camera”.
6. Add Image for your Target
Here you will upload and choose your target image that will trigger the projection. You may choose any image you would like for your target image, as long as it is within 2mb, and is a .png or .jpg file. In the menu, you can see the rating or chance of compatibility with the program of the image you chose. 5 stars represents that it is an image that can be detected relatively easily for the application. You want to get that 5 stars! Pictures with a lot of contrast in colors are best, it can’t look too similar or it will be hard for the application to distinguish. You also have to download the images as a package to import into the application later.
7. Import Image Target Database into the Application
Here you will upload the database of images that you just downloaded into the application for you to use.
8. Set the Imported Image as the Target Image
Now you will have to select “ImageTarget” and set the image you would like as the target image.
9. Add 3D Text to your Heirarchy
Here you can choose what you would like to project onto the target image. This can be anything you want, it is all up to your imagination. In my case, I decided to project the Japanese word for chair, いす onto an image of a chair.
10. Design the Projected Object/Word
If you chose a word, here is where you can choose what the word says, the font, the size, etc. It is also important to change the rotation so that the image is facing the same way as the target image. Otherwise, it will project in a weird angle.
Now you can project the image!
If you click the play button, and “Game”, it will present you the camera in which you can display your target image. Once you display your target image, and the application has detected it, you should be able to see the projected word!
With technology like this, language immersion can be taken to the next level! Now, Jessica can simply pull out her phone (on silent of course, we’re in a library) and look at the Japanese word label for chair through the camera on her phone.
There are so many different applications for this that could help transform the way we look at things around us for the better. In schools, this could be used to increase the vocabulary of the things around you 10x! In public transit, this could make information so much easier to understand. The station names etc. could all be translated in a moment and on your screen. Even if you just want to be able to communicate better in the native language of your best friend, or family member.
Next I plan to continue building on my projects with AR, with a focus on improving language learning!
Resources:
- https://www.youtube.com/watch?v=MtiUx_szKbI
- https://medium.com/@avseoul/unity3d-vuforia-ar-augmented-reality-101-how-to-setup-vuforia-for-your-very-first-ar-project-88d7ae7140ea
- A company that uses VR for language immersion! Mondly
Thank you!
My name is Alyssa Gould, and I’m passionate about the intersection between Artificial Intelligence and Second Language Acquisition!
Feel free to contact me at alyssa25g25@gmail.com for questions or anything!
