AR on your Phone with MagicaVoxel, Wikitude and PhoneGap

The PhoneGap Speeder!

Let me float an idea by you: an augmented reality app on your phone with cool 3D models made in voxel art. “What, explain how!?”, you ask? Well, allow me to show you the process. But first, perhaps you should see this video of it all in action:

Let’s get some software!

First things first, you will need some software. Oh don’t worry — everything listed here is free to use (until you actually need to publish an app. I’ll explain later). But the first thing you should download is MagicaVoxel.

MagicaVoxel is this wonderful program that allows you to create really fun 3D voxel art really quickly without worrying about the complexity of a 3D editor. It’s made by @ephtracy and the license is free to use for personal and commercial use. Thanks @ephtracy! Just look at some of the cool stuff you can make with it!

The second program you’ll need is Blender. You could just make all of your assets in Blender if you really wanted to, but I find that my 3D skills in Blender to be pretty meh. Really all you need Blender are for some intermediate steps to export models made in MagicaVoxel into the Wikitude format.

Then you’ll need the PhoneGap Developer App with the version being at least 1.7.9 on either Android or iOS. Download now! In the PhoneGap Developer App, there is a plugin especially made for augmented reality experiences made by Wikitude. So you’ll be able to preview on device the PhoneGap project you are creating on your dev machine.

Finally, the Wikitude encoder is needed to convert the final version of the voxel art models into the Wikitude .wt3 file format. You may download the encoder here.

Ok, now that you have everything, it’s time to piece them all together. The idea is:

1. You’ll make something cool in MagicaVoxel
2. Export it to Blender to set the model’s lighting and scale
3. Then export it to the Wikitude encoder to be finally used in your PhoneGap project.

So to sum up the software needed, download these:

• MagicaVoxel
• Blender
• PhoneGap Developer App 1.7.9 or greater(iOS or Android)
• Wikitude Encoder

Making and Exporting Models in MagicaVoxel

This section could be its own article, but the quick and down-low on MagicaVoxel is that you construct your 3D models out of cubes. Stacking these cubes together with different colours is the essence of voxel art. Think of 8-bit pixel art but in 3D! Check out some of the things I’ve made in MagicaVoxel:

Some cool stuff I’ve made in MagicaVoxel. For more, see https://www.instagram.com/timkimtimkim/

You may want to check out some tutorials on YouTube on making models if you’re not familiar with MagicaVoxel.

For the purpose of this article, let’s assume you have your model all ready. Here are the steps to export your model:

1. Switch to Render mode
2. Under the Export tab, hit bake and that should export your model as a ply file. Baking a model basically saves the lighting and the shadows of the model when you render it. So make sure to adjust the lighting and shadows of your model until you’re happy with the results.
3. Now that you have your baked model, it’s time to open it up in Blender.

The green rectangles indicate where to click for Render and bake.

Importing and Adjusting the Model in Blender

Now it’s time to adjust the model in Blender. This is a bit of an annoying but necessary step since MagicaVoxel doesn’t cleanly export to the Wikitude format. So to import in the MagicaVoxel models:

1. In a new file in Blender, delete the starting cube and lamp. That way we have a clean project to work with.
2. Go to File > Import >Stanform (.ply) and import in that baked model that we had from the previous step.
3. I tend to find the models to be way too big so use the Scale tool to make the model much smaller. Also, you might want to set the Geometry to Origin so the model is the center of the 3D view.
4. Switch to Material mode and select your model. Then under the object’s Materials tab, create a new Material. Select Shadeless and Vertex Color Paint to be checked. With that, your model should look like it did when you rendered it.
5. Now let’s export it to a format the Wikitude encode can read. Go to File > Export > FBX (.fbx). If you really wanted to, you can choose to export it as an obj file instead to be used in other 3D environments (eg, WebGL) instead of using it in Wikitude. But perhaps that’s for another article ;)

Or if you prefer, a video tutorial of steps 1–4:

Encode the Model into Wikitude’s Format

If all has gone well, this step should be pretty easy. Start up the Wikitude3dEncoder and do following:

1. File > Open that fbx file we exported from Blender.
2. Inspect your model in the viewer to make sure it looks ok. There are number of things you can do in the encoder program such as setting and viewing animations. To learn more, visit the Wikitude’s developer docs.
3. When it all looks good, File > Export to wt3 which should generate your .wt3 file to be used in your PhoneGap project.

Import the Model into your PhoneGap Project

If you don’t have an existing PhoneGap project using Wikitude, you can use a template that I created to help you get started. The template is the Speeder demo that is shown in the video at the start of this article. The idea is that you’ll replace the Speeder model with your own model and get a feel for using Wikitude and its AR experience.

To create the PhoneGap project with the Wikitude template:

$ phonegap create myApp --template phonegap-wikitude-speeder-template

Once that’s all downloaded and ready to go, let’s add your model that you created. Navigate to the newly created project and go to www > pgday > assets > models. In that directory, you can see an existing landspeeder.wt3 file. Place in your new your model file there.

The project structure and where the landspeeder.wt3 model resides

Then in www > pgday > js > 3dmodelsinstanttracking.js, we’ll add the reference to the new model.

Where 3dmodelsinstanttracking.js is in the project

If you search for “assets/models/landspeeder.wt3” in 3dmodelsinstanttracking.js, that should take you to the reference where we adding a new model using Wikitude’s api. Simply replace landspeeder.wt3 with your model’s file name. Now it’s time to see if your model worked ok.

addModel: function (xpos, ypos) {
        if (World.isTracking()) {
            // replace landspeeder.wt3 with your model's file path
            var model = new AR.Model("assets/models/landspeeder.wt3", {
                onLoaded: this.loadSpeeder,
                scale: {
                    x: 0.005,
                    y: 0.005,
                    z: 0.005
                },
                translate: {
                    x: xpos,
                    y: ypos,
                    z: 4
                },
                rotate: {
                    z: Math.random() * 360
                },
            })

Now it’s time to preview your changes in the PhoneGap Developer App. Open the app on your phone and run the phonegap serve command into your Terminal:

$ phonegap serve
[phonegap] executing 'cordova prepare' ...
[phonegap] completed 'cordova prepare'
[phonegap] starting app server...
[phonegap] listening on 10.58.133.107:3000
[phonegap] listening on 169.254.253.181:3000
[phonegap]
[phonegap] ctrl-c to stop the server
[phonegap]

Type the IP address it gives you into your phone and it should launch the demo. If all goes well, the Speeder model that drops in the demo should be your own model!

Conclusion

I hope by going through this article that you will learn some cool 3D stuff and be able to import assets to your phone. Even if you don’t do the Wikitude or PhoneGap stuff, I think being able to export assets from MagicaVoxel to Blender is a pretty good base for using those assets in other settings.

Going forward, there should be more articles from me about using 3D and PhoneGap. There’s a lot of cool stuff happening right now and I think the dam is ready to be busted wide open!

If you decide to continue down the Wikitude route, Wikitude’s API offers all sorts of ways to add different animations, models, and effects. I would recommend heading over to the Wikitude SDK API reference docs and see the kind of things you can make for yourself. More over, if you finally do convince yourself to make a commercial app for purchase and download with Wikitude’s technology, you should sign yourself up for a license.