My first AR exploration with Processing
follow along to create your first AR experiment :)
In this experiment, we are going to explore marker-based AR.
The system I’m using here is Windows 10.
Time
approx. 30 min
Tools
a laptop with a camera, a printer, Processing, and the NyARToolKit. (Don’t panic if this doesn’t sound familiar. I’m going to walk you through the setup step by step.)
Step 1: Download Processing
Processing is a flexible software sketchbook and a language for learning how to code within the context of the visual arts. It is open-source, free to download, easy to use, and very well documented. Processing is available for Linux, Mac OS X, and Windows. You can select your choice to download here.
Wish I’d known: Pay close attention to the version of Processing you are using, as sometimes you could run into serious compatibility issues with libraries or tutorials based on different versions. For this article, I’m using Processing 3.5.4. When I first started out I was going to follow this amazing tutorial by Amnon Owed, only to find myself ending up with a monstrosity of incompatibility errors. I later realized that although Amnon’s tutorial is great, it was written in 2011 and he was using Processing 1.5.1.
But don’t panic! If you’re following along and seeing the following welcome page, you’re all set for this step!
Step 2: Download and Setup NyARToolKit for Processing.
NyARToolKit (Copyright ©2008–2016 Ryo Iizuka) is a class library for AR application built with Java, based on the ARToolkit 5.2.3. In this experiment, we are using the NyARTookKit for Processing. Simply put, it is a contributed library/ toolkit designed for Processing that gives us the tools to explore AR interactions. It is also free and open-sourced.
You can download NyArToolKit for Processing here on GitHub.
(Don’t forget to extract the ZIP file after you download.)
Next, we want to add this library we just downloaded to Processing.
To do this, we first navigate to the sketchbook location on your laptop (where all the Processing files are stored). If you don’t know where this is, simply go to your Processing application (PDE), and then go to File > Preferences … You can copy the sketchbook location from there.
Now we are in the sketchbook location, you should be able to several folders.
The one we’re looking for is called libraries. If you don’t see one, simply create an empty folder and name it libraries.
Lastly, we want to drag the downloaded and extracted library folder (named nyar4psg) into the libraries folder.
To see if the library is successfully added to Processing, restart the application (PDE) and go to File > Examples > Contributed Libraries, see if you can find the nyar4psg folder.
Wish I’d Known: Another crucial library that we’ll be using is the Video library in Processing.
Although the video library is developed and maintained by the Processing Foundation, due to its size, it must still be downloaded separately through the contributions manager. The Video and Sound libraries need to be downloaded through the Library Manager. Select “Add Library…” from the “Import Library…” submenu within the Sketch menu.
In the contribution manager pop-up window, you’ll be able to find and download the Video library.
Hurray! You are all done with setting up! Now let’s play with the marker-based AR :)
Step 3: Print out the Markers, and Have Fun!
Markers are black-and-white images that help the NyARToolKit to determine the three-dimensional positions and directions in the real world and in real-time. You can find them in the library you downloaded. Print out these markers.
nyar4psg > data> …
There are many different examples offered in the NyARTookKit library that you can experiment with. The one we’re going to try here is called rotation.
Go back to the Processing application (PDE) and navigate to File > Examples> Contributed Libraries> nyar4psg> rotation, and double click to launch the rotation sketch (file).
Click on the Run icon in the top left corner, and you’ll see a pop-up window of a real-time video from your camera.
Hold up the markers to your camera, and see the AR cubes dance!
Congratulations on completing your first AR experiment!
Here are some resources if you want to further explore:
Learn more about Processing from their official tutorials, or my favorite Processing tutorials by Daniel Shiffman.
Check out Google AR/VR courses on Coursera
Check out this tutorial from the Processing Foundation to learn how to use Processing to build mobile AR applications for Android*
*For the next story in this publication, I’m going to explore using Unity to build mobile AR applications.
Hope you had fun reading :)
