How to build an Augmented Reality App using the Foursquare API + Mapbox + ARKit

Earlier this month, we saw this tweet from Aaron Ng showing how he used the Mapbox Unity SDK, Apple’s ARKit, and Foursquare to visualize his check-ins. We wanted to break down the steps required for you to do this and turn more magic into reality for others.

This is a brief technical tutorial on building out a Foursquare powered Augmented Reality app using Unity w/the Mapbox Unity SDK, ARKit and Foursquare’s API. We’ll be building an iOS app displaying nearby coffee shops using Foursquare Venue Search.

Demo using Foursquare API, Mapbox Unity SDK and ARKit.

Note: At the time of writing, this project requires an iPhone 6S (or later) running a pre-release of iOS 11.3 w/Metal, a pre-release version of Xcode 9, and a patched version of Unity (Unity 5.6.1.p1 or later). Some experience with both Unity and iOS development is helpful, but not required.

Additionally we’ll continue to produce content as the Mapbox SDK evolves to support streaming map providers 😍.

Getting Started

Before getting started, you’ll need to:

1. Install Unity (Unity 5.6.1.p1 or later)
2. Create an empty Unity Project
3. Install, Configure and Setup the Mapbox Unity SDK.
4. Install the ARKit Plugin.

Step 1: Setup your environment

To get started, check out this guide to learn how to make a Unity scene with mesh terrain and Mapbox map imagery and the following to hotwire your app with ARKit.

As a reference point during the tutorial here is our Unity Scene Hierarchy

Further to this here is a breakdown of our GameObjects and details.

1. LocationProvider is shipped with the Mapbox Unity SDK
2. Map has three additional components (BuildMapAtLocation.cs, AbstractMap.cs, RangeTileProvider)
3. MainCamera has two additional components (UnityARVideo.cs, UnityARCameraManager.cs)
4. Player is added with PositionWithLocationManager.cs
5. Attribution provides attribution to Mapbox and Foursquare.
6. FoursquareFactory is added with FoursquareProvider.cs

Step 2: Setup the Foursquare API

In order to call the Foursquare API we can utilize UnityWebRequest to send a request to the Foursquare API and then use the built in JsonUtility to parse the response. We’ve chosen to keep this sample simple to find “coffee” utilizing our Venue Search API.

In the below we setup the function GetFoursquareVenueData to call the Foursquare API.

Once we do this we now need to parse the response with some models. In order to simplify things, we’ve included some sample models below. You can also utilize our extensive C-Sharp library too if preferred.

Now that we have our models setup we need to connect them to our GetFoursquareVenueData function.

Step 3: Create Foursquare Provider

In order to utilize the FoursquareAPI we plan to utilize Mapbox’s LocationProvider. This allows us to easily convert between the Unity World and Earth World (latitude and longitude). Additionally we could leverage the real-world location of the phone via LocationService.

We start by creating a GameObject named “FoursquareFactory” and attaching a “FoursquareProvider” which we will use as a function to display and surface Foursquare data.

Here you will need to add your CLIENT_ID and CLIENT_SECRET after signing up for a Foursquare Developer Account via developer.foursquare.com.

We can then call the function GetFoursquareVenueData inside our FoursquareProvider and create a Foursquare Map Object for venues near the phone.

Here is the final script for FoursquareProvider.

Finally let’s link everything together by creating a GameObject inside Unity with the name Player.

We then can attach the script PositionWithLocationProvider to this GameObject. We make some minor modifications here to allow us to push LocationData (lat,lng) over to the FoursquareProvider.

Step 4: Build and Deploy

You can now build your Unity Project and deploy this magic to your phone/tablet.

Given the complexity of all these steps, we’d encourage you to reach out to us @FoursquareAPI or via Stackoverflow for questions, tips and tricks.

A huge thank you to Mapbox, Unity and Apple for their efforts in this space, without their work these demos would not be possible.

At Foursquare, our mission is to make the world more delightful through location intelligence products including building illustrations like the one above. If this sounds like fun feel free to reach out and learn more.

Happy Building!