Building AR Apps with Flutter and ARwayKit

Mayank Raj
ARWAY
Published in
6 min readMar 14, 2021

--

ARwayKit + Flutter
ARwayKit + Flutter

Flutter is a great toolkit for building beautiful applications natively compiled for mobile, web, and desktop from a single codebase. But what if we want to use Flutter to build cross-platform World Scale AR apps?

Well, now that’s possible using the power of ARwayKit! Using ARwayKit with Flutter allows the developers to use their existing Android/iOS apps built using Flutter and then integrate ARwayKit into it as a scene rather than rebuilding the app from the ground up in Unity.‌

Thanks to ARwayKit, developers can now build unique AR experiences in a single application, using Unity’s power, while benefiting from Flutter's native performance and battery life preservation!

How does it work?

This methodology leverages Unity’s Unity as a Library feature that lets you manage when and how to load/activate/unload the Unity runtime library within your existing native application, thus saving a lot of time that otherwise would be wasted on building an app from scratch.

Integrating this with Flutter using flutter_unity_widget, we bring your App to the next level by combining the power of ARwayKit with Microsoft’s Azure Spatial Anchors to create some genuinely outstanding AR experiences in the real world!

Components of ARwayKit + Flutter:‌

  • Flutter Project: Your new/existing Flutter project used for the integration with ARWAY Unity SDK.
  • ARWAY Web Studio: included in ARwayKit, it’s a one-stop place for creating and editing maps, adding floor plans for navigation along with overlaying 3D assets, waypoints, destinations, text, and pictures over it.
  • ARWAY SDK: included in ARwayKit, it’s a Unity-based SDK for linking the AR experiences created in the Web Studio and adding them to the Flutter application as a page/scene.

How to develop using ARwayKit + Flutter?

1. Download the Flutter project‌

Step 1: Download the ARwayKit_SDK_FlutterSamples project from GitHub as a ZIP. After sign-up, you can find the link in the Developer Portal Downloads section.

Download the project as ZIP
Download the project as a ZIP

Step 2: Extract the .zip file and open the project with Android Studio or VS Code.

2. Configure the Flutter project

Step 1: Open the pubspec.yaml file and add flutter_unity_widget to it under the dependencies section.

Add Flutter Dependencies
Add Flutter Dependencies

We are using the flutter_unity_widget package to integrate the ARWAY Unity SDK with Flutter — https://pub.dev/packages/flutter_unity_widget

Step 2: After adding the dependencies, you need to fetch them into your Flutter project. You can fetch the dependencies by -

• From the terminal: Run flutter pub get. OR
• From Android Studio/IntelliJ: Click Packages get in the action ribbon at the top of pubspec.yaml.
• From VS Code: Click Get Packages located on the right side of the action ribbon at the top of pubspec.yaml.

Step 3: Then add an import statement for using the package inside your Dart code.

import 'package:flutter_unity_widget/flutter_unity_widget.dart';

We have created sample Dart files with unity pre-integrated to help you get started quickly with the project. The Dart files are located in the lib folder within the Flutter project.

3. Configure the Unity Project

Step 1: Add the Unity project located inside the Flutter Samples project into the Unity Hub. Your project files should look like this.

├── android
├── ios
├── lib
├── test
├── unity
│ └── <Your Unity Project> // Example: ARwayKit-SDK
├── pubspec.yml
├── README.md
Add project to Unity Hub
Add project to Unity Hub.

Step 2: Open the project with Unity 2019.4.3f1 or above.

Import theFlutterUnityPackage.unitypackage if you don't see a FlutterUnityIntegration folder inside the ARwayKit-SDK project.

Step 3: Open the Package Manager (Window -> Package Manager) and install the following components if not already installed -
AR Foundation >= 4.1.1 (2019.4 verified)
ARCore XR Plugin >= 4.1.1 (2019.4 verified)
ARKit XR Plugin >=4.1.1 (2019.4 verified)
Unity XR management >= 3.2.16 (2019.4 verified)
TextMesh Pro >= 2.1.1 (2019.4 verified)

Step 4: Go to Player Settings (Edit -> Project Settings -> Player -> Other Settings)
• Turn off the Auto-Graphics API.

For iOS:
• Set Graphics API to Metal.
• Set Target Minimum Version to 12.0.

For Android:
• Set Graphics API to OpenGLES3.
• Set Minimum API Level to 24 (Android 7.0).
• Disable Multithreaded Rendering

Step 5: Add the scenes you wish to build under the Unity build settings.

Add scenes to Unity Build Settings
Add scenes to Unity Build Settings.

DO NOT press the Build or Build And Run buttons. It’s not needed here.‌

Step 6: Now, this is a crucial step. Instead of building the app, go to the Unity menu and then export the Unity project with Flutter/Export Android (for Unity versions 2019.3 and up, which uses the new Unity as a Library export format), or Flutter/Export IOS menu.

Do not use Flutter/Export <Platform> plugin as it was specially added to work with flutter_unity_cli for larger projects.

Export to Flutter
Export to Flutter

Android will export unity project to android/unityLibrary.
IOS will export unity project to ios/UnityLibrary.

NOTE: The build unity export script automatically sets things up for you, so you don’t have to do anything for android. But if you want to manually set it up, continue, else skip to iOS.‌

4. How to use ARWAY SDK Scenes?

To use the scenes in the ARWAY Unity SDK or the Azure Spatial Anchors, kindly look at the documentation for those scenes linked below.

Using ARWAY SDK — https://docs.arway.app/arway-sdk/sample-scenes

Using Azure Spatial Anchors + ARwayKit — https://docs.arway.app/azure-spatial-anchors/quickstart-in-unity

5. iOS Platform Only

Step 1: Open your ios/Runner.xcworkspace (workspace!, not the project) in Xcode and add the exported project in the workspace root (with a right-click in the Navigator, not on an item -> Add Files to “Runner” -> add the unityLibrary/Unity-Iphone.xcodeproj file

Step 2: Select the Unity-iPhone/Data folder and change the Target Membership for Data folder to UnityFramework (Optional)

Step 3: Add this to your Runner/Runner/Runner-Bridging-Header.h

#import "UnityUtils.h"

Step 4: Add to Runner/Runner/AppDelegate.swift before the GeneratedPluginRegistrant call:

InitArgs(CommandLine.argc, CommandLine.unsafeArgv)

For example

import UIKit
import Flutter
@UIApplicationMain
@objc class AppDelegate: FlutterAppDelegate {
override func application(
_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
) -> Bool {
InitArgs(CommandLine.argc, CommandLine.unsafeArgv)
GeneratedPluginRegistrant.register(with: self)
return super.application(application, didFinishLaunchingWithOptions: launchOptions)
}
}

Or, when using Objective-C, your main.m should look like this:

#import "UnityUtils.h"
int main(int argc, char * argv[]) {
@autoreleasepool {
InitArgs(argc, argv);
return UIApplicationMain(argc, argv, nil, NSStringFromClass([AppDelegate class]));
}
}

Step 5: Opt-in to the preview of the embedded view by adding a boolean property to the app’s Info.plist file with the key io.flutter.embedded_views_preview and the value YES.‌

Step 6: Add UnityFramework.framework as a Library to the Runner project.

6. ‌Build And Run:‌

Step 1: Run the app on Android with the following command. For iOS, build an XCode project to deploy it to an iOS device from your choice IDE.

$ flutter run

Step 2: After the app build completes, you’ll see the starter app on your device.

Start Screen
Start Screen

Step 3: Now press the Open ARWAY SDK button to open the Unity Scenes you’ve added before exporting the Unity project within the Flutter app itself. It should look something like this depending upon the scene you’re building.

Unity Screen
Unity Screen

And that’s it! Now you can get started with creating unique AR experiences using ARwayKit + Azure Spatial Anchors + Flutter. What are you waiting for?

For more details on how the integration with Flutter works, check out the ARWAY Documentation.

Try ARwayKit — Sign Up Now!

ARwayKit is on the horizon — and you and your team can be among the first to get access to this innovative tool. With ARwayKit, you’ll be able to boost user engagement, create unforgettable and unique experiences for augmented reality users, and realize your team’s highest spatial AR goals. Sign up today and get started building.

--

--

Mayank Raj
ARWAY

Merging Realities | Building the Metaverse with the power of Blockchain & Gaming!