Building AR Apps with Flutter and ARwayKit
Flutter is a great toolkit for building beautiful applications natively compiled for mobile, web, and desktop from a single codebase. But what if we want to use Flutter to build cross-platform World Scale AR apps?
Well, now that’s possible using the power of ARwayKit! Using ARwayKit with Flutter allows the developers to use their existing Android/iOS apps built using Flutter and then integrate ARwayKit into it as a scene rather than rebuilding the app from the ground up in Unity.
Thanks to ARwayKit, developers can now build unique AR experiences in a single application, using Unity’s power, while benefiting from Flutter's native performance and battery life preservation!
How does it work?
This methodology leverages Unity’s Unity as a Library feature that lets you manage when and how to load/activate/unload the Unity runtime library within your existing native application, thus saving a lot of time that otherwise would be wasted on building an app from scratch.
Integrating this with Flutter using flutter_unity_widget, we bring your App to the next level by combining the power of ARwayKit with Microsoft’s Azure Spatial Anchors to create some genuinely outstanding AR experiences in the real world!
Components of ARwayKit + Flutter:
- Flutter Project: Your new/existing Flutter project used for the integration with ARWAY Unity SDK.
- ARWAY Web Studio: included in ARwayKit, it’s a one-stop place for creating and editing maps, adding floor plans for navigation along with overlaying 3D assets, waypoints, destinations, text, and pictures over it.
- ARWAY SDK: included in ARwayKit, it’s a Unity-based SDK for linking the AR experiences created in the Web Studio and adding them to the Flutter application as a page/scene.
How to develop using ARwayKit + Flutter?
1. Download the Flutter project
Step 1: Download the ARwayKit_SDK_FlutterSamples project from GitHub as a ZIP. After sign-up, you can find the link in the Developer Portal Downloads section.
Step 2: Extract the .zip file and open the project with Android Studio or VS Code.
2. Configure the Flutter project
Step 1: Open the pubspec.yaml file and add flutter_unity_widget to it under the dependencies section.
We are using the flutter_unity_widget package to integrate the ARWAY Unity SDK with Flutter — https://pub.dev/packages/flutter_unity_widget
Step 2: After adding the dependencies, you need to fetch them into your Flutter project. You can fetch the dependencies by -
• From the terminal: Run flutter pub get. OR
• From Android Studio/IntelliJ: Click Packages get in the action ribbon at the top of pubspec.yaml.
• From VS Code: Click Get Packages located on the right side of the action ribbon at the top of pubspec.yaml.
Step 3: Then add an import statement for using the package inside your Dart code.
import 'package:flutter_unity_widget/flutter_unity_widget.dart';We have created sample Dart files with unity pre-integrated to help you get started quickly with the project. The Dart files are located in the lib folder within the Flutter project.
3. Configure the Unity Project
Step 1: Add the Unity project located inside the Flutter Samples project into the Unity Hub. Your project files should look like this.
├── android
├── ios
├── lib
├── test
├── unity
│ └── <Your Unity Project> // Example: ARwayKit-SDK
├── pubspec.yml
├── README.mdStep 2: Open the project with Unity 2019.4.3f1 or above.
Import theFlutterUnityPackage.unitypackage if you don't see a FlutterUnityIntegration folder inside the ARwayKit-SDK project.
Step 3: Open the Package Manager (Window -> Package Manager) and install the following components if not already installed -
• AR Foundation >= 4.1.1 (2019.4 verified)
• ARCore XR Plugin >= 4.1.1 (2019.4 verified)
• ARKit XR Plugin >=4.1.1 (2019.4 verified)
• Unity XR management >= 3.2.16 (2019.4 verified)
• TextMesh Pro >= 2.1.1 (2019.4 verified)
Step 4: Go to Player Settings (Edit -> Project Settings -> Player -> Other Settings)
• Turn off the Auto-Graphics API.
For iOS:
• Set Graphics API to Metal.
• Set Target Minimum Version to 12.0.
For Android:
• Set Graphics API to OpenGLES3.
• Set Minimum API Level to 24 (Android 7.0).
• Disable Multithreaded Rendering
Step 5: Add the scenes you wish to build under the Unity build settings.
DO NOT press the Build or Build And Run buttons. It’s not needed here.
Step 6: Now, this is a crucial step. Instead of building the app, go to the Unity menu and then export the Unity project with Flutter/Export Android (for Unity versions 2019.3 and up, which uses the new Unity as a Library export format), or Flutter/Export IOS menu.
Do not use
Flutter/Export <Platform> pluginas it was specially added to work withflutter_unity_clifor larger projects.
Android will export unity project to
android/unityLibrary.
IOS will export unity project toios/UnityLibrary.NOTE: The build unity export script automatically sets things up for you, so you don’t have to do anything for android. But if you want to manually set it up, continue, else skip to iOS.
4. How to use ARWAY SDK Scenes?
To use the scenes in the ARWAY Unity SDK or the Azure Spatial Anchors, kindly look at the documentation for those scenes linked below.
Using ARWAY SDK — https://docs.arway.app/arway-sdk/sample-scenes
Using Azure Spatial Anchors + ARwayKit — https://docs.arway.app/azure-spatial-anchors/quickstart-in-unity
5. iOS Platform Only
Step 1: Open your ios/Runner.xcworkspace (workspace!, not the project) in Xcode and add the exported project in the workspace root (with a right-click in the Navigator, not on an item -> Add Files to “Runner” -> add the unityLibrary/Unity-Iphone.xcodeproj file
Step 2: Select the Unity-iPhone/Data folder and change the Target Membership for Data folder to UnityFramework (Optional)
Step 3: Add this to your Runner/Runner/Runner-Bridging-Header.h
#import "UnityUtils.h"Step 4: Add to Runner/Runner/AppDelegate.swift before the GeneratedPluginRegistrant call:
InitArgs(CommandLine.argc, CommandLine.unsafeArgv)For example
import UIKit
import Flutter
@UIApplicationMain
@objc class AppDelegate: FlutterAppDelegate {
override func application(
_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
) -> Bool {
InitArgs(CommandLine.argc, CommandLine.unsafeArgv)
GeneratedPluginRegistrant.register(with: self)
return super.application(application, didFinishLaunchingWithOptions: launchOptions)
}
}Or, when using Objective-C, your main.m should look like this:
#import "UnityUtils.h"
int main(int argc, char * argv[]) {
@autoreleasepool {
InitArgs(argc, argv);
return UIApplicationMain(argc, argv, nil, NSStringFromClass([AppDelegate class]));
}
}Step 5: Opt-in to the preview of the embedded view by adding a boolean property to the app’s Info.plist file with the key io.flutter.embedded_views_preview and the value YES.
Step 6: Add UnityFramework.framework as a Library to the Runner project.
6. Build And Run:
Step 1: Run the app on Android with the following command. For iOS, build an XCode project to deploy it to an iOS device from your choice IDE.
$ flutter runStep 2: After the app build completes, you’ll see the starter app on your device.
Step 3: Now press the Open ARWAY SDK button to open the Unity Scenes you’ve added before exporting the Unity project within the Flutter app itself. It should look something like this depending upon the scene you’re building.
And that’s it! Now you can get started with creating unique AR experiences using ARwayKit + Azure Spatial Anchors + Flutter. What are you waiting for?
For more details on how the integration with Flutter works, check out the ARWAY Documentation.
Try ARwayKit — Sign Up Now!
ARwayKit is on the horizon — and you and your team can be among the first to get access to this innovative tool. With ARwayKit, you’ll be able to boost user engagement, create unforgettable and unique experiences for augmented reality users, and realize your team’s highest spatial AR goals. Sign up today and get started building.
