Part 1 — Show Unity3D view in React-Native application. Yes it’s possible!
Story (Android only)
I needed to integrate a Vuforia / Unity3D augmented reality view into a React-Native application on a recent project. You know, the simplest path is not simple when you do not know the path to achieve your goal. In fact, my first prototype was difficult to realize. Specifically, I encountered a black screen problem as described on this link StackOverflow.
This first article concerns the integration between Unity3D and React-Native. How to add the Unity View in your React-Native application. Attention: This article concerns only the solution for Android developers. Also, you will not find any technicality concerning Unity3D and Vuforia, except for some export details of the Unity project for Android. Please refer to the Vuforia and Unity documentation for more details. I am not an expert in Unity3D, so I prefer to let the experts guide you.
The second part will be about the communications between React-Native and Unity3D. It’s possible to invoke Unity3D from React-Native or the opposite. (Coming soon!)
Project Structure
OK! Let’s start first with the general presentation of the architecture of the solution.
Unity Project : The project contains the 3D that you want to show in your React-Native application.
Unity Android exported project : The results of the Android export in Unity.
React-Native Project : The destination project of your Unity View!
- React-Native Android Project : The Android project generated by React-Native. Here is the most important part of the project.
On disk you can imagine your file structure like It describe. 3 projects side by side and the React-Native Android Project as a children of React-Native project. Anyway, that project is a children by default.
THE Code
Now the real thing, THE code! Yeah :-)
As I can generate an Android project from Unity with a UnityPlayer view displayed in an activity, I thought that the approach to take would be to create a Native UI component for React-Native containing the UnityPlayer view that I could use in my React-Native component.
But you can not just instantiate UnityPlayer in the Manager and return it. As you can see in the UnityPlayerActivity in the project by Unity, the UnityPlayer depends on the lifetime of the activity. That is the most important part to know. UnityPlayer can not work properly if not used in the Root Activity of the React-Native application. By default, it’s the MainActivity.
Native UI Component
So You should instantiate UnityPlayer in the MainActivity of your React-Native Android project and copy exactly the implementation of the UnityPlayerActivity. The result should look like this.
public class MainActivity extends ReactActivity {
private UnityPlayer mUnityPlayer;
public UnityPlayer getUnityPlayer() {
return mUnityPlayer;
}
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
getWindow().setFormat(PixelFormat.RGBX_8888); // <--- This makes xperia play happy
mUnityPlayer = new UnityPlayer(this);
}
public void onDestroy() {
super.onDestroy();
mUnityPlayer.quit();
}
public void onResume() {
super.onResume();
mUnityPlayer.resume();
}
// Low Memory Unity
@Override public void onLowMemory() {
super.onLowMemory();
mUnityPlayer.lowMemory();
}
// Trim Memory Unity
@Override public void onTrimMemory(int level) {
super.onTrimMemory(level);
if (level == TRIM_MEMORY_RUNNING_CRITICAL)
{
mUnityPlayer.lowMemory();
}
}
// This ensures the layout will be correct.
@Override public void onConfigurationChanged(Configuration newConfig) {
super.onConfigurationChanged(newConfig);
mUnityPlayer.configurationChanged(newConfig);
}
// Notify Unity of the focus change.
@Override public void onWindowFocusChanged(boolean hasFocus) {
super.onWindowFocusChanged(hasFocus);
mUnityPlayer.windowFocusChanged(hasFocus);
}
// For some reason the multiple keyevent type is not supported by the ndk.
// Force event injection by overriding dispatchKeyEvent().
@Override public boolean dispatchKeyEvent(KeyEvent event)
{
if (event.getAction() == KeyEvent.ACTION_MULTIPLE)
return mUnityPlayer.injectEvent(event);
return super.dispatchKeyEvent(event);
}
// Pass any events not handled by (unfocused) views straight to UnityPlayer
@Override public boolean onKeyUp(int keyCode, KeyEvent event) { return mUnityPlayer.injectEvent(event); }
@Override public boolean onKeyDown(int keyCode, KeyEvent event) { return mUnityPlayer.injectEvent(event); }
@Override public boolean onTouchEvent(MotionEvent event) { return mUnityPlayer.injectEvent(event); }
/*API12*/ public boolean onGenericMotionEvent(MotionEvent event) { return mUnityPlayer.injectEvent(event); }
/**
* Returns the name of the main component registered from JavaScript.
* This is used to schedule rendering of the component.
*/
@Override
protected String getMainComponentName() {
return "protoreactnative";
}
}Now you want to make UnityPlayer available to your React-Native component. The first thing to do, create a ViewManager and the ReactPackage:
public class CustomUnityViewManager extends SimpleViewManager<UnityPlayer> {
public static final String REACT_CLASS = "UnityContainerView";
@Override
public String getName() {
return REACT_CLASS;
}
@Override
protected UnityPlayer createViewInstance(ThemedReactContext reactContext) {
MainActivity activity = (MainActivity)reactContext.getCurrentActivity();
return activity.getUnityPlayer();
}
}public class CustomReactPackage implements ReactPackage {
@Override
public List<NativeModule> createNativeModules(ReactApplicationContext reactContext) {
return Arrays.<NativeModule>asList(
new CommBridge(reactContext) //This will be the subject of the next article.
);
}
@Override
public List<ViewManager> createViewManagers(ReactApplicationContext reactContext) {
return Arrays.<ViewManager>asList(
new CustomUnityViewManager()
);
}
}
In your ViewManager you return the UnityPlayer provided by your MainActivity that is connected to the life cycle of the activity.
Gradle
The next question is, hey man how I can build it without any dependencies to the unity exported result?
You need to configure Gradle to point on the output of Unity.
- In the project level gradle add flatDir in repositories add:
flatDir {
dirs 'libs'
}2. In the app level gradle add require dependencies
3. Be sure that ‘compileSdkVersion’ and ‘buildToolsVersion’ are the same as the version use in Unity.
4. Add required dependencies that point on Unity build output:
// Unity Build output
implementation files('../../[BuildOutput]/libs/unity-classes.jar')
implementation files('../../[BuildOutput]/libs/VuforiaWrapper.aar')5. Copy ‘assets’ folder from unity generated project to react native project:
task copyUnityAssets << {
copy {
from '../../[BuildOutput]/src/main/assets'
into 'src/main/assets'
}
copy {
from '../../[BuildOutput]/src/main/jniLibs'
into 'src/main/jniLibs'
}
}
preBuild.dependsOn(copyUnityAssets)React-Native
Visual Studio Code is my prefered IDE for that part.
In your index.android.js register your component like that
import { AppRegistry } from "react-native";
import AppRoot from "./scripts/react/root";
AppRegistry.registerComponent("protoreactnative", () => AppRoot);In your React-Native component just use it!
const UnityContainerView = requireNativeComponent("UnityContainerView", null);
class ArPage extends React.Component {
render() {
return (
<View style={{ flex: 1, backgroundColor: "red" }}>
<UnityContainerView style={{ flex: 1 }} />
</View>
);
}
}The result is awesome
In our case, we wanted to do augmented reality. The result is great!
I hope that tutorial you will help your awesome project!
Good luck :-)
Part 2 of the tutorial available : https://medium.com/@beaulieufrancois/part-2-show-unity3d-view-in-react-native-application-138219323cbc
Project based on :
Unity 2017.2.0f3
React 16.0.0.0-alpha12
React-native 0.48.4
Special thanks to fortinmike for your knowledge! He is the IOS Master, I share his iOs integration notes.
