How to implement AR based App using React Native?

As the “Artificial Intelligence” term can be mixed up with other related concepts. AR(Augmented Reality) & VR(Virtual Reality) are not the same.
AR is a blended projection of a virtual object in the real world. and VR is a projection of a virtual world to our eyes,

At the time of writing this article, ViroReact is the best option to develop AR/VR app in React Native.

ViroReact is based on 2 APIs that dominate the world of Augmented & Virtual Reality for mobile phones: ARKit for iOS and ARCore for Android.

Viro, AR/VR

ViroReact is an open-source development platform for rapidly building AR/VR applications using React Native. Use a single code base for your AR and VR apps.

Viro is provided two distinct packages: ViroCore and ViroReact.

ViroReact supports:
1. AR: ARKit and ARCore

2. VR: Gear VR, Daydream, and Cardboard (iOS and Android),

The platform is composed of two main components:

1. A high-performance native 3D rendering engine, and
2. A custom extension of React for AR and VR development.

If you are looking to build a new AR/VR application, or add AR/VR features to your existing applications, the ViroReact platform enables you to create powerful mobile AR/VR applications using a single code base. Write once, run everywhere!

Open Source

As of Viro React v2.17.0, we are now open source under the MIT License. Add, customize or extend Viro any you want. The source code is available here.

No need to register a special API Key to use it. Feel free to contribute, this project is great and we must maintain it anyway

Setup Viro Installation instructions

If you are starting a new project, please consider using our Starter Kit as a basis for your app. This is a blank React Native project setup with Viro.

Installation

1. git clone https://github.com/ViroCommunity/starter-kit.git
2. cd starter-kit
3. npm install
4. npx pod-install (iOS)
5. npx react-native run-android or npx react-native run-ios

NOTE: The variant arguments are not needed for debug or release.

How to Install Viro in an existing project?

The steps below are for manually installing and linking the library to an existing React Native project. We do not yet support auto-linking.

$ npm install --save @viro-community/react-viro

Linking (You must do this — we do not support auto-linking)

If you’re unsure about which file to edit or where to put specified lines, we have added links to how this is done in our starter-kit repo.

» iOS linking

» Android linking

NOTE: It doesn’t work on my iOS simulator! AR does not work on iOS simulators.

What is a Scene?

Applications in ViroReact consist of Scenes, represented either by ViroScene or ViroARScene components. Scenes are the 3D equivalent of the Views found in most 2D application frameworks. They contain all the content that ViroReact renders in AR/VR: UI controls, 3D objects, lights, and more.

An application in ViroReact typically contains one or more of these Scene components contained in a ViroSceneNavigator or ViroARSceneNavigator.

Basic Scene

A simple <ViroScene> is provided below. The scene contains a single <ViroText> object, which displays the text "Hello World".

<ViroScene>
  <ViroText text="Hello World" position={[0, -.1, -1]} />
</ViroScene>

How to Detect planes & Create an AR Plane?

One method for placing objects in the real world is by placing them on a plane. We can use either ViroARPlane or ViroARPlaneSelector component to accomplish this. When the AR system detects a plane, the Viro platform attempts to attach it to any declared ViroARPlane components and continually keeps the virtual plane anchored to the detected real-world plane. On the other hand, the ViroARPlaneSelector component enables developers to allow their users to select the plane that they want the developer to use.

Let’s use ViroARPlaneSelector. Paste the following code right aboveViro3DObject in your scene.

<ViroARPlaneSelector />

Save your file and reload the testbed app. In addition to the previous scene, you should now see planes appear as you move around your room. In our real world, both the table and floor plane were detected as shown below:

Let’s add some cool 3D models to our basic AR app.

3D objects are not as simple as 2D images; they are usually a mesh of a model (VRX or OBJ) file that defines geometry, plus a few images to define textures that are embedded over top.

Position the object in space, give it a scale and specify the type — note there are a lot of different object types, and they are all handled differently in viro. For more information on 3D objects, view this doc

Place the Emoji on the Plane

Continuing to in the documentation I started adding 3D objects you need to import components Viro3DObject, ViroAmbientLight and ViroSpotLight.

Add the following to your import statement

import {
    ...
  Viro3DObject,
  ViroAmbientLight,
  ViroSpotLight,
} from 'react-viro';

Once you are done importing creates a new Viro3DObject and import the provided files by viro into your project as the source. Specify the position, scale, and type — vrx is a converted 3d object file type

With AR, we oftentimes want objects to be placed about the real world. Using the planes we identified earlier, let’s place our emoji on a plane. Place the Viro3DObject code within ViroARPlaneSelector. The modified code should look like this:

<ViroARPlaneSelector>
  <Viro3DObject
    source={require('./emoji_smile.vrx')}
    position={[0, .1, 0]}
    scale={[.2, .2, .2]}
    type="VRX"
    dragType="FixedDistance" onDrag={()=>{}} />
</ViroARPlaneSelector>

Save the file and reload the testbed app. Now that we have placed the 3D Object inside the ViroARPlaneSelector, when a plane is tapped, the emoji will be placed on the selected plane and the planes will disappear.

Move 3D Object in Real World

To make our emoji drag along real-world surfaces, we need to replace ViroARPlaneSelector with a ViroNode, set the dragType to "FixedToWorld", and add an empty anonymous function to let the platform know that we want this object to drag. Copy and paste the code below replacing: <ViroARPlaneSelector> … </ViroARPlaneSelector>

<ViroNode position={[0,0,-1]} dragType="FixedToWorld" onDrag={()=>{}} >
  <Viro3DObject
    source={require('./emoji_smile.vrx')}
    position={[0, .1, 0]}
    scale={[.2, .2, .2]}
    type="VRX" />
</ViroNode>

Save your file and reload the testbed app. The emoji should appear right in front of you. Drag the emoji towards a plane, notice how it moves along real-world surfaces. (Make sure to scan the room to give ARKit a chance to identify planes so the emoji can be moved along it.)

Positioning Objects in a Scene

Viro uses a right-handed coordinate system, where the direction of view is along the negative z-axis. The point of view can be modified by changing the Camera. By default, the camera is positioned at [0, 0, 0] and looks in the direction [0, 0, -1]. In frameworks with only 3DOF (3 degrees of freedom) support (like mobile VR), the camera stays at [0,0,0] until moved by the developer with the end user only able to control the rotation of the camera. On the other hand, 6DOF (6 degrees of freedom) supported frameworks allow the end-user to move about their world and move the camera in the Viro Scene in response (like in AR).

Next Steps

You should now have a basic interactive AR app (final code posted below). Check out our Code Samples for other AR examples or continue adding functionality to this scene.
- Try adding a shadow to the emoji. Check out the Lighting and Materials guide for details.

Conclusion

That’s it! In this tutorial, you learned the basics of using ViroReact. Specifically, you learned how to create a basic VR application that displays a scene, an image, and a text.

ViroReact is a very promising solution for creating VR and AR apps in React Native. It allows you to develop VR/AR apps faster because of React Native’s fast life to reload feature.