47Billion
Published in

47Billion

How to code Augmented Reality Mobile Application with Android ARCore SDK — Part 2

In the previous blog, I described the basics of AR and how to set up Android Studio to start with AR Core SDK to develop AR App. In this blog, we will develop an AR app for Home Decor step by step.

  1. Add Sceneform plugin in Android Studio.
  2. Select minimum SDK for AR projects to API 24 that is Android 7.0 (Nougat)
  3. Add Gradle dependencies for ARCore SDK and Sceneform
  4. Configure Android Manifest for Camera permission, OpenGL, and AR requirement
  5. Create a layout including ARFragment.

All the above requirements are explained in detail in the previous blog [ How To Code Augmented Reality Mobile Application With Android ARCore SDK — Part 1].

Import 3D Model

Now after basic setup, we need to import 3D models. You can create your 3D model or directly import 3D models from poly.google.com

Select a 3D model and download the original OBJ file.

We will not copy these models to our project directly in resources. Instead, we will create a sample data directory and put them there. Sceneform will convert them to the format that our app will understand.

In the app folder, create a directory named sampledata/models.

Copy images of the models in a drawable folder under res.

We require sceneform plugin in android studio to import 3d models in our project.

Now import the models using a command in build.Gradle

sceneform.asset (
'sampledata/models/Chair2.obj',
'default',
'sampledata/models/Chair2.sfa',
'src/main/res/raw/chair'
)
sceneform.asset (
'sampledata/models/Lamp.obj',
'default',
'sampledata/models/Lamp.sfa',
'src/main/res/raw/lamp'
)
sceneform.asset (
'sampledata/models/Houseplant.obj',
'default',
'sampledata/models/Houseplant.sfa',
'src/main/res/raw/houseplant'
)

Sync gradle and rebuild the project, it will create raw folder and 3d model are imported into raw folder.

Loading models from resources

As 3D models are imported to the raw folder we have to load the selected model into the ArFragment

fun loadModels(callback : (ModelRenderable) -> Unit) {
val modelRenderable = ModelRenderable.builder()
.setSource(this, selectedId)
.build()
CompletableFuture.allOf(modelRenderable)
.thenAccept() {
callback(modelRenderable.get())
}
.exceptionally {
Toast.makeText(this,"Unable to load Models", Toast.LENGTH_LONG).show()
null
}
}

Here selectedId is the id of the 3D model selected from UI layout by the user. It will be like R.raw.piano if piano is selected by the user.

We will add the 3D model to ArFragment when user double tap on the fragment

fun addNodeToScene(anchor: Anchor, modelRenderable: ModelRenderable) {
var anchorNode = AnchorNode(anchor)
modelNode = TransformableNode(arFragment.transformationSystem)
.apply {
renderable = modelRenderable
localScale = Vector3(0.1f, 0.1f, 0.1f)
localPosition = Vector3(0.0f, 0.0f, 0.0f)
setParent(anchorNode)
getCurrentScene().addChild(anchorNode)
select()
}
modelNode.setOnTapListener{_,_ ->
if
(modelNode.isTransforming) {
if(buttonLayout.isVisible) {
buttonLayout.visibility = View.GONE
} else {
buttonLayout.visibility = View.VISIBLE
}
}
}
}
fun getCurrentScene () : Scene = arFragment.arSceneView.sceneprivate fun setupDoubleTapArPlaneListener() {
arFragment.setOnTapArPlaneListener { hitResult,_,_ ->
loadModels { modelRenderable ->
addNodeToScene(hitResult.createAnchor(),modelRenderable);
}
}
}

Double tap listener we will register in onCreate of Activity

override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
arFragment = fragment as ArFragment
buttonLayout = layoutInflater.inflate(
R.layout.button_layout, null) as LinearLayout
setupDoubleTapArPlaneListener()
}

We have to add buttons for proper positioning of our 3D object. Here is the complete code

button_layout.xml

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:gravity="center"
android:background="#fff"
android:orientation="horizontal"
>
<ImageView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:src="@drawable/leftarrow"
android:layout_marginRight="10dp"
android:layout_marginLeft="10dp"
android:onClick="leftClicked"
android:id="@+id/button1"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginRight="10dp"
android:layout_marginLeft="10dp"
android:src="@drawable/uparrow"
android:onClick="upClicked"
android:id="@+id/button2"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginRight="10dp"
android:layout_marginLeft="10dp"
android:src="@drawable/downarrow"
android:onClick="downClicked"
android:id="@+id/button3"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginRight="10dp"
android:layout_marginLeft="10dp"
android:src="@drawable/rightarrow"
android:onClick="rightClicked"
android:id="@+id/button4"
/>
</LinearLayout>

activity_main.xml

<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity"
>
<fragment
android:name="com.google.ar.sceneform.ux.ArFragment"
android:id="@+id/fragment"
android:layout_width="match_parent"
android:layout_height="match_parent"
/>
<include layout="@layout/button_layout" />
<LinearLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_gravity="bottom"
android:layout_marginTop="10dp"
>
<HorizontalScrollView
android:layout_width="match_parent"
android:layout_height="100dp"
android:id="@+id/horizontalScrollView"
>
<LinearLayout
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:orientation="horizontal"
>
<ImageView
android:layout_width="wrap_content"
android:layout_height="100dp"
android:src="@drawable/piano"
android:onClick="pianoSelected"
android:id="@+id/button1"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="100dp"
android:src="@drawable/lamp"
android:onClick="lampSelected"
android:id="@+id/button2"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="100dp"
android:src="@drawable/chair"
android:onClick="chairSelected"
android:id="@+id/button3"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="100dp"
android:src="@drawable/plant"
android:onClick="plantSelected"
android:id="@+id/button4"
/>
</LinearLayout>
</HorizontalScrollView>
</LinearLayout>
</FrameLayout>

MainActivity.kt

class MainActivity : AppCompatActivity() {    lateinit var buttonLayout: LinearLayout
lateinit var arFragment : ArFragment
var selectedId = R.raw.piano
var x : Float = 0.0f
var y : Float = 0.0f
lateinit var modelNode : TransformableNode
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
arFragment = fragment as ArFragment
buttonLayout = layoutInflater.inflate(R.layout.button_layout, null) as LinearLayout
setupDoubleTapArPlaneListener()
}
fun leftClicked(v : View){
x = modelNode?.localPosition.x - 0.1f;
modelNode?.apply {
localPosition = Vector3(x, y , 0.0f)
}
}
fun rightClicked(v : View){
x = 0.1f + modelNode?.localPosition.x;
modelNode?.apply {
localPosition = Vector3(x, y , 0.0f)
}
}
fun upClicked(v : View){
y = modelNode?.localPosition.y + 0.1f;
modelNode?.apply {
localPosition = Vector3(x, y , 0.0f)
}
}
fun downClicked(v : View){
y = modelNode?.localPosition.y - 0.1f;
modelNode?.apply {
localPosition = Vector3(x, y , 0.0f)
}
}
fun pianoSelected(v : View){
selectedId = R.raw.piano
}
fun lampSelected(v : View){
selectedId = R.raw.lamp
}
fun chairSelected(v : View){
selectedId = R.raw.chair
}
fun plantSelected(v : View){
selectedId = R.raw.houseplant
}
fun loadModels(callback : (ModelRenderable) -> Unit) {
val modelRenderable = ModelRenderable.builder()
.setSource(this, selectedId)
.build()
CompletableFuture.allOf(modelRenderable)
.thenAccept() {
callback(modelRenderable.get())
}
.exceptionally {
Toast.makeText(this,"Unable to load Models", Toast.LENGTH_LONG).show()
null
}
}
fun addNodeToScene(anchor: Anchor, modelRenderable: ModelRenderable) {
var anchorNode = AnchorNode(anchor)
modelNode = TransformableNode(arFragment.transformationSystem)
.apply {
renderable = modelRenderable
localScale = Vector3(0.1f, 0.1f, 0.1f)
localPosition = Vector3(0.0f, 0.0f, 0.0f)
setParent(anchorNode)
getCurrentScene().addChild(anchorNode)
select()
}
}
fun getCurrentScene () : Scene = arFragment.arSceneView.scene

private fun setupDoubleTapArPlaneListener() {
arFragment.setOnTapArPlaneListener { hitResult,_,_ ->
loadModels { modelRenderable ->
addNodeToScene(hitResult.createAnchor(),modelRenderable);
}
}
}
}

Output

Originally published at https://47billion.com on January 21, 2021.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store