In-depth study of AR-Augmented Image class file from ARCore

Photo by Joan Gamell on Unsplash

Firstly, AugmentedImageActivity uses several variables. Here’s what they are being used for and the number refers to the function they are used in:

• surfaceView: Represents a GLSurfaceView for rendering graphics using OpenGL.(#1, #3, #4)
• fitToScanView: Represents an ImageView used to display an image.(#1, #3, #11)
• glideRequestManager: Manages the image loading and caching using the Glide library.(#1)
• installRequested: A boolean flag used to track whether an installation is requested.(#1, #3)
• session: Represents an AR (Augmented Reality) session, used for tracking and rendering AR objects.(see here)(#2,#3,#4, #9, #10, #12)
• messageSnackbarHelper: Provides helper methods for showing snackbar messages.(#2, #3, #10, #11)
• displayRotationHelper: Assists with handling changes in display rotation.(#1, #4, #8, #9)
• trackingStateHelper: Provides helper methods for tracking the state of AR tracking.(#9)
• backgroundRenderer: Responsible for rendering the background of the AR scene. (#7, #9)
• augmentedImageRenderer: Responsible for rendering augmented images in the AR scene.(#7 ,#11)
• shouldConfigureSession: A boolean flag indicating whether the session needs to be configured.(#3)
• useSingleImage: A boolean flag indicating whether a single image or a pre-generated image database is used.(#12)
• augmentedImageMap: A map that stores augmented images and their associated anchor points, using the index of the augmented image as the key.(#11)

Apart from these there are several functions that are being used:

1. onCreate(Bundle savedInstanceState): This method is called when the activity is created. It initializes various variables and views, sets up the OpenGL rendering context, loads an image into `fitToScanView` using Glide, and sets the `installRequested` flag to false.

2. onDestroy(): This method is called when the activity is destroyed. It’s a simple function that explicitly closes the ARCore session to release native resources.

3. onResume(): This method is called when the activity is resumed. It checks if the AR session is null and attempts to create a new session. It handles various exceptions related to ARCore installation and device compatibility. It also configures the AR session and starts the camera preview. (P.s. Most of the errors related to camera permissions are handled by this function.)

4. onPause(): This method is called when the activity is paused. It pauses the AR session and releases the camera resources.

5. onRequestPermissionsResult(int requestCode,String[] permissions, int[] results):This method is called when the app requests camera permissions from the user. It displays a Toast message if the camera permission is denied and prompts the user to go to the app’s settings to grant the required permission.

6. onWindowFocusChanged(boolean hasFocus) : This method is called when the window focus of the activity changes. It sets the activity to full screen mode using a helper function.

7. onSurfaceCreated(GL10 gl,EGLConfig config): This method is called when the OpenGL surface is created. It uses the backgroundRenderer and sets the background color for the OpenGL context and prepares rendering objects, such as creating textures for the camera image and augmented images.

8. onSurfaceChanged(GL10 gl,int width, int height):This method is called when the OpenGL surface changes its size. It updates the display rotation helper and sets the viewport to match the new size.

9. onDrawFrame(GL10 gl): This method is called when a new frame needs to be rendered in the OpenGL context. It clears the screen, updates the ARCore session, obtains the camera and projection matrices, and renders the camera preview and augmented images.
P.s. I recommend you to check the code of this function(here) as it’s explained there well in detail.

10. configureSession(): This method configures the AR session, sets the focus mode to auto, and attempts to set up the augmented image database. This uses the setupAugmentedImageDatabase(Congig config) (see #12) to set up the augmented image database.

11. drawAugmentedImages(Frame frame,float[] projmtx,float[] viewmtx,float[] colorCorrectionRgba): This method is responsible for drawing augmented images on the screen. It iterates through the updated augmented images and updates their tracking state.
For newly detected images, it creates anchors for tracking, and for stopped images, it removes them from the rendering list. Finally, it calls the augmentedImageRenderer to draw the augmented images that are currently being tracked.

12. setupAugmentedImageDatabase(Congig config): This method sets up the augmented image database for ARCore. It uses the function mentioned below loadAugmentedImageBitmap() to load an image in object “augmentedImageBitmap” and add it to augmentedImageDatabase.
Alternavitly, if useSingeImage is set to FALSE then it loads an already existing database and adds it to object augmentedImageDatabase.

13. loadAugmentedImageBitmap(): loadAugmentedImageBitmap() method uses BitmapFactory.decodeStream() to load an augmented image bitmap from the assets folder. It opens an input stream to the file "default.jpg" located in the assets folder and passes that stream to BitmapFactory.decodeStream() to decode the image file into a Bitmap object.

These were all the functions and variables used in the AugmentedImagesActivity provided as a sample app by ARCore-Google.