Getting Started with Filament on Android

9 min readApr 3, 2020

Filament is Google’s open source physically-based renderer. It’s great for when you need to add 3D capabilities to your application without the overhead of an entire game engine.

Filament can be used across a variety of platforms (including iOS and the web), but it is particularly well-suited to Android. It has a fairly small core library that can be loaded quickly, which is essential to creating smooth experiences on mobile devices.

In this post we will walk through the process of creating a simple Android application that displays a glTF 2.0 model like the one above, allowing users to pan and zoom using familiar gestures. The result will be similar to sample-gltf-viewer, which you can find in the Filament repo on GitHub.

Utility Layers

The various Filament layers that are available with Maven are summarized below. In this tutorial we will be using all of these layers except filamat.

filament-utils High-level Android utilities, including helpers for Kotlin.
gltfio Materials and importers for glTF 2.0 model files.
filament Core renderer.
filamat Enables generation of materials at run time.

The following diagram depicts the Maven packages that we will use and some of the classes they provide. Applications can use whichever level of abstraction they need.

In general, the utility layers do not prevent interaction with Filament’s core objects. For example, FilamentAsset is just an unstructured container of entities, materials, and textures. It provides very little functionality on its own. Similarly, ModelViewer has properties that expose the asset, engine, scene, view, and camera.

Some exceptions to this rule are the grayed-out boxes in the above diagram, which are hidden to ModelViewer clients.

Application Skeleton

To get started, create a new project in Android Studio using the Empty Activity template. Select Kotlin in the configuration dialog. I hope you like Kotlin as much as I do, but Filament supports Java and C++ too. You can select any minimum SDK version at API 19 or higher.

Open the root-level build.gradle and add mavenCentral() into the top repositories block. Next, add the following dependencies to your app-level gradle file.

The next time Android Studio syncs to Gradle, it will download the necessary archives from the central Maven repository. This can be manually triggered by selecting Sync Project With Gradle Files in the File menu or toolbar.

Next open MainActivity.kt and add the following.

This loads in the native code for the filament-utils layer, which in turn loads the native code for the other two layers. If you were using only the core Filament classes, you would instead say Filament.init().

At this point, you should be able to run the app without errors.

Next, add the following fields to your activity class after the companion object, and replace the pre-supplied onCreate method with the one below.

You’ll also need to add some import lines, but Android Studio can generate these for you. Next, let’s allow Filament to render during the frame callback. Add the following lines to your activity class.

At this point, if you run the app, the Filament engine is instantiated and an OpenGL context is created. As proof you should see something like this in your Android log:

I/Filament: FEngine (64 bits) created at 0x79099f2840

Adding Assets

Next let’s add a couple glTF models into the project, as well as some KTX files for the environment (more on these later). For convenience here’s a zip of the assets we used for this tutorial. You can unzip these into your main folder such that the resulting file structure looks like the one below.

Next make the following additions to MainActivity. This uses Android’s AssetManager facility to read the contents of the glb file into a ByteBuffer and pass it to ModelViewer.

Note the call to transformToUnitCube(). This tells the model viewer to transform the root node of the scene such that it fits into a 1x1x1 cube centered at the origin.

Let’s also add support for landscape orientation. Open AndroidManifest.xml and add the following two attributes to the <activity> tag.

android:screenOrientation=
    "fullSensor"android:configChanges=
    "orientation|screenSize|screenLayout|keyboardHidden"

The screenOrientation attribute allows the viewport to rotate into all four orientations, while configChanges stipulates that the activity should be resized rather than recreated.

At this point you should be able to run the app and see a 3D model. It should also respond to a one-finger tumble gesture, two-finger pan, and pinch-to-zoom.

However the lighting is a bit dark. That’s because it has only one light source, which is a simple directional light created by ModelViewer. To make the scene look much nicer, we need to add an indirect light source and a skybox. This brings us to the next section.

Image-Based Light and Skybox

Filament supports rendering with image-based lighting, or IBL. This uses an environment map to approximate the lighting all directions.

At run time, we need to create an IndirectLight object by loading a KTX file that contains a set of floating-point images. Together these images comprise all the mipmap levels and cubemap faces that make up the environment. In a sense these are not visible images; it is more accurate to them of them as data that can be used to approximate the indirect lighting in the scene.

On the other hand, the Skybox object can be loaded from a KTX that does contain visible images. Filament provides an offline tool called cmgen that can consume an equirectangular image and produce these two files in one fell swoop, as depicted below.

The asset bundle that we added to the project already contains the necessary KTX files, so let’s carry on and add some code to load them into the scene. (Jump to the bottom of the post to see how to use cmgen for making your own cubemaps.)

Now if you run your app, the scene should be much more compelling.

JSON-based glTF

The DamagedHelmet model that we loaded is a .glb file, so all its texture resources and vertex buffers are embedded in a single file. To load a file with a .gltf extension, we will need to pass in a callback that tells ModelViewer how to load an external resource from a URI string. Try making the following changes. This will load the BusterDrone model (as seen at the top of the post) along with its external resources.

Buster Drone was created by LaVADraGoN and obtained from Sketchfab. It is licensed under the creative commons (CC BY-NC).

If you try running the app on a slow device or emulator, you’ll notice that the individual meshes gradually pop in while the app is fully interactive. This is due to the asynchronous API in ResourceLoader, which allows texture decoding to occur in the background.

Applying Animation

The gltfio library includes an Animator object that can be accessed through FilamentAsset. To see this in action, try replacing your frameCallback as follows.

The key piece in the above snippet is the call to applyAnimation. This takes two arguments: an index into the model’s list of animation definitions, and the the elapsed time for that particular animation. In glTF, animations typically represent actions. For example, animation 0 might be a walk cycle, while animation 1 is a run cycle.

Since this model uses morphing rather than skinning, the call to updateBoneMatrices() is not strictly required. We included it to show best practice. In glTF, skinning is orthogonal to animation (you can have one without the other) which is why this is not done automatically.

In addition to applyAnimation and updateBoneMatrices, the animator interface offers some simple queries:

int getAnimationCount()
float getAnimationDuration(int index)
String getAnimationName(int index)

Diving Deeper

At this point, using only ~100 lines of code, we’ve created a fairly sophisticated Android application that can render, animate, and tumble a 3D scene. However Filament is much more than a glTF viewer, so let’s dive a little deeper and play with some of its core API’s.

The individual objects that comprise a Filament scene are part of an entity-component system (ECS). This allows the renderer to efficiently traverse the scene in a data-oriented way, and allows composition of behaviors and attributes without an unwieldy class hierarchy.

Filament does not provide a “node” type like a classic scene graph, instead it provides transformable components that can be composed into a tree. Thus, for each node in the glTF hierarchy, the gltfio loader creates an entity, and to each of these entities it adds a transformable component. Additionally, if any glTF node has an associated mesh, then the loader adds a renderable component to its corresponding entity.

Setting a transform

To illustrate using the ECS, let’s modify the application to make the drone continuously spin around the Z axis. To achieve this effect, we’ll need to grab the transform of the root entity.

The root is the only transformable entity that does not correspond to a particular node in the glTF file. It’s created by the loader to allow transformation of the whole asset.

Try adding the following snippet to the inside of your doFrame function.

Woops, this won’t build; the root entity does not expose getTransform and setTransform methods! That’s because entities in Filament are just integers. Remember, in an ECS system, entities are not strongly-typed objects. We need to extract the transformable component from the root and use that instead. Add the following two helper methods to your class. They use Kotlin extension functions to allow for a more natural syntax than what the low-level ECS provides.

Now the app should build and run, and the drone should spin around.

Names, Materials, and Visibility

As another example of using the ECS, let’s try hiding the floor disk and disabling the emissive tail lights on the back of the drone. Try adding the following to the end of the onCreate method.

Some of the entities in the asset do not have a renderable component so we check for zero at the top of the loop. To hide the floor disk, we check each entity name against a known string (this is the name that the artist gave to this particular entity), and call setLayerMask to hide it from the view.

The layer mask on the renderable works in tandem with a visibility mask that’s set in View. The setLayerMask method takes two bitmasks: a list of bits to affect, and the replacement values for those bits. In this case we want to hide the renderable from everything so we’re setting all visibility bits to zero.

Filament’s layer masking facility is meant only for simple use cases. Another way of hiding an entity is by calling scene.removeEntity, which would work only after the progressive load is complete. In this case, we use setLayerMask because we want to hide it shortly after calling loadModelGltf.

To disable the emissive red tail lights, we call setParameter on the first primitive of every mesh in the asset. This tells Filament to change the value of a material parameter (also known as a shader uniform), and in this case we’re setting the red-green-blue values of the emissive multiplier to zero. You can also use setParameter to swap one texture for another using one of the following parameter strings.

baseColorMap
metallicRoughnessMap
normalMap
occlusionMap
emissiveMap

To see the entire list of glTF material parameters, look in the Filament source tree for ubershader.mat.in.