Realistic reflections and environment textures in ARKit

Ivan Nesterenko
Jun 13, 2018 · 3 min read

Reflection is an essential part of our everyday environment. Naturally, for a realistic augmented reality experience, virtual objects should contain it too. Starting from ARKit 2.0 developers can use automatically generated environment textures for realistic reflections.

“In 3D asset rendering, environment textures are the basis for image-based lighting algorithms where surfaces can realistically reflect light from their surroundings. ARKit can generate environment textures during an AR session using camera imagery, allowing SceneKit or a custom rendering engine to provide realistic image-based lighting for virtual objects in your AR experience.”

Apple doc.

Environment texture gathers scene information in the form of a cube map; this texture later is used as a reflection probe. Because it’s impossible to have a fully complete cube map in realistic use cases(basically, to complete cube map the user has to complete 360-degree panorama), ARKit builds up missing parts of the cube map texture with machine learning.

Let’s take a look at a small demo of the reflective object.

As you can see, the sphere reflects real world objects that are placed on the table, which in my opinion is ultimately awesome 🤩

For this demo, I created a sphere and a reflective material for it.

I created a PBR material with a metalness = 1 and roughness = 0, here is a cheatsheet of how different materials look like depending on these values.

Metalness and roughness are two of the basic components for any physical based rendered material. You can check this article for more info about PBR lighting in SceneKit.

To enable environment texture map generation for an AR session all we have to do is set the environmentTexturingproperty of ARSession configuration:

From this point, ARKit will automatically create AREnvironmentProbeAnchor objects. AREnvironmentProbeAnchor provides environmental lighting information for a specific area of space.

AREnvironmentProbeAnchor is a subclass of ARAnchor, which means that they have the same lifecycle. Its extent property represents the area around the anchor’s position which can be later used when projecting its texture for parallax correction. The texture is represented by environmentTexture property.

Currently, the environment texturing supports two settings:

  • For EnvironmentTexturing.automatic ARKit automatically creates, positions, and adds AREnvironmentProbeAnchor objects to the session.
  • ForEnvironmentTexturing.manual you identify points in the scene for which you want light probe texture maps by creating AREnvironmentProbeAnchor objects and adding them to the session.

If configuration.environmentTexturing is set to .automatic you don’t have to work with AREnvironmentProbeAnchor directly to achieve great results.

That’s it about the environment texturing and reflections in ARKit. You can find the source code of the reflective sphere that was shown earlier here. I hope you learned something new from this small post!

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store