How can you use the “fourth camera” of iPhone 12

Sajjad Hussain
Nov 19 · 5 min read
Image for post
Image for post

This is not the first time Apple has used a lidar scanner in its own products. The iPad Pro was equipped with this new technology six months ago.

Aside from various professional terms, in layman’s terms, the lidar scanner on the iPhone 12 Pro series and iPad Pro is a device that can quickly detect the distance of an object and record depth information.

If you use an iPhone or iPad that is not equipped with a lidar scanner, you will be reminded to “move the device to start” or “return to the previous area to continue” when entering an AR app such as “Rangefinder”.

However, if you use the iPad Pro or iPhone 12 Pro series models equipped with a lidar scanner, you do not need to move the device to find the plane, just enter the app to quickly identify the plane, generate objects or measure distances.

In addition to assisting in shooting and measuring height, what use is it for? Today, i recommends several AR apps that can be applied to lidar scanners to see what new experiences it can bring us.

1. 3D Scanner APP: Scan your room

If you want to introduce the room layout, what form will you adopt?

Is it a verbal description or a drawing to illustrate? Why not try to use the 3D Scanner APP to scan the generated model and show it directly.

The operation of the 3D Scanner APP is very simple. Open the app, click the shooting button on the right, and move the iPad to scan walls and objects. Thanks to the existence of the lidar scanner, the entire scanning modeling process is very smooth and relatively accurate.

Although the resolution is not very high, compared with other 3D scanning modeling apps in the past, it is a very big improvement. After the scan is over, you can drag, zoom, and recolor the scanned object, and edit the measurement.

You can also click to share, save the 3D model as “OBJ” and other common model formats, click on the “Sketchfab” at the bottom, after registering and logging in, you can upload the model to the official website.

In the website, you can still re-render the model and click to share the exclusive address of the generated model. Even if your friend does not install the 3D Scanner APP and does not have a registered account, you can also view it directly in the browser through the generated link The 3D model you scanned.

3D Scanner APP can do more than just scan the room layout, you can also use it to scan items on the table, outdoor construction vehicles, and you can also use AR to change the location of the building and the vehicle.

2. Jig Space: Use AR to learn the structure

When we were in school, we learned about cell classification and mechanical structure through presentations made by teachers, which were flat, boring and non-interactive. Today, AR that could only be experienced in science exhibitions can finally be realized through iPad.

Open Jig Space, and you can see the “Jig Library” with clear classification and rich content. In this library, you can select the content of interest to view.

Because of the presence of the lidar scanner, Jig Space can be positioned on the plane very quickly, and objects can be placed on the plane with one click.

Through operations such as zooming and dragging, we can view the appearance structure and every detail of three-dimensional objects from multiple angles. At the same time, there is a text description of the corresponding interface below the picture.

Click the arrow on the right side of the screen to enter the analysis interface, and click on the corresponding area to display the name of the structure, allowing us to have a more comprehensive and in-depth understanding of the AR items displayed.

In addition to the very rich structure display and teaching, there are also very interesting introductions to scientific principles in Jig Space, from the structure of traditional locks to the electronic movement of batteries. If you need to explain this kind of knowledge to children, then Jig Space is a very Convenient and practical teaching tools.

3. Reality Composer: Build your own AR scene

Unlike the previous apps, the Reality Composer officially developed by Apple is not an AR app for scanning or teaching aids. With it, you can build an AR scene from scratch.

Open the app, and you will see a recent project interface as when the iWork suite is opened. Click the “+” in the upper right corner to select the anchored new scene of the AR scene. The editing interface of Reality Composer is similar to iWork. The left side is a scene area with square modules, and the right side is the control area of ​​the model.

As a demonstration, let’s make an AR scene of “a ball hitting a square mountain”. First click on the “+” at the top, add a ball, and select “Steel” in the “Material” on the right, so that we have an iron ball. In the same way, we can also create other objects.

Click the behavior tool in the upper right corner, add the trigger method and trigger result of the sphere, and you can make it move. Here, we can set the speed and direction of the sphere’s movement, and set it to “click and roll forward”.

However, when you click to play, the sphere will pass directly through the block without impact. This is because we have not added a movement type to the square. After setting the movement type of the block to “dynamic”, you can see the animation of the block being knocked apart by the sphere.

Next, we copy the square to form a square mountain. Remember, when copying, we must remember to click the “…” in the upper right corner and select “Copy with behavior”. In this way, the “round ball hits the square mountain” AR scene we created has completed the most critical step.

Click the play button in the upper right corner to see how it works.

After creating the scene and animation effect, click the “AR” button at the top to display the newly created AR effect in the real scene.

During the entire production process, we did not program or specifically learn some editing methods, and completed the creation of an AR scene completely through a visual interface. Reality Composer also has a very rich model, from shelf cabinets to battery notebooks, from axe brooms to house furniture. If you understand this app in depth, you can create a lot of rich and interesting AR scenes.

Although AR is not new, because of the addition of lidar scanners, AR has higher recognition accuracy and faster speed. Whether it is using AR to try shoes or travel from room to outer space, the new lidar scanner allows AR apps have changed from “usable” to “easy to use”.

The potential of the lidar scanner does not stop there. Maybe in the not-too-distant future, we can use it to obtain accurate three-dimensional data of the human body, and achieve more perfect fitting and makeup; you can also use the photos and videos obtained Information, quickly implement 3D modeling, making the production of games and animations easier.

Even we can create a self exactly like the real world in the virtual world…Perhaps, the lidar scanner can open up a new application system, and this day is not far away.

God Hardware

For hardware discussion

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store