Tapdance your way to a new kitchen!

Refurbishing your bathroom or kitchen is an exciting (yet daunting) prospect. The thought of finally getting that elegant granite island worktop you’ve always dreamed of is really thrilling, until you realise you’re not even sure whether an island is going to fit in your octagonal shaped kitchen. That’s the moment when the reality hits you: it’s time to measure the room.

You dust off your measuring tape, call your partner or friend to hold the other end and start measuring. Sharpening a pencil, you note down the figures and roughly draw the floor plan on a piece of paper. Then, a few days later, you have the tiresome task of deciphering your handwriting when it comes to picking the right cabinets. The whole process is boring and time consuming, and certainly not the way you imagined your exciting new project starting.

Fig. 1: ARKit logo

However, it’s not all doom and gloom as we have technology on our side! We’ve already talked about the differences between ARKit and ARCore in a previous blog post. Today, we’re focusing on how Augmented Reality can make our lives easier in the process of refurbishing our kitchen or bathroom. Specifically, we’ll look at how three apps, already available on the Apple Store, allow you to measure the floor plan of your room simply by tapping on your iOS device.

Let’s get technical: what does ARKit do?

ARKit’s core is a very powerful technique called visual-inertial odometry. From the ARKit official documentation:

“This process combines information from the iOS device’s motion sensing hardware with computer vision analysis of the scene visible to the device’s camera. ARKit recognizes notable features in the scene image, tracks differences in the positions of those features across video frames, and compares that information with motion sensing data. The result is a high-precision model of the device’s position and motion.”

Another interesting feature of ARKit is detecting horizontal planes (the new version even supports detection of vertical planes!). That is necessary for augmented reality applications, where most likely we’ll want to place objects on the floor or other horizontal surfaces. What ARKit is doing in practice, is detecting interesting feature points in the image and tracking them across different images. It’s then possible to triangulate them into 3D points and try to fit horizontal planes. If a plane has a reasonable amount of unique interesting feature points, ARKit is going to detect it.

Fig. 2: IKEA app. Once ARKit detects the floor, it is possible to virtually place objects there.

The plane detection feature, combined with the possibility of knowing where the device is at any time (60 times every second, to be specific), makes it easy to develop an app that will measure your room for you, with just a little help from your side.

Comparing floor plan estimation apps

There are already a few apps out there that implement the idea of measuring your floor plan using ARKit. Below, we compare three: TapMeasure, PLNAR and MagicPlan.

The user journey is the same for all three apps:

• Point the device towards the floor

Fig. 3: Point the device towards the floor. Up left: TapMeasure; Up right: PLNAR; Bottom: MagicPlan.

• Get close to the first corner of the room and tap on it, then record all the other corners until the entire floor plan has been covered.

Fig. 4: Tap on the corners of the room. Up left: TapMeasure; Up right: PLNAR; Bottom: MagicPlan.

• TapMeasure offers a couple more features: it allows you to swipe the plane of the floor up to the ceiling and measure the height of the room, and allows you to tap on detected rectangles (which most likely will be windows and doors) and get their dimensions.

Fig. 5: TapMeasure allows you to estimate the height of the ceiling, detect and measure rectangular regions.

When you’re done, you’ll have a very nice looking floor plan ready on your device to be shown to your trusted designer.

Fig. 6: Floor plan. Up left: TapMeasure; Up right: PLNAR; Bottom: MagicPlan.

Let’s put these apps to test!

How accurate is the final floor plan estimated this way? To find out, we measured our 4m x 3.85m meeting room five times with each app. We took approximately 1 minute and 45 seconds for each scan. We report here the average error for each scan measured as the average of the absolute differences between the real dimension of each wall and the estimated one:

where w(i) is the real length of the i-th wall and l(i) is the estimated length of the i-th wall. Besides, we report the standard deviation on the error computed as:

We also report the maximum error for each scan and, for TapMeasure, the error in the height estimation:

Table 1: Average and maximum error and standard deviation reported for each app, for each capture.

The table shows that the error is comparable among the apps. We observed that the more carefully we tapped the corners, the more accurate the floor plan was. However, it looks like ARKit has still some small issues with localisation. As you can see in Fig. 8 and Fig. 9, once the user has moved around the room for a while, the corners are displayed in the wrong position. These same issues sometimes lead TapMeasure to get a very wrong height of the room (in capture 5, there was an error in height of 59 cm).

Fig. 7: After moving around, the localisation is not really accurate anymore.

Overall, the average error is definitely acceptable, and even though a human can certainly do better, it takes at least 10 minutes for two people to measure the same room manually.

Conclusion

In this blog we’ve compared three different apps for iOS devices that promise to estimate your floor plan quickly and accurately. In light of our experiments, we can say that these apps do a good enough job for design purposes, but are not accurate enough to build to.

In summary, the advantages of using ARKit-based apps over measuring the room manually are:

• It takes less time than measuring the room manually.
• It’s accurate enough to give you a good sense of what’s possible.
• You only need one person and an iPhone (or an iPad).
• You’ll have the floor plan immediately saved on your phone, without the need to draw and decipher later.
• TapMeasure also allows you to measure the height of the ceiling as well as rectangular objects attached to the wall such as windows and doors.

However, there are still a few challenges to overcome:

• If a corner is hidden by a fridge, for example, the user can only try to guess its actual position and inevitably get a 3D point that’s wrong. In this regard, it is possible to improve the estimation of the corner using the vertical plane detection offered by the newer version of ARKit, even though such detection only works when there are visual features on the wall.
• The accuracy depends a lot on the precision of the user in tapping the corners. An optimised UI could probably help there, even though there is always going to be a small error due to the fact that a corner point on the image is tiny! Some computer vision tricks could help as well to refine the corner position on the image.
• The performance of ARKit depends a lot on light conditions and feature detection. For example, if the floor has a uniform color and no features can be detected, all these apps will just fail.
• Localisation in ARKit is not perfect yet, which means you may see the corner of the room in the wrong position and, therefore, will accumulate greater errors in subsequent measurements.