Building a Room Scanning App with the RoomPlan API in iOS
Guide to Integrate the RoomPlan API into Your iOS App
What is RoomPlan?
RoomPlan API is the latest addition by Apple powered by ARKit. It allows user to scan their room using a lidar-enabled iPhone or iPad to generate a parametric 3d model of the room and its room-defining objects.
Why use RoomPlan?
Previously Apple released scene reconstruction API, which gives a coarse understanding of the geometric structure of our space but demands a lot of complexities in implementation.
RoomPlan can manage those complexities by implementing machine learning and computer vision algorithms to detect walls, doors, openings, windows, and objects.
Where can it be used?
- Interior design apps: Wall colour changes and accurately calculate the amount of paint required to repaint a room.
- Architecture apps: Allow someone to preview and edit changes to their room’s layout in real-time.
- Real estate apps: Seamlessly enable agents to capture floor plans and 3d models of a listing.
- E-commerce apps: Engage customers through product visualization in their physical spaces.
Prerequisites
- Xcode 12.5 or later: This version of Xcode includes the necessary tools and frameworks for building AR apps and using the RoomPlan API.
- Supported device for Room Scanning: iOS devices containing the LiDAR sensor are supported. For example, iPhone 12 Pro, iPhone 12 Pro Max, iPhone 13 Pro, iPhone 13 pro max, and iPad Pro (2020 or later) are supported.
How to use RoomPlan?
The basic workflow consists of three parts:
- Scan: Set up and start the session, display progress and instruction
- Process: Process scanned data and receive model
- Export: Generate and export the usd/usdz model file
There are two ways of implementing RoomPlan in our application, one by using RoomCaptureSession and another by using RoomCaptureView.
Let’s look at how we can implement it in both ways.
RoomCaptureSession
Step-1 Setup RoomCaptureSession:
Add ARView, instantiate a RoomCaptureSession instance and assign self to the delegate property of this session by extending the current ViewController with RoomCaptureSessionDelegate. We will need to create our custom visualizer which updates the user on the progress of the scanned room.
Step-2 Implement necessary delegate methods of RoomCaptureSession:
- didUpdate: Called every time when any surface or object is detected or updated while scanning
- didProvide: Provides instructions regarding the scanning process which includes warnings regarding distance, speed, light adjustment, and low texture
Other than these methods we also have didAdd, didChange, didRemove, didStartWith, and didEndWith methods.
This was all about the scan part, onto the process and export part, we will use RoomBuilder class to process the scan data and generate the final 3d models.
Step-3 Capture and export the final model:
The didEndWith method of RoomCaptureSessionDelegate is called after the session has been stopped and provides captured room data or error if it fails to capture. We will use this method to process and export our model
The CapturedRoom obtained here contains both the surfaces and objects which have some unique and some common properties.
Unique properties of a surface:
- Curve - radius, startAngle, and endAngle
- Edge - left, right, top, and bottom
- Category - wall, opening, window, and door
Unique properties of the object:
- Category - table, chair, sofa, bed, and a few more
Common properties between a surface and an object:
- Dimension - height, width and length of the surface/object
- Confidence - three levels of confidence for the scanned surface/object
- Transform - the 3d transformation matrix
- Identifier - a unique identifier
- export(to: URL) - exports the generated 3d model to provided url
This export function allows us to export this CapturedRoom into a usd/usdz file(3D file format that displays 3D and AR content on iOS devices without having to download special apps) and share or save the model to files.
RoomCaptureView
RoomCaptureView is a UIView subclass that we can easily place in our app. It handles the presentation of world space, scanning feedback, real-time room model generation, coaching and user guidance.
It provides:
- Animated lines which outline detected surfaces and objects in real-time
- Interactive 3d model generation at the bottom of the view which provides an overview of our scanning progress
- Coaching instructions to guide us to the best possible scanning results
Let’s see how we can implement RoomCaptureView in our application,
Step-1 Setup RoomCaptureView:
Create a RoomCaptureView reference in our ViewController and add it to the view hierarchy.
Step-2 Start RoomCaptureSession:
Create a RoomCaptureSession.Configuration instance which is used to start the session. We can enable/disable coaching using this variable.
Step-3 Capture and export the final model:
Implement methods of RoomCaptureViewDelegate to optionally opt out of the scanning process and export the processed model. Assign self to the delegate property of RoomCaptureView after extending ViewController by RoomCaptureViewDelegate.
- shouldPresent - returns a boolean value that indicates whether the adopter wishes to post-process and present the scan results.
- didPresent - provides the delegate with the post-processed scan results once the view presents them.
Here is the output screenshot of a room scanned using RoomPlan API. It detected 6 chairs, a table, a television, a door, a window, and surrounding walls.
You might be wondering how cool it would be if we could place real objects instead of static blocks. Actually, it is possible to do so, we already have the CapturedRoom data structure with us, and we can use this data to design our own scene with custom objects.
To create a model with custom objects, we need to create our own scene view and add nodes of the surfaces/objects in it.
Let’s see how we can achieve this custom objects 3d model.
- Add predefined 3d models of all types of objects i.e. table, chair, sofa, and others to the bundle of the project. For surfaces, we can use the default SCNBox as a node.
- Just as we receive the CapturedRoom data structure after scanning. For surfaces, we can create a SCNBox instance having width and height defined from the surface’s dimension property and we can set the length(thickness) according to our need.
- For objects, first of all, fetch the object from the bundle based on the category of the object. Create a scene, clone the node, and apply the transform. Finally, add those nodes to the scene view.
- At last, for exporting, we can use the write(to: URL) method of scene view to export our model at the required url and later on, we can share this model with others.
This is how it looks with custom predefined objects
Additional enhancements:
- We can store the nodes added to the scene view and let the user position and scale the node runtime after scanning and before exporting it for a better and more accurate model.
- Provide multiple options of the objects for the user to choose between. For example, we can have 3 different chairs - office chair, dining chair, and normal chair and let the user swap the chair model after scanning and before exporting to make it more sensible.
- We can create a camera node and add it to the scene view for navigating inside the room.
Limitations
- Problem with multiple rooms scanning: It can cause inaccurate data and device overheating on longer scans.
- Cannot detect odd surface shapes: It fails to detect curved edges of surfaces as it assumes all shapes to be either square or rectangular.
- Sometimes detects phantom objects: RoomPlan can sometimes see something that isn’t actually there.
- Limited number of objects detected: Currently RoomPlan is only capable of detecting 16 types of household objects.
Conclusion
In this article, we learned how to implement RoomPlan API in our application in two different ways. We also learned how to get an attractive custom 3d model out of the static 3d model generated by RoomPlan. I hope this article was appropriate and provided detailed information on how to implement RoomPlan with a proper example.
Thank you for reading this guide on implementing the RoomPlan API in iOS. I hope this post has provided you with a comprehensive understanding of the RoomPlan API and how you can use it to create amazing AR room-scanning apps for iOS. Here is the sample video of navigation inside a 3d model generated using RoomPlan.
Thank you so much for investing your valuable time in reading this article:
Here’s what you can do next:
Give 👏 claps on the article and follow the author for more insightful stories.
Read 📚 more valuable content pieces in Simform Engineering Publication.
