The Make of 3D Seat Map: Engineering Episode
People think that using 3D in Application is too far-fetched and hard to start learning. But using 3D in iOS is not as hard as many people think it is. Since iOS 8, Apple has built a strong framework called SceneKit into iOS as a way to make it easier for the apps we write to work with 3D graphics.
People who are already familiar with SceneKit may see it as a framework for making games that only work on the iOS platform. This engine is different from Unity and Unreal Engines. Unity and Unreal Engines work on multiple platforms, which makes it worth the investment to use them to make games.
But 3D graphics can also be used with commercial products, which is not something that is done very often. So, this project is a study of how to use the SceneKit Framework to book seats on the Shinkansen, a high-speed train in Japan.
This article continues from the other article (3D Doesn’t Always Mean Games or AR: 3D Seat Map), with the main focus on Engineering. The readers can choose to read each article on its own or read them all in order.
Overview
From the point of view of iOS engineers, many may not feel ready to start learning about 3D because they think it’s hard enough to combine a 2D UIView with animation. In reality, though, the only real difference between 3D and 2D is that 3D has an extra Z-axis. Anyway, if we know how 3D geometry and materials work and how some of SceneKit works, we can already do a lot.
The highlight of this Shinkansen 3D ticket reservation prototype is the feature of ticket reservation from the view created from SceneKit (SCNView), which functions as a canvas to accommodate drawing. The picture mentioned above is a render picture of chairs and models that are the components. For example, the structure of train cars, the walls, the floor, etc.
However, to draw objects onto the Canvas (SCNView), the app needs to have the 3D model data. In this case, those 3D models are created in Autodesk 3DsMax. Using the 3D Model data file in the application via SceneKit in this Prototype Application is efficient and flexible. The author uses various techniques and processes to develop. It includes the method of loading model files from the server that allows the application not to require those 3D models into the bundle. There are also data management techniques, placing objects according to different positions through the backend, and the 3D Modeling Technique can respond when selected as a UIButton button.
Loading 3D Model from the Server via NodeFactory
The 3D model is the key to the 3D Graphic display. Without the 3D model, nothing can be displayed. The 3D model file has a variety of formats, for example, obj, dae, glTF, an OpenFormat or USDZ, SCN, which is Apple’s signature file. The 3D Model file is similar to the image file; that is, those files are saved separately from the source code and are used to display visual effects for symbolic and aesthetic purposes.
The use of 3D models can be utilized from various sources. For example, calls from the bundle application, internet browsing via URL, as well as image files. This application applies a method of loading from the internet to increase flexibility, which allows developers to edit 3D Model files directly through the server without having to modify the application. This includes storing 3D Models in the application results in large size of the application. Meanwhile, the author also allows the system to support using 3D models in local files as well.
The author has created an object named NodeFactory to be a tool to load 3D model from the URL to be stored as prototype before being cloned in the next order, which is the origin of the word factory, as if this was a factory for producing a 3D model that had been created as a different SCN Node.
Designing Steps & Functions for NodeFactory
The main functions of the NodeFactory are divided into two main functions, loading the 3D model, to be used in the Prototype Operation, and in creating a new 3D Model for displaying results from the new prototype that has been downloaded.
The Process of Loading 3D Model & Applying as Prototype Model
Receiving input through the initialization of NodeFactory. The argument received can support 2 types of variables as follows;
- An array of ModelData, which struct ModelData consists of the reference name and the URL of the 3D model file. Then sends the data that has been downloaded to Step 2.
- URL: If it is a URL, it is downloaded and parsed as an array of ModelData. Then the information is sent to download at Step 2 as well
Downloading 3D Model from Acquired ModelData
- When downloading every model, it is stored in the form of key-value [String: SCNNode] named PrototypeNodes.
- The key is taken from the name variable in struct ModelData. The value is the SCNNode derived from the download from the resource variable in the struct ModelData to be a prototype for further cloning.
Callback Notification
- After the work has been completed, there is a notification that the prototype has been successfully saved via Callback, to inform the user that the object is ready to call the create (name :) command, to create a node from the prototype.
Steps to Take the Prototype nodes to Create a New Node
After NodeFactory has finished loading all files, the onComplete closure Callback is called. In this step, the user can create a new node from the Object NodeFactory via function create (name :), by which the property name that is used to create each node is the name variable that is in ModelData.
Operation Process of the Create Function
- The primary function is to clone and copy SCNNode from the Node Prototype that was previously loaded at the Initialize Object NodeFactory. This function clones the SCNNode and the child of each SCNNode, including Geometry (Recursive function).
- When the clone is completed, the final step is to return the node that is finished as designed, by returning the SCNNode.
SeatMapSceneView
SeatMapSceneView is the dominant view in the display of 3D contents, which is the display of models, chairs, and railways.
SeatMapSceneView is responsible for placing the node precisely according to the position according to the information received. Besides, it also works to send the seat number information through delegates, when the address is selected by the user. Therefore SeatMapSceneView is the main page of User Flow in this project.
The Set-Up
SeatMapSceneView is a subclass of SCNView. The unique property in SCNView that is different from other UIView is the scene that contains SceneGraph. The scene in this project consists of two primary nodes, including contentNode and cameraNode.
- contentNode is an empty node (SCNNode) that is ready for the 3D model (node) be packed and displayed in the next order scene.
- cameraNode is a sub-class of SCNNode that contains the SCNCamera by rendering the image from the camera location that has been created. The cameraNode used in this project is designed to support the Parallax Effect (UIKit Dynamic Visual Effect) when the device has the same tilt as the Parallax Effect of the background in iOS springboard. The unique feature of this cameraNode is that the camera’s Pivot Point position is not at the center of the camera but in the position of the object being displayed. (Please refer to the sample picture for better understanding.)
Below is an example of utilizing two primary nodes in the SeatMapSceneView.
The prototype, Sample Box Placement, & Selection Capabilities
The primary function of SeatMap is the three-dimensional seat map display that allows users to select seats through 3D graphics displayed in UIKit. The creation of the interaction UI can be used in subclasses such as UIButton or UIControl to respond to touch or select objects. However, as SceneKit does not contain ready-made functions that support the responsive interaction of users, the author has to create those special functions manually.
As previously mentioned, the author experimented by starting using the box image. Because at the time, 3D model of the chair was not completed, and the user’s responsive interaction was tested, by allowing those boxes to be touched and changed colors according to the various states of tapping.
Box Placement
For this project, the author wants to specify that the position of the train chair is not stored in the codebase (HardCoded), which means that the location of the chair in the form of JSON can be loaded from the internet and can be edited later, without needing to edit the codebase. The author, therefore, designed the structure named TransformedModelEntity, which the structure holds the value, namely ModelEntity, Position, and Rotation of the node.
Remark: Model Entity is a structure that specifies the name and URL used for loading the 3D Model through NodeFactory described in the topic.
With TransformedModelEntity conforming to the protocol Codable, we can convert the return values from JSON. Therefore, JSON is designed as created above.
After creating the codable struct and creating the data in JSON, the next experiment is to create that data and place the node in the correct position using the subclass of SCNNode in the example code below. The code creates a 0.5 size cube box according to the number, and place it in the specified position via the JSON data.
The Change of State When the Box Touched
As mentioned above, the author wants to have interaction. The nine selections have a behavior similar to UIButton or UIControl, with the unique main component of UIControl is various states. For example, highlighted, selected, and enabled. The author, therefore, adds variables. isHighlighted into the class. When this value is modified, this value is passed through the didSet. The setupTheme [1] function is called to change color. This function changes the diffuse color to normal yellow and red when touching [2].
At this point, BoxTesterNode supports state isHighlighted. However, SCNNode does not have touch support functions such as UIView or UIControl. The author must check to detect the user’s touch screen manually using the hitTest function. This function receives CGPoint of the location that the user touches the screen (X, Y). This function returns hitTestResult, which contains the node at the point that receives the CGPoint (X, Y) value, which is the point that the user touches on the screen.
The code above is a code that is designed to test the state of the chair. The time is tapped and released. The chair that is touching is changed. isHighlighted to true [1]. However, if the finger is released, it changes the value of isHighlighted to false [2] instead, which makes the box that we touch has changed the color as we want.
InteractiveNode
InteractiveNode is a class that was sub-classed from SCNNode by changing the variable factors to be able to collect the state value similarly to UIControl. There are 4 collectible states, namely, Normal, Highlighted, Selected, and Disabled, which are controlled by 3 variables, including, isEnabled, isHighlighted, and isSelected.
The Change of Material upon the Change of State
Changing the state in the example of BoxTesterNode is only changing the color of diffuse that has been pre-defined via the code. In actual usage, the author has designed each node to be changed according to each state. In doing so, the author found that the material in the SCNNode cannot be set to hide/show. However, SCNNode can store the material in the form of an array, which can contain unlimited material.
Based on such limitations, the author, therefore, designed a 3D model that contains various state materials by using the suffix name of the material as an indicator of the material that is used in various states. In that display, the author wrote a command to extract the material with the name suffix that does not match the current state, leaving only the material with the suffix name to be used in the display.
The image below is an example of setting the suffix of the material name that corresponds to the state.
Technically, each node has all the material stored in the variable of that node. When changing the state node, it automatically selects each state of the material.
Transformation upon Changes of State
In addition to the material being changed according to each state, the author has designed a transformation of position, rotation, and the scale of each node must also change according to the state that has changed. For example, the angle of the chair’s backrest is adjusted to lean down when the chair is touched.
This interaction can be done by assigning value to the Hardcode Node that changes when the state transformation of that node is changed. However, the author sees that those transformed value may not be a constant that can be used in every model, which is inconvenient if every time engineering changes, the code must be modified. As a result, the author designed each node in the 3D model contain empty nodes as child nodes that store position values and transformative rotations. This empty node is allowed to be defined by the 3D artist, who is the model creator. For example, in this case, if you want the wall to lean down the ligament when touching the 3D Artist chair, you must create an empty node that has a degree of leaning on the desired position and then assigning those nodes to the child nodes of the backer. Those empty nodes must be named according to different states, as well as the name used as a suffix in the name of the material.
The working principle in the application is similar to changing the material according to the state. When ReservableNode was called to change the State Function updateTransform is called within the function that modifies the transform of each node in the ReservableNode — reading the value from an empty node named after the State name contained in the ChildNode of each node and using the Transform value of that node.
Next Steps
Improvement of Cloning Material
Because the NodeFactory’s cloning mechanism does not clone material, materials in the scene are unique and not reused. I feel that the material in the same model should be reused, which is thought to help save memory and time when initializing StaticNode classes.
Framework Development
I think that the above-mentioned classes, especially NodeFactory, StaticNode, and InteractiveNode, can be reused in other projects in the future. That means these classes could be split up and refactored into a framework. If so, it might be easier to use in different ways, and I think it might also help people who are just starting to learn about SceneKit.
Conclusion
From the author’s point of view, the logic and architecture of the 3D-Seat Booking Program made with SceneKit aren’t very different from other kinds of apps. Only a few 3D techniques make it different. So, you need to know the basic rules of 3D, like how geometry works, how to change transforms, and how different materials can be used in different ways.
With a willingness to learn how SceneKit works, we find out that making apps with a 3D interface is not that hard.
