Smart Porters: A Haptic Learning Environment

Team name: Fancy Carp

Names: Lu Zhao, Geliang Tian, Zhuoluo Ma

Contact: zhaol@bit.edu.cn

Learning Description

The learning system, Smart Porters is developed to teach the basic physical mechanics including the buoyancy force, the wind force and the friction to the students. The correlative phenomena of the three selected knowledge points are common in our daily life, but usually we cannot learn what the physical meanings and influence factors of the different physical forces are, and how these forces influence us. Understanding and grasping the theories and properties of these physics knowledge will facilitate people’s life and work. By designing Smart Porters, learners can study the physics principles in the theory and experience the forces with this system.

Learn the knowledge of buoyancy with “Smart Porters”

In “Smart Porters”, three parts are designed to present the buoyancy, the wind force and the friction between two objects respectively. In each part, with simulating forces added to the haptic device (Haply) of this system, learners could feel the force feedback when they operate to move a box in the screen to the other side by using the handle of the Haply. When the box changes its position in the screen, the value and the direction of the force will also changes.

The scenes of the three sections of Smart Porters

In the buoyancy force part, students should learn the basic principles and formulas of the buoyancy. Lifting the box from the water or pulling it into the water will make the gravity or buoyancy generated, and learners could get feedback of the corresponding forces by operating the system. The buoyancy force increases with the submerged depth of the box. Only find the places that the gravity of the box and buoyancy are equal, the learner can get the minimum force feedback that the box can be moved in the horizontal direction without much effort.

In the wind force part of the learning environment, the knowledge of the cause of the wind and the wind force scale are introduced to students, and the computing method of the wind load with wind velocity should also be learnt. In the virtual wind zone, we set the wind velocity changes in the linear relative to the display screen coordinates. The student should observe and feel the changed force to find a proper area that the box can overcome the wind resistance and be moved easily.

For the friction education part, materials with different friction coefficient and the normal force of the object which is effected by the angles of the incline that the object is placed on are specified as the parameters for students to calculate the friction force. If a more rough and steeper incline is selected, the system will generate a distinctly stronger friction force to users.

To create an enjoyable learning experience, we design a game scenario to make the learning system get the entertainment function. The learning environment is named “Smart Porters” because students should act as porters and carry the box from one place to another place. All the works students ought to do with “Smart Porters” are to learn the mechanics of wind force, buoyancy force and friction force and then figure out the path of least resistance to carry the box. Combing education and fun can deepen the students’ impression of the knowledge and arouse studying initiative, which is considered as one of the emphasis of the whole system design.

System

Smart Porters is a real-time interactive system used for students to learn the basic physics mechanics. It needs to realize functions of display and producing feedback force, thus, we constitute the system using Haply as the force feedback device, and a Microsoft Surface Pro4 as the display and process device. For Haply is a 2 degree-of-freedom force-feedback device, it is suitable for installing the display on Haply so that student participants can get a more direct-viewing feeling when they interact with the system. MS SP4 is powerful enough to satisfy the requirement of processing. More importantly, it is slim that can be easily placed between Haply. The main interactive method is touch control and use Haply.

The mounted system of Smart Porters

For the reason of the limited workspace of Haply, the area of haptic interaction on the display screen is downsized, which ensure the image frame keep up with the force feedback. When the handle of Haply captures the box in the image and is shifted on the screen, the corresponding forces on the object will be calculated according to the current screen position of the box; also, the haptic will be rendered and presented to student participants.

The interface of Smart Porters

To explain the knowledge of the physics mechanics to learners more clearly, principles and formulas are specifically shown under the haptic interaction area. The formulas can be checked during or before the haptic operation. The law of buoyancy force that is equal to the weight of displaced fluid first defined by Archimedes is introduced. As for the knowledge of wind force, a generic computing method of wind force that is associated with the parameters of the projected area of the object, wind speed, air density and the drag coefficient is explained. How to analyze and calculate the friction force with friction coefficient of different materials and angle of inclinations is also stated in the third part of Smart Porters. By providing values of the parameters, students can calculate the current wind force on the object and feel the corresponding force feedback at the same time.

To make the learning fun, apart from making different terrains in the scene, we design an ice carving Kylin put in the moved box, and a timer is added in the system, so students ought to complete the transit task quickly to avoid the melt of the ice carving when they master the learning content. In addition, the different magnitude of the forces can not only reflect in the force feedback generated by Haply, but also make the character’s energy in the game expend at different speeds. Burnout of the energy will make the character rest for 5 seconds, which may result in failure to finish the task on time.

In Smart Porters, three education sections are designed as three levels of the game. A Guide section is set before the three education sections to lead learners to know how to complete the levels. Some announcement and tips are provided on the system interface, as well as a Minimap is pasted to prompt current position of the character.

The interface of Smart Porters is developed with Unity, and the mainly scripts are finished with C# language.

Hardware

The whole learning environment consist of a Haply and a SP4. In order to attach the Haply on the tablet computer, we changed the height of the motor stand to make it become 8.5mm longer, that is also the same as the thick of SP4. The new motor stand is produced by a 3D printer.

The new printed motor stand

We make the SP4 fixed on the table with 4 two-side suckers attached to the device and the table.

The fixed SP4 and two-side suckers we assembled

Software

The software of Smart Porters is developed by Unity. The interface of the software is achieved by using uGUI, which is a graphical user interface system of Unity. This system allows us to create user interfaces conveniently and intuitively. The picture materials for UI are drawn with Adobe Photoshop CC.

Based on the provided hAPI, all the command scripts of the Haply are rewritten with C#, for which is one of editing language of Unity.

Our code is available at https://github.com/Luellaisme/Smart-Porters.

Reflection

To ensure real-time feedback, the communication script for the Unity scene should be refreshed at a high rate of about 1 kHz. However, the frame rate of the scene constructed by Unity will decrease with the increasing complexity of the scene. Low frame rate will lead to a lack of experience in the game, and associated variables of the objects in scenes will also be affected. It results in anomalies of the force feedback. Thus, a proper refresh rate of the Fixedupdate function in Unity script should be found to guarantee the normal force feedback. We set the refresh rate of the communication as 500Hz by using the Fixedupdate function, and other rendering of the scene is still achieved with the Update function. The lower refresh rate provided by the Update function ensures smoother pictures when the system runs.

When communication between Haply and PC is tested, we found that force feedback instructions can be sent by PC, and position information of the handle can be transmitted from Haply to PC. To make it easy to capture the box in scene and ensure the object in scene can be moved consistent with the handle of Haply, we figure out a method that set a certain range around the object, so that the object is easy to be find although the handle of Haply is not at the precise location of the object. However, the tolerance area results in inconsistent movement of the box when operate the handle. In order to reduce the distance between the moved handle and the object of each frame, it is necessary to find a proper range value. The value is adjusted based on experience.

The Haply cannot know whether user’s operation causes the movement of the handle, so we define a force buffer area in our system. The area is placed around the box in scene, and the size should be determined by upper limit value of the force feedback. Once the handle is moved out of the area, then the box in scene will follow the handle. In other words, only when the handle locates between the box-captured area and force buffer area, the box can be moved. If the handle enters the force buffer area, the box will stop moving, and the feedback force will be attenuated based on the distance between the handle and the box. This work benefit to obtain more stable force feedback even the Haply changes the force suddenly; in addition, this method can also avoid autonomous moving even if Haply loses user’s control, which is conducive to presenting the simulation of the three physics forces.

During the test, we found Haply does not work properly occasionally, and the communication between Haply and PC is unstable; in addition, the force feedback is not uniform. The above problems can be solved by reconnecting Haply or powering it off. The connecting port of Haply development board is apt to get loose, so it should be avoided repeatedly plugging.

The development time spent on Smart Porters is approximately one month, including the create of the game scene and realize the haptic interaction function. Because we are not familiar with the function of Haply at first, the learning content we selected are not be demonstrated perfectly. According to the developed system, we find that the force feedback with Haply performs better in simulating buoyancy and wind force. Next, we will continue exploring more diversified teaching content on buoyancy and wind force.