Build a web app to test fine motor coordination with Leap Motion : Part 3 (Programming Tutorial)
Modifying the code to enable drawing by a hand in the air
Introduction
In this final tutorial of the series, we will use the Leap Motion Controller to track finger motion in the air as it traces our Archimedes spiral.
Refer to Part 1 and Part 2 for prequisite steps.
Step 1. Install Leap Motion Controller
Download the Leap Motion software to be able to obtain data from the controller. You will need to connect the device to your computer during this process. The App Home should launch automatically where you can explore the “Playground” and download other Leap-enabled apps.
Note there is an option in the Settings (on Mac, found within the Leap icon in the top menu bar; in Windows found in the Leap system tray icon) to automatically adjust the tracking space based on where the hands are moving. I recommend un-checking “Automatic Interaction Height” for now so that the Leap coordinate space stays consistent with the html canvas coordinate space.
Step 2. Import the Leap Motion library
- Duplicate the part2 folder in your project folder and rename it part3, then rename the file in the new folder from cursor-spiral.html to leap-spiral.html.
- Import the Leap Motion JavaScript library into leap-spiral.html (Line 18 below) like you did with jQuery and jCanvas, and update the
<h1>heading (Line 9). The Leap library must appear above spiral.js, analysis.js, and canvas.js.
- I highly recommend browsing through the official API Overview here. The rest of Leap’s JavaScript documentation is here.
Step 3. Delete the code handling mouse events
In Part 2, we used mousemove, mouseover, and mouseout methods to trigger actions to draw the path. Now we will take away responsibility from the mouse.
- In part3/canvas.js, look for the
drawArccode to draw thestartCirclenear Line 36. Delete themouseoverandmouseoutmethods, and also the comma at the end of the line above'name: 'startCircle', becausenameis now the last property of that object.
2. Look for the $('canvas').drawArc method that draws the targetCircle and also delete its mouseover and mouseoutmethods, and the comma at the end of name: 'targetCircle',.
3. Scroll down to find the method$(‘canvas’).mousemove() and delete the whole thing (21 lines of code).
(I intentionally did not include the code for items #2 and #3, but you can check your work here if you need some help).
Step 4. Add finger position and diagnostic layers
Let’s add two layers to the user interface that will provide feedback about what Leap Motion is doing. Drop in the following code after the resetText layer and before function addUserPathLayer().
The first method draws a text box in the top left of the window, containing the default text “Leap” that will change to coordinates when the system finds a finger (controlled by code in the next step).
The second method draws a small purple circle called leapCursor that will be invisible by default, then become visible and follow a finger’s position while the finger is detected (controlled by code in the next step). Providing some type of visual feedback of finger position is essential for a good user interface.
Step 5. Modify canvas.js to listen to Leap Motion
By importing the Leap Motion library to leap-spiral.html (done in Step 2), your application now has access to all the controller’s real time tracking data, in literally one line of code. Add the following after the complete resetPath(){} function in canvas.js:
Leap.loop({}, function(frame) { /* custom code goes here */ });Leap.loop takes two parameters: an object {} in which initialization options can be set, and a “callback” function function(frame){ }that will be run every time a new frame of data comes in. The controller’s data sampling frame rate varies but is approximately 90 frames per second. You can test this by choosing Visualizer… in Leap Motion’s menu (found in the menu bar on Mac or the system tray on Windows), and pressing h to see the live frame rates in the top left corner.
Complete the code inside the Leap.loop() callback method
The logical flow to the code loop is similar to Part 2's:
- convert the coordinate in space to a coordinate on the html
canvasand move the fingertip cursor there. - if the fingertip cursor has collided with
startCircle, start path tracking. - if the fingertip cursor has collided with
targetCircle, stop path tracking and analyze the path. - if the user is currently path tracking and the position has moved, add that point to the path points array.
- update the
leapxytext box to print the canvas coordinates
Type or paste the code below into your canvas.js.
Add a collision detection method
The last coding task is to build collisionTest, a function called in the Leap.loop. Collision detection is a common requirement of games to monitor when two objects have come in contact with each other. Our function will take two round canvas layer objects as parameters, and test if the distance between their centers is less than the sum of their radii. In that case, the objects have collided.
Add thecollisionTest function just below the full Leap.loop in canvas.js. The code calculates the differences between the objects’ x and y positions, then finds the magnitude of the distance between the two points using the Pythagorean theorem.
The last line in the function, return vectorMagnitude < sumOfRadii;, is a simplified syntax and works like so: vectorMagnitude < sumOfRadii is a boolean comparison, meaning JavaScript checks if the statement is accurate and if so, returns true, otherwise, returns false. The statement simplifies to return true or return false.
return vectorMagnitude < sumOfRadii;// is the same as:
if(vectorMagnitude < sumOfRadii){
return true;
}else{
return false;
}
Now save the scripts and refresh the page. Hopefully there are no errors in the JavaScript console. Move your finger over the tracking area and a purple finger should appear. Functionality should be the same as Part 2, except it will be a little harder to trace the line.
As before, you can always download the finished applications here.
Conclusion
You have now created an interactive clinical web application! We touched on syntax for HTML, CSS, JavaScript, jQuery, jCanvas and the Leap Motion controller.
Now that your application is working, experiment with changing code such as the sizes of circles, number of loops of the spiral, colors, etc. Please leave questions and comments below!
Where to go from here?
We just scratched the surface of the tracking capabilities available with Leap Motion Controller. You can also monitor forearm, hand, joint and finger position in 3-dimensional space relatively easily. Here are some other clinically relevant applications you could create:
- Finger-to-on-screen-target: Track a user’s finger path to a target in 3D space and compare it to a straight line path (i.e. the magnitude of the 3D vector) using a linear regression, like what was covered in Part 1 and 2.
- Interactive goniometer: Measure joint angles and use this data to control parameters on screen. While the controller is not as precise as a physical goniometer, the tracking will suffice if your application just needs to know if a joint is relatively open or relatively closed.
- Occupational Therapy Tasks: Virtual interactive tasks can make rehabilitation more fun and engaging.
- Virtual Reality with Headsets: Leap Motion Orion is a software library designed for interacting with a virtual environment displayed in a headset. While developing a 3D environment is certainly more complex than making this tutorial’s app, the fundamental process is similar: the developer writes code to creates objects and methods to control the 3D scene. Unity3D is a free and powerful 3D engine with plenty of great tutorials for getting started. Mirror therapy is one relevant rehabilitation use.
