Build a web app to test fine motor coordination with Leap Motion : Part 1 (Programming Tutorial)
Developing the app structure with HTML, CSS, and Javascript
Introduction
Leap Motion controller is a low cost USB infrared sensor that can track position of hands, fingers and joints in 3D space. The device’s tracking reliability has been studied and is used in virtual rehabilitation projects (e.g. Putrino Lab, VirtualRehab). The controller features an application programming interface (API) that allows programmers to interact with data in a more “human” way. For example, rather than having to develop your own algorithm to interpret the meaning of raw infrared sensor voltages, the Leap Motion API can tell you the existence and positions of “arms”, “hands”, fingers”, and “joints” in space. APIs make devices accessible to a wider audience.
In this series of three tutorials, I will show you how to build a web application that records a user’s finger position as it draws a line and Archimedes spiral, then runs basic analysis for accuracy. The spiral test has been validated for use in evaluating essential tremors and Parkinson’s disease, and published algorithms exist for quantifying test performance.
The first two tutorials will record and analyze the path of a mouse cursor. In the final tutorial, we will plug in a Leap Motion Controller and change some code to replace the mouse control with a user’s finger in the air.
Below is an animation of the end goal of the Part 1 tutorial. You can try the Part 1 live demo here.
Prerequisites
This tutorial is designed for beginners. I will explain much of the code in this text and in code comments, though you may want to skim through some other fundamental tutorials on HTML, CSS and JavaScript code. Examples:
- HTML: W3schools.com, Codecademy
- CSS: CSSbasics.com
- Javascript: W3schools.com
A quick Google search of unfamiliar terms or syntax in the code will usually provide an answer. Or, ask questions in the article comments below. The link to completed, working code is available at the end of the tutorial.
Requirements
- Cost: Parts 1 and 2 can be completed with free software. Part 3 uses the same free tools plus the Leap Motion Controller, which is $79.99 in the company store. (disclaimer: I have no affiliation with the company)
- Text editor: I recommend SublimeText 3 or Atom.
- Web browser: this tutorial uses Chrome, but you could also use Safari or Firefox
Step 1: Setup your project folder structure
Create a new folder called leap-tracer somewhere on the hard drive, then create a subfolder named part1. In the next step you will create a file named cursor-line.html within the part1 folder.
Step 2: Create an HTML file
HTML stands for markup language. It is not a programming language but rather a way to create the structure of a document (I.e. where headings, images, sections etc. appear in relation to each other).
Begin by writing the structure of the page. Start your text editor (Sublime or Atom), and open a new blank file if needed. Add the following code:
You have made html, head, and body elements. Notice that for every opening “tag” such as <html> and <body>, there is a closing tag such as </body> and </html>. You can add comments that are ignored by the browser by surrounding them by <!-- and --> like in line 1. Extra lines and tabs do not affect the code, but whitespace makes it easier to read.
Save this file as cursor-line.html in the leap-tracer/part1 folder. Check your work by opening cursor-line.html in a web browser (in Finder or Explorer, right click on the filename and choose Open with > Chrome). You should see a blank white page, because there is no content within the <body>. Keep the file open and return to the text editor.
Add a <title> element within <head> </head>; the text within it shows up in the browser tab but not the actual page. Within the body, add some heading elements like <h1> and <h2> with your text between the tags. Header elements range from h1 to h6, with h1 being the most important (and usually the most visually prominent).
Save this file and refresh the page in Chrome. It should look something like this:
Now, create a <canvas> element within the <body> after the headings. The canvas is an area for drawing that uses code rather than a brush. Also add the attributes width="800" and height="400" to the canvas element. An attribute is a property defined within an element tag. Finally, add an <h3> header with attribute id=”results” for the results output. We can refer to this header by its id later to output the performance report.
You will not see a difference if you refresh the browser, because the <canvas> is blank and the<h3> element is empty. In the next steps, we will add style to the page and create the programming logic.
Step 2. Add a Cascading Style Sheet
A website uses Cascading Style sheets (CSS) to customize the aesthetic presentation of the HTML — e.g. font size, color, text alignment, margins, borders, etc. Below is an example of CSS syntax that applies centered text-alignment and 36px font-size for all <h1> elements found in an HTML file:
h1 {
text-align: center;
font-size: 36px;
/* this is a comment for humans */
}Take a look at CSS Basics for a more thorough introduction. Open a new file in your text editor and add style to the body, headings h1, h2, h3 and the canvas box:
Save the file as style.css in the same folder as cursor-line.html. Now we need to link the HTML file to the stylesheet. Switch over to cursor-line.html and add a <link> tag in the <head> element:
href stands for hypertext reference. Here we tell the browser to import the css file you just wrote called style.css. Save the file and refresh the page in your browser. You should see changes in formatting.
Step 3. Create your program logic in JavaScript
JavaScript is a programming language that allows you to interact with HTML elements. For example, you can perform calculations, check conditions, and change the HTML in real-time. The browser will use a JavaScript file as the brains behind your application. We will use a framework called jQuery that works on top of JavaScript to make interacting with HTML even easier. Similarly, we will use a framework called jCanvas that looks like jQuery and allows us to interact with canvas easily.
Find canvas.js and save to your project
The script file is more complicated than the html and css files, so copy the text in canvas.js available here and paste it into a blank new file in your text editor. Save it as canvas.js in your project’s part1 folder. I have commented throughout the script to explain what is happening, and I will highlight a few parts below.
If you have not done any programming before, you can still continue! I recommend reading one page each about variables, arithmetic, and objects.
Drawing in canvas.js using jCanvas methods
Just as an artist plans the location, color, and thickness of a stroke then paints it on a canvas, the HTML canvas uses code to plan a stroke, and then a second step to draw that stroke. jCanvas simplifies the code needed by doing a lot of the work behind the scenes. Below is example code to draw a circle (do not add these lines to canvas.js). Anything to the right of // is a comment for humans and ignored by Javascript.
See the jCanvas documentation for more examples. In canvas.js, you will find the green and blue circles created by drawArc(), a grey line created by drawLine(), and text added by drawText().
Creating the red path to track the cursor
We want tracking to begin when the cursor hits the green start circle, and a path to be drawn following the cursor to the blue circle, then tracking should stop. Look at Line 100 in canvas.js. I attached a function assigned to the $('canvas')'s mousemove property. To create a line path, pass coordinates that make up the line as properties such as x1, y1, x2, y2 etc. Every time the mouse changes position, a new point is added to the line path. These very small changes in distance are connected by line segments.
The tracking code will only run if isTracking is true, which is set in themouseover function for the green circle (Line 53).
When the cursor reaches the blue circle target, we want to stop tracking by setting and run automated analysis. To do this, we add a function assigned to the mouseover property of the blue circle (See Lines 71–81) to set isTracking = false; and run analyzePerformance().
Basic Analysis
A simple way to analyze the user’s performance is to calculate the average deviation from the path. The gray guide line is parallel to the x-axis which makes the math easy — at every x position between the start and target points, calculate the y distance of the path from the guideline. Take a look at analyzePerformance() starting at Line 153 in canvas.js. Here, we calculate the average deviation (the sum of the distances divided by the number of points recorded). Note that the function takes the absolute value of each deviation since the path may cross above and below the guide line.
Admittedly, average deviation is a crude metric for testing tremor, but this basic calculation works for an introductory project.
Add script tags to the HTML file
The last step is to tell the browser it needs to import the JavaScript files by adding <script> elements to cursor-line.html, just before the closing </body> tag:
Switch to the browser and refresh your page. The canvas should be functional!
The full code from all three tutorials is available to download on Github.
Troubleshooting
If your code is not working as expected, take a look at the browser console to see if there any errors. In Chrome, choose View -> Developer -> Javascript Console. Error messages will tell you the file and line number where the error is occurring. If you are still having trouble, download the tutorial code and compare it to your files. And feel free to ask questions in the comments below.
Conclusion
By completing this tutorial, you will have created a working interactive HTML canvas app that responds to a user’s cursor. In Part 2, we will modify the code to change the stimulus from a straight line to an Archimedes spiral.
