On the Pulse
Pen-based Computing in the Laboratory
During the summer of 2001, our nascent department was preparing to welcome its first class of students back to their sophomore year. After a freshman year that saw courses in basic electronics, computing, solution chemistry, biology and systems theory, our matriculating majors were about to launch into a pair of courses that placed them squarely in the scientific laboratory. As part of the process of acquiring laboratory data, we decided to introduce handheld personal digital assistant (PDA) devices into the curriculum.
Our initial classes were small enough to fit within our instrumentation budget, such that we could equip each sophomore with a Compaq “iPAQ” PDA. These devices were fairly new technology in 2001 and their use would achieve multiple goals:
- to reward our new majors for serving a guinea pigs while we tweaked the curriculum
- to allow our majors to display their new toys to roommates and friends in a subtle marketing campaign
- to permit us to evaluate the use of the PDA as an information device within the laboratory
The PDAs came equipped with a color touch screen, microphone, speaker, IR port, RS-232/USB docking/charging station, and some basic applications to manage Microsoft Outlook contacts and calendar, a basic word processor, and a non-scientific calculator. The PDA served well as an information retrieval device for contacts and appointments, but that is about where it stopped. It had three primary modes for entering information into the device, namely, the serial ports, the touchscreen, and the microphone. The serial ports naturally had the highest bandwidth, but required the device to be docked or within line of sight to either a desktop or notebook computer — great while you were at the office, but not in the laboratory. The microphone performed as expected and created digital audio files, but management of files named “Recording1.wav” and “Recording2.wav” were not of much use.
The largest disappointment was the touchscreen. While adequate for retrieving information by pecking at menu buttons with the stylus, its near-zero baud rate was of no use for entering information into the device. The handwriting recognition software was adequate, but it had to be trained, and input was limited to a small input box. It was possible to write anywhere on the screen using a program called the “transcriber,” but the awkwardness of writing on a device that is raised relative to the surface of the laboratory bench made that difficult and uncomfortable.
We purchased PCMCIA accessory sleeves for the PDAs that facilitated the use of standard wireless cards, while subsequent models sprouted integrated WiFi and Bluetooth radios. However, these only enhanced the ability of the PDA to retrieve information, not assist in its collection. The recent arrival of Apple’s iPhone and Google’s G1 smart phones subsumed the functions of the PDA and also included cellular phone and G3 wireless service. Our PDA program had some success with porting LabVIEW data acquisition applications to the hand-held devices, but we did not find a killer application.
In the summer of 2008, we searched for a PDA replacement for use by our new batch of sophomores. The university had no plans to distribute and support iPhones (much to the dismay of the students). However, after some sleuthing, we discovered the logical successor to the PDA — a digital “smartpen,” specifically, the “Pulse” smartpen from Livescribe. This hand-held computer sports a 32-bit ARM 9 processor, 1 or 2 GB of flash memory, and runs applications written in Java. It does have a 96x18 pixel monochrome OLED display, but it is not a touch screen. It instead uses standard paper that has been printed with a screen pattern of 100-µm diameter dots. The specific pattern was developed and patented by Anoto Group AB in Sweden and consists of over 6.7 x 10²⁰ dots that are spaced on a regular grid having a square cell size of 0.3 mm. Each dot is placed near the intersection of the grid lines and is offset in one of four positions: above, below, left or right of the grid vortex. A six by six matrix of these offset dots produces a position-addressable grid with an area exceeding 4.6 million square kilometers, corresponding to about 73 trillion letter-size pages. Instead of poking around with the stylus on a 320 x 240 pixel touch screen, the smartpen can smoothly track its position at a sub-mm resolution on a canvas having the combined size of Europe and Asia.
The truly amazing feature of this computer is that it is an actual pen. After switching out the laboratory notebook for one printed on “dot paper,” the smartpen’s black, blue, or red ink cartridges can be used to write manual laboratory notes as always. Turning the smartpen “on” by pressing its single button, energizes a small infrared camera near the tip of the pen that captures the dot pattern at over 70 Hz and records the position of the tip while it is in contact with the paper. When the smartpen is subsequently docked in its USB charging cradle, the strokes of the smartpen are uploaded to the host computer and displayed as “digital ink” on an electronic copy of the paper. Each page of the notebook contains a unique subsection of the dot pattern such that the smartpen is not only “aware” of what is being written, but on which page of the notebook it was written. This feature in itself is pretty nifty as electronic backup for manual laboratory notes, but that is not the real power of the pen.
When placed in “record mode,” the smartpen energizes its internal microphone and records audio while you write notes or simply dictate into the pen. Experimental color commentary that would be too time consuming to document using text can simply be described orally by the researcher as they write down specific major details of the experiment. After the smartpen is docked, the combined digital ink/audio file is uploaded and the spatial location on the page acts as a temporal index to the audio file. If you would like to review what was dictated while you were writing the sample name, you simply click on that area of the page, and the audio file is replayed from that bookmark. The files also are stored in the memory of the smartpen, so you can review the audio information by tapping the specific location in the laboratory notebook and listening to the audio replay through the smartpen’s speaker or stereo headphones.
So the Pulse is a neat “gadget.” Students can record information in the laboratory and during lecture and replay their notes — something that was nearly impossible with the PDA. The Livescribe desktop software available for both Windows and Mac provides a comprehensive interface to the images, audio and file management on the smartpen and is a vast improvement over the Activesync utility. Students also can upload their notes to the Web and share them via Facebook.
But, the real killer applications for the laboratory are yet to come. Livescribe has released their SDK to developers, and folks can create applications for the smartpen using Java in either Netbeans or Eclipse. One of the hurdles in electronic notebook adoption is the loss of the paper notebook. The smartpen utilizes the paper notebook, but simply records what is being written. While the Livescribe Pulse is sold with consumer applications focused toward information sharing, in-house security-focused applications can be developed, such as signature recognition for password management and encrypted recording.
Forthcoming Bluetooth-equipped smartpens will permit researchers to use their paper notebook as the user interface for laboratory instrumentation. It is technically feasible to write the name of an instrument in the notebook along with the sample name and some analysis parameters and simply write “run.” The wireless smartpen would then beam the information to the instrument and the data would be sent to the user’s file server.
Anoto Group also licenses sections of their dot pattern for dedicated use. Livescribe uses a specific section of the pattern along with pre-printed images to permit the user to adjust smartpen settings, query its status and run other utilities such as a scientific calculator and keyboard, simply by tapping the pen on the printed image. Laboratory informatics managers will be able to create printed forms that prompt the user for sample information and analysis parameters that can be filled in by the user. After the user places a check mark in the “run” box, the analysis would be performed, and the completed form could serve as a record of the analysis in both paper and electronic page. In an era in which laboratories are moving toward paperless record archival, a smartpen can serve as a low-cost alternative to costly and fragile touchscreen notebooks. In fact, the computer-as-a-pen form factor may not be a laptop substitute, but an emerging essential data acquisition peripheral.
This material originally appeared as a Contributed Editorial in Scientific Computing 26:1 January/February 2009, pg. 32.
William L. Weaver is an Associate Professor in the Department of Integrated Science, Business, and Technology at La Salle University in Philadelphia, PA USA. He holds a B.S. Degree with Double Majors in Chemistry and Physics and earned his Ph.D. in Analytical Chemistry with expertise in Ultrafast LASER Spectroscopy. He teaches, writes, and speaks on the application of Systems Thinking to the development of New Products and Innovation.
