From Force Feedback Manipulanda to Surface Haptics: Part III

(In which we unveil the future of haptic interface)

As a quick refresher, my last post in this series introduced the notion of “surface haptics” — programmable haptic effects that you can feel on physical surfaces such as touchscreens. I went on to explain my preferred approach to implementing surface haptics by controlling the friction between the fingertips and the surface. That friction has two aspects: the amount of friction, and direction in which it acts. The surface haptic technology that Tanvas is now commercializing controls the amount of friction, but not yet the direction. Which brings us to this post.

An ongoing project in the Northwestern University lab that Michael Peshkin and I run is “active forcing”, in which we vary both the amount and the direction of friction acting on the fingertip. To appreciate the significance of this, do a simple experiment: run your fingertip along the sharp edge of a tabletop (a laminate desktop works well) with your eyes closed. It’s pretty easy, right? In this experiment, there simply must be some forces acting on the finger, and mostly those forces act perpendicular to the edge. If there were no forces, your finger would wander off in its own direction. Don’t believe me? Now try holding your finger exactly one inch from the edge and following parallel to it, again with your eyes closed. You might be good, but you sure aren’t perfect. You need those forces to keep you on the straight and narrow!

But here’s the more important lesson from that experiment: the forces act mainly perpendicular to the edge, yet you move mainly parallel to the edge. Friction, sadly, cannot do this: friction forces must always act exactly opposite your direction of motion, never perpendicular to it. So, we’re stuck. We can’t use friction control to simulate an edge (or anything else where the forces aren’t opposite the direction of motion).

Ah, but that’s only if the motion comes entirely from your finger. What if the surface is moving too? For instance, imagine putting your finger down on a moving conveyor belt. Your finger gets pulled along by friction. That’s because friction depends on the relative motion of the finger and surface. So, the way to accomplish active forcing is to move the surface.

Of course, conveyor belts are not one of the more sought-after features for smartphone touchscreens, so we had to think of a more, ah, elegant approach. What we settled on was the idea of vibrating the screen side-to-side and then synchronizing the friction to those vibrations. Suppose, for instance, that we wanted to push the finger to the left. In that case, we’d vibrate the screen left-right and increase the friction every time it was moving leftward, decreasing as it moved back rightward. A finger placed on the screen is, on average, pushed leftward.

This really works! The trick is to use really small and fast vibrations so that you don’t see, feel or hear them, and at the same time increase/decrease friction enough that there is a sizable average force pushing the finger. A recent doctoral student, Joe Mullenbach (who is now at Tanvas), demonstrated some amazingly large forces as the accompanying video shows.

The eShiver is a surface haptic device capable of active forcing. In this video, there are two types of frequencies: there is the pushing frequency which is set to 0.5, 10 or 20 Hz. That’s the frequency at which the finger is pushed back and forth. And there is the “operating frequency”, which is 1000 Hz. That’s the frequency at which the surface vibrates side-to-side. Ideally, it would be such a high frequency that you would neither feel nor hear it. 1000 Hz is high enough that you don’t feel it, but sadly not high enough that you don’t hear it. In fact, there is a distinct hum due to this frequency. Ongoing work aims at pushing the operating frequency to greater than 20,000 Hz so that it is inaudible.

There is a boatload of work that needs to be done before this type of active pushing is ready for commercialization, but I find it an incredibly exciting prospect. In addition to the things we can already do with variable friction, there will be many new tricks in the bag. Like tracing your finger along virtual tabletops! Actually, I’m sort of serious. All day long you use your fingertips to locate surfaces and edges, and you do it so seamlessly that it is rarely worth a second thought. The keys on a keyboard, the pages of a book, the buttons of a shirt, a pocket, the free end of a roll of tape, the top of a Ziploc bag, and so on.

So that’s what you have to look forward to, a few years down the road: virtual Ziploc bags on your smartphone. And if you don’t find that exciting, don’t worry: active forcing technology will let the UI designer look to both the physical world and her own vast imagination for inspiration. The possibilities are endless and seriously, seriously awesome.