There and Back Again
The Many Flavors of Image Stabilization
The first permanent photograph was created around 1826 by French scientist Joseph Nicéphore Niépce, and is currently housed in the Harry Ransom Center at the University of Texas at Austin. Coined heliography (Greek for “sun writing”), Niépce made special notice of the bleaching and stiffening effects sunlight has on organic polymers — much like the discoloration and cracking experienced by plush toys and plastic figurines placed on the rear deck of an automobile. He ultimately used a thin film of petroleum tar coated on a pewter plate as a recording medium. After exposing the film to several hours of sunlight shining through a pinhole camera, Niépce developed the film by washing away the unhardened tar with paint thinner to reveal a recorded image of the view from his home near Chalon-sur-Saône in Burgundy. Although the basic process is very similar to that of modern photography, the long exposure times required by heliography made it impractical for subjects other than still life.
Over the succeeding decades, exposure times have decreased dramatically with advances in film technology — even today’s disposable flash cameras have an average exposure time of one one-hundredth of a second. However, the optimal length of exposure is determined by a combination of film sensitivity, camera design, lens configuration, lighting and subject distance. In a portrait studio, the camera is often mounted on a sturdy tripod and the subjects are asked to hold their pose in order to minimize the amount of motion blur introduced into the photograph. Digital photography replaces the photochemical film with a matrix of photosensitive electronic circuits known as a charge coupled device (CCD). After exposure, the digital image is recorded by reading the voltage values of each CCD picture element (pixel) and then the solid-state CCD is reset for the next image. In addition to obviating the film development step, digital camera design permits the elimination of the camera tripod support system as well. Digital camera manufacturers are offering image stabilization systems that fix the spatial position of the CCD even though the handheld camera body may be moving or shaking. For example, Konica Minolta incorporates an angle speed sensor into cameras equipped with its anti-shake (AS) system. While the shutter is opened, the sensor measures changes in camera position and moves the CCD relative to the camera by actuating the internal smooth impact drive mechanism (SIDM).
Video (motion) cameras can benefit from image stabilization technology as well. Although each frame of a video sequence may not suffer from motion blur, stationary observers (such as those seated in a movie theater) may become disoriented if the recording camera is shaking or unsteady. Developed by Garret Brown in 1973, the Steadycam system utilizes a collection of gimbals and springs to isolate the inertial frame of the camera from that of the camera support. The system works equally well with film and digital video cameras, but is impractical for use with consumer video products, including digital video cameras, video phones and PDAs. The miniaturization of personal electronic devices also makes Konica Minolta AS technology cumbersome as it masks the solid-state nature of the CCD.
Enabled by emerging digital signal processors (DSPs), digital image stabilization (DIS) provides motion compensation without physical movement of any camera component. The basic DIS process involves capturing an image using the camera’s large CCD and only recording a subsection of the digital image. For example, a 3000 x 2000 pixel image is recorded from a 3016 x 2016 pixel CCD, resulting in a frame of eight unused pixels around the image. If the video camera is moved slightly before the next image is acquired (due to shaking), the original 3000 x 2000 scene is still captured, but is now located at a different position within the large CCD. The DSP compares the locations of objects in the new image with their locations in the previous one and determines if changes are the result of camera shake using its internal algorithms. The new image is stabilized by selecting a different 3000 x 2000 CCD pixel region centered on the position of the previous subsection. In this manner, the virtual 3000 x 2000 CCD is moved electronically.
Recently, Daniel A. Tazartes, Director of the Navigation and Applied Sensors Technology Center (NASTC) at Northrop Grumman, and his team, have taken image stabilization technology to the next level of complexity. While seated theater goers are stationary, helicopter pilots are decidedly not. A stabilized video camera on the ground or mounted to the helicopter can be very disorienting when displayed on a vibrating monitor and viewed by a vibrating pilot. NASTC has placed accelerometers on the camera, the display, and the pilot’s helmet and these measurements and the video stream are fed into a processing unit. The processed video image is translated along the horizontal and vertical directions of the monitor to induce vibration and jitter. Viewed by a stationary observer, the display looks erratic, but appears stabilized to the pilot as the movement is synchronized to the relative motion of the vibrating display and the helmet-mounted accelerometer. New applications are arriving at a fast pace, so it may take some time for this technology to stabilize.
This material originally appeared as a Contributed Editorial in Scientific Computing 23:9 August 2006, pg. 14.
William L. Weaver is an Associate Professor in the Department of Integrated Science, Business, and Technology at La Salle University in Philadelphia, PA USA. He holds a B.S. Degree with Double Majors in Chemistry and Physics and earned his Ph.D. in Analytical Chemistry with expertise in Ultrafast LASER Spectroscopy. He teaches, writes, and speaks on the application of Systems Thinking to the development of New Products and Innovation.
