Creating the perfect AR projector
Augmented reality glasses promise to revolutionize how we interact with data as it projects virtual objects into our world. The miniature projectors integrated into these glasses have so far failed to deliver the performance required for everyday use. Here VitreaLab presents the first “perfect” AR projector that excels on all necessary metrics to create breakthrough AR glasses. The future of AR is bright!
Augmented reality glasses are tasked to perform something conceptually simple: overlaying an image over the real world within a glasses form factor. Achieving this deceivingly simple goal though is extremely challenging because it should be achieved with high image quality, long battery lifetime, tiny size, and most crucially, high brightness, which is essential for outdoor use.
Optimizing any one of these factors impacts all the others (and many more), meaning that to date no AR glasses that are truly compelling have ever reached the market. At VitreaLab a new technology has been developed that enables for the first time the combination of standard LCoS projector technology with ultra-bright laser light within the tiny size required for AR glasses. It relies heavily on a new type of optical integrated photonics chip and represents a breakthrough in laser optics and despeckling technology.
Before we get started on VitreaLab’s technology, we will review some useful optics background.
AR Combiners and Pupil Expansion
To create an image in AR glasses two components are necessary, (a) a projector that creates the image and (b) a combiner that takes the image from the projector and combines/overlays it with the real world.
While there are many ways of creating a combiner, in fact, a simple piece of window glass could be used as one, the leading technological approaches share three important features:
- They are light and compact,
- Made from a single piece of glass,
- Implement eye box expansion via pupil replication.
The advantage of the first two features is self-explanatory as in only a few scenarios it is appealing to the user to wear a heavy, helmet-like, device on the head.
Eyebox expansion is a sophisticated optical feature that requires a more detailed description. The eye box describes the area around the eye onto which the image is projected and typically has a box-like shape. If it is very small, simply rotating the eye would move it out of the eye box and the image will no longer be seen. Therefore at least a few millimeters of eye box size are necessary (e.g. 10x10mm). Furthermore, eyeboxes are necessary to create a large field of view, which otherwise would be limited by vignetting. Larger eye boxes also allow viewers with different pupil distances to use the same AR glasses.
A large eye box is achieved using pupil expansion, where the optical combiner replicates the projected image multiple times. Pupil expansion is therefore an important tool to achieve both the form factor and performance required from AR glasses. The main disadvantage of combiners with pupil expansion is that they are very inefficient: the replication of the image is equivalent to a projector that has to project onto 100 screens. This means that most of the light inserted into the combiner does not reach the viewer. In order to compensate for the combiner's inefficiency, extremely bright projectors are necessary in order to deliver enough light to the viewers’ eye.
AR Projectors
AR projectors work very similarly to everyday projectors used at home or at work. However, they need to be a lot smaller and more efficient, which creates many challenges. The following technological approaches are being developed or are already in use in AR glasses:
- Liquid Crystal on Silicon (LCoS) with LED illumination
- Laser Beam Scanning (LBS)
- MicroLED
LCoS with LED LCoS is a reflective display technology using liquid crystals to manipulate light polarization and generate an image. Since it is reflective, light has to be bounced off it to generate the actual image, meaning that a front light is necessary. This is typically done via red, green, and blue LEDs working time-sequentially. LCoS with LED illumination is a mature and frequently used technology (e.g. Hololens 1, MagicLeap 1 and 2, Avegant) showing excellent image quality and frame rate. The technology has downsides in terms of size and brightness. As the power of LEDs is limited for a given size, and the size is ruled by the etendue of the optical system, simply increasing the brightness with more or larger LEDs is not possible. Another significant disadvantage is that all pixels have to be illuminated at all times, even if just 10% of them are actually used.
Laser Beam Scanning Laser Beam Scanning (LBS) projectors use collimated red, green, and blue laser beams that are bounced off a fast-moving mirror. Similarly to cathode-ray tube (CRT) displays, they build up an image line by line and rely on the eye to integrate over time the lines into a full image. The advantages of LBS projectors are their small size, lightweight, and that only pixels that are ON are actually consuming energy. However, these devices are affected by poor resolution, low frame rate, and overall bad image quality. Brightness can be significantly higher than LCoS with LED, and the efficiency depends on the image level (i.e. how many pixels are actually on), which can lie above and below LCoS with LED depending on the image content.
MicroLED MicroLED is an upcoming technology that uses miniature LEDs as pixels. Its main advantage is very high brightness while retaining a compact form factor. The technology is still very immature, in particular for the tiny pixel sizes required for AR. The (sub-)pixel size has to be even smaller than for LCoS, as LCoS uses color sequential technology which is not possible for microLED (apart from Porotech’s technology), meaning that pixels have to be 3x smaller. Additionally, since LEDs emit light over a wide angle (180°), a lens that is much bigger than the sub-pixel is necessary to direct the light toward the exit pupil, further shrinking the possible size of the microLED resulting in lower brightness and efficiency. Expected wall-plug efficiencies of the microLEDs are on the low single percent level and a collection efficiency into the combiner of more than 30% seems daunting. MicroLED remains a promising solution to the brightness problem for AR, even though most likely at a lower energy efficiency than competing solutions that are already on the market. To date, it remains unclear when the first microLED display suitable for AR that features full color, as well as competitive efficiency and brightness, will be presented.
VitreaLab’s Breakthrough
The takeaway from the discussion above is that to date no perfect projector solution for AR exists as no technology is able to deliver:
- Small size
- High brightness
- High efficiency
- High resolution
- Strong colors
- Low price
At VitreaLab we set out to overcome these challenges. We started by realizing that LCoS can match all of the above’s requirements and that it is only the currently used LED illumination technology that produces all of the downsides in terms of size, brightness and energy efficiency. Luckily, at VitreaLab we have a new photonic tool in our hands that helps us to overcome this illumination challenge: the Quantum Light Chip.
Quantum Light Chip The Quantum Light Chip is a small piece of glass that emits a very dense array of laser beams from its surface. The light itself is provided by red, green, and blue laser diodes attached to the glass facet.
The main difference between a Quantum Light Chip and an LED is the collimation of the output light, which in optical terms means that it has a much lower etendue. This matters a lot for projectors, as the light from all of the pixels has to be collected by a lens (see below). When using an LED, light is emitted over a 180° angle and a lens has to be placed on top of it to collimate the beams (make them “parallel”) and guide them to the projector lens. These lenses can only collect part of the light, leading to reduced efficiency and unwanted stray light that reduces image quality.
The same problem does not exist for laser illumination, as laser diodes, and with it, the Quantum Light Chip emits light over a small angle spectrum. All light emitted from the laser diodes can therefore be collected by the projector lens, leading to large improvements in efficiency!
Local Illumination Technique A problem of many LCoS projectors, independent of their illumination source, is their low wall-plug efficiency in scenarios where just a few pixels are active. When showing a black image, the LCoS is still uniformly illuminated with white light, making it a lot less efficient than a self-emissive technology like microLED. Depending on the image content, the number of pixels that are actually “ON” can vary significantly, but it is reasonable to assume that for AR the number is fairly small (e.g. 10–30%). A way to overcome this issue is to use a smart illumination technique that adapts to the image content. If we illuminate only the part of the LCoS that is showing an image a lot of energy can be saved.
This is easily possible with the Quantum Light Chip: the illumination can be segmented into different zones which allows for Local Illumination Technique, the local dimming for AR glasses. As can be seen below this technique leads to a large reduction in energy consumption.
Comparing MicroLED and VitreaLab’s Quantum Light Engine
MicroLED was for a long time seen as the ideal display technology for AR due to its self-emissive nature and high brightness. In practice, these conceptual advantages have failed to turn into successful products due to the challenge of assembling millions of 1–2µm size LEDs and microlenses onto a single substrate. High-current backplane issues, pixel brightness non-uniformities, and manufacturing price are common issues hindering adoption.
In the meantime, LCoS-based projectors have seen massive improvements based on their mature technology. LED LCoS projectors have been reduced to small sizes previously thought impossible, thanks to ever-shrinking pixel sizes and the removal of the polarizing beam splitter. Now, with the introduction of laser illumination by VitreaLab, microLED can be outperformed in virtually any metric, and even more importantly, in a timeframe of months and not years. See here for a recent microLED review.
In 2023 VitreaLab will show AR projectors that have a higher resolution, higher efficiency, smaller size, and much better image quality than any microLED projector, or any other AR projector for that matter, has shown to date.
Bonus: Speckle - The Nemesis of Laser Displays
As is clear from the discussion above, combining LCoS with laser illumination solves a lot of problems, and naturally, multiple intents have been made to make it work. The unsolvable problem all of them encountered is laser speckle, i.e. image artifacts created by the coherence of the laser illumination.
Laser speckles can easily be seen by anyone at home by shining a laser pointer at the wall: the rough wall reflection will introduce random scattering on the coherent laser beam. This creates a grainy reflection pattern which obviously reduces the image quality of a display system.
A similar effect can be observed when an LCoS is illuminated by a laser beam. An effective method for reducing this effect is to quickly move around the laser beam though this will not reduce this visible speckle enough to be comfortable to the eye.
Only when replacing the illumination source from a single laser beam to hundreds of beams, as is possible with the Quantum Light Chip, does the speckle vanish and the image looks pleasant to the eye. Our tests demonstrate that with a single laser beam speckle contrast remains >30%, which is unsuitable for display applications, while when using VitreaLab’s approach it can be reduced to <5%, which is generally accepted as no longer visible to the eye.
