70.2 / E.

Buckley

70.2: Invited Paper: Holographic Laser Projection Technology

Edward Buckley
Light Blue Optics Inc., 4775 Centennial Blvd., Suite 103, Colorado Springs, Colorado 80919, USA

Abstract
Light Blue Optics holographic laser projection technology exploits the physical process of two-dimensional diffraction to form video images. Such a method of image projection and display has long been desired, but was never previously realised, due to computational complexity and the poor quality of the resultant images. LBOs approach represents an entirely new approach to hologram generation and display, matched to human visual perceptive factors, which overcomes both of these problems enabling for the first time a high quality, real-time holographic projector.

Fxy =F [huv ]

(2)

To achieve video-rate holographic display, a dynamicallyaddressable display element is required to display the holograms; LCOS microdisplays are well suited to the task, and are widely available. Principles of LBOs holographic display technology Despite the potential advantages of a holographic display, previous attempts at constructing such a system have foundered on two fundamental technical problems. The first difficulty is that of calculating a hologram huv such that, when the microdisplay showing huv is illuminated by coherent light, a high quality image Fxy is formed. A reasonable first guess might be to compute the inverse Fourier transform of Fxy to obtain the desired hologram huv. However, the result of this calculation would be fully complex and there is no liquid crystal (LC) material in existence which can independently modulate both amplitude Auv and phase uv where huv = Auv exp juv. Even if such a material became available, the result contains amplitude components which would absorb incident light and reduce system efficiency. A much better approach, and the key to LBOs unique efficiency advantage, is to quantise the hologram huv to a set of phase only values uv. Performing this operation on huv whilst maintaining high image quality in Fxy is absolutely non-trivial, and requires computation to mitigate the effects of information lost in the quantisation. Until LBOs inception, there was no hologram generation method in existence that could simultaneously produce images of sufficient quality for video-style images, whilst calculating the holograms quickly enough to allow real-time image display. The key to LBOs technology lies in the way in which image noise, due to hologram quantisation, is made perceptually insignificant. The key to achieving this is the definition of a new measure of image quality, which is typically measured by defining some metric J, which is minimised when comparing the projected image Fxy with respect to a target image Txy within some region . An obvious choice of metric is the mean-squared error (MSE) measure

1.

Background

A holographic display is based on the rather beautiful result that, when a hologram h(u,v) is illuminated by coherent collimated light of wavelength , the complex field F(x,y) formed in the back focal plane of the lens of focal length f due to Fraunhofer diffraction from the pattern h(u,v) is the two-dimensional spatial Fourier transform of the hologram pattern:

F ( x, y ) =

h(u, v) exp{ j 2f (ux + vy )}dudv

(1)

This relationship is illustrated in Figure 1 below.

h(u,v)

F(x,y)

f f

Figure 1 The relationship between hologram h(u,v) and image F(x,y) present at the back focal plane of a lens of focal length f, when illuminated by coherent monochromatic light of wavelength . The hologram h(u,v) can typically be realised as a fixed, etched structure in which case, it is called a diffractive optical element (DOE) or displayed on dynamically addressable liquid crystal on silicon (LCOS) microdisplays. In either case, the principal attraction is the ability to generate arbitrary, complex-valued optical fields from a small, thin device. The behaviour of such systems are well understood, often being employed, for example, in telecommunications applications for laser beam shaping or splitting [1-5] and wavelength selection [6-9]. Over the last thirty years, there have been several attempts [10-18] at adapting such systems for the purposes of image formation. In principle, this is relatively easily achieved. If the continuous hologram pattern is then replaced by a display element with pixel size then the image Fxy formed in the focal plane of the lens is related to the pixellated hologram pattern huv by the discrete Fourier transform F

J = Fxy Txy

(3)

where is a normalising constant chosen to minimise equation (3) which seems intuitively satisfying, since zero MSE implies a perfect reconstruction. Unfortunately, this metric is particularly insensitive for the low MSE values typically encountered in holographicallygenerated images. An effective demonstration of the deficiency of the MSE measure is provided in the following example, in which three images Fxy1, Fxy2 and Fxy3 are generated from a target image Txy. Image Fxy1 is equivalent to Txy except for a small contrast change, Fxy2 contains additive Gaussian noise of variance 2, and Fxy3 exhibits both a change in contrast and additive noise. If the change in contrast is given by c and the mean value of the image pixels is , then the MSE metric for each of the images can be shown to be

[], and is written as

1074 SID 08 DIGEST

ISSN/008-0966X/08/3902-1074-$1.00 2008 SID

70.2 / E. Buckley
J = ( + c ) + 2
2

(4)

The resultant images are shown in Figure 2, together with MSE figures calculated using equation (4). Although Fxy1exhibits the highest perceptual image quality, and Fxy3 the lowest, the MSE metrics in fact indicate the opposite. It is clear from equation (4) and Figure 2 that MSE is in fact dominated by mean image errors caused by the contrast change, rather than the additive Gaussian noise which corresponds with poor perceptual image quality.

A simple, practical method for the creation of the time-averaged percept of equation (5) relies upon the properties of the human visual system. The eye, because it responds to intensity, is a squarelaw detector and due to its composition has a finite response time. Kelly et al. [20] performed a series of experiments using flickering veridical fields to deduce the temporal frequency characteristics of the eye, which resulted in the frequency response curves of Figure 3. Since the rod and cone structures respond slightly differently to flicker, there are disparities between pure luminous and chromatic (red-green) flicker responses - nevertheless, the frequency response of the human eye can be well approximated by a brick wall filter function with a temporal bandwidth of approximately 25 Hz.

Normalised response

(a)

Fxy1:

J = 103

(b)

Fxy2:

J = 81

(c)

Fxy3:

0.1

J = 67

Chromatic flicker Luminous flicker Brick-wall response

Figure 2 The poor correspondence of the MSE metric with image quality. Whilst image Fxy1 exhibits the best image quality and Fxy3 the worst, the MSE metric J indicates the reverse. It has previously been shown [19] that a hologram generation approach optimised for the display of video-style images should instead attempt to minimise noise variance 2 in order to generate images with perceptually pleasing properties. This result immediately suggests a method of noise reduction by temporal averaging. Indeed it is simple to show that the noise variance of the average of N video subframes, where each subframe contains the same image but with different and independent additive noise (of some arbitrary distribution), falls as 1/N. To demonstrate how this applies to a holographic display, we consider a holographic display which generates N video subframes which are the result of some, as yet unspecified, hologram generation algorithm. The intensity of the ith displayed image is
(i I = Fxy ) , and has mean and variance 2 and i = 1,..., N. If the 2

0.01 0.1 1 10 Frequency (Hz) 100

Figure 3

Temporal frequency response of the eye to luminous and chromatic flicker (after Kelly et al [28]). The curves show that the eye can be modelled as a brickwall filter function with temporal bandwidth of approximately 25 Hz.

Using this approximation and accounting for the square-law response, then the time-averaged intensity percept Vxy is approximately equal to the integral of the veridical field |Fxy|2 within a 40 ms time window, and can be expressed as

V xy (t ) =

t 0.04

F ( )d
2 xy

(7)

average of all such subframes is displayed, the time-averaged percept is

V xy =

1 N

F
i =1

(i ) 2 xy

(5)

and, from the central limit theorem, it follows that the variance of this time-averaged field is given by

Var V xy =

[ ]

2
N

If a microdisplay is used to show N subframes within this 40 ms period, then the integral of equation (7) becomes the summation of equation (5). Hence, by displaying N frames quickly enough to exploit the limited temporal bandwidth of the eye, a human subject will perceive an image which is the average of N noisy subframes. What remains is to design a hologram generation algorithm that has the capability to generate N sets of holograms both efficiently and in real time. LBO has developed and patented proprietary algorithms for this purpose which, crucially, can implemented in a custom silicon chip drawing little power. Practical holographic projector realisation Previous sections described the theoretical basis for a holographic projector. A practical realisation is rather simple, and is shown in the schematic of Figure 4. A desired image is converted into sets of holograms by LBOs proprietary algorithms and displayed on a phase modulating microdisplay which is time-sequentially illuminated by red, green and blue laser light of wavelengths r, g and b respectively. The

(6)

which is N times smaller than the variance of each individual subframe F

(i ) 2 xy

. Hence, a reduction in the noise variance of a

video frame can be achieved by displaying the average of N subframes. The display of multiple subframes per video frame is the key to LBOs approach, and the only method of achieving high quality holographic projection demonstrated thus far to the authors knowledge.

SID 08 DIGEST 1075

70.2 / E. Buckley
subsequent diffraction pattern passes through a simple demagnification lens pair L1 and L2, thereby generating the required output image. The resultant instantaneous projected image is a direct consequence of Fourier optics and since the image is created in the far field (or Fraunhofer regime) unlike conventional projectors, in which the image is always created by means of converging rays, the image is in focus at all distances from the lens L2. additional benefit of a very low laser modulation frequency gives a clear power consumption advantage compared to scanned beam systems. Since greyscale is formed by the hologram, and because the phase-modulating microdisplay directs a fixed proportion of light into the image, it is only necessary to modulate the laser sources with respect to average scene brightness. This results in a laser modulation frequency of the order of 1 kHz, compared to tens of MHz for a scanning system. A projector based on LBOs technology can be made very small indeed because the fundamental properties of diffraction allow for a huge degree of miniaturisation. Although imaging projectors can be made small, the resultant image size must either shrink in proportion or be subject to severe aberrations, which can only be reduced through the use of complex and expensive lens systems. By comparison, the design of the LBO optical system is very straightforward, requiring only a simple and inexpensive demagnification lens pair (L1 and L2 in Figure 4). This can be achieved since LBO technology employs diffraction at feature sizes of the order of tens of wavelengths; as a consequence, a very small image is presented to the output optics, enabling the size and complexity of the projection lens to be dramatically reduced. The diffractive effect also becomes larger for smaller feature sizes. By choosing the lenses appropriately, the system is capable of ultrawide projection angles in excess of 100; that is, the projected image is 12 across at only 5 from the projector aperture. This throw angle is unmatched by any other projection technology, and allows for novel use cases such as table-down projection and ultrathin rear-projection displays. In addition, it potentially enables Class 1 laser safety criteria to be met without severely compromising image brightness perhaps the single most important requirement for consumer electronic applications. Unlike conventional projectors, the use of laser sources in conjunction with LBOs phase-modulating holographic technique guarantees that the projected image is in focus everywhere at all distances from the projector. Moreover, there is no requirement for the projection surface to be flat, allowing for some very unusual projection geometries and novel use cases for automotive and consumer electronics applications. LBOs system achieves these benefits using no moving parts; the opto-mechanical assembly of Figure 4 is entirely solid state. Furthermore, due to the Fourier relationship between hologram and image, the microdisplay is tolerant to the entire range of defects found in LCOS processes - from individual dead pixels, through to row, column and area defects. LBOs technology is tolerant of dust, debris and component misalignment and, since the microdisplay directly controls the incident optical wavefront, the system is able to correct for optical aberrations by suitable modification of the hologram. Laser speckle One of the huge advantages of LBOs technology is the ability to substantially reduce laser speckle, a very unpleasant artefact which makes the image sparkle due to interference of coherent light in the retina. Although there have been several demonstrations of speckle reduction [22-25], thus far only LBO has demonstrated the possibility for speckle reduction within the projection optics of a miniature laser projector. Indeed, unlike other laser projection technologies, LBOs speckle reduction method does not require a special projection screen.

L2 L1

Microdisplay

r g
Figure 4 - Part of the optical assembly of a LBO projector, consisting of LCOS microdisplay, lens pair L1 and L2, and laser light sources. LBOs system employs frame-sequential colour, in which sets of holograms huv(i) i = 1,...,N are calculated and displayed for each wavelength r, g and b as shown in the schematic of Figure 5. Since colour planes are displayed and illuminated at the subframe rate, however, the system does not suffer from colour breakup. Properties of holographic laser projection LBOs holographic laser projection technology employs a very different approach to image generation and display and has some unique benefits compared to other display technologies. LBOs proprietary technology enables the manufacture of small, robust, power efficient projectors for digital signage, automotive head-up displays and, ultimately, for embedding into the next generation of mobile phones. LBO technology is highly power efficient, by virtue of the use of a phase-modulating microdisplay which removes the need for a front polariser and associated 50% energy loss. The combination of laser illumination and a phase-modulating hologram provides a highly power efficient method of projection, since no light is blocked in the system. Hence, if in the projected image only one pixel is illuminated, then only the laser power required to illuminate that pixel is used. This has significant implications for the display of video-style images, which are conservatively estimated as having an average pixel value of one-half the maximum [21], or an average intensity of one-quarter the maximum. As a result, a conventional projector utilising 100 mW of optical power (for example) could produce the same brightness video image as a LBO projector using just 25 mW. Conversely, far brighter images could be obtained from a LBO projector for the same electrical power consumption. The

1076 SID 08 DIGEST

70.2 / E. Buckley
(1) huv (i) huv (N) huv

Vxy =

Red channel Blue channel Green channel

1 N

F
i =1

2 (i ) xy

Figure 5

Frame-sequential colour formation in LBOs holographic laser projection technology, in which multiple subframes Fxy(i) = F[huv(i)] are displayed per video frame.

Speckle can be removed by effectively destroying coherence, a process which can be achieved in at least two ways. The first, and perhaps most obvious, is to employ several sources, each of which are incoherent with respect to each other; such an approach has previously been demonstrated [26,27] using multiple laser emitters. The same effect can also be achieved by displaying multiple images, the pixels of which exhibit identical amplitudes but independent phase components. This has the same effect as employing multiple sources, in that the coherence is reduced by phase averaging, but has the advantage of a rather more compact and cost-effective optical system. The proof of this fact follows that outlined by Goodman [28], who showed that a simple measure of speckle contrast C can be obtained by calculating the ratio of the speckle intensity standard deviation s to the average field intensity E[Vxy]:

CN =

2 N s 1 = C N V N

(10)

giving a N1/2 fold reduction in speckle contrast. Evidently, some speckle reduction comes for free with LBOs multiple holograms per video frame approach to image display. Whilst this approach is useful, the N1/2 dependence means that this approach is limited by the law of diminishing returns. Since N cannot be increased indefinitely, further time-varying speckle reduction methods such as moving diffusers, diffractive optical elements [29-31] or ultrasonic modulation [32,33] - can be used to remove speckle, provided that the operation is performed multiple times per laser dwell period. This is straightforward in LBOs system because the laser modulation frequency is low, and all pixels are formed simultaneously. In addition, the intermediate image plane formed between L1 and L2 in Figure 4 makes it simple to embed the speckle reduction mechanism in the projector optics; the high laser modulation frequency and lack of an image plane would certainly make this speckle reduction technique difficult to implement in a scanned-beam system. A demonstration of the efficacy of LBOs combined speckle reduction techniques is shown in Figure 6, in which laser speckle is substantially reduced in the projected image without loss of focal depth or resolution.

C=

E V xy

[ ]

(8)

In his analysis, Goodman considered a veridical field consisting of speckle only; here, the analysis is extended to cover a case more representative of a laser projection system, in which the veridical field is subject to additive laser speckle, but additionally contains an image component of mean pixel value V. In this case, since Goodman has already shown that amplitude speckle statistics are Rayleigh distributed, it is straightforward to derive that the amplitude statistics of the total field are Ricean distributed and, as shown in Appendix A, the speckle contrast ratio then becomes

Conclusion

2 s V

(9)

assuming that the mean of the image samples is much greater than the speckle mean. This measure can also be applied to a holographic display which is capable of generating and displaying multiple holograms per video frame. If N subframes are displayed within the eyes integration period, the eye acts to add N independent speckle patterns on an intensity basis, and the new speckle contrast CN becomes

LBO technology represents a revolutionary approach to the projection and display of information, exploiting the physical process of two-dimensional diffraction to form video images. LBOs patented approach to hologram generation and display overcomes the problems of poor image quality and computational complexity previously associated with holographic displays. This allows, for the first time, the realisation of a high quality, real-time holographic projector. Such an approach provides many compelling approaches compared to competing laser projection systems in terms of speckle, efficiency, eye safety, fault tolerance, robustness and cost.

SID 08 DIGEST 1077

70.2 / E. Buckley

(a)

(b)

(c) (d) Figure 6 Sample projected image (a) without speckle removal (b) using a combination of speckle reduction techniques. The close-ups (c) and (d) clearly show substantial speckle reduction.

Appendix A
Speckle occurs because, under illumination by coherent light, the light reflected from the projection surface consists of the sum of contributions from many scattering areas. Consider a P x 1 vector of independent identically distributed (i.i.d) random variables h1, h2, ..., hn each of which has some arbitrary probability distribution (PDF) of fh (xi1, ..., xn). The central limit theorem states that the sum of these variables will tend to the Normal distribution, regardless of the distribution of the variables themselves, provided that P is large enough. It can therefore be assumed that the real and imaginary parts of the complex laser speckle field Fxy are both independent identically distributed (i.i.d) zero-mean Gaussian random variables. Hence, the statistics of the speckle intensity samples Ixy = |Fxy|2 correspond to the negative exponential distribution, which has a probability density function fI(x) given by

f A (x ) =

x2 exp 2 2 s
x
2 s

(12)

In the presence of an image component of mean value V, however, the amplitude components become Ricean distributed so that the PDF fA(x) is described by

f A (x ) =

x2 + V 2 exp 2 s2 x
2 s

xV I 0 2 s

(13)

where I0(x) is the modified Bessel function of the first kind with order zero. The mean and variance of the intensity samples Ixy = Axy2, E[Axy2] and Var[Axy2] respectively, can be shown to be

x 1 f I (x ) = exp s s

[ ] Var[A ] = 4 (V
2 xy 2 s
2 s 2 xy 2

2 E Axy = V 2 + 2 s2 2

(14)

+ s2

)
2 s V

(15)

(11)

and hence the speckle contrast ratio is given by

C=

2 Var Axy

It is straightforward to show that the amplitude component samples Axy = Ixy are Rayleigh distributed with PDF

[ ] 2 (V + ) = E [A ] V + 2

2 1/ 2 s 2 s

(16)

1078 SID 08 DIGEST

70.2 / E. Buckley 7
1.

References
T. Dresel, M. Beyerlein, and J. Schwider, Design of computer generated beam-shaping holograms by iterative finite-element mesh adaption, Applied Optics, vol. 35, pp. 68656874, 1996. J. Bengtsson, N. Eriksson, and A. Larsson, Small-feature size fan-out kinoform etched in GaAs, Applied Optics, vol. 35, pp. 801, 1996. Y. S. Jean-Numa Gillet, Irregular spot array generator with trapezoidal apertures of varying heights, Optics Communications, vol. 166, 1999. J.-N. Gillet and Y. Sheng, Multiplexed computer-generated holograms with polygonal-aperture layouts optimized by genetic algorithm, Applied Optics, vol. 42, pp. 298307, 2003. P. E. Keller and A. F. Gmitro, Computer-generated holograms for optical neural networks: on-axis versus off-axis geometry, Applied Optics, vol. 32, pp. 1304, 1993. B. Layet, I. G. Cormack, and M. R. Taghizadeh, Stripe color separation with diffractive optics, Applied Optics, vol. 38, pp. 71937201, 1999. B.-Z. Dong, R. Liu, G.-Z. Yang, and B.-Y. Gu, Design of diffractive phase elements that generate monochromatic or color point and ring patterns, J. Opt. Soc. Am. A, vol. 15, pp. 480, 1998. G. Yang, B. Dong, B. Gu, J. Zhuang, and O. K. Ersoy, Gerchberg- Saxton and Yang-Gu algorithms for phase retrieval in a nonunitary transform system: a comparison, Applied Optics, vol. 33, pp. 209, 1994. B.-Z. Dong, G.-Q. Zhang, G.-Z. Yang, B.-Y. Gu, et al., Design and fabrication of a diffractive phase element for wavelength demultiplexing and spatial focusing simultaneously, Applied Optics, vol. 35, pp. 6859 6864, 1996. L. B. Lesem and P. M. Hirsch, The Kinoform: A new wavefront reconstruction device, IBM J. Res. and Dev., vol. 13, pp. 150155, 1969. Y. Takaki and J. Hojo, Computer-generated holograms to produce high-density intensity patterns, Applied Optics, vol. 38, pp. 2189 2195, 1999. K. Heggarty and R. Chevalier, Signal window minimum average error algorithm for computer-generated holograms, J. Opt. Soc. Am. A, vol. 15, pp. 625635, 1998. A. Kirk, K. Powell, and T. Hall, A generalisation of the error diffusion method for binary computer generated hologram design, Optics Communications, vol. 92, 1992. M. Taghizadeh, Best technical diffractive optical element, in Diffractive Beauty Competition, OSA Diffractive Optics and Micro-Optics Conf., Hawaii, 1998. M. Taghizadeh, Best technical diffractive optical element, in Diffractive Beauty Competition, OSA Diffractive Optics and Micro-Optics Conf., Quebec City, 2000. T. Ito and T. Shimobaba, One-unit system for electroholography by use of a special-purpose computational chip with a high-resolution liquid-crystal display toward a three-dimensional television, Optics Express, vol. 12, 2004.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17. T.-C. Poon, B. W. Schilling, M. H. Wu, K. Shinoda, and Y. Suzuki, Real-time two-dimensional holographic imaging by using an electron beam- addressed spatial light modulator, Optics Letters, vol. 18, pp. 63, 1993. 18. F. Mok, J. Diep, H.-K. Liu, and D. Psaltis, Real-time computer generated hologram by means of liquid-crystal television spatial light modulator, Optics Letters, vol. 11, pp. 748, 1986. 19. A. Cable, E. Buckley, N. Lawrence, P. Mash, T. Wilkinson, Real-time binary hologram generation for high-quality video projection, Proc. SID Symposium 53.1 (35), pp. 1431-1433, 2004. 20. D. H. Kelly and D. van Norren, Two-band model of heterochromatic flicker, J. Opt. Soc. Am., vol. 67, no. 8, pp. 10811090, 1977. 21. F. P. M. Budzelaar, C. N. Cordes, J. J. L. Hoppenbrouwers, N. C. van der Vaart, W. H. M. van Beek, F. J. Vossen, A. A. M. Hoevenaars and R. G. H. Boom, Video processing for active-matrix polymer OLED TV, Eurodisplay 2005 Digest, 18.2, pp. 484-487, Edinburgh 22. D. Gabor, Laser speckle and its elimination, IBM J. Res. Dev., pp. 509514, 1970. 23. L. Wang, T. Tschudi, T. Halldorsson, and P. R. Petursson, Speckle reduction in laser projection systems by diffractive optical elements, Applied Optics, vol. 37, pp. 17701775, 1998. 24. J. I. Trisnadi, Speckle contrast reduction in laser projection displays, Tech. Rep., 2003. 25. J. I. Trisnadi, Hadamard speckle contrast reduction, Optics Letters, vol. 29, no. 1, pp. 1113, 2004. 26. A. Mooradian, S. Antikichev, B. Cantos, G. Carey, M. Jansen, S. Hallstein, W. Hitchens, D. Lee, J.-M. Pelaprat, R. Nabiev, G. Niven, A. Shchegrov, A. Umbrasas and J. Watson, High power extended vertical cavity surface emitting diode lasers and arrays and their applications, Micro-Optics Conference 2005, Tokyo. 27. B. Dingel, S. Kawata, and S. Minami, Speckle reduction with virtual incoherent laser illumination using a modified fiber array, Optik (Stuttgart) 94, 132136, 1993. 28. J. W. Goodman, Some fundamental properties of speckle, J. Opt. Soc. Am. A, vol. 66, 1976. 29. S. Lowenthal and D. Joyeux, Speckle removal by a slowly moving diffuser associated with a motionless diffuser, J. Opt. Soc. Am. 61, 847851, 1971. 30. L. L. Wang, T. Tschudi, T. Halldorsson, and P. R. Petursson, Speckle reduction in laser projection systems using diffractive optical elements, Appl. Opt. 37, 11701175, 1998. 31. Trisnadi, J., Speckle contrast reduction in laser projection displays, Silicon Light Machines white paper 32. Y. Imai and Y. Ohtsuka, Laser speckle reduction by ultrasonic modulation, Opt. Commun. 27, 1822, 1978. 33. Wang, L., Tschudi, T., Halldorsson, T., Petursson, P., Speckle reduction in laser projections with ultrasonic waves, Opt. Eng. 39(6) 16591664, 2000

SID 08 DIGEST 1079