The design and the analysis of an off-axis (50°) diffractive imaging optical system is presented in this paper. A 10°x15° field of view is considered. The optical system is composed of two diffractive optical elements. A static diffractive optical element having a frozen phase transfer function is used to perform a virtual point in the considered field of view. A dynamic diffractive optical element having an adapted calculated phase transfer function is used to compensate for aberrations of the static element. Using a sequential creation of virtual image points and considering human eye characteristics, it is shown that a nine points virtual image can be obtained with current technology. Moreover, it is presented that aberrations can be compensated whatever the position of the virtual point in the 10°x15° field of view. Finally, using rigorous coupled wave analysis, it is shown that an average diffraction efficiency of 79% can be reached across the considered field of view with a standard deviation of nearly 5%.
©2011 Optical Society of America
Augmented reality (AR) supplements the real world with computer-generated objects that appear to coexist in the same space as the real world. AR technology has been used in fields as varied as medicine, robotics, manufacturing, machine repair, aircraft simulations, personal information systems, entertainment, gaming and education . AR can be potentially applied to all senses, but only sight is considered in this paper. An AR scene can be visualized by using projection displays , handheld displays  or head-worn displays (HWDs) [4,5].
The present paper deals with HWD technology. An AR scene can be created with an HWD by using a see-through approach (video/optical)  or a projection approach . In the video see-through approach the virtual environment is replaced by a video feed of reality; the AR is integrated into the digitized images. In the optical see-through approach, AR information is directly superimposed with a combiner onto the real-world perception. In the projection approach, an AR image is projected onto real objects and retro-reflected back to the eye of the user.
The present paper deals with optical see-through HWDs. They generally have off-axis configurations and therefore suffer for high geometrical aberrations as Gauss conditions are not respected. The compensation for their aberrations is thus fundamental to obtain high quality images. Several designs applying aspheric, diffractive, holographic or free-form surfaces have been proposed to improve compactness and imaging quality of see-through HWDs. For instance, Cakmacki  realized an off-axis magnifier using a free-form mirror as a combiner and a diffractive optical element in relay optics. Nagahara  developed a wide field of view catadioptrical HWD achieving a horizontal view angle of 180° to cover the human peripheral field of vision. Zheng  designed and realized an off-axis see-through HWD combining a tilted reflective combiner with an x-y polynomial surface in relay lenses to compensate for aberrations. Cheng  used a free-form prism to design a see-through HWD with a low f-number and a wide field of view. Ando  developed a stereoscopic HWD using an holographic optical element (HOE) as a combiner. Kasai  proposed a practical method for the design and the fabrication of HOEs as off-axis imaging systems. Amitai  proposed a HWD having an holographic combiner recorded with complex wavefronts to compensate for aberrations. In all these approaches, an image created by a microdisplay is relayed by lenses and superposed on the real environment by the combiner. The optical system is optimized to simultaneously minimize the aberrations for the entire image. However, with such a simultaneous approach, aberration compensation is not in general uniform and/or very close to the diffraction limit over the whole field of view.
In this paper, the design of an optical see-through HWD associating a static diffractive optical element (DOE) and a dynamic DOE (Fig. 1 ) is proposed. These DOEs are used to construct point-by-point a nine points virtual image in a sequential way over a 10°x15° field of view. The virtual image is displayed at a finite distance. The sequential construction of the virtual image has the advantage to perform an optimized correction of aberrations for each and every point of the nine points image by adapting the phase function of the dynamic DOE.
The proposed optical system and the principles of the sequential creation of the virtual image are introduced in section 2. A discussion is done in section 3 to evaluate the feasibility of the proposed setup considering current technology. A method to determine and calculate the optimized PTF of the dynamic DOE is developed in section 4; the resulting aberrations compensation is also presented. The performances of the system in term of diffraction efficiency are discussed in section 5.
2. Description of the optical system and image creation
2.1 General description
An incident monochromatic plane wave ΦC1 of wavelength λ0 is diffracted, at the first order of diffraction, and modulated by a dynamic DOE labeled 1 in order to generate a complex wave ΦD1 (Fig. 2 ). This complex wave acts as a reading wave ΦC2 for a static DOE labeled 2. The direction of the optical axis of the system is modified at the surface of DOE 2. Its normal is at an angle α1 to the optical axis before its break and at an angle α2 after its break. The reading wave ΦC2 is backward-diffracted, at the first order of diffraction, by the DOE 2 into a diverging wave ΦD2 until the eye of the user. The user will thus see, through DOE 2, a virtual point I superposed into the real environment. According to the description, DOE 2 is an imaging element. It is a reflective analogical element having a frozen phase transfer function (PTF) ΦPTF2. This PTF has a simple form to ease the fabrication. The position and the aberrations of the image point I, especially coma and astigmatism [15–17], depend on the reading wave ΦC2. DOE 1 is a dynamic computer-generated diffractive optical element. It is used to generate at the first order of diffraction a complex wavefront that compensates for the aberrations. Its PTF is dynamic. It is optimized for each position of the image point I to minimize aberrations. The method of calculation is developed in section 4.
2.2 Specifications of the virtual image
When the reading wave ΦC2 is modified, the form and the position of I in the virtual space are modified. It is thus possible to construct an image sequentially “point by point” by controlling the reading wave. This sequential approach presents advantages as it allows an adapted correction of aberrations for each and every point of the image scene rather than a global and simultaneous optimization for the entire image. The properties of the virtual image are defined from the physiological characteristics of the eye. The sequential creation of the virtual image uses the afterimage effect which is around 50ms. As a consequence, all the points constituting the virtual image have to be displayed in a shorter time. According to the chosen technology for the dynamic DOE, this specification limits the number of points of the image (see section 3). The smallest distance separating two side-by-side points of the image can be defined thanks to the resolving power of the eye which is around 1 arc minute. At a display distance D of one meter, the separation distance is approximately 0.3mm. As a consequence, considering a 10°x15° field of view, nearly 600x900 points can be potentially displayed in a 180mm x 270mm area. It is therefore possible to display a virtual image with a limited number of points at different positions over the 10°x15° field of view. The studied case is limited to a virtual image with nine points.
3. Technological discussion
From a practical point of view, spatial light modulators (SLMs) can be used as dynamic elements. Development of SLMs has been an active research field with numerous potential applications . Usually, SLMs are based on liquid crystal displays (LCD) and digital micromirrors devices. In this paper, only LCDs are considered as they can be used in reflection or in transmission. Nematic (twisted or parallel) and ferroelectric (smectic C* or smectic A*) phases are commonly used. They have different physical properties and offer therefore different performances. A brief description of the characteristics of these technologies is presented. Current technology allows the implementation of the proposed setup. Moreover, some practical limitations are discussed.
3.1 Modulation mode
Each liquid crystal allows light modulation because of their birefringence properties. When a voltage is applied, the internal structure of the liquid crystal is modified and a modulation is obtained (amplitude or phase). Theory and experimental characterization of twisted-nematic LCD (TN-LCD) have shown that amplitude and phase modulations are generally coupled. It is possible to obtain an amplitude-mostly or a phase-mostly modulation mode (2π or greater) by adding adjusted polarization elements [19,20]. Parallel-nematic LCD (PAL-LCD) behaves as a tunable retardation plate and a pure dephasing of 2π is possible . Ferroelectric LCDs (F-LCD) are more suitable for amplitude modulation with generally a binary phase modulation (0 – π) . Multiple-level phase-only modulation mode obtained by cascading of F-LCDs has been shown . A continuous 0-2π phase modulation concept with F-LCD has also been proposed .
3.2 Response time
Whatever the technology used for the dynamic element, the number of points of the virtual image will be limited. As discussed in section 2, the display time must be less than 50ms. The operating rate of the LCD SLM is limited by the response time of the liquid crystal which is the summation of optical rise time and fall time . The response time and therefore the frequency rate depends on the liquid crystal phase. TN-LCDs have fast optical rise time (~50-1000µs) but slow fall time (~2-30ms). F-LCDs are much more interesting because smectic C* liquid crystals have a response time around 50-150µs. Smectic A* are even faster with response time around 200ns.
The maximal number of points contained in the virtual image as a function of the frequency rate is presented in Fig. 3 . The different technologies are shown in relation with their speed limits. With TN-LCD, the virtual image is limited to 10 points. The number of points that can be displayed with smectic C* F-LCD varies between 100 and 2000. Finally, smectic A* F-LCD can potentially display the greatest number of points, up to 250 000.
3.3 Pixel pitch
The SLM is a matrix of separated pixels on which a voltage is applied to perform a local modulation (see Fig. 4 ). The pixel pitch p of the SLM is an important parameter. Indeed, it defines the resolution limit R = (2p)−1 of the dynamic element and thus the practical implementation of the corrective phase transfer function which calculation is presented in section 4.
Among commercially available LCD SLMs, pixel pitch is in a range from 5µm to 32µm. As a consequence, the resolution of the dynamic element is currently limited from 15lp/mm to 100lp/mm.
Table 1 summarizes the previous discussed points for each liquid crystal type.
In our case, three requirements are needed for the optical system. First, nine points have to be displayed in a time shorter than 50ms. Then, a continuous phase only modulation is necessary for the dynamic element. Finally, the resolution of the dynamic element must be as high as possible. Such requirements can be obtained with a nematic LCD. Their slow frequency rate is enough for sequential generation of nine points in a time shorter than 50ms. Moreover, they can perform a nearly continuous phase-only modulation (more than 2π). As it is shown in section 4.4, resolution is sufficient to perform the expected correction. For applications requiring a higher number of points, a solution based on ferroelectric LCDs would have to be considered due to their high frequency rate.
4. Determination of the PTFs of the dynamic reconfigurable element
In this part, the method used to determine and calculate the theoretical corrective PTF of the dynamic DOE for a given point I of the virtual image is developed. Results obtained with and without calculated aberration compensation are discussed.
Assumptions done in this section are: (1) DOEs are pure phase elements, absorption is not considered; (2) DOEs are considered infinitely thin as corresponding thicknesses are very small comparing to the displaying distance of virtual points; (3) each image wavefront is assumed to be aberration-free, i.e. a spherical diverging wave; (4) wavefronts are described locally by a set of rays normal to the local tangent planes according to the eikonal description. It is therefore possible to link the phase Φ and the unit propagation vector u of the local ray by the following relation:
where n is the average refractive index of the modulated medium and λ0 the recording and readout wavelength in vacuum. This relation is valid only for slowly varying wavefronts.
4.2 Phase transfer function of the static element
The static element is obtained by the recording of a superposition of two counter-propagating laser beams (Fig. 5a ): a collimated plane wave, as reference beam and a diverging spherical wave, as object beam. The reference beam is directed to the recording hologram plate with an incidence angle θR while the object wave is centered at point S.
The corresponding phase function in the plane of the static element is described by
where ΦO and ΦR are respectively the phase distributions in the plane of the static element of the object wave and the reference wave. Inserting the various geometrical parameters and taking the constant phase term in such way that ΦPTF2(O) = 0, the recording FTP can be written as
where k0 = 2π/λ0, P is one point at the surface of static element and d(A,B) is the distance between two points A and B. The restitution geometry is presented on Fig. 5b. The diffracted wave phase at the first diffraction order is given by
where ΦC2 and ΦD2 are respectively the phase distributions of the reading wave and the diffracted wave in the plane of the static element. A reconstructed wave deviates from the desired spherical one excepted in the case where restitution and reference recording waves are identical. Strong primary aberrations i.e. coma and astigmatism are present.
4.3 Method of determination of the correcting phase transfer functions
In this section the different steps for the determination of the ideal PTF of the dynamic DOE for a given point Ikl of the virtual image are presented. Obviously, it is the same approach for each point of the virtual image. A reverse approach has been used. Light is propagated into the optical system from a virtual point Ikl to the plane of the dynamic DOE, where we want to determine the two-dimensional phase distribution (Fig. 6 ).
The first step consists in defining by ray-tracing a sampling grid of N points in the plane of each DOE. These two grids are used to obtain the phase of the different waves and thus the PTF of the dynamic DOE. The surface of the static DOE is uniformly sampled into N Pij points. N rays originating from the point Ikl are traced to the points Pij. These N rays are locally orthogonal to the ideal spherical diverging diffracted wave. Their associated unit propagation vectors u D2 are given by
Ray-tracing formula in vectorial form for a diffractive element is then applied to obtain the unit propagation vectors u C2 of the rays locally orthogonal to the ideal reading wave of the static DOE:
are respectively the unit propagation vector of the rays locally orthogonal to the object and the reference waves used to define the PTF of the static DOE (Fig. 5a). At this stage, the sampling grid in the plane of the dynamic DOE can be defined. Indeed, its sampled points Qij are at the intersection of the rays directed by u C2 with the plane of the dynamic DOE.
Phase distributions can be expressed into the coordinate system (Oxyz) at points Pij (xij P, yij P, zij P) and QN (xij Q, yij Q, zij Q). The phase distribution ΦD2 id of the ideal spherical diverging diffracted wave in the plane of the static DOE, is expressed as
We can thus calculate the phase of the ideal reading wave in the plane of the dynamic DOE:
Equation (10) is available with the local planar approximation. The ideal PTF of the dynamic DOE can finally be numerically expressed at points QN:
where ΦD1 = ΦC2 and ΦC1 is an incident monochromatic plane wave. We change the reference coordinate system to express ΦPTF1 in the coordinate system (O1x1y1z1). Then we realize a X-Y polynomial interpolation to obtain an analytical form of the PTF of the dynamic DOE:
With this analytical form, a particular set of coefficients akl is obtained for each position of the virtual image point I. Moreover, this description is relevant to determine the local spatial frequencies νX and νY along the direction (O1x1) and (O1y1). As discussed in section 3.3, the practical realization of the PTF is limited by the SLM pixel pitch. The absolute values of these spatial frequencies are defined as the following:
A monochromatic light wave with λ0 = 514.5nm is considered for the recording of the static element as well as for its reconstruction. The incidence of the recording reference plane is θR = 50°. The source point S of the recording object wave is at the position (−421.2mm, 0mm, 903.3mm). As it is shown on Fig. 2, the angle α2 between the normal of the static element and the normal of the dynamic element is 50°. Their centers O1 and O2 are distant of 50 mm.
The image used for simulation is presented in Fig. 8 . It is a 3x3 matrix centered in point I22 which is located at the same place than the point S used for the recording geometry. The eight other points are located at peripheral positions of the 10°x15° field of view. The coordinates of these nine points are listed in Table 2 . Aberrations are maximal at the peripheral points. As a consequence, if the aberrations compensation is obtained for these points it can also be achieved for the points inside the 10°x15° field of view since they introduce fewer aberrations.
The method developed in section 4.3 has been used to calculate corrective PTF of the dynamic DOE for the nine points of the image. The analytical form (Eq. (12)) has been obtained by applying a 3rd order polynomial interpolation. Two configurations for the incident monochromatic wave have been considered for the calculation of spatial frequencies (Fig. 9 ): (a) the incident wave used to lighten the dynamic DOE is a plane wave having a normal incidence whatever the position of the virtual image point I, the beam is both deflected and modulated by the dynamic DOE; (b) the incident wave is a plane wave with an oblique incidence (θ1, δ1) (Fig. 7) that depends on the position of the point I of the virtual image. In this case, the unit propagation vector uC1 is assumed to be identical to the main unit propagation vector uD1 of the wave diffracted by the dynamic DOE. The deflection is assumed to be done before the dynamic DOE by a system which is not studied in the present paper.
Modulation transfer functions (MTFs) with and without aberration compensation (Fig. 10 ) have been obtained using the CodeV® software. As expected, the MTFs of the uncorrected setup (blue curves) are far from the limit of diffraction excepted for the central point of the image. On the other hand, the MTFs of the optimized system (red curves) are very close to the limit of diffraction whatever the position of the point in the 10°x15° field of view.
5. Diffraction efficiency
In order to present a complete description of the performance of the optimized studied system, the diffraction efficiency (DE) of the proposed setup has been calculated over the 10°x15° field of view.
5.1 Theory and assumptions
Considering the geometry of the setup, the diffraction is conic and Kogelnik’s approximated coupled wave theory cannot be used . DE calculation has been done using the rigorous coupled wave analysis (RCWA) theory . It is valid for a planar grating lit by an obliquely incident plane wave.
The DE of the whole setup is obtained by multiplying the dynamic element DE with the static element one. In this paper, only static element DE is considered, that is to say dynamic element DE is assumed to be constant (100%) whatever the position of the image point in the 10°x15° field of view. The static element is assumed to be a volume hologram, i.e. its thickness is not neglected.
Here, RCWA cannot be used directly for two reasons. The static hologram is not a planar grating and reading waves have a complex form. A local description is therefore used (Fig. 11 ).
The static element is divided into N elementary rectangles each one centered on a point Pij. Because of the slow variation of the phase function, each area is considered as a localized planar grating having a local period Λ and a local slanted angle Φ . In order to have a plane wave description, reading wave are described locally by a set of rays normal to the local tangent planes according to the eikonal description. A local grating vector K and a local slanted angle Φ can thus be calculated. RCWA can therefore be used to calculate local DE ηij as defined in . The total diffraction efficiency η for a given position of the virtual point in the 10°x15° field of view is then obtained using
where N is the total number of Pij points considered.
Using the description presented in section 5.1, the diffraction efficiency of the optimized system has been calculated over the 10°x15° field of view (Fig. 12 ). It has been sampled into 81x81 uniformly distributed virtual image points. A transverse electric polarization state (ψ = 90°) has been considered. The static hologram of 16.2mm x 17.2mm dimension has been divided into 100 elementary rectangles.
By optimizing the index modulation and the thickness of the hologram, an average diffraction efficiency of 79% can be reached over the considered field of view with a standard deviation close to 5%.
The design of an off-axis see-through display associating two diffractive optical elements has been presented in this paper. A dynamic method to display sequentially a virtual image having nine points in a 10°x15° field of view has been proposed. The feasibility of this approach has been discussed considering the current state of the technology. A nematic LCD SLM can be used to perform the dynamic setup. It has been shown that aberrations compensation of the virtual image can be done whatever the position of its points into the 10°x15° field of view. These compensations allow to obtain MTFs close to the limit of diffraction. With an optimal analog holographic recording for the static element, relatively high diffraction efficiency (79%) with good uniformity (5% standard deviation) has been obtained across the considered field of view.
References and links
1. D. W. F. Van Krevelen and R. Poelman, “A survey of augmented reality technologies, applications and limitations,”Int. J. Virtual Reality 9, 1–20 (2010).
2. E. Buckley, D. Stindt, and R. Isele, “14.4: Novel human-machine interface (HMI) design enabled by holographic laser projection,” SID Symp. Dig. 40(1), 172–177 (2009). [CrossRef]
3. R. T. Azuma, “A survey of augmented reality,” Presence (Camb. Mass.) 6, 355–385 (1997).
4. O. Cakmakci and J. Rolland, “Head-worn displays: a review,” J. Display Technol. 2(3), 199–216 (2006). [CrossRef]
5. J. Rolland and O. Cakmakci, “Head-worn displays: The future through new eyes,” Opt. Photonics News 20(4), 20–27 (2009). [CrossRef]
6. J. P. Rolland, R. L. Holloway, and H. Fuchs, “A comparison of optical and video see-through head-mounted displays,” Proc. SPIE 2351, 293–307 (1995). [CrossRef]
9. H. Nagahara, Y. Yagi, and M. Yachida, “A wide-field-of-view catadioptrical head-mounted display,” Electon. Commun. Jpn. 89, 33–43 (2006).
10. Z. Zheng, X. Liu, H. Li, and L. Xu, “Design and fabrication of an off-axis see-through head-mounted display with an x-y polynomial surface,” Appl. Opt. 49(19), 3661–3668 (2010). [CrossRef] [PubMed]
11. D. Cheng, Y. Wang, H. Hua, and M. M. Talha, “Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism,” Appl. Opt. 48(14), 2655–2668 (2009). [CrossRef] [PubMed]
12. T. Ando, T. Matsumoto, H. Takahashi, and E. Shimizu, “Head mounted display for mixed reality using holographic optical elements,” Mem. Fac. Eng. Osaka City Univ. 40, 1–6 (1999).
13. I. Kasai, Y. Tanijiri, T. Endo, and H. Ueda, “A practical see-through head mounted display using a holographic optical element,” Opt. Rev. 8(4), 241–244 (2001). [CrossRef]
15. D. L. Dickensheets, “Imaging performance of off-axis planar diffractive lenses,” J. Opt. Soc. Am. A 13(9), 1849–1858 (1996). [CrossRef]
17. K. Goto and M. Kitaoka, “Aberrations in nonparaxial holography,” J. Opt. Soc. Am. A 5(3), 397–402 (1988). [CrossRef]
18. P. Ambs, J. Otόn, M. S. Millán, A. Jaulin, and L. Bigué, “Spatial light modulators for information processing: applications and overview,” in AIP Conference Proceedings of Sixth International Workshop on Information Optics, B. Javidi, and J.A. Benediktsson eds.(Reykjavik, Islande, 2007), 226–233.
19. C. Soutar and K. Lu, “Determination of the physical properties of an arbitrary twisted-nematic liquid crystal cell,” Opt. Eng. 33(8), 2704–2712 (1994). [CrossRef]
20. M. S. Mahmud, I. Naydenova, and V. Toal, “Implementation of phase-only modulation utilizing a twisted nematic liquid crystal spatial light modulator,” J. Opt. A, Pure Appl. Opt. 10(8), 085007 (2008). [CrossRef]
21. J. Otón, P. Ambs, M. S. Millán, and E. Pérez-Cabré, “Dynamic calibration for improving the speed of a parallel-aligned liquid-crystal-on-silicon display,” Appl. Opt. 48(23), 4616–4624 (2009). [CrossRef] [PubMed]
22. P. Ambs and L. Bigué, “Characterization of an analog ferroelectric spatial light modulator. Application to dynamic diffractive optical elements and optical information processing,” presented at the Fourth Euro American Workshop on Optoelectronic Information Processing, Valencia, Spain, 2001.
23. S. E. Broomfield, M. A. A. Neil, and E. G. S. Paige, “Programmable multiple-level phase modulation that uses ferroelectric liquid-crystal spatial light modulators,” Appl. Opt. 34(29), 6652–6665 (1995). [CrossRef] [PubMed]
24. D. Engström, M. J. O’Callaghan, C. Walker, and M. A. Handschy, “Fast beam steering with a ferroelectric-liquid-crystal optical phased array,” Appl. Opt. 48(9), 1721–1726 (2009). [CrossRef] [PubMed]
25. Boulder Nonlinear Systems, “Liquid crystal reference,” white paper (Boulder Nonlinear Systems, 2001), http://www.bnonlinear.com/papers/LCReference.pdf.
26. H. Kögelnik, “Coupled wave theory for thick hologram gratings,” Bell Syst. Tech. J. 48, 2909–2947 (1969).
27. M. G. Moharam and T. K. Gaylord, “Three-dimensional vector coupled-wave analysis of planar-grating diffraction,” J. Opt. Soc. Am. 73(9), 1105–1112 (1983). [CrossRef]
28. Y. Luo, J. Castro, J. K. Barton, R. K. Kostuk, and G. Barbastathis, “Simulations and experiments of aperiodic and multiplexed gratings in volume holographic imaging systems,” Opt. Express 18(18), 19273–19285 (2010). [CrossRef] [PubMed]