Open Access
Add to BibSonomyAbstract
The field-of-view (FOV) of a simple imaging system can be dramatically improved using a liquid crystal spatial light modulator (SLM). A SLM can be used to correct the off-axis aberrations that often limit the useful FOV of an imaging system giving near diffraction-limited performance at much larger field angles than would otherwise be possible. Foveated imaging refers to the variation in spatial resolution across the image caused by using the SLM in this application, and it is useful in reducing bandwidth requirements for data transmission.
©2001 Optical Society of America
1. Introduction
One of the most difficult tasks for an optical designer of wide field-of-view (FOV) imaging systems is maintaining high image quality across the entire FOV. Complex designs utilizing multiple optics are typically used in fast (i.e. low f/#) systems to minimize off-axis aberrations, but even the best, well-corrected systems have a limited FOV [1]. Because of the multiple optical elements that are required, these systems are often more expensive, heavier, and bulkier than narrower FOV systems. The complexity can be minimized with a slower system but at the expense of either light gathering ability or overall length. Currently, there exists a growing need for small, lightweight imaging sensors with wide FOV and high data transmission rates. There are a variety of military and civilian applications including surveillance, threat detection, and the remote operation of unmanned vehicles that could benefit by using such systems.
In this letter, we present a concept for reducing the size and complexity of a fast, wide FOV system, while maintaining near diffraction-limited performance over a limited spatial and spectral region. An additional benefit of this design is that it lends itself to reducing the data transmission bandwidth, which dictates the frame rate and resolution at which images can be transmitted. Image compression techniques, ranging from the standard JPEG [2] to clever techniques that maintain high resolution over a limited area of interest [3], attempt to increase the rates at which images can be transmitted. Our imaging technique will increase the total area coverage of a system while maintaining high resolution over a spatially limited region of interest. Systems utilizing this technique are ideally suited for a novel compression technique called foveated imaging [4,5] that can be used to reduce the data transmission bandwidth of a system.
2. Wide field-of-view imaging
The key to the proposed wide FOV system is the use of liquid crystal spatial light modulators (SLM). SLMs are compact, inexpensive devices that can be used to manipulate optical wavefronts at up to kHz rates with minimal electrical requirements [6–8]. Here we present initial results showing how a SLM can be used to correct off-axis aberrations in a fast, wide FOV imaging system over a limited spectral bandwidth.
A pixilated, liquid-crystal SLM used in a monochromatic application is the transmissive analogue of a segmented deformable mirror [9]; it imposes a user-controlled, spatially varying optical path across the wavefront. In the case of a nematic liquid crystal, applying a small voltage to an individual pixel changes the index of refraction in the direction of propagation, thus changing the optical retardance of that pixel. The optical path difference (OPD) created by applying the voltage is simply the change in index of refraction in the direction of propagation, Δnz, multiplied by the thickness of the liquid crystal, z, (OPD=Δnz z). The ability of the SLM to correct aberrations and increase the useable FOV of an imaging system depends on the maximum OPD (i.e. dynamic range) of the SLM, as well as the total number of pixels. If the phase errors of the wavefront are greater than the dynamic range of the SLM, correction can still be done modulo 2π [8,9].
Aberration correction is accomplished by systematically altering the optical path across the entire wavefront. By applying the appropriate voltage to each pixel, the optical path can be adjusted to correctly compensate the aberrated wavefront. However, because aberrations vary as a function of field angle, only a single angle can be perfectly corrected at any given time. Centered about that angle, there will exist a range of field angles that will be well corrected. That range will depend on the amount of aberration and the degree of correction required for the particular imaging application. Thus, there will be a limited region of interest within the entire FOV of the image that will be highly resolved. Because the aberrations at each field angle can be determined in advance, either with ray trace calculations or direct measurements, the set of voltages required to correct the wavefront at a given field angle can be stored as a look-up table. Dynamic control is accomplished by simply changing the voltages to correct for a new field angle of interest.
Figure 1 shows a representative system modeled in ZEMAX® with ray bundles propagating from several different field angles. In this simple example, two positive meniscus lenses (Lens 1: focal length=3.8 cm and diameter=2.4 cm, Lens 2: focal length=4.0 cm and diameter=3.0 cm) and an SLM are used to collect and focus light onto the image plane. This imaging system was designed around the SLM, rather than simply placing the SLM at a pupil plane of an existing system, which resulted in an extremely fast system (f/#=2.4) with theoretically diffraction-limited performance out to +/- 45°. Without the SLM, this simple system is too fast to achieve diffraction-limited performance at any field angle.
The SLM is modeled as an electrically addressed, transmissive device with 2048×2048 pixels and is placed at a pupil plane. By placing the SLM at a pupil plane, the phase of the wavefront at any field angle can be corrected. The phase correction produced by the SLM is Fig. 1. Optical layout for fast imaging system (f/#=2.4) with a +/-45° field-of-view specifically dependent on the chosen field angle, and the phase retardance required for each pixel is determined by the optical path at that angle. In Figure 1, the SLM is addressed to correct aberrations at the largest field angle of 45°, and all rays at that field angle are brought to a focus in the image plane by the combination of the lens and SLM. Aberrations at field angles less than 45° are not correctly compensated by the SLM and therefore ray bundles will be aberrated in the image plane, as shown in Figure 1. This causes areas outside the region of interest to appear blurry or out-of-focus, as seen in the image shown in Figure 2. Here image analysis software in ZEMAX® is used with the system shown in Figure 1 to create a foveated image from an aerial picture of an airport. The edges of the original image are defined to be at 90° full field (+/- 45°). The SLM is addressed to correct a field angle toward the bottom of the image so that the region immediately around the airplane near the bottom is highly resolved. Even with degraded resolution, coarse features throughout the picture are still identifiable.
The OPD ray fans in Figure 3 show the wavefront error to the edge of the pupil in both Y and X at field angles of (a) 0° (on-axis) and (b) 45° with no correction by the SLM. On-axis there are about 30 waves of aberration at the edge of the pupil, and that increases to about 150 waves at 45° in the X direction. By applying the correct voltages to the individual pixels of the SLM, the aberrations can theoretically be reduced to nearly zero for any angle within the FOV of the system. Figure 4 shows the OPD ray fans, (a) on-axis and (b) at 45°, after the SLM is addressed to correct the wavefront at 45°. The on-axis performance is severely degraded in this case, as expected, due to the fact that the aberrations are functionally dependent on the field angle. But the OPD ray fans at 45° shows less than 0.05 waves of residual aberrations. On-axis correction can be achieved by simply readdressing the SLM, as shown in Figure 5(a). Here, near diffraction-limited performance is obtained on-axis by correcting the aberrations at a field angle of 0°, but the wavefront at 45° is not corrected, as shown in Figure 5(b).
Our discussion to this point applies to quasi-monochromatic (e.g. actively illuminated) systems. Unfortunately, using a SLM for wavefront correction inherently limits the useable spectral bandwidth. As we increase the spectral bandpass of the system, the fidelity of correction will slowly decrease. If the wavefront aberration is less than the dynamic range of the SLM, the spectral bandwidth is simply limited by dispersion in the liquid crystal. In this case, passive imaging over a finite spectral bandwidth is possible. For the opposite case where the wavefront aberration is greater than the dynamic range, correction can still be done modulo 2π [8,9]. This is the case for our example system where there are approximately 150 waves of aberration at 45° in the X direction, see Figure 3. Unfortunately, correction done in this manner will significantly limit the spectral bandwidth that can be used. We are currently investigating the trade space between image quality and spectral bandwidth.
3. Foveated imaging
Foveated imaging is a consequence of our proposed system and simply refers to the variation in resolution across the final image. The term originates from the operation of the human eye, where a limited area within a few degrees of the point of gaze is highly resolved and resolution falls off rapidly with increasing field angle. Foveated imaging occurs in our system because it is only possible to correct for a single field angle at any given time. Thus, within a large FOV image, only a limited region of interest is well corrected and appears in focus, while peripheral areas are aberrated and appear blurred, as seen in Figure 2. However, the field angle, and thus the region of interest, can be changed dynamically on a millisecond time scale by electrically readdressing the SLM, such that any area within the FOV of the system can be highly resolved. The reduction in resolution outside the region of interest will depend on the particular system.
In addition to broader area coverage available with a wide FOV system, foveated imaging can reduce bandwidth requirements for transmitting digital images because only the area of interest contains high-resolution data. A spatially varying filter or software algorithm could be used to bin pixels outside the area of interest as appropriate for the system, reducing the total amount of data that must be transmitted. In fact, retaining high-resolution information in peripheral areas may actually be useless to an observer/operator. If one cleverly designs an observer’s display to match the encoding properties of the eye, the variable resolution will be unnoticeable to the observer, and useless data does not have to be processed and transmitted by the system [4].
Although resolution is reduced outside the region of interest, there is still useful information within the entire FOV. An observer/operator might detect movement or a glint in a peripheral region outside the highly resolved area and redirect the region of interest by readdressing the SLM. The effect is similar to someone turning their head after seeing “something out of the corner of their eye”. By combining this concept with eye-tracking techniques to determine the point of gaze [4], eye movements of the observer/operator could directly control the addressing of the SLM. Thus, in the case of remote operation of an unmanned vehicle, if something enters the imaging systems FOV, the operator automatically readdresses the SLM to focus on the potential threat by simply looking in that direction. This creates a so-called “telepresence” where the observations of the operator are as though he is in the unmanned vehicle viewing the scene directly.
4. Inexpensive system
Another benefit to using this concept is the potential for using cheap optics. By utilizing SLMs, not only can the useable FOV be increased and the complexity reduced compared to a conventional system, but also inexpensive plastic lenses could be used to reduce the cost and weight of the system. Plastic lenses are typically cheap, easily fabricated, lightweight, and poor in optical quality, which would likely make them unacceptable in most imaging applications. In this case, however, the SLM could be used to correct any optical deficiencies both on- and off-axis.
5. Conclusion
We have presented a technique for increasing the useable field-of-view of an imaging system by simply adding a liquid crystal spatial light modulator at a pupil plane in an optical design. This technique will allow near diffraction-limited performance over a region of interest, and this region can be shifted in real-time anywhere within the field-of-view of the system. The creation of variable resolution or foveated images can be used to reduce bandwidth requirements for data transmission.
Acknowledgments
The authors would like to acknowledge useful discussions with Mark Kramer, Steve Tucker, and Stan Peplinski.
References and links
1. See for example, R.R. Shannon, The art and science of optical design (Cambridge University Press, New York, 1997), Ch. 7. [CrossRef]
2. G.K. Wallace, “The JPEG still picture compression standard,” Communications of the ACM 34, 30–44 (1991). [CrossRef]
3. N. Tsumura, C. Endo, H. Haneishi, and Y. Miyake, “Image compression and decompression based on gazing area,” Proc. SPIE 2657, 361–367 (1996). [CrossRef]
4. W.S. Geisler and J.S. Perry, “A real-time foveated multi-resolution system for low-bandwidth video communication,” Proc. SPIE 3299, 294–305 (1998). [CrossRef]
5. E. Chang, S. Mallat, and C.K. Yap, “Wavelet foveation,” Applied and Computational Harmonic Analysis 9, 312–35 (2000). [CrossRef]
6. S.R. Restaino, J.T. Baker, D.C. Dayton, D. Werling, and S.L. Browne, “State of the art in liquid crystal technologies for wavefront compensation: an AFRL perspective,” Proc. SPIE 4167, 157–161 (2000).
7. P.F. McManamon, T.A. Dorschner, D.L. Corkum, L.J. Friedman, D.S. Hobbs, M. Holtz, S. Liberman, H.Q. Nguyen, D.P. Resler, R.C. Sharp, and E.A. Watson, “Optical phased array technology,” Proc. IEEE 84, 268–298 (1996). [CrossRef]
8. G. D. Love, “Wave-front correction and production of Zernike modes with a liquid-crystal spatial light modulator,” Appl. Opt. 36, 1517–1524 (1997). [CrossRef] [PubMed]
9. D.S. Acton, “Correction of static optical errors in a segmented adaptive optical system,” Appl. Opt. 34, 7965–7968 (1995). [CrossRef] [PubMed]
