Abstract

We describe the mathematical and conceptual foundations for a novel methodology for jointly optimizing the design and analysis of the optics, detector, and digital image processing for imaging systems. Our methodology is based on the end-to-end merit function of predicted average pixel sum-squared error to find the optical and image processing parameters that minimize this merit function. Our approach offers several advantages over the traditional principles of optical design, such as improved imaging performance, expanded operating capabilities, and improved as-built performance.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. C. Fales, F. Huck, and R. Samms, “Imaging system design for improved information capacity,” Appl. Opt. 23, 872-888(1984).
    [Crossref] [PubMed]
  2. W. T. Cathey and E. R. Dowski, “New paradigm for imaging systems,” Appl. Opt. 41, 6080-6092 (2002).
    [Crossref] [PubMed]
  3. G. E. Johnson, A. K. Macon, and G. M. Rauker, “Computational imaging design tools and methods,” Proc. SPIE 5524, 284-294 (2004).
    [Crossref]
  4. R. Narayanswamy, G. E. Johnson, P. E. X. Silviera, and H. B. Wach, “Extending the imaging volume for biometric iris recognition,” Appl. Opt. 44, 701-712 (2005).
    [Crossref] [PubMed]
  5. P. Silviera and R. Narayanswamy, “Signal-to-noise analysis of task-based imaging systems with defocus,” Appl. Opt. 45, 2924-2934 (2006).
    [Crossref]
  6. P. D. Welch, “The use of fast Fourier transforms for the estimation of power spectra: a method based on time averaging over short modified periodograms,” IEEE Trans. Audio Electroacoust. 15, 70-73 (1967).
    [Crossref]
  7. J. W. Goodman, Introduction to Fourier Optics, 2nd ed. (McGraw-Hill, 1986).
  8. R. E. Fischer and B. Tadic-Galeb, Optical System Design (McGraw-Hill, 2000).
  9. W. J. Smith, Modern Optical Engineering (McGraw-Hill, 2000).
  10. A. K. Jain, Fundamentals of Digital Image Processing, 1st ed. (Prentice Hall, 1989).
  11. M. D. Robinson and D. G. Stork, “Joint design of lens system and digital image processing,” Proc. SPIE 6342, 63421G(2006).
  12. D. G. Stork and M. D. Robinson, “Information-based methods for optics/image processing co-design,” in 5th International Workshop on Information Optics, Vol. 860 of AIP Conference Proceedings, G.Cristóbal, B.Javidi, and S.Vallmitjana, eds. (AIP Press, 2006), pp. 125-135.
  13. ZEMAX Optical Design Program User's Guide (Zemax Development Corporation, 2004).
  14. P. Maeda, P. B. Catrysse, and B. A. Wandell, “Integrating lens design with digital camera simulation,” Proc. SPIE 5678, 48-58 (2005).
    [Crossref]

2006 (2)

P. Silviera and R. Narayanswamy, “Signal-to-noise analysis of task-based imaging systems with defocus,” Appl. Opt. 45, 2924-2934 (2006).
[Crossref]

M. D. Robinson and D. G. Stork, “Joint design of lens system and digital image processing,” Proc. SPIE 6342, 63421G(2006).

2005 (2)

P. Maeda, P. B. Catrysse, and B. A. Wandell, “Integrating lens design with digital camera simulation,” Proc. SPIE 5678, 48-58 (2005).
[Crossref]

R. Narayanswamy, G. E. Johnson, P. E. X. Silviera, and H. B. Wach, “Extending the imaging volume for biometric iris recognition,” Appl. Opt. 44, 701-712 (2005).
[Crossref] [PubMed]

2004 (1)

G. E. Johnson, A. K. Macon, and G. M. Rauker, “Computational imaging design tools and methods,” Proc. SPIE 5524, 284-294 (2004).
[Crossref]

2002 (1)

1984 (1)

1967 (1)

P. D. Welch, “The use of fast Fourier transforms for the estimation of power spectra: a method based on time averaging over short modified periodograms,” IEEE Trans. Audio Electroacoust. 15, 70-73 (1967).
[Crossref]

Cathey, W. T.

Catrysse, P. B.

P. Maeda, P. B. Catrysse, and B. A. Wandell, “Integrating lens design with digital camera simulation,” Proc. SPIE 5678, 48-58 (2005).
[Crossref]

Dowski, E. R.

Fales, C.

Fischer, R. E.

R. E. Fischer and B. Tadic-Galeb, Optical System Design (McGraw-Hill, 2000).

Goodman, J. W.

J. W. Goodman, Introduction to Fourier Optics, 2nd ed. (McGraw-Hill, 1986).

Huck, F.

Jain, A. K.

A. K. Jain, Fundamentals of Digital Image Processing, 1st ed. (Prentice Hall, 1989).

Johnson, G. E.

R. Narayanswamy, G. E. Johnson, P. E. X. Silviera, and H. B. Wach, “Extending the imaging volume for biometric iris recognition,” Appl. Opt. 44, 701-712 (2005).
[Crossref] [PubMed]

G. E. Johnson, A. K. Macon, and G. M. Rauker, “Computational imaging design tools and methods,” Proc. SPIE 5524, 284-294 (2004).
[Crossref]

Macon, A. K.

G. E. Johnson, A. K. Macon, and G. M. Rauker, “Computational imaging design tools and methods,” Proc. SPIE 5524, 284-294 (2004).
[Crossref]

Maeda, P.

P. Maeda, P. B. Catrysse, and B. A. Wandell, “Integrating lens design with digital camera simulation,” Proc. SPIE 5678, 48-58 (2005).
[Crossref]

Narayanswamy, R.

Rauker, G. M.

G. E. Johnson, A. K. Macon, and G. M. Rauker, “Computational imaging design tools and methods,” Proc. SPIE 5524, 284-294 (2004).
[Crossref]

Robinson, M. D.

M. D. Robinson and D. G. Stork, “Joint design of lens system and digital image processing,” Proc. SPIE 6342, 63421G(2006).

D. G. Stork and M. D. Robinson, “Information-based methods for optics/image processing co-design,” in 5th International Workshop on Information Optics, Vol. 860 of AIP Conference Proceedings, G.Cristóbal, B.Javidi, and S.Vallmitjana, eds. (AIP Press, 2006), pp. 125-135.

Samms, R.

Silviera, P.

Silviera, P. E. X.

Smith, W. J.

W. J. Smith, Modern Optical Engineering (McGraw-Hill, 2000).

Stork, D. G.

M. D. Robinson and D. G. Stork, “Joint design of lens system and digital image processing,” Proc. SPIE 6342, 63421G(2006).

D. G. Stork and M. D. Robinson, “Information-based methods for optics/image processing co-design,” in 5th International Workshop on Information Optics, Vol. 860 of AIP Conference Proceedings, G.Cristóbal, B.Javidi, and S.Vallmitjana, eds. (AIP Press, 2006), pp. 125-135.

Tadic-Galeb, B.

R. E. Fischer and B. Tadic-Galeb, Optical System Design (McGraw-Hill, 2000).

Wach, H. B.

Wandell, B. A.

P. Maeda, P. B. Catrysse, and B. A. Wandell, “Integrating lens design with digital camera simulation,” Proc. SPIE 5678, 48-58 (2005).
[Crossref]

Welch, P. D.

P. D. Welch, “The use of fast Fourier transforms for the estimation of power spectra: a method based on time averaging over short modified periodograms,” IEEE Trans. Audio Electroacoust. 15, 70-73 (1967).
[Crossref]

Appl. Opt. (4)

IEEE Trans. Audio Electroacoust. (1)

P. D. Welch, “The use of fast Fourier transforms for the estimation of power spectra: a method based on time averaging over short modified periodograms,” IEEE Trans. Audio Electroacoust. 15, 70-73 (1967).
[Crossref]

Proc. SPIE (2)

G. E. Johnson, A. K. Macon, and G. M. Rauker, “Computational imaging design tools and methods,” Proc. SPIE 5524, 284-294 (2004).
[Crossref]

P. Maeda, P. B. Catrysse, and B. A. Wandell, “Integrating lens design with digital camera simulation,” Proc. SPIE 5678, 48-58 (2005).
[Crossref]

Other (7)

J. W. Goodman, Introduction to Fourier Optics, 2nd ed. (McGraw-Hill, 1986).

R. E. Fischer and B. Tadic-Galeb, Optical System Design (McGraw-Hill, 2000).

W. J. Smith, Modern Optical Engineering (McGraw-Hill, 2000).

A. K. Jain, Fundamentals of Digital Image Processing, 1st ed. (Prentice Hall, 1989).

M. D. Robinson and D. G. Stork, “Joint design of lens system and digital image processing,” Proc. SPIE 6342, 63421G(2006).

D. G. Stork and M. D. Robinson, “Information-based methods for optics/image processing co-design,” in 5th International Workshop on Information Optics, Vol. 860 of AIP Conference Proceedings, G.Cristóbal, B.Javidi, and S.Vallmitjana, eds. (AIP Press, 2006), pp. 125-135.

ZEMAX Optical Design Program User's Guide (Zemax Development Corporation, 2004).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1
Fig. 1

Top, in the traditional electro-optical imaging system design methodology, the optical subsystem and the image processing subsystem are designed and optimized sequentially. This method may give a possibly complex optical system and a high-quality optical image. Bottom, a joint optimization method often produces a smaller but lower-quality optical subsystem together with a more complex image processing subsystem, yielding a final digital image that is of quality equal to or better than that produced by the system designed by traditional methods.

Fig. 2
Fig. 2

Left, some of the image patches taken from a collection of 300 dpi PostScript documents used to estimate the PSD. Right, a vertical slice through the PSD estimated using Welch’s method. The spike near 0.2 cycles/pixel corresponds to the approximate line spacing in the collection of documents.

Fig. 3
Fig. 3

Examples of geometrically constrained digital filters. These digital filters vary in their computational complexity (function of the number of taps) as well as their ability to approximate the ideal Wiener filter spectral response of Eq. (7).

Fig. 4
Fig. 4

General software components of joint digital–optical compensation software. For each iteration of the optimization process, a function call is made to the UDOP module, which computes the predicted RMSE for the current state of the optical design. During computation of the predicted MSE, the UDOP software uses the Zemax ray-tracing capability to compute the needed wavefront error functions used to compute H tot ( ω ) .

Fig. 5
Fig. 5

Graph, wavefront error (OPD-RMS) as a function of the focal distance. According to the effective focal length of the lens system, the detector should be placed at a distance of 84.75 mm from the lens. This corresponds to the focal point minimizing the OPD-RMS wavefront error. Below the graph are portions of the simulated captured image. We observe that the minimal OPD-RMS image appears to have the sharpest resolution.

Fig. 6
Fig. 6

Solid curve, experimental RMSE image quality measure after the ideal Wiener filter is applied to each of the simulated images captured at different focal depths. Dashed curve, RMSE predicted by using Eq. (9). We observe that the experimental RMSE closely follows the predicted RMSE. We also observe that the ideal focal distance from an end-to-end perspective is 86 mm . The images at the bottom demonstrate the reduction in imaging artifacts when the RMSE optimal focal distance is chosen.

Fig. 7
Fig. 7

Comparison of experimental RMSE performance for our test image using the complete ( optical + digital ) imaging systems produced by using three different design approaches as a function of the digital filter size. Dotted curve, performance of the traditional sequential image system design in which the lens system is first designed to minimize OPD-RMS wavefront error followed by a subsequent design of the image processing filters. Dashed curve, performance of the optimistic MSE-based design where the lens was first designed by using the optimistic MSE prediction of Eq. (9) followed by subsequent design of geometry-constrained digital filters. We see that this design produces about a 10% improvement in performance over the traditional sequential approach. Solid gray curve, performance of multiple imaging systems, each optimized in a joint fashion while considering the geometry constraints on the digital filter. These systems offer an additional 10% improvement in performance while almost matching the experimental RMSE achieved when the ideal Wiener filter is applied.

Fig. 8
Fig. 8

Comparison of polychromatic OPD-RMS wavefront error over the field of view for the lens system optimized based on OPD-RMS (solid) and based on the optimistic MSE (dashed). The digital–optical design shows significantly worse optical performance in terms of the OPD-RMS wavefront error.

Fig. 9
Fig. 9

Left column, captured and processed images for the traditionally designed optical system. The captured opti cal image for the traditional system shows some blurring but minimal lateral chromatic aberration and coma. The processed image, however, shows the ringing characteristic of information loss due to zero crossings in the MTF. Right column, captured and processed images for the jointly designed system. The captured images show significant coma and lateral chromatic aberration as evidenced by the vertical smearing of the image. After image processing, however, the image shows high contrast with almost no visible artifacts.

Fig. 10
Fig. 10

Comparison of cumulative distribution functions for the wavefront error (OPD-RMS) computed at 0%, 70%, and 100% of the image field for both the sequential (solid) and the joint (dashed) compensation strategies. As expected, the sequential compensation approach based on minimizing wavefront error produces an as-built lens system with much less wavefront error than the joint approach, which ignores the intermediate optical performance.

Fig. 11
Fig. 11

Comparison of cumulative distribution functions for the predicted RMSE performance. In this case, the joint compensation strategy produces much higher-quality imaging systems. Adjusting both the digital and the optical compensation parameters jointly produces systems with much higher yield. In some cases, the improvement suggests nearly 2 × the yield for the jointly compensated systems over the sequentially compensated systems.

Fig. 12
Fig. 12

MTFs for a simple optical system designed with equal OPD-RMS (five waves). Even though the optical aberrations are roughly equivalent, the MTFs differ significantly. This difference in MTFs has profound implications in the context of digital image processing: zeros in an MTF mean that some information is lost and cannot be recovered through image processing, even in principle.

Tables (2)

Tables Icon

Table 1 General Imaging Specifications for a 300 dpi Document Scanner

Tables Icon

Table 2 Optical Specifications for the Triplet Lens System

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

s ideal ( k ) = [ B T ( x ) * P ( s obj ( z , λ ) ) ] | k = T x , λ = λ 0 = [ B T ( x ) * s proj ( x , λ ) ] | k = T x , λ = λ 0 = [ s img ( x , λ ) ] | k = T x , λ = λ 0 ,
s ( x - x ˜ ) h opt ( x , x ˜ ) d x ˜ ,
h opt ( x , x ˜ ) | A ( p ) e j OPD ( p , x ) e j 2 π x ˜ p d p | 2 ,
[ H ] j k = h tot ( x = T j , x ˜ = T k ) .
y = Hs + n ,
min c E n , s [ Ry - s 2 ] ,
R ( ω ) = H tot * ( ω ) P s ( ω ) | H tot ( ω ) | 2 P s ( ω ) + σ 2 ,
MSE ( Θ ) = P s ( ω ) | H tot ( ω , Θ o ) R ( ω , Θ d ) - 1 | 2 + | R ( ω , Θ d ) | 2 σ 2 d ω ,
MSE ( Θ ) = P s ( ω ) σ 2 | H tot ( ω , Θ o ) | 2 P s ( ω ) + σ 2 d ω .

Metrics