Abstract

Optical imaging systems in which the lens and sensor are free to rotate about independent pivots offer greater degrees of freedom for controlling and optimizing the process of image gathering. However, to benefit from the expanded possibilities, we need an imaging model that directly incorporates the essential parameters. In this work, we propose a model of imaging which can accurately predict the geometric properties of the image in such systems. Furthermore, we introduce a new method for synthesizing an omnifocus (all-in-focus) image from a sequence of images captured while rotating a lens. The crux of our approach lies in insights gained from the new model.

© 2017 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Analysis of imaging for laser triangulation sensors under Scheimpflug rule

Antonin Miks, Jiri Novak, and Pavel Novak
Opt. Express 21(15) 18225-18235 (2013)

Generalized sine condition

Tamer T. Elazhary, Ping Zhou, Chunyu Zhao, and James H. Burge
Appl. Opt. 54(16) 5037-5049 (2015)

The entrance pupil of the human eye: a three-dimensional model as a function of viewing angle

Cathleen Fedtke, Fabrice Manns, and Arthur Ho
Opt. Express 18(21) 22364-22376 (2010)

References

  • View by:
  • |
  • |
  • |

  1. A. Walther, The Ray and Wave Theory of Lenses, 1st ed. (Cambridge University, 2006).
  2. R. Kingslake and R. B. Johnson, Lens Design Fundamentals, 2nd ed. (Academic, 2009).
  3. R. R. Shannon, The Art and Science of Optical Design, 1st ed. (Cambridge University, 1997).
  4. J. E. Greivenkamp, Field Guide to Geometrical Optics (SPIE Publications, 2003).
  5. A. Hornberg, Handbook of Machine Vision, 1st ed. (Wiley-VCH, 2006).
  6. P. Rangarajan, Pushing the Limits of Imaging Using Patterned Illumination (Southern Methodist University, 2014).
  7. N. Xu, K.-H. Tan, H. Arora, and N. Ahuja, Generating Omnifocus Images Using Graph Cuts and a New Focus Measure (IEEE, 2004), pp. 697–700.
  8. R. Jacobson, S. Ray, G. G. Attridge, and N. Axford, Manual of Photography, 9th ed. (Focal, 2000).
  9. C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden, Pyramid Methods in Image Processing (RCA Engineers, 1984).
  10. L. C. G. Brown, “A survey of image registration techniques,” ACM Comp. Surv. 24, 325–376 (1992).
    [Crossref]
  11. A. Criminisi, Accurate Visual Metrology from Single and Multiple Uncalibrated Images (University of Oxford, 1999).
  12. ZEMAX, Optical Design Program, User’s Manual (ZEMAX Development Corporation, 2011).
  13. I. Sinharoy, C. Holloway, and J. Stuermer, “PyZDDE,” in Zenodo (2016).
  14. I. Sinharoy, cosi2016_omnifocus: release of simulation code, files and dataset [Software] (2016), Zenodo. http://doi.org/10.5281/zenodo.59647 .

1992 (1)

L. C. G. Brown, “A survey of image registration techniques,” ACM Comp. Surv. 24, 325–376 (1992).
[Crossref]

Ahuja, N.

N. Xu, K.-H. Tan, H. Arora, and N. Ahuja, Generating Omnifocus Images Using Graph Cuts and a New Focus Measure (IEEE, 2004), pp. 697–700.

Anderson, C. H.

C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden, Pyramid Methods in Image Processing (RCA Engineers, 1984).

Arora, H.

N. Xu, K.-H. Tan, H. Arora, and N. Ahuja, Generating Omnifocus Images Using Graph Cuts and a New Focus Measure (IEEE, 2004), pp. 697–700.

Attridge, G. G.

R. Jacobson, S. Ray, G. G. Attridge, and N. Axford, Manual of Photography, 9th ed. (Focal, 2000).

Axford, N.

R. Jacobson, S. Ray, G. G. Attridge, and N. Axford, Manual of Photography, 9th ed. (Focal, 2000).

Bergen, J. R.

C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden, Pyramid Methods in Image Processing (RCA Engineers, 1984).

Brown, L. C. G.

L. C. G. Brown, “A survey of image registration techniques,” ACM Comp. Surv. 24, 325–376 (1992).
[Crossref]

Burt, P. J.

C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden, Pyramid Methods in Image Processing (RCA Engineers, 1984).

Criminisi, A.

A. Criminisi, Accurate Visual Metrology from Single and Multiple Uncalibrated Images (University of Oxford, 1999).

Greivenkamp, J. E.

J. E. Greivenkamp, Field Guide to Geometrical Optics (SPIE Publications, 2003).

Holloway, C.

I. Sinharoy, C. Holloway, and J. Stuermer, “PyZDDE,” in Zenodo (2016).

Hornberg, A.

A. Hornberg, Handbook of Machine Vision, 1st ed. (Wiley-VCH, 2006).

Jacobson, R.

R. Jacobson, S. Ray, G. G. Attridge, and N. Axford, Manual of Photography, 9th ed. (Focal, 2000).

Johnson, R. B.

R. Kingslake and R. B. Johnson, Lens Design Fundamentals, 2nd ed. (Academic, 2009).

Kingslake, R.

R. Kingslake and R. B. Johnson, Lens Design Fundamentals, 2nd ed. (Academic, 2009).

Ogden, J. M.

C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden, Pyramid Methods in Image Processing (RCA Engineers, 1984).

Rangarajan, P.

P. Rangarajan, Pushing the Limits of Imaging Using Patterned Illumination (Southern Methodist University, 2014).

Ray, S.

R. Jacobson, S. Ray, G. G. Attridge, and N. Axford, Manual of Photography, 9th ed. (Focal, 2000).

Shannon, R. R.

R. R. Shannon, The Art and Science of Optical Design, 1st ed. (Cambridge University, 1997).

Sinharoy, I.

I. Sinharoy, C. Holloway, and J. Stuermer, “PyZDDE,” in Zenodo (2016).

Stuermer, J.

I. Sinharoy, C. Holloway, and J. Stuermer, “PyZDDE,” in Zenodo (2016).

Tan, K.-H.

N. Xu, K.-H. Tan, H. Arora, and N. Ahuja, Generating Omnifocus Images Using Graph Cuts and a New Focus Measure (IEEE, 2004), pp. 697–700.

Walther, A.

A. Walther, The Ray and Wave Theory of Lenses, 1st ed. (Cambridge University, 2006).

Xu, N.

N. Xu, K.-H. Tan, H. Arora, and N. Ahuja, Generating Omnifocus Images Using Graph Cuts and a New Focus Measure (IEEE, 2004), pp. 697–700.

ACM Comp. Surv. (1)

L. C. G. Brown, “A survey of image registration techniques,” ACM Comp. Surv. 24, 325–376 (1992).
[Crossref]

Other (13)

A. Criminisi, Accurate Visual Metrology from Single and Multiple Uncalibrated Images (University of Oxford, 1999).

ZEMAX, Optical Design Program, User’s Manual (ZEMAX Development Corporation, 2011).

I. Sinharoy, C. Holloway, and J. Stuermer, “PyZDDE,” in Zenodo (2016).

I. Sinharoy, cosi2016_omnifocus: release of simulation code, files and dataset [Software] (2016), Zenodo. http://doi.org/10.5281/zenodo.59647 .

A. Walther, The Ray and Wave Theory of Lenses, 1st ed. (Cambridge University, 2006).

R. Kingslake and R. B. Johnson, Lens Design Fundamentals, 2nd ed. (Academic, 2009).

R. R. Shannon, The Art and Science of Optical Design, 1st ed. (Cambridge University, 1997).

J. E. Greivenkamp, Field Guide to Geometrical Optics (SPIE Publications, 2003).

A. Hornberg, Handbook of Machine Vision, 1st ed. (Wiley-VCH, 2006).

P. Rangarajan, Pushing the Limits of Imaging Using Patterned Illumination (Southern Methodist University, 2014).

N. Xu, K.-H. Tan, H. Arora, and N. Ahuja, Generating Omnifocus Images Using Graph Cuts and a New Focus Measure (IEEE, 2004), pp. 697–700.

R. Jacobson, S. Ray, G. G. Attridge, and N. Axford, Manual of Photography, 9th ed. (Focal, 2000).

C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden, Pyramid Methods in Image Processing (RCA Engineers, 1984).

Supplementary Material (1)

NameDescription
» Code 1       cosi2016_omnifocus: Release of simulation code, files and dataset [Software].

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1.
Fig. 1. Schematic of the general optical system with the lens pivoted at { C } , the sensor plane pivoted at { I } , and the object plane pivoted at { O } .
Fig. 2.
Fig. 2. Points of intersection (POI) of chief rays with the sensor plane. The set of chief rays originate from two parallel planes at two different depths from the lens such that, in the absence of lens rotation, their POIs perfectly overlap in the image plane (due to differing transverse magnification). The red and blue markers represent the POIs of the chief ray originating from the planes nearer and further from the lens, respectively. For all subplots shown above, the lens is rotated by 10° and 3° about the x axis and y axis, respectively. The top row—subplots (a), (b), (c)—shows the POIs when the lens is pivoted away (5 mm) from the center of the entrance pupil. The bottom row—subplots (d), (e), (f)—shows the same POIs when the lens is rotated about the center of the entrance pupil. The left, middle, and right columns correspond to lenses with pupil magnification m p equal to 0.55, 1.0, and 2.0, respectively. We can observe that: (1) the POIs from different depths warp by different degrees, causing parallax when the lens is rotated about a point away from the entrance pupil, (2) the nature of geometric distortion induced by lens rotation depends on the pupil magnification. Specifically, if m p = 1 , the all image points experience the same amount of scaling and transverse shift.
Fig. 3.
Fig. 3. Chief rays traced from a grid of points in the object plane through an ideal lens tilted about a point d e = 5    mm away from the entrance pupil along the optical axis to the tilted image plane.
Fig. 4.
Fig. 4. Image simulation using Zemax and PyZDDE: (a) schematic of setup, (b) captured image for α = 8 ° , (c) focus-measure using Laplacian of Gaussian (LoG) filter showing the regions in focus, (d) resulting composite image, and (e) focus-measure of the composite image showing all three depths in focus.

Tables (2)

Tables Icon

Table 1. Comparison of Numerically Computed Image Points using Eq. (12) and Ray Traced Image Points in Zemax for the Optical System Shown in Fig. 3

Tables Icon

Table 2. Verification of Eq. (20) and Eq. (21) for Focusing on a Tilted Object Plane by Tilting a Lens about the Entrance Pupil

Equations (22)

Equations on this page are rendered with MathJax. Learn more.

tan ( ω ) tan ( ω ´ ) = h ´ e h e y Ω y ´ Ω ´ .
tan ( ω ) tan ( ω ´ ) = m p .
l = [ l , m , n ] T = [ cos ( φ ) sin ( ω ) , sin ( φ ) sin ( ω ) , cos ( ω ) ] T l ´ = [ l ´ , m ´ , n ´ ] T = [ cos ( φ ´ ) sin ( ω ´ ) , sin ( φ ´ ) sin ( ω ´ ) , cos ( ω ´ ) ] T .
l ´ = 1 m p n ´ n l , m ´ = 1 m p n ´ n m , n ´ = ± m p 1 + ( m p 2 1 ) n 2 n .
l ´ = ± 1 1 + ( m p 2 1 ) n 2 M p l ,
l ´ L = 1 1 + ( m p 2 1 ) n R 2 M p R T l .
l ´ = 1 1 + ( m p 2 1 ) n R 2 R M p R T l ,
k ( λ ) = d ´ e r , 3 + λ 1 + ( m p 2 1 ) n R 2 R M p R T l .
λ = ( z ´ o d ´ e n ^ i T    r , 3 ) 1 + ( m p 2 1 ) n R 2 n ^ i T R M p R T l .
x ´ = d ´ e r , 3 + ( n ^ i ( 3 ) z ´ o d ´ e n ^ i T r , 3 ) n ^ i T R M p R T l R M p R T .
x ´ = d ´ e r , 3 + ( n ^ i ( 3 ) z ´ o d ´ e n ^ i T r , 3 ) n ^ i T R M p R T ( x d e r , 3 ) R M p R T ( x d e r , 3 ) .
x ´ I = R i T ( d ´ e r , 3 t i ) + ( n ^ i ( 3 ) z ´ o d ´ e n ^ i T r , 3 ) n ^ i T R M p R T ( x C d e r , 3 ) R i T R M p R T ( x C d e r , 3 ) .
1 m p u + m p u ´ = 1 f .
u ˜ = z o ( n ^ o T c ^ z ) + d e ( n ^ o T r , 3 ) n ^ o T l ,
u ´ ˜ = z ´ o ( n ^ i T c ^ z ) d ´ e ( n ^ i T r , 3 ) n ^ i T l ´ .
n ^ o T l m p [ z o ( n ^ o T c ^ z ) d e ( n ^ o T r , 3 ) ] ( l · o ^ ) + m p n ^ i T ( R M p R T l ) [ z ´ o ( n ^ i T c ^ z ) d ´ e ( n ^ i T r , 3 ) ] ( R M p R T l ) · o ^ = 1 f .
l T [ n ^ o m p [ z o n ^ o ( 3 ) d e ( n ^ o T r , 3 ) ] + R M p R T n ^ i [ z ´ o n ^ i ( 3 ) d ´ e ( n ^ i T r , 3 ) ] r , 3 f ] = 0 .
n ^ o m p [ z o n ^ o ( 3 ) d e ( n ^ o T r , 3 ] + R M p R T n ^ i [ z ´ o n ^ i ( 3 ) d ´ e ( n ^ i T r , 3 ) ] = r , 3 f .
n ^ ˜ o m p [ z o d e ( n ^ ˜ o T r , 3 ) ] + R M p R T n ^ ˜ i [ z ´ o d ´ e ( n ^ ˜ i T r , 3 ) ] = r , 3 f .
z ´ o = d cos α + m p z o f ( m p cos 2 α + sin 2 α ) m p z o cos α + f ,
tan β = sin α [ m p z o + f ( 1 m p ) cos α ] f ( m p cos 2 α + sin 2 α ) .
x ´ n = [ ( d cos α z ´ o ) d z ´ o 0 0 0 ( d cos α z ´ o ) d z ´ o d sin α 0 0 1 ] Inter-image homography ,    H ( α , 0 ) x ´ 0 ,

Metrics