Abstract

An imaging system design procedure for miniature wide-angle computational cameras with subsequent software correction of distortion is described. Such miniature wide-angle computational cameras have a broad range of applications, including eye-tracked extraocular cameras for retinal prostheses, and also wearable visual aids for the blind and those with low vision. As significant (typically barrel) distortion is commonplace in wide-field-of-view imaging systems, digital post-processing is often employed to generate rectilinear output images. Relaxation of the constraint on distortion during the optical system design process is shown to allow for improved optimization of other image-degrading aberrations. Analysis of the effects of distortion on the software-corrected final image during optical design is accomplished by using comprehensive image quality metrics such as the correlation coefficient and the spatial frequency response. Selection of a surprisingly large exact initial distortion value as a constraint allows for the design of a miniature wide-angle imaging system that yields significantly enhanced final image quality.

© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Calibration of a trinocular system formed with wide angle lens cameras

Carlos Ricolfe-Viala, Antonio-Jose Sanchez-Salmeron, and Angel Valera
Opt. Express 20(25) 27691-27696 (2012)

Wide-angle catadioptric lens with a rectilinear projection scheme

Gyeong-il Kweon, Seung Hwang-bo, Geon-hee Kim, Sun-cheol Yang, and Young-hun Lee
Appl. Opt. 45(34) 8659-8673 (2006)

Wide-angle camera with multichannel architecture using microlenses on a curved surface

Wei-Lun Liang, Hui-Kai Shen, and Guo-Dung J. Su
Appl. Opt. 53(17) 3696-3705 (2014)

References

  • View by:
  • |
  • |
  • |

  1. N. R. B. Stiles, B. P. McIntosh, P. J. Nasiatka, M. C. Hauer, J. D. Weiland, M. S. Humayun, and A. R. Tanguay, Jr., “An Intraocular Camera for Retinal Prostheses: Restoring Sight to the Blind,” Chapter 20 in Optical Processes in Microparticles and Nanostructures, Advanced Series in Applied Physics, Volume 6, A. Serpenguzel and A. Poon, eds., (World Scientific, 2010); pp. 385–429.
  2. B. P. McIntosh, “Intraocular and Extraocular Cameras for Retinal Prostheses: Effects of Foveation by Means of Visual Prosthesis Simulation,” Ph.D. Thesis, University of Southern California, (May, 2015).
  3. F. E. Sahin, “Novel Imaging Systems for Intraocular Retinal Prostheses and Wearable Visual Aids,” Ph.D. Thesis, University of Southern California, (December, 2015).
  4. C. Pernechele, “Hyper Hemispheric Lens,” Opt. Express 24(5), 5014–5019 (2016).
    [Crossref] [PubMed]
  5. F. Remondino and C. Fraser, “Digital Camera Calibration Methods: Considerations and Comparisons,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci 36, 266–272 (2006).
  6. Z. Zhang, “Flexible Camera Calibration by Viewing a Plane from Unknown Orientations,” in Proceedings of the Seventh IEEE International Conference on Computer Vision (IEEE, 1999), pp. 666–673.
    [Crossref]
  7. J. A. Gohman, “Wide Angle Lens System Having a Distorted Intermediate Image,” U.S. Patent 7,009,765, (2006).
  8. J. Gauvin, M. Doucet, M. Wang, S. Thibault, and B. Blanc, “Development of New Family of Wide-Angle Anamorphic Lens with Controlled Distortion Profile,” Proc. SPIE 5874, 587404 (2005).
    [Crossref]
  9. S. Thibault, J. Parent, H. Zhang, and P. Roulet, “Design, Fabrication and Test of Miniature Plastic Panomorph Lenses for 180° Field of View,” in Proceedings of the International Design Conference: Classical Optics (2014), (Optical Society of America, 2014), paper IM2A.3.
  10. Theia Technologies SY125A Specification Sheet. Available Online: http://www.theiatech.com/s/SY125_spec_sheet_R5.pdf
  11. H. Sun, Lens Design: A Practical Guide, (CRC, 2016), Chap. 5.
  12. W. J. Smith, Modern Optical Engineering, (McGraw-Hill, 2000).
  13. M. Reiss, “The Cos4 Law of Illumination,” J. Opt. Soc. Am. 35(4), 283–288 (1945).
    [Crossref] [PubMed]
  14. V. N. Mahajan, Optical Imaging and Aberrations: Part 1. Ray Geometrical Optics (SPIE, 1998), Chap. 2.
  15. R. E. Fischer and B. Tadic-Galeb, Optical System Design (McGraw-Hill, 2000), Chap. 23.
  16. J. A. Seibert, J. M. Boone, and K. K. Lindfors, “Flat-Field Correction Technique for Digital Detectors,” Proc. SPIE 3336, 348–354 (1998).
    [Crossref]
  17. M. M. Roossinov, “Wide Angle Orthoscopic Anastigmatic Photographic Objective,” U.S. Patent 2,516,724, (1950).
  18. CODE V Software, Synopsys Inc., Pasadena, California.
  19. R. Kingslake, A History of the Photographic Lens (Academic, 1989), Chap. 10.
  20. A. Bovik, Handbook of Image and Video Processing (Elsevier Academic, 2005), p. 400.
  21. R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).
  22. S. Winkler, Digital Video Quality: Vision Models and Metrics (John Wiley & Sons, 2005), pp. 54–55.
  23. A. Goshtasby, Image Registration: Principles, Tools and Methods (Springer, 2012), pp. 9–12.
  24. Photography – Electronic Still Picture Cameras – Resolution Measurements, ISO Standard 12233:2000.
  25. P. Burns, “Slanted-Edge MTF for Digital Camera and Scanner Analysis,” in Proceedings of the IS&T PICS Conference (2000) (Society for Imaging Science and Technology, 2000), pp. 135–138.

2016 (1)

2007 (1)

R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).

2006 (1)

F. Remondino and C. Fraser, “Digital Camera Calibration Methods: Considerations and Comparisons,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci 36, 266–272 (2006).

2005 (1)

J. Gauvin, M. Doucet, M. Wang, S. Thibault, and B. Blanc, “Development of New Family of Wide-Angle Anamorphic Lens with Controlled Distortion Profile,” Proc. SPIE 5874, 587404 (2005).
[Crossref]

1998 (1)

J. A. Seibert, J. M. Boone, and K. K. Lindfors, “Flat-Field Correction Technique for Digital Detectors,” Proc. SPIE 3336, 348–354 (1998).
[Crossref]

1945 (1)

Blanc, B.

J. Gauvin, M. Doucet, M. Wang, S. Thibault, and B. Blanc, “Development of New Family of Wide-Angle Anamorphic Lens with Controlled Distortion Profile,” Proc. SPIE 5874, 587404 (2005).
[Crossref]

Boone, J. M.

J. A. Seibert, J. M. Boone, and K. K. Lindfors, “Flat-Field Correction Technique for Digital Detectors,” Proc. SPIE 3336, 348–354 (1998).
[Crossref]

Doucet, M.

J. Gauvin, M. Doucet, M. Wang, S. Thibault, and B. Blanc, “Development of New Family of Wide-Angle Anamorphic Lens with Controlled Distortion Profile,” Proc. SPIE 5874, 587404 (2005).
[Crossref]

Findlater, K.

R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).

Fraser, C.

F. Remondino and C. Fraser, “Digital Camera Calibration Methods: Considerations and Comparisons,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci 36, 266–272 (2006).

Gauvin, J.

J. Gauvin, M. Doucet, M. Wang, S. Thibault, and B. Blanc, “Development of New Family of Wide-Angle Anamorphic Lens with Controlled Distortion Profile,” Proc. SPIE 5874, 587404 (2005).
[Crossref]

Gow, R. D.

R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).

Grant, L.

R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).

Hart, J.

R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).

Lindfors, K. K.

J. A. Seibert, J. M. Boone, and K. K. Lindfors, “Flat-Field Correction Technique for Digital Detectors,” Proc. SPIE 3336, 348–354 (1998).
[Crossref]

Mcleod, S. J.

R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).

Nicol, R. L.

R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).

Pernechele, C.

Reiss, M.

Remondino, F.

F. Remondino and C. Fraser, “Digital Camera Calibration Methods: Considerations and Comparisons,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci 36, 266–272 (2006).

Renshaw, D.

R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).

Seibert, J. A.

J. A. Seibert, J. M. Boone, and K. K. Lindfors, “Flat-Field Correction Technique for Digital Detectors,” Proc. SPIE 3336, 348–354 (1998).
[Crossref]

Thibault, S.

J. Gauvin, M. Doucet, M. Wang, S. Thibault, and B. Blanc, “Development of New Family of Wide-Angle Anamorphic Lens with Controlled Distortion Profile,” Proc. SPIE 5874, 587404 (2005).
[Crossref]

Wang, M.

J. Gauvin, M. Doucet, M. Wang, S. Thibault, and B. Blanc, “Development of New Family of Wide-Angle Anamorphic Lens with Controlled Distortion Profile,” Proc. SPIE 5874, 587404 (2005).
[Crossref]

Zhang, Z.

Z. Zhang, “Flexible Camera Calibration by Viewing a Plane from Unknown Orientations,” in Proceedings of the Seventh IEEE International Conference on Computer Vision (IEEE, 1999), pp. 666–673.
[Crossref]

IEEE Trans. Electron Dev. (1)

R. D. Gow, D. Renshaw, K. Findlater, L. Grant, S. J. Mcleod, J. Hart, and R. L. Nicol, “A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance,” IEEE Trans. Electron Dev. 54(6), 1321–1329 (2007).

Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci (1)

F. Remondino and C. Fraser, “Digital Camera Calibration Methods: Considerations and Comparisons,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci 36, 266–272 (2006).

J. Opt. Soc. Am. (1)

Opt. Express (1)

Proc. SPIE (2)

J. Gauvin, M. Doucet, M. Wang, S. Thibault, and B. Blanc, “Development of New Family of Wide-Angle Anamorphic Lens with Controlled Distortion Profile,” Proc. SPIE 5874, 587404 (2005).
[Crossref]

J. A. Seibert, J. M. Boone, and K. K. Lindfors, “Flat-Field Correction Technique for Digital Detectors,” Proc. SPIE 3336, 348–354 (1998).
[Crossref]

Other (19)

M. M. Roossinov, “Wide Angle Orthoscopic Anastigmatic Photographic Objective,” U.S. Patent 2,516,724, (1950).

CODE V Software, Synopsys Inc., Pasadena, California.

R. Kingslake, A History of the Photographic Lens (Academic, 1989), Chap. 10.

A. Bovik, Handbook of Image and Video Processing (Elsevier Academic, 2005), p. 400.

S. Thibault, J. Parent, H. Zhang, and P. Roulet, “Design, Fabrication and Test of Miniature Plastic Panomorph Lenses for 180° Field of View,” in Proceedings of the International Design Conference: Classical Optics (2014), (Optical Society of America, 2014), paper IM2A.3.

Theia Technologies SY125A Specification Sheet. Available Online: http://www.theiatech.com/s/SY125_spec_sheet_R5.pdf

H. Sun, Lens Design: A Practical Guide, (CRC, 2016), Chap. 5.

W. J. Smith, Modern Optical Engineering, (McGraw-Hill, 2000).

V. N. Mahajan, Optical Imaging and Aberrations: Part 1. Ray Geometrical Optics (SPIE, 1998), Chap. 2.

R. E. Fischer and B. Tadic-Galeb, Optical System Design (McGraw-Hill, 2000), Chap. 23.

S. Winkler, Digital Video Quality: Vision Models and Metrics (John Wiley & Sons, 2005), pp. 54–55.

A. Goshtasby, Image Registration: Principles, Tools and Methods (Springer, 2012), pp. 9–12.

Photography – Electronic Still Picture Cameras – Resolution Measurements, ISO Standard 12233:2000.

P. Burns, “Slanted-Edge MTF for Digital Camera and Scanner Analysis,” in Proceedings of the IS&T PICS Conference (2000) (Society for Imaging Science and Technology, 2000), pp. 135–138.

N. R. B. Stiles, B. P. McIntosh, P. J. Nasiatka, M. C. Hauer, J. D. Weiland, M. S. Humayun, and A. R. Tanguay, Jr., “An Intraocular Camera for Retinal Prostheses: Restoring Sight to the Blind,” Chapter 20 in Optical Processes in Microparticles and Nanostructures, Advanced Series in Applied Physics, Volume 6, A. Serpenguzel and A. Poon, eds., (World Scientific, 2010); pp. 385–429.

B. P. McIntosh, “Intraocular and Extraocular Cameras for Retinal Prostheses: Effects of Foveation by Means of Visual Prosthesis Simulation,” Ph.D. Thesis, University of Southern California, (May, 2015).

F. E. Sahin, “Novel Imaging Systems for Intraocular Retinal Prostheses and Wearable Visual Aids,” Ph.D. Thesis, University of Southern California, (December, 2015).

Z. Zhang, “Flexible Camera Calibration by Viewing a Plane from Unknown Orientations,” in Proceedings of the Seventh IEEE International Conference on Computer Vision (IEEE, 1999), pp. 666–673.
[Crossref]

J. A. Gohman, “Wide Angle Lens System Having a Distorted Intermediate Image,” U.S. Patent 7,009,765, (2006).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1 (Left) Schematic diagram of the Roossinov wide-field-of-view lens, from [17]. (Center) Schematic diagram of a lens designed and optimized to have −5% distortion at a 60° half field angle. (Right) Schematic diagram of a lens designed and optimized to have −45% distortion at a 60° half field angle. The last two elements were merged into a doublet as a result of successive optimization cycles.
Fig. 2
Fig. 2 (Top Left) Simulation input image. (Top Right) Input image simulated through the lens with 40% barrel distortion at full field ( ± 60° diagonal FOV). (Bottom Left) The same image as in the Top Right, following dewarping to correct for the barrel distortion. The image is not flat field corrected to show the variation in illumination at the image plane. (Bottom Right) The same image as in the Bottom Left, following flat field correction.
Fig. 3
Fig. 3 (Left) Variation of illumination as a function of the field angle for the dewarped images resulting from eight different designs, each with different degrees of barrel distortion at full field ( ± 60° diagonal FOV). (Right) The correlation coefficients for a set of lens designs with different degrees of barrel distortion at full field ( ± 60° diagonal FOV).
Fig. 4
Fig. 4 (Left) Correlation coefficient variation as a function of field angle for lens designs with different degrees of barrel distortion at full field ( ± 60° diagonal FOV). (Right) Correlation coefficient variation as a function of field angle for lens designs with different degrees of barrel distortion at full field ( ± 60° diagonal FOV), after flat field correction. Correlation coefficients in quadrants corresponding to the same field angles are averaged, and the resulting values are plotted in the graphs. Note that these two figures have different y-axis scales and offsets.
Fig. 5
Fig. 5 Spatial frequency corresponding to a modulation of 0.5 for lens designs with different degrees of barrel distortion at full field ( ± 60° diagonal FOV), derived from simulated images after dewarping and flat field correction. Corresponding regions shown in the plot are highlighted on the overlaid image with the associated color.
Fig. 6
Fig. 6 (Top Left) Image simulated through the lens design with 10% barrel distortion at full field following dewarping and flat field correction. (Bottom Left) Center patch of the image shown on the Top Left. (Top Right) Image simulated through the lens design with 50% barrel distortion at full field following dewarping and flat field correction. (Bottom Right) Center patch of the image shown on the Top Right. The higher resolution in the central patch of the 50% barrel distortion design is readily apparent in comparison with that of the 10% barrel distortion design. The reference wavelengths employed for both image simulations are 470 nm, 540 nm, and 630 nm.

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

Distortion(%)= h r h i h i ×100
PSNR=10 log R 2 MSE
MSE= m n ( A mn B mn ) 2 m×n
r= m n ( A mn A ¯ )( B mn B ¯ ) ( m n ( A mn A ¯ ) 2 )( ( m n B mn B ¯ ) 2 )

Metrics