Abstract

Fisheye lens can provide a wide view over 180°. It then has prominence advantages in three dimensional reconstruction and machine vision applications. However, the serious deformation in the image limits fisheye lens’s usage. To overcome this obstacle, a new rectification method named DDM (Digital Deformation Model) is developed based on two dimensional perspective transformation. DDM is a type of digital grid representation of the deformation of each pixel on CCD chip which is built by interpolating the difference between the actual image coordinate and pseudo-ideal coordinate of each mark on a control panel. This method obtains the pseudo-ideal coordinate according to two dimensional perspective transformation by setting four mark’s deformations on image. The main advantages are that this method does not rely on the optical principle of fisheye lens and has relatively less computation. In applications, equivalent pinhole images can be obtained after correcting fisheye lens images using DDM.

© 2012 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H. Bakstein and T. Pajdla, “Panoramic mosaicing with a field of view lens,” in Proceedings of IEEE Conference on Omnidirectional Vision (IEEE, 2002), pp. 60–67.
  2. Y. Jia, H. Lu, and A. Xu, “Fish-eye lens camera calibration for stereo vision system,” Chinese J Comput. 23(11), 1215–1219 (2002).
  3. J. Willneff and O. Wenisch, “The calibration of wide-angle lens cameras using perspective and non-perspective projections in the context of realtime tracking applications,” Proc. SPIE 8085, 80850S–80850S-9 (2011).
    [CrossRef]
  4. A. Parian and A. Gruen, “Panoramic camera calibration using 3D straight lines,” presented at ISPRS Panoramic Photogrammetry Workshop, Berlin, Germany, 24–25 Feb. 2005.
  5. S. Abraham and W. Forstner, “Fish-eye-stereo calibration and epipolar rectification,” ISPRS J Photogramm. 59(5), 278–288 (2005).
    [CrossRef]
  6. P. Sturm and S. Maybank, “On plane-based camera calibration: a general glgorithm, singularities, applications,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1999), pp. 432–437.
  7. A. Heikkil, “Geometric camera calibration using circular control points,” IEEE T Pattern Anal. 22(10), 1066–1077 (2000).
    [CrossRef]
  8. M. Grossberg and S. Nayar, “The raxel imaging model and ray-based calibration,” Int J Comput Vision 61(2), 119–137 (2005).
    [CrossRef]
  9. I. Akio, Y. Kazukiyo, M. Nobuya, and K. Yuichiro, “Calibrating view angle and lens distortion of the nikon fisheye converter FC-E8,” J Forest Res. 9(3), 177–181 (2004).
    [CrossRef]
  10. Z. Zhang, “A flexible new technique for camera calibration,” IEEE T Pattern Anal. 22(11), 1330–1334 (2000).
    [CrossRef]
  11. D. Schneider, E. Schwalbe, and H. Maas, “Validation of geometric models for fisheye lenses,” ISPRS J Photogramm. 64(3), 259–266 (2009).
    [CrossRef]
  12. J. Kannala and S. Brandt, “A generic camera calibration method for fish-eye lenses,” in Proceedings of International Conference on Pattern Recognition (IEEE, 2004), pp. 10–13.
  13. J. Kannala and S. Brandt, “A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses,” IEEE T Pattern Anal. 28(8), 1335–1340 (2006).
    [CrossRef]
  14. D. Gennery, “Generalized camera calibration including fish-eye lenses,” Int J Comput Vision 68(3), 239–266 (2006).
    [CrossRef]
  15. V. Orekhov, B. Abidi, C. Broaddus, and M. Abidi, “Universal camera calibration with automatic distortion model selection,” in Proceedings of International Conference on Image Processing (IEEE, 2007), pp. 397–400.
  16. Z. Kang, L. Zhang, and S. Zlatanova, “An automatic mosaicking method for building facade texture mapping using a monocular close-range image sequence,” ISPRS J Photogramm. 65(3), 282–293 (2010).
    [CrossRef]

2011

J. Willneff and O. Wenisch, “The calibration of wide-angle lens cameras using perspective and non-perspective projections in the context of realtime tracking applications,” Proc. SPIE 8085, 80850S–80850S-9 (2011).
[CrossRef]

2010

Z. Kang, L. Zhang, and S. Zlatanova, “An automatic mosaicking method for building facade texture mapping using a monocular close-range image sequence,” ISPRS J Photogramm. 65(3), 282–293 (2010).
[CrossRef]

2009

D. Schneider, E. Schwalbe, and H. Maas, “Validation of geometric models for fisheye lenses,” ISPRS J Photogramm. 64(3), 259–266 (2009).
[CrossRef]

2006

J. Kannala and S. Brandt, “A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses,” IEEE T Pattern Anal. 28(8), 1335–1340 (2006).
[CrossRef]

D. Gennery, “Generalized camera calibration including fish-eye lenses,” Int J Comput Vision 68(3), 239–266 (2006).
[CrossRef]

2005

S. Abraham and W. Forstner, “Fish-eye-stereo calibration and epipolar rectification,” ISPRS J Photogramm. 59(5), 278–288 (2005).
[CrossRef]

M. Grossberg and S. Nayar, “The raxel imaging model and ray-based calibration,” Int J Comput Vision 61(2), 119–137 (2005).
[CrossRef]

2004

I. Akio, Y. Kazukiyo, M. Nobuya, and K. Yuichiro, “Calibrating view angle and lens distortion of the nikon fisheye converter FC-E8,” J Forest Res. 9(3), 177–181 (2004).
[CrossRef]

2002

Y. Jia, H. Lu, and A. Xu, “Fish-eye lens camera calibration for stereo vision system,” Chinese J Comput. 23(11), 1215–1219 (2002).

2000

Z. Zhang, “A flexible new technique for camera calibration,” IEEE T Pattern Anal. 22(11), 1330–1334 (2000).
[CrossRef]

A. Heikkil, “Geometric camera calibration using circular control points,” IEEE T Pattern Anal. 22(10), 1066–1077 (2000).
[CrossRef]

Abidi, B.

V. Orekhov, B. Abidi, C. Broaddus, and M. Abidi, “Universal camera calibration with automatic distortion model selection,” in Proceedings of International Conference on Image Processing (IEEE, 2007), pp. 397–400.

Abidi, M.

V. Orekhov, B. Abidi, C. Broaddus, and M. Abidi, “Universal camera calibration with automatic distortion model selection,” in Proceedings of International Conference on Image Processing (IEEE, 2007), pp. 397–400.

Abraham, S.

S. Abraham and W. Forstner, “Fish-eye-stereo calibration and epipolar rectification,” ISPRS J Photogramm. 59(5), 278–288 (2005).
[CrossRef]

Akio, I.

I. Akio, Y. Kazukiyo, M. Nobuya, and K. Yuichiro, “Calibrating view angle and lens distortion of the nikon fisheye converter FC-E8,” J Forest Res. 9(3), 177–181 (2004).
[CrossRef]

Bakstein, H.

H. Bakstein and T. Pajdla, “Panoramic mosaicing with a field of view lens,” in Proceedings of IEEE Conference on Omnidirectional Vision (IEEE, 2002), pp. 60–67.

Brandt, S.

J. Kannala and S. Brandt, “A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses,” IEEE T Pattern Anal. 28(8), 1335–1340 (2006).
[CrossRef]

J. Kannala and S. Brandt, “A generic camera calibration method for fish-eye lenses,” in Proceedings of International Conference on Pattern Recognition (IEEE, 2004), pp. 10–13.

Broaddus, C.

V. Orekhov, B. Abidi, C. Broaddus, and M. Abidi, “Universal camera calibration with automatic distortion model selection,” in Proceedings of International Conference on Image Processing (IEEE, 2007), pp. 397–400.

Forstner, W.

S. Abraham and W. Forstner, “Fish-eye-stereo calibration and epipolar rectification,” ISPRS J Photogramm. 59(5), 278–288 (2005).
[CrossRef]

Gennery, D.

D. Gennery, “Generalized camera calibration including fish-eye lenses,” Int J Comput Vision 68(3), 239–266 (2006).
[CrossRef]

Grossberg, M.

M. Grossberg and S. Nayar, “The raxel imaging model and ray-based calibration,” Int J Comput Vision 61(2), 119–137 (2005).
[CrossRef]

Gruen, A.

A. Parian and A. Gruen, “Panoramic camera calibration using 3D straight lines,” presented at ISPRS Panoramic Photogrammetry Workshop, Berlin, Germany, 24–25 Feb. 2005.

Heikkil, A.

A. Heikkil, “Geometric camera calibration using circular control points,” IEEE T Pattern Anal. 22(10), 1066–1077 (2000).
[CrossRef]

Jia, Y.

Y. Jia, H. Lu, and A. Xu, “Fish-eye lens camera calibration for stereo vision system,” Chinese J Comput. 23(11), 1215–1219 (2002).

Kang, Z.

Z. Kang, L. Zhang, and S. Zlatanova, “An automatic mosaicking method for building facade texture mapping using a monocular close-range image sequence,” ISPRS J Photogramm. 65(3), 282–293 (2010).
[CrossRef]

Kannala, J.

J. Kannala and S. Brandt, “A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses,” IEEE T Pattern Anal. 28(8), 1335–1340 (2006).
[CrossRef]

J. Kannala and S. Brandt, “A generic camera calibration method for fish-eye lenses,” in Proceedings of International Conference on Pattern Recognition (IEEE, 2004), pp. 10–13.

Kazukiyo, Y.

I. Akio, Y. Kazukiyo, M. Nobuya, and K. Yuichiro, “Calibrating view angle and lens distortion of the nikon fisheye converter FC-E8,” J Forest Res. 9(3), 177–181 (2004).
[CrossRef]

Lu, H.

Y. Jia, H. Lu, and A. Xu, “Fish-eye lens camera calibration for stereo vision system,” Chinese J Comput. 23(11), 1215–1219 (2002).

Maas, H.

D. Schneider, E. Schwalbe, and H. Maas, “Validation of geometric models for fisheye lenses,” ISPRS J Photogramm. 64(3), 259–266 (2009).
[CrossRef]

Maybank, S.

P. Sturm and S. Maybank, “On plane-based camera calibration: a general glgorithm, singularities, applications,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1999), pp. 432–437.

Nayar, S.

M. Grossberg and S. Nayar, “The raxel imaging model and ray-based calibration,” Int J Comput Vision 61(2), 119–137 (2005).
[CrossRef]

Nobuya, M.

I. Akio, Y. Kazukiyo, M. Nobuya, and K. Yuichiro, “Calibrating view angle and lens distortion of the nikon fisheye converter FC-E8,” J Forest Res. 9(3), 177–181 (2004).
[CrossRef]

Orekhov, V.

V. Orekhov, B. Abidi, C. Broaddus, and M. Abidi, “Universal camera calibration with automatic distortion model selection,” in Proceedings of International Conference on Image Processing (IEEE, 2007), pp. 397–400.

Pajdla, T.

H. Bakstein and T. Pajdla, “Panoramic mosaicing with a field of view lens,” in Proceedings of IEEE Conference on Omnidirectional Vision (IEEE, 2002), pp. 60–67.

Parian, A.

A. Parian and A. Gruen, “Panoramic camera calibration using 3D straight lines,” presented at ISPRS Panoramic Photogrammetry Workshop, Berlin, Germany, 24–25 Feb. 2005.

Schneider, D.

D. Schneider, E. Schwalbe, and H. Maas, “Validation of geometric models for fisheye lenses,” ISPRS J Photogramm. 64(3), 259–266 (2009).
[CrossRef]

Schwalbe, E.

D. Schneider, E. Schwalbe, and H. Maas, “Validation of geometric models for fisheye lenses,” ISPRS J Photogramm. 64(3), 259–266 (2009).
[CrossRef]

Sturm, P.

P. Sturm and S. Maybank, “On plane-based camera calibration: a general glgorithm, singularities, applications,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1999), pp. 432–437.

Wenisch, O.

J. Willneff and O. Wenisch, “The calibration of wide-angle lens cameras using perspective and non-perspective projections in the context of realtime tracking applications,” Proc. SPIE 8085, 80850S–80850S-9 (2011).
[CrossRef]

Willneff, J.

J. Willneff and O. Wenisch, “The calibration of wide-angle lens cameras using perspective and non-perspective projections in the context of realtime tracking applications,” Proc. SPIE 8085, 80850S–80850S-9 (2011).
[CrossRef]

Xu, A.

Y. Jia, H. Lu, and A. Xu, “Fish-eye lens camera calibration for stereo vision system,” Chinese J Comput. 23(11), 1215–1219 (2002).

Yuichiro, K.

I. Akio, Y. Kazukiyo, M. Nobuya, and K. Yuichiro, “Calibrating view angle and lens distortion of the nikon fisheye converter FC-E8,” J Forest Res. 9(3), 177–181 (2004).
[CrossRef]

Zhang, L.

Z. Kang, L. Zhang, and S. Zlatanova, “An automatic mosaicking method for building facade texture mapping using a monocular close-range image sequence,” ISPRS J Photogramm. 65(3), 282–293 (2010).
[CrossRef]

Zhang, Z.

Z. Zhang, “A flexible new technique for camera calibration,” IEEE T Pattern Anal. 22(11), 1330–1334 (2000).
[CrossRef]

Zlatanova, S.

Z. Kang, L. Zhang, and S. Zlatanova, “An automatic mosaicking method for building facade texture mapping using a monocular close-range image sequence,” ISPRS J Photogramm. 65(3), 282–293 (2010).
[CrossRef]

Chinese J Comput.

Y. Jia, H. Lu, and A. Xu, “Fish-eye lens camera calibration for stereo vision system,” Chinese J Comput. 23(11), 1215–1219 (2002).

IEEE T Pattern Anal.

A. Heikkil, “Geometric camera calibration using circular control points,” IEEE T Pattern Anal. 22(10), 1066–1077 (2000).
[CrossRef]

Z. Zhang, “A flexible new technique for camera calibration,” IEEE T Pattern Anal. 22(11), 1330–1334 (2000).
[CrossRef]

J. Kannala and S. Brandt, “A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses,” IEEE T Pattern Anal. 28(8), 1335–1340 (2006).
[CrossRef]

Int J Comput Vision

D. Gennery, “Generalized camera calibration including fish-eye lenses,” Int J Comput Vision 68(3), 239–266 (2006).
[CrossRef]

M. Grossberg and S. Nayar, “The raxel imaging model and ray-based calibration,” Int J Comput Vision 61(2), 119–137 (2005).
[CrossRef]

ISPRS J Photogramm.

D. Schneider, E. Schwalbe, and H. Maas, “Validation of geometric models for fisheye lenses,” ISPRS J Photogramm. 64(3), 259–266 (2009).
[CrossRef]

S. Abraham and W. Forstner, “Fish-eye-stereo calibration and epipolar rectification,” ISPRS J Photogramm. 59(5), 278–288 (2005).
[CrossRef]

Z. Kang, L. Zhang, and S. Zlatanova, “An automatic mosaicking method for building facade texture mapping using a monocular close-range image sequence,” ISPRS J Photogramm. 65(3), 282–293 (2010).
[CrossRef]

J Forest Res.

I. Akio, Y. Kazukiyo, M. Nobuya, and K. Yuichiro, “Calibrating view angle and lens distortion of the nikon fisheye converter FC-E8,” J Forest Res. 9(3), 177–181 (2004).
[CrossRef]

Proc. SPIE

J. Willneff and O. Wenisch, “The calibration of wide-angle lens cameras using perspective and non-perspective projections in the context of realtime tracking applications,” Proc. SPIE 8085, 80850S–80850S-9 (2011).
[CrossRef]

Other

A. Parian and A. Gruen, “Panoramic camera calibration using 3D straight lines,” presented at ISPRS Panoramic Photogrammetry Workshop, Berlin, Germany, 24–25 Feb. 2005.

P. Sturm and S. Maybank, “On plane-based camera calibration: a general glgorithm, singularities, applications,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1999), pp. 432–437.

J. Kannala and S. Brandt, “A generic camera calibration method for fish-eye lenses,” in Proceedings of International Conference on Pattern Recognition (IEEE, 2004), pp. 10–13.

H. Bakstein and T. Pajdla, “Panoramic mosaicing with a field of view lens,” in Proceedings of IEEE Conference on Omnidirectional Vision (IEEE, 2002), pp. 60–67.

V. Orekhov, B. Abidi, C. Broaddus, and M. Abidi, “Universal camera calibration with automatic distortion model selection,” in Proceedings of International Conference on Image Processing (IEEE, 2007), pp. 397–400.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1

Principles of different lens. (a) the projections of different lens, p, p1, p2, p3 and p4 are perspective projection, stereographic projection, equidistance projection, equisolid angle projection and orthogonal projection, r, r1, r2, r3 and r4 are their corresponding distances between the image points and principal point; (b) difference between pinhole lens and fisheye lens. For fisheye lens, the actual image is the projection of perspective image on hemisphere surface to image plane.

Fig. 2
Fig. 2

DDM establishment based on 2D control panel, (a) the control panel image taken using fisheye camera; (b) the established digital deformation model.

Fig. 3
Fig. 3

Deformation correction of fisheye image using DDM, (a) building image taken by fisheye lens; (b) corrected image using DDM.

Fig. 4
Fig. 4

Fisheye lens Calibration.

Fig. 5
Fig. 5

Image after Correction using DDM.

Tables (1)

Tables Icon

Table 1 Distance comparisons after correction using DDM

Equations (11)

Equations on this page are rendered with MathJax. Learn more.

r = f tan θ , ( perspective projection )
r = 2 f tan ( θ / 2 ) ( stereographic projection ) r = f θ ( equidistance projection ) r = 2 f sin ( θ / 2 ) ( equisolid angle projection ) r = f sin ( θ ) ( orthogonal projection )
[ x ^ y ^ 1 ] = λ [ f x s x 0 0 0 f y y 0 0 0 0 1 0 ] [ R T 0 T 1 ] [ X Y Z 1 ] ,
x ^ = A 1 X + A 2 Y + A 3 Z + A 4 A 9 X + A 10 Y + A 11 Z + 1 y ^ = A 5 X + A 6 Y + A 7 Z + A 8 A 9 X + A 10 Y + A 11 Z + 1 .
{ Δ x = ( x x 0 ) × k 1 [ ( x x 0 ) 2 + ( y y 0 ) 2 ] + ( x x 0 ) × k 2 [ ( x x 0 ) 2 + ( y y 0 ) 2 ] 2 Δ y = ( y y 0 ) × k 1 [ ( x x 0 ) 2 + ( y y 0 ) 2 ] + ( y y 0 ) × k 2 [ ( y y 0 ) 2 + ( y y 0 ) 2 ] 2
[ x ^ y ^ ] = [ x x 0 Δ x y y 0 Δ y ] .
x ^ = B 1 X + B 2 Y + B 3 B 7 X + B 8 Y + 1 y ^ = B 4 X + B 5 Y + B 6 B 7 X + B 8 Y + 1 .
{ B 1 = A 1 A 11 Z + 1 B 2 = A 2 A 11 Z + 1 B 3 = A 3 Z + A 4 A 11 Z + 1 B 4 = A 5 A 11 Z + 1 B 5 = A 6 A 11 Z + 1 B 6 = A 7 Z + A 8 A 11 Z + 1 B 7 = A 9 A 11 Z + 1 B 8 = A 10 A 11 Z + 1
x x 0 Δ x = B 1 X + B 2 Y + B 3 B 7 X + B 8 Y + 1 y y 0 Δ y = B 4 X + B 5 Y + B 6 B 7 X + B 8 Y + 1
x Δ x = C 1 X + C 2 Y + C 3 C 7 X + C 8 Y + 1 y Δ y = C 4 X + C 5 X + C 6 C 7 X + C 8 Y + 1
d x = x x d y = y y

Metrics