Abstract

There is an increasing interest in wide-angle imaging of the environment using curved reflective surfaces. With this comes the need for appropriate filtering and processing of the acquired images. Here we present a technique for homogeneous, fast filtering of panoramic images captured using a camera and a wide-angle-imaging reflective surface. Imaging of the panoramic environment onto a two-dimensional (2-D) plane necessarily introduces spatial distortions such as stretching and bending that vary with the viewing direction. Therefore, if the panoramic image is to be filtered homogeneously in all viewing directions, it is necessary to match the filtering to the distortions. We show how this can be accomplished. The image acquired by the camera is first digitally unwarped and represented in Cartesian coordinates representing azimuth and elevation. The mappings of patches of uniform size and shape on the viewsphere are then established. Next, for each filter patch the local mappings of great circles along two principal axes (along the local longitudinal and elevational directions) on the image plane are determined. The mappings of these great circles are used to perform the 2-D convolution required by the filtering operation. Convolution along the directions of local, mutually orthogonal great circles permits the filtering to be carried out in a quasi-separable fashion, resulting in increased computational speed and efficiency. Examples of homogeneous filtering using this procedure are presented.

© 2000 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. Y. Yagi, W. Nishii, K. Yamazawa, M. Yachida, “Stabilization for mobile robot by using omnidirectional optic flow,” in Proceedings of the International Conference on Intelligent Robots and Systems, 1996 (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 618–624.
  2. D. Wettergreen, M. Bualat, D. Christian, K. Schwehr, H. Thomas, D. Tucker, E. Zbinden, “Operating Nomad during the Atacama Desert trek,” in Proceedings of the International Conference on Field and Service Robotics, 1997 (Panther, Canberra, Australia, 1997), pp. 80–87.
  3. J. Hong, X. Tan, B. Pinette, R. Weiss, E. M. Riseman, “Image-based homing,” in Proceedings of the International Conference on Robotics and Automation (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1991), pp. 620–625.
  4. J. S. Chahl, M. V. Srinivasan, “Reflective surfaces for panoramic imaging,” Appl. Opt. 36, 8275–8285 (1997).
    [CrossRef]
  5. S. Nayar, “Omnidirectional video camera,” in Proceedings of the 1997 DARPA Image Understanding Workshop (Morgan Kaufmann, Los Altos, Calif., 1997), pp. 1431–1437.
  6. S. Bogner, D. Southwell, S. Penzes, C. Brosinsky, R. Anderson, “Progress in video immersion using panospheric imaging,” in Cockpit Dispalys V: Displays for Defense Applications, D. G. Hopper, ed., Proc. SPIE3363, 394–406 (1998).
  7. M. Ollis, H. Herman, S. Singh, “Analysis and design of panoramic stereo vision vision using equi-angular pixel cameras,” (Carnegie Mellon University, Pittsburgh, Pa., 1999).
  8. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, 1968).
  9. D. H. Ballard, C. M. Brown, Computer Vision (Prentice-Hall, Englewood Cliffs, N.J., 1982).
  10. G. A. Horridge, “The compound eye of insects,” Sci. Am. 237, 108–120 (1977).
    [CrossRef]
  11. K. Skifstad, R. Jain, “Range estimation from intensity gradient analysis,” Mach. Vision Appl. 2, 81–102 (1989).
    [CrossRef]
  12. M. V. Srinivasan, “Generalised gradient schemes for the measurement of two-dimensional image motion,” Biol. Cybern. 63, 421–431 (1990).
    [CrossRef]
  13. M. V. Srinivasan, “An image interpolation technique for the computation of optic flow and egomotion,” Biol. Cybern. 71, 401–415 (1994).
    [CrossRef]

1997

1994

M. V. Srinivasan, “An image interpolation technique for the computation of optic flow and egomotion,” Biol. Cybern. 71, 401–415 (1994).
[CrossRef]

1990

M. V. Srinivasan, “Generalised gradient schemes for the measurement of two-dimensional image motion,” Biol. Cybern. 63, 421–431 (1990).
[CrossRef]

1989

K. Skifstad, R. Jain, “Range estimation from intensity gradient analysis,” Mach. Vision Appl. 2, 81–102 (1989).
[CrossRef]

1977

G. A. Horridge, “The compound eye of insects,” Sci. Am. 237, 108–120 (1977).
[CrossRef]

Anderson, R.

S. Bogner, D. Southwell, S. Penzes, C. Brosinsky, R. Anderson, “Progress in video immersion using panospheric imaging,” in Cockpit Dispalys V: Displays for Defense Applications, D. G. Hopper, ed., Proc. SPIE3363, 394–406 (1998).

Ballard, D. H.

D. H. Ballard, C. M. Brown, Computer Vision (Prentice-Hall, Englewood Cliffs, N.J., 1982).

Bogner, S.

S. Bogner, D. Southwell, S. Penzes, C. Brosinsky, R. Anderson, “Progress in video immersion using panospheric imaging,” in Cockpit Dispalys V: Displays for Defense Applications, D. G. Hopper, ed., Proc. SPIE3363, 394–406 (1998).

Brosinsky, C.

S. Bogner, D. Southwell, S. Penzes, C. Brosinsky, R. Anderson, “Progress in video immersion using panospheric imaging,” in Cockpit Dispalys V: Displays for Defense Applications, D. G. Hopper, ed., Proc. SPIE3363, 394–406 (1998).

Brown, C. M.

D. H. Ballard, C. M. Brown, Computer Vision (Prentice-Hall, Englewood Cliffs, N.J., 1982).

Bualat, M.

D. Wettergreen, M. Bualat, D. Christian, K. Schwehr, H. Thomas, D. Tucker, E. Zbinden, “Operating Nomad during the Atacama Desert trek,” in Proceedings of the International Conference on Field and Service Robotics, 1997 (Panther, Canberra, Australia, 1997), pp. 80–87.

Chahl, J. S.

Christian, D.

D. Wettergreen, M. Bualat, D. Christian, K. Schwehr, H. Thomas, D. Tucker, E. Zbinden, “Operating Nomad during the Atacama Desert trek,” in Proceedings of the International Conference on Field and Service Robotics, 1997 (Panther, Canberra, Australia, 1997), pp. 80–87.

Goodman, J. W.

J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, 1968).

Herman, H.

M. Ollis, H. Herman, S. Singh, “Analysis and design of panoramic stereo vision vision using equi-angular pixel cameras,” (Carnegie Mellon University, Pittsburgh, Pa., 1999).

Hong, J.

J. Hong, X. Tan, B. Pinette, R. Weiss, E. M. Riseman, “Image-based homing,” in Proceedings of the International Conference on Robotics and Automation (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1991), pp. 620–625.

Horridge, G. A.

G. A. Horridge, “The compound eye of insects,” Sci. Am. 237, 108–120 (1977).
[CrossRef]

Jain, R.

K. Skifstad, R. Jain, “Range estimation from intensity gradient analysis,” Mach. Vision Appl. 2, 81–102 (1989).
[CrossRef]

Nayar, S.

S. Nayar, “Omnidirectional video camera,” in Proceedings of the 1997 DARPA Image Understanding Workshop (Morgan Kaufmann, Los Altos, Calif., 1997), pp. 1431–1437.

Nishii, W.

Y. Yagi, W. Nishii, K. Yamazawa, M. Yachida, “Stabilization for mobile robot by using omnidirectional optic flow,” in Proceedings of the International Conference on Intelligent Robots and Systems, 1996 (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 618–624.

Ollis, M.

M. Ollis, H. Herman, S. Singh, “Analysis and design of panoramic stereo vision vision using equi-angular pixel cameras,” (Carnegie Mellon University, Pittsburgh, Pa., 1999).

Penzes, S.

S. Bogner, D. Southwell, S. Penzes, C. Brosinsky, R. Anderson, “Progress in video immersion using panospheric imaging,” in Cockpit Dispalys V: Displays for Defense Applications, D. G. Hopper, ed., Proc. SPIE3363, 394–406 (1998).

Pinette, B.

J. Hong, X. Tan, B. Pinette, R. Weiss, E. M. Riseman, “Image-based homing,” in Proceedings of the International Conference on Robotics and Automation (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1991), pp. 620–625.

Riseman, E. M.

J. Hong, X. Tan, B. Pinette, R. Weiss, E. M. Riseman, “Image-based homing,” in Proceedings of the International Conference on Robotics and Automation (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1991), pp. 620–625.

Schwehr, K.

D. Wettergreen, M. Bualat, D. Christian, K. Schwehr, H. Thomas, D. Tucker, E. Zbinden, “Operating Nomad during the Atacama Desert trek,” in Proceedings of the International Conference on Field and Service Robotics, 1997 (Panther, Canberra, Australia, 1997), pp. 80–87.

Singh, S.

M. Ollis, H. Herman, S. Singh, “Analysis and design of panoramic stereo vision vision using equi-angular pixel cameras,” (Carnegie Mellon University, Pittsburgh, Pa., 1999).

Skifstad, K.

K. Skifstad, R. Jain, “Range estimation from intensity gradient analysis,” Mach. Vision Appl. 2, 81–102 (1989).
[CrossRef]

Southwell, D.

S. Bogner, D. Southwell, S. Penzes, C. Brosinsky, R. Anderson, “Progress in video immersion using panospheric imaging,” in Cockpit Dispalys V: Displays for Defense Applications, D. G. Hopper, ed., Proc. SPIE3363, 394–406 (1998).

Srinivasan, M. V.

J. S. Chahl, M. V. Srinivasan, “Reflective surfaces for panoramic imaging,” Appl. Opt. 36, 8275–8285 (1997).
[CrossRef]

M. V. Srinivasan, “An image interpolation technique for the computation of optic flow and egomotion,” Biol. Cybern. 71, 401–415 (1994).
[CrossRef]

M. V. Srinivasan, “Generalised gradient schemes for the measurement of two-dimensional image motion,” Biol. Cybern. 63, 421–431 (1990).
[CrossRef]

Tan, X.

J. Hong, X. Tan, B. Pinette, R. Weiss, E. M. Riseman, “Image-based homing,” in Proceedings of the International Conference on Robotics and Automation (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1991), pp. 620–625.

Thomas, H.

D. Wettergreen, M. Bualat, D. Christian, K. Schwehr, H. Thomas, D. Tucker, E. Zbinden, “Operating Nomad during the Atacama Desert trek,” in Proceedings of the International Conference on Field and Service Robotics, 1997 (Panther, Canberra, Australia, 1997), pp. 80–87.

Tucker, D.

D. Wettergreen, M. Bualat, D. Christian, K. Schwehr, H. Thomas, D. Tucker, E. Zbinden, “Operating Nomad during the Atacama Desert trek,” in Proceedings of the International Conference on Field and Service Robotics, 1997 (Panther, Canberra, Australia, 1997), pp. 80–87.

Weiss, R.

J. Hong, X. Tan, B. Pinette, R. Weiss, E. M. Riseman, “Image-based homing,” in Proceedings of the International Conference on Robotics and Automation (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1991), pp. 620–625.

Wettergreen, D.

D. Wettergreen, M. Bualat, D. Christian, K. Schwehr, H. Thomas, D. Tucker, E. Zbinden, “Operating Nomad during the Atacama Desert trek,” in Proceedings of the International Conference on Field and Service Robotics, 1997 (Panther, Canberra, Australia, 1997), pp. 80–87.

Yachida, M.

Y. Yagi, W. Nishii, K. Yamazawa, M. Yachida, “Stabilization for mobile robot by using omnidirectional optic flow,” in Proceedings of the International Conference on Intelligent Robots and Systems, 1996 (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 618–624.

Yagi, Y.

Y. Yagi, W. Nishii, K. Yamazawa, M. Yachida, “Stabilization for mobile robot by using omnidirectional optic flow,” in Proceedings of the International Conference on Intelligent Robots and Systems, 1996 (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 618–624.

Yamazawa, K.

Y. Yagi, W. Nishii, K. Yamazawa, M. Yachida, “Stabilization for mobile robot by using omnidirectional optic flow,” in Proceedings of the International Conference on Intelligent Robots and Systems, 1996 (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 618–624.

Zbinden, E.

D. Wettergreen, M. Bualat, D. Christian, K. Schwehr, H. Thomas, D. Tucker, E. Zbinden, “Operating Nomad during the Atacama Desert trek,” in Proceedings of the International Conference on Field and Service Robotics, 1997 (Panther, Canberra, Australia, 1997), pp. 80–87.

Appl. Opt.

Biol. Cybern.

M. V. Srinivasan, “Generalised gradient schemes for the measurement of two-dimensional image motion,” Biol. Cybern. 63, 421–431 (1990).
[CrossRef]

M. V. Srinivasan, “An image interpolation technique for the computation of optic flow and egomotion,” Biol. Cybern. 71, 401–415 (1994).
[CrossRef]

Mach. Vision Appl.

K. Skifstad, R. Jain, “Range estimation from intensity gradient analysis,” Mach. Vision Appl. 2, 81–102 (1989).
[CrossRef]

Sci. Am.

G. A. Horridge, “The compound eye of insects,” Sci. Am. 237, 108–120 (1977).
[CrossRef]

Other

Y. Yagi, W. Nishii, K. Yamazawa, M. Yachida, “Stabilization for mobile robot by using omnidirectional optic flow,” in Proceedings of the International Conference on Intelligent Robots and Systems, 1996 (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 618–624.

D. Wettergreen, M. Bualat, D. Christian, K. Schwehr, H. Thomas, D. Tucker, E. Zbinden, “Operating Nomad during the Atacama Desert trek,” in Proceedings of the International Conference on Field and Service Robotics, 1997 (Panther, Canberra, Australia, 1997), pp. 80–87.

J. Hong, X. Tan, B. Pinette, R. Weiss, E. M. Riseman, “Image-based homing,” in Proceedings of the International Conference on Robotics and Automation (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1991), pp. 620–625.

S. Nayar, “Omnidirectional video camera,” in Proceedings of the 1997 DARPA Image Understanding Workshop (Morgan Kaufmann, Los Altos, Calif., 1997), pp. 1431–1437.

S. Bogner, D. Southwell, S. Penzes, C. Brosinsky, R. Anderson, “Progress in video immersion using panospheric imaging,” in Cockpit Dispalys V: Displays for Defense Applications, D. G. Hopper, ed., Proc. SPIE3363, 394–406 (1998).

M. Ollis, H. Herman, S. Singh, “Analysis and design of panoramic stereo vision vision using equi-angular pixel cameras,” (Carnegie Mellon University, Pittsburgh, Pa., 1999).

J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, 1968).

D. H. Ballard, C. M. Brown, Computer Vision (Prentice-Hall, Englewood Cliffs, N.J., 1982).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1

(a) Basic metrics of the panoramic imaging system. (b) Implementation.

Fig. 2
Fig. 2

(a) Panoramic image captured from a reflective surface of gain 7 using a 35-mm camera. (b) Digitally unwarped image after a rectangular-to-polar coordinate transform of the image in (a).

Fig. 3
Fig. 3

(a) White, homogeneous filtering of the imaging plane; black, attempted correction for nonuniform sampling by keeping the azimuthal width of each filter equal. (b) Both techniques are unsuccessful, with filtering being highly inhomogeneous when seen on the world sphere.

Fig. 4
Fig. 4

(a) Filters must be of constant size and shape in the viewsphere regardless of angle of elevation or azimuth. (b) Resulting filters map onto the image plane as shown. (c) In the unwarped image the filters are of constant shape for any given angle of elevation, with elevational expanse constant for each filter.

Fig. 5
Fig. 5

Mapping of the filters in the unwarped image shows that the shape of the filters is invariant to rotation along the azimuthal axis with this representation of the image, although their shape changes for different elevations. All filters have the same elevational expanse because of the properties of the reflective surfaces used. The individual curves to the left-hand side of the image show examples of mappings of the individual great arcs that form the horizontal axis of the filters. The shape of the mappings of the great arcs varies with elevation but not with azimuth. The advantage of this representation is that computationally costly trigonometric computations need not be performed.

Fig. 6
Fig. 6

(a) Original image. The image resulting from homogeneous, separable filtering of the image with (b) an averaging (square) filter of width 5°, (c) a Gaussian filter of standard deviation 5°, and (d) a DOG filter of negative component 5° and positive component 2°.

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

r6=r06cos 6η,
θ=θc+tan-1[tan(θc+u)/cos ϕc],
ϕ=cos-1[cos(θc+u)sin ϕc],
θ=θc,ϕ=ϕc+v,

Metrics