Abstract

This paper proposes an approach to rectifying two images of the same scene captured by cameras at general positions so that the results form a stereo pair that satisfies the constraints of the stereoscopic visualization platforms. This is unlike conventional image rectification research that primarily focuses on making stereo matching easier but pays little attention to 3D viewing. The novel derivation of the rectification algorithm also has an intuitive physical meaning that is not available from conventional approaches. Practical issues related to wide-baseline rectification and operation range of the proposed method are analyzed. Both simulated and real data experiments are used to assess the performance of the proposed algorithm.

© 2008 Optical Society of America

PDF Article

References

  • View by:
  • |
  • |
  • |

  1. P. May, “A survey of 3-D display technologies,” Inf. Disp. 32, 28-33 (2005).
  2. http://sharp-world.com/products/device/catalog/pdf/lcd200807_e.pdf (October 2008).
  3. A. Fusiello, E. Trucco, and A. Verri, “A compact algorithm for rectification of stereo pairs,” Mach. Vision Appl. 12, 16-22 (2000).
    [CrossRef]
  4. N. Ayache and C. Hansen, “Rectification of images for binocular and trinocular stereovision,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 1988), pp. 11-16.
  5. N. Ayache and C. Hansen, “Trinocular stereo vision for robotics,” IEEE Trans. Pattern Anal. Mach. Intell. 13, 73-85 (1991).
    [CrossRef]
  6. R. Hartley, “Theory and practice of projective rectification,” Int. J. Comput. Vis. 35, 1-16 (1999).
  7. F. Isgrò and E. Trucco, “On robust rectification for uncalibrated images,” in Proceedings of International Conference on Image Analysis and Processing (ICIAP, 1999), pp. 297-302.
  8. C. Loop and Z. Zhang, “Computing rectifying homographies for stereo vision,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1999), pp. 125-131.
  9. M. Pollefeys, R. Koch and L. Van Gool, “A simple and efficient rectification method for general motion,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 1999), pp. 496-501.
  10. Z. Chen, C. Wu, and H. T. Tsui, “A new image rectification algorithm,” Pattern Recogn. Lett. 24, 251-260 (2003).
    [CrossRef]
  11. D. V. Papadimitriou and T. J. Dennis, “Epipolar line estimation and rectification for stereo image pairs,” IEEE Trans. Image Process. 5, 672-676 (1996).
    [CrossRef]
  12. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge U. Press, 2003).
  13. H. H. P. Wu, Y. H. Yu, W. C. Chen, “Projective rectification based on relative modification and size extension for stereo image pairs,” IEE Proc. Vision Image Signal Process. 152623-633 (2005).
    [CrossRef]
  14. J. Mallon, and P. F. Whelan, “Projective rectification from the fundamental matrix,” Image Vis. Comput. 23, 643-650 (2005).
    [CrossRef]
  15. R. I. Hartley, “In defense of the eight-point algorithm,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 580-593 (1997).
    [CrossRef]
  16. Z. Zhang, “Determining the epipolar geometry and its uncertainty: a review,” Int. J. Comput. Vis. 27, 161-195 (1998).
    [CrossRef]
  17. L. B. Stelmach, W. J. Tam, F. Speranza, R. Renaud, and T. Martin, “Improving the visual comfort of stereoscopic images,” Proc. SPIE 5006, 269-282 (2003).
    [CrossRef]
  18. P. Neri, H. Bridge, and D. J. Heeger, “Stereoscopic processing of absolute and relative disparity in human visual cortex,” J. Neurophysiol. 92, 1880-1891 (2004).
  19. D. Regan, C. J. Erkelens, and H. Collewijn, “Necessary conditions for the perception of motion-in-depth,” Invest. Ophthalmol. Visual Sci. 27, 584-597 (1986).
  20. F. L. Kooi and A. Toet, “Visual comfort of binocular and 3D displays,” Displays 25, 99-108 (2004).
    [CrossRef]
  21. J. Zhou and B. Li, “Rectification with intersecting optical axes for stereoscopic visualization,” in Proceedings of International Conference on Pattern Recognition (IAPR, 2006), pp. 17-20.
  22. S. Pastoor, “Human factors of 3D imaging: results of recent research at Heinrich-Hertz-Institut Berlin,” in Proceedings of the International Display Workshop (IDW, 1995), pp. 66-72.
  23. http://www.public.asu.edu/~bli24/Rectification.html.
  24. C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. A. J. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation,” in Proceedings of SIGGRAPH (ACM, 2004), pp. 600-608.
  25. The Persistence of Vision Raytracer, http://www.povray.org.

2005 (3)

P. May, “A survey of 3-D display technologies,” Inf. Disp. 32, 28-33 (2005).

H. H. P. Wu, Y. H. Yu, W. C. Chen, “Projective rectification based on relative modification and size extension for stereo image pairs,” IEE Proc. Vision Image Signal Process. 152623-633 (2005).
[CrossRef]

J. Mallon, and P. F. Whelan, “Projective rectification from the fundamental matrix,” Image Vis. Comput. 23, 643-650 (2005).
[CrossRef]

2004 (2)

P. Neri, H. Bridge, and D. J. Heeger, “Stereoscopic processing of absolute and relative disparity in human visual cortex,” J. Neurophysiol. 92, 1880-1891 (2004).

F. L. Kooi and A. Toet, “Visual comfort of binocular and 3D displays,” Displays 25, 99-108 (2004).
[CrossRef]

2003 (2)

L. B. Stelmach, W. J. Tam, F. Speranza, R. Renaud, and T. Martin, “Improving the visual comfort of stereoscopic images,” Proc. SPIE 5006, 269-282 (2003).
[CrossRef]

Z. Chen, C. Wu, and H. T. Tsui, “A new image rectification algorithm,” Pattern Recogn. Lett. 24, 251-260 (2003).
[CrossRef]

2000 (1)

A. Fusiello, E. Trucco, and A. Verri, “A compact algorithm for rectification of stereo pairs,” Mach. Vision Appl. 12, 16-22 (2000).
[CrossRef]

1999 (1)

R. Hartley, “Theory and practice of projective rectification,” Int. J. Comput. Vis. 35, 1-16 (1999).

1998 (1)

Z. Zhang, “Determining the epipolar geometry and its uncertainty: a review,” Int. J. Comput. Vis. 27, 161-195 (1998).
[CrossRef]

1997 (1)

R. I. Hartley, “In defense of the eight-point algorithm,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 580-593 (1997).
[CrossRef]

1996 (1)

D. V. Papadimitriou and T. J. Dennis, “Epipolar line estimation and rectification for stereo image pairs,” IEEE Trans. Image Process. 5, 672-676 (1996).
[CrossRef]

1991 (1)

N. Ayache and C. Hansen, “Trinocular stereo vision for robotics,” IEEE Trans. Pattern Anal. Mach. Intell. 13, 73-85 (1991).
[CrossRef]

1986 (1)

D. Regan, C. J. Erkelens, and H. Collewijn, “Necessary conditions for the perception of motion-in-depth,” Invest. Ophthalmol. Visual Sci. 27, 584-597 (1986).

Ayache, N.

N. Ayache and C. Hansen, “Trinocular stereo vision for robotics,” IEEE Trans. Pattern Anal. Mach. Intell. 13, 73-85 (1991).
[CrossRef]

N. Ayache and C. Hansen, “Rectification of images for binocular and trinocular stereovision,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 1988), pp. 11-16.

Bridge, H.

P. Neri, H. Bridge, and D. J. Heeger, “Stereoscopic processing of absolute and relative disparity in human visual cortex,” J. Neurophysiol. 92, 1880-1891 (2004).

Chen, W. C.

H. H. P. Wu, Y. H. Yu, W. C. Chen, “Projective rectification based on relative modification and size extension for stereo image pairs,” IEE Proc. Vision Image Signal Process. 152623-633 (2005).
[CrossRef]

Chen, Z.

Z. Chen, C. Wu, and H. T. Tsui, “A new image rectification algorithm,” Pattern Recogn. Lett. 24, 251-260 (2003).
[CrossRef]

Collewijn, H.

D. Regan, C. J. Erkelens, and H. Collewijn, “Necessary conditions for the perception of motion-in-depth,” Invest. Ophthalmol. Visual Sci. 27, 584-597 (1986).

Dennis, T. J.

D. V. Papadimitriou and T. J. Dennis, “Epipolar line estimation and rectification for stereo image pairs,” IEEE Trans. Image Process. 5, 672-676 (1996).
[CrossRef]

Erkelens, C. J.

D. Regan, C. J. Erkelens, and H. Collewijn, “Necessary conditions for the perception of motion-in-depth,” Invest. Ophthalmol. Visual Sci. 27, 584-597 (1986).

Fusiello, A.

A. Fusiello, E. Trucco, and A. Verri, “A compact algorithm for rectification of stereo pairs,” Mach. Vision Appl. 12, 16-22 (2000).
[CrossRef]

Hansen, C.

N. Ayache and C. Hansen, “Trinocular stereo vision for robotics,” IEEE Trans. Pattern Anal. Mach. Intell. 13, 73-85 (1991).
[CrossRef]

N. Ayache and C. Hansen, “Rectification of images for binocular and trinocular stereovision,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 1988), pp. 11-16.

Hartley, R.

R. Hartley, “Theory and practice of projective rectification,” Int. J. Comput. Vis. 35, 1-16 (1999).

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge U. Press, 2003).

Hartley, R. I.

R. I. Hartley, “In defense of the eight-point algorithm,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 580-593 (1997).
[CrossRef]

Heeger, D. J.

P. Neri, H. Bridge, and D. J. Heeger, “Stereoscopic processing of absolute and relative disparity in human visual cortex,” J. Neurophysiol. 92, 1880-1891 (2004).

Isgrò, F.

F. Isgrò and E. Trucco, “On robust rectification for uncalibrated images,” in Proceedings of International Conference on Image Analysis and Processing (ICIAP, 1999), pp. 297-302.

Kang, S. B.

C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. A. J. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation,” in Proceedings of SIGGRAPH (ACM, 2004), pp. 600-608.

Koch, R.

M. Pollefeys, R. Koch and L. Van Gool, “A simple and efficient rectification method for general motion,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 1999), pp. 496-501.

Kooi, F. L.

F. L. Kooi and A. Toet, “Visual comfort of binocular and 3D displays,” Displays 25, 99-108 (2004).
[CrossRef]

Li, B.

J. Zhou and B. Li, “Rectification with intersecting optical axes for stereoscopic visualization,” in Proceedings of International Conference on Pattern Recognition (IAPR, 2006), pp. 17-20.

Loop, C.

C. Loop and Z. Zhang, “Computing rectifying homographies for stereo vision,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1999), pp. 125-131.

Mallon, J.

J. Mallon, and P. F. Whelan, “Projective rectification from the fundamental matrix,” Image Vis. Comput. 23, 643-650 (2005).
[CrossRef]

Martin, T.

L. B. Stelmach, W. J. Tam, F. Speranza, R. Renaud, and T. Martin, “Improving the visual comfort of stereoscopic images,” Proc. SPIE 5006, 269-282 (2003).
[CrossRef]

May, P.

P. May, “A survey of 3-D display technologies,” Inf. Disp. 32, 28-33 (2005).

Neri, P.

P. Neri, H. Bridge, and D. J. Heeger, “Stereoscopic processing of absolute and relative disparity in human visual cortex,” J. Neurophysiol. 92, 1880-1891 (2004).

Papadimitriou, D. V.

D. V. Papadimitriou and T. J. Dennis, “Epipolar line estimation and rectification for stereo image pairs,” IEEE Trans. Image Process. 5, 672-676 (1996).
[CrossRef]

Pastoor, S.

S. Pastoor, “Human factors of 3D imaging: results of recent research at Heinrich-Hertz-Institut Berlin,” in Proceedings of the International Display Workshop (IDW, 1995), pp. 66-72.

Pollefeys, M.

M. Pollefeys, R. Koch and L. Van Gool, “A simple and efficient rectification method for general motion,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 1999), pp. 496-501.

Regan, D.

D. Regan, C. J. Erkelens, and H. Collewijn, “Necessary conditions for the perception of motion-in-depth,” Invest. Ophthalmol. Visual Sci. 27, 584-597 (1986).

Renaud, R.

L. B. Stelmach, W. J. Tam, F. Speranza, R. Renaud, and T. Martin, “Improving the visual comfort of stereoscopic images,” Proc. SPIE 5006, 269-282 (2003).
[CrossRef]

Speranza, F.

L. B. Stelmach, W. J. Tam, F. Speranza, R. Renaud, and T. Martin, “Improving the visual comfort of stereoscopic images,” Proc. SPIE 5006, 269-282 (2003).
[CrossRef]

Stelmach, L. B.

L. B. Stelmach, W. J. Tam, F. Speranza, R. Renaud, and T. Martin, “Improving the visual comfort of stereoscopic images,” Proc. SPIE 5006, 269-282 (2003).
[CrossRef]

Szeliski, R.

C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. A. J. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation,” in Proceedings of SIGGRAPH (ACM, 2004), pp. 600-608.

Tam, W. J.

L. B. Stelmach, W. J. Tam, F. Speranza, R. Renaud, and T. Martin, “Improving the visual comfort of stereoscopic images,” Proc. SPIE 5006, 269-282 (2003).
[CrossRef]

Toet, A.

F. L. Kooi and A. Toet, “Visual comfort of binocular and 3D displays,” Displays 25, 99-108 (2004).
[CrossRef]

Trucco, E.

A. Fusiello, E. Trucco, and A. Verri, “A compact algorithm for rectification of stereo pairs,” Mach. Vision Appl. 12, 16-22 (2000).
[CrossRef]

F. Isgrò and E. Trucco, “On robust rectification for uncalibrated images,” in Proceedings of International Conference on Image Analysis and Processing (ICIAP, 1999), pp. 297-302.

Tsui, H. T.

Z. Chen, C. Wu, and H. T. Tsui, “A new image rectification algorithm,” Pattern Recogn. Lett. 24, 251-260 (2003).
[CrossRef]

Uyttendaele, M.

C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. A. J. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation,” in Proceedings of SIGGRAPH (ACM, 2004), pp. 600-608.

Van Gool, L.

M. Pollefeys, R. Koch and L. Van Gool, “A simple and efficient rectification method for general motion,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 1999), pp. 496-501.

Verri, A.

A. Fusiello, E. Trucco, and A. Verri, “A compact algorithm for rectification of stereo pairs,” Mach. Vision Appl. 12, 16-22 (2000).
[CrossRef]

Whelan, P. F.

J. Mallon, and P. F. Whelan, “Projective rectification from the fundamental matrix,” Image Vis. Comput. 23, 643-650 (2005).
[CrossRef]

Winder, S. A. J.

C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. A. J. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation,” in Proceedings of SIGGRAPH (ACM, 2004), pp. 600-608.

Wu, C.

Z. Chen, C. Wu, and H. T. Tsui, “A new image rectification algorithm,” Pattern Recogn. Lett. 24, 251-260 (2003).
[CrossRef]

Wu, H. H. P.

H. H. P. Wu, Y. H. Yu, W. C. Chen, “Projective rectification based on relative modification and size extension for stereo image pairs,” IEE Proc. Vision Image Signal Process. 152623-633 (2005).
[CrossRef]

Yu, Y. H.

H. H. P. Wu, Y. H. Yu, W. C. Chen, “Projective rectification based on relative modification and size extension for stereo image pairs,” IEE Proc. Vision Image Signal Process. 152623-633 (2005).
[CrossRef]

Zhang, Z.

Z. Zhang, “Determining the epipolar geometry and its uncertainty: a review,” Int. J. Comput. Vis. 27, 161-195 (1998).
[CrossRef]

C. Loop and Z. Zhang, “Computing rectifying homographies for stereo vision,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1999), pp. 125-131.

Zhou, J.

J. Zhou and B. Li, “Rectification with intersecting optical axes for stereoscopic visualization,” in Proceedings of International Conference on Pattern Recognition (IAPR, 2006), pp. 17-20.

Zisserman, A.

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge U. Press, 2003).

Zitnick, C. L.

C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. A. J. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation,” in Proceedings of SIGGRAPH (ACM, 2004), pp. 600-608.

Displays (1)

F. L. Kooi and A. Toet, “Visual comfort of binocular and 3D displays,” Displays 25, 99-108 (2004).
[CrossRef]

IEE Proc. Vision Image Signal Process. (1)

H. H. P. Wu, Y. H. Yu, W. C. Chen, “Projective rectification based on relative modification and size extension for stereo image pairs,” IEE Proc. Vision Image Signal Process. 152623-633 (2005).
[CrossRef]

IEEE Trans. Image Process. (1)

D. V. Papadimitriou and T. J. Dennis, “Epipolar line estimation and rectification for stereo image pairs,” IEEE Trans. Image Process. 5, 672-676 (1996).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (2)

R. I. Hartley, “In defense of the eight-point algorithm,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 580-593 (1997).
[CrossRef]

N. Ayache and C. Hansen, “Trinocular stereo vision for robotics,” IEEE Trans. Pattern Anal. Mach. Intell. 13, 73-85 (1991).
[CrossRef]

Image Vis. Comput. (1)

J. Mallon, and P. F. Whelan, “Projective rectification from the fundamental matrix,” Image Vis. Comput. 23, 643-650 (2005).
[CrossRef]

Inf. Disp. (1)

P. May, “A survey of 3-D display technologies,” Inf. Disp. 32, 28-33 (2005).

Int. J. Comput. Vis. (2)

R. Hartley, “Theory and practice of projective rectification,” Int. J. Comput. Vis. 35, 1-16 (1999).

Z. Zhang, “Determining the epipolar geometry and its uncertainty: a review,” Int. J. Comput. Vis. 27, 161-195 (1998).
[CrossRef]

Invest. Ophthalmol. Visual Sci. (1)

D. Regan, C. J. Erkelens, and H. Collewijn, “Necessary conditions for the perception of motion-in-depth,” Invest. Ophthalmol. Visual Sci. 27, 584-597 (1986).

J. Neurophysiol. (1)

P. Neri, H. Bridge, and D. J. Heeger, “Stereoscopic processing of absolute and relative disparity in human visual cortex,” J. Neurophysiol. 92, 1880-1891 (2004).

Mach. Vision Appl. (1)

A. Fusiello, E. Trucco, and A. Verri, “A compact algorithm for rectification of stereo pairs,” Mach. Vision Appl. 12, 16-22 (2000).
[CrossRef]

Pattern Recogn. Lett. (1)

Z. Chen, C. Wu, and H. T. Tsui, “A new image rectification algorithm,” Pattern Recogn. Lett. 24, 251-260 (2003).
[CrossRef]

Proc. SPIE (1)

L. B. Stelmach, W. J. Tam, F. Speranza, R. Renaud, and T. Martin, “Improving the visual comfort of stereoscopic images,” Proc. SPIE 5006, 269-282 (2003).
[CrossRef]

Other (11)

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge U. Press, 2003).

N. Ayache and C. Hansen, “Rectification of images for binocular and trinocular stereovision,” in Proceedings of IEEE International Conference on Pattern Recognition (IEEE, 1988), pp. 11-16.

http://sharp-world.com/products/device/catalog/pdf/lcd200807_e.pdf (October 2008).

F. Isgrò and E. Trucco, “On robust rectification for uncalibrated images,” in Proceedings of International Conference on Image Analysis and Processing (ICIAP, 1999), pp. 297-302.

C. Loop and Z. Zhang, “Computing rectifying homographies for stereo vision,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1999), pp. 125-131.

M. Pollefeys, R. Koch and L. Van Gool, “A simple and efficient rectification method for general motion,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 1999), pp. 496-501.

J. Zhou and B. Li, “Rectification with intersecting optical axes for stereoscopic visualization,” in Proceedings of International Conference on Pattern Recognition (IAPR, 2006), pp. 17-20.

S. Pastoor, “Human factors of 3D imaging: results of recent research at Heinrich-Hertz-Institut Berlin,” in Proceedings of the International Display Workshop (IDW, 1995), pp. 66-72.

http://www.public.asu.edu/~bli24/Rectification.html.

C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. A. J. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation,” in Proceedings of SIGGRAPH (ACM, 2004), pp. 600-608.

The Persistence of Vision Raytracer, http://www.povray.org.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Metrics