Abstract

We propose a new method for rectifying a geometrical distortion in the elemental image set and extracting an accurate lens lattice lines by projective image transformation. The information of distortion in the acquired elemental image set is found by Hough transform algorithm. With this initial information of distortions, the acquired elemental image set is rectified automatically without the prior knowledge on the characteristics of pickup system by stratified image transformation procedure. Computer-generated elemental image sets with distortion on purpose are used for verifying the proposed rectification method. Experimentally-captured elemental image sets are optically reconstructed before and after the rectification by the proposed method. The experimental results support the validity of the proposed method with high accuracy of image rectification and lattice extraction.

© 2010 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. G. Lippmann, “La photographie integrále,” C.R. Acad. Sci Ser. IIc: Chim. 146, 446–451 (1908).
  2. B. Lee, J.-H. Park, and S.-W. Min, Digital Holography and Three-Dimensional Display, T.-C. Poon, ed. (Springer US, 2006), Chap. 12.
  3. J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
    [CrossRef] [PubMed]
  4. M. C. Forman, N. Davies, and M. McCormick, “Continuous parallax in discrete pixilated integral three-dimensional displays,” J. Opt. Soc. Am. A 20(3), 411–420 (2003).
    [CrossRef]
  5. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997).
    [CrossRef] [PubMed]
  6. J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, “Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification,” Appl. Opt. 43(25), 4882–4895 (2004).
    [CrossRef] [PubMed]
  7. J.-H. Park, S. Jung, H. Choi, and B. Lee, “Detection of the longitudinal and the lateral positions of a three-dimensional object using a lens array and joint transform correlator,” Opt. Mem. Neural Networks. 11, 181–188 (2002).
  8. G. Passalis, N. Sgouros, S. Athineos, and T. Theoharis, “Enhanced reconstruction of three-dimensional shape and texture from integral photography images,” Appl. Opt. 46(22), 5311–5320 (2007).
    [CrossRef] [PubMed]
  9. D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D object using a depth conversion technique,” J. Opt. Soc. Korea 12(3), 131–135 (2008).
    [CrossRef]
  10. M. Kawakita, H. Sasaki, J. Arai, F. Okano, K. Suehiro, Y. Haino, M. Yoshimura, and M. Sato, “Geometric analysis of spatial distortion in projection-type integral imaging,” Opt. Lett. 33(7), 684–686 (2008).
    [CrossRef] [PubMed]
  11. J. Arai, M. Okui, M. Kobayashi, and F. Okano, “Geometrical effects of positional errors in integral photography,” J. Opt. Soc. Am. A 21(6), 951–958 (2004).
    [CrossRef]
  12. A. Aggoun, “Pre-processing of integral images for 3-D displays,” J. Display Technol. 2(4), 393–400 (2006).
    [CrossRef]
  13. N. P. Sgouros, S. S. Athineos, M. S. Sangriotis, P. G. Papageorgas, and N. G. Theofanous, “Accurate lattice extraction in integral images,” Opt. Express 14(22), 10403–10409 (2006).
    [CrossRef] [PubMed]
  14. J.-J. Lee, D.-H. Shin, and B.-G. Lee, “Simple correction method of distorted elemental images using surface markers on lenslet array for computational integral imaging reconstruction,” Opt. Express 17(20), 18026–18037 (2009).
    [CrossRef] [PubMed]
  15. R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital Image Processing using MATLAB (Prentice Hall, 2004), Chap. 10.
  16. D. Liebowitz, and A. Zisserman, “Metric Rectification for Perspective Images of Planes,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Santa Barbara, CA, USA, June 23–25, 1998), p.482.
  17. R. Hartley, and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. (Cambridge University Press, Cambridge, 2000).
  18. J. Delon, A. Desolneux, J.-L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE Trans. Image Process. 16(1), 253–261 (2007).
    [CrossRef] [PubMed]
  19. J. F. Canny, “A computational approach for edge detection,” Trans. Pat. Anal. Mach. Intell. 8, 679–698 (1986).
  20. http://epaperpress.com/ptlens/ .

2009 (2)

2008 (2)

2007 (2)

G. Passalis, N. Sgouros, S. Athineos, and T. Theoharis, “Enhanced reconstruction of three-dimensional shape and texture from integral photography images,” Appl. Opt. 46(22), 5311–5320 (2007).
[CrossRef] [PubMed]

J. Delon, A. Desolneux, J.-L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE Trans. Image Process. 16(1), 253–261 (2007).
[CrossRef] [PubMed]

2006 (2)

2004 (2)

2003 (1)

2002 (1)

J.-H. Park, S. Jung, H. Choi, and B. Lee, “Detection of the longitudinal and the lateral positions of a three-dimensional object using a lens array and joint transform correlator,” Opt. Mem. Neural Networks. 11, 181–188 (2002).

1997 (1)

1908 (1)

G. Lippmann, “La photographie integrále,” C.R. Acad. Sci Ser. IIc: Chim. 146, 446–451 (1908).

Aggoun, A.

Arai, J.

Athineos, S.

Athineos, S. S.

Choi, H.

J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, “Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification,” Appl. Opt. 43(25), 4882–4895 (2004).
[CrossRef] [PubMed]

J.-H. Park, S. Jung, H. Choi, and B. Lee, “Detection of the longitudinal and the lateral positions of a three-dimensional object using a lens array and joint transform correlator,” Opt. Mem. Neural Networks. 11, 181–188 (2002).

Davies, N.

Delon, J.

J. Delon, A. Desolneux, J.-L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE Trans. Image Process. 16(1), 253–261 (2007).
[CrossRef] [PubMed]

Desolneux, A.

J. Delon, A. Desolneux, J.-L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE Trans. Image Process. 16(1), 253–261 (2007).
[CrossRef] [PubMed]

Forman, M. C.

Haino, Y.

Hong, K.

Hoshino, H.

Jung, S.

J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, “Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification,” Appl. Opt. 43(25), 4882–4895 (2004).
[CrossRef] [PubMed]

J.-H. Park, S. Jung, H. Choi, and B. Lee, “Detection of the longitudinal and the lateral positions of a three-dimensional object using a lens array and joint transform correlator,” Opt. Mem. Neural Networks. 11, 181–188 (2002).

Kawakita, M.

Kim, E.-S.

Kim, Y.

Kobayashi, M.

Lee, B.

Lee, B.-G.

Lee, J.-J.

Lippmann, G.

G. Lippmann, “La photographie integrále,” C.R. Acad. Sci Ser. IIc: Chim. 146, 446–451 (1908).

Lisani, J.-L.

J. Delon, A. Desolneux, J.-L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE Trans. Image Process. 16(1), 253–261 (2007).
[CrossRef] [PubMed]

McCormick, M.

Okano, F.

Okui, M.

Papageorgas, P. G.

Park, J.-H.

Passalis, G.

Petro, A. B.

J. Delon, A. Desolneux, J.-L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE Trans. Image Process. 16(1), 253–261 (2007).
[CrossRef] [PubMed]

Sangriotis, M. S.

Sasaki, H.

Sato, M.

Sgouros, N.

Sgouros, N. P.

Shin, D.-H.

Suehiro, K.

Theofanous, N. G.

Theoharis, T.

Yoshimura, M.

Yuyama, I.

Appl. Opt. (4)

C.R. Acad. Sci Ser. IIc: Chim. (1)

G. Lippmann, “La photographie integrále,” C.R. Acad. Sci Ser. IIc: Chim. 146, 446–451 (1908).

IEEE Trans. Image Process. (1)

J. Delon, A. Desolneux, J.-L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE Trans. Image Process. 16(1), 253–261 (2007).
[CrossRef] [PubMed]

J. Display Technol. (1)

J. Opt. Soc. Am. A (2)

J. Opt. Soc. Korea (1)

Opt. Express (2)

Opt. Lett. (1)

Opt. Mem. Neural Networks. (1)

J.-H. Park, S. Jung, H. Choi, and B. Lee, “Detection of the longitudinal and the lateral positions of a three-dimensional object using a lens array and joint transform correlator,” Opt. Mem. Neural Networks. 11, 181–188 (2002).

Other (6)

B. Lee, J.-H. Park, and S.-W. Min, Digital Holography and Three-Dimensional Display, T.-C. Poon, ed. (Springer US, 2006), Chap. 12.

R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital Image Processing using MATLAB (Prentice Hall, 2004), Chap. 10.

D. Liebowitz, and A. Zisserman, “Metric Rectification for Perspective Images of Planes,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Santa Barbara, CA, USA, June 23–25, 1998), p.482.

R. Hartley, and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. (Cambridge University Press, Cambridge, 2000).

J. F. Canny, “A computational approach for edge detection,” Trans. Pat. Anal. Mach. Intell. 8, 679–698 (1986).

http://epaperpress.com/ptlens/ .

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1

Integral imaging which is sequentially processed in accordance with two procedures of (a) pickup and (b) reconstruction.

Fig. 2
Fig. 2

Stratification of the projective image transformation into a cascade process of pure-projective (Hp ), affine (Ha ), and similarity (Hs ) transforms.

Fig. 3
Fig. 3

Preprocessing images of (a) an object, (b) a distorted elemental image set acquired by known projective transformation matrices, and (c) a tetragonal edge image with four peak lines of Hough transform.

Fig. 4
Fig. 4

Correction procedures by rectifications (a) of the pure-projective distortion into an affine geometry elemental image set with a parallelogram shape, and (b) of the affine distortion into a metric geometry elemental image set with a rectangular shape.

Fig. 5
Fig. 5

(a) A projective profile of the compensated elemental image set along the longitudinal direction in units of pixel and (b) an interpolated projective profile by a factor of 10 plotted with an impulse train of detected lens pitch

Fig. 6
Fig. 6

A rectified elemental image set with extracted lens lattice lines on it, which is obtained by the proposed method of projective image transformation.

Fig. 7
Fig. 7

Photograph and optical arrangement of the experimental setup for picking up the 3D object images.

Fig. 8
Fig. 8

(a) 3D objects of a textured box (up) and a letter ‘2’ candle (down) used in the experiments, and (b) their corresponding elemental image sets optically captured with geometrical distortions on the CCD plane.

Fig. 9
Fig. 9

Sequential corrections of distortions in the pickup procedure for the elemental image set captured optically from a textured box object: (a) initial distortion detection with Hough transform peaks, (b) affine geometry recovered image, (c) metric geometry recovered image, and (d) extracted lens lattice lines in the close-up image of an inserted upper left corner in (c).

Fig. 10
Fig. 10

Sequential corrections of distortions in the pickup procedure for the elemental image set captured optically from a letter ‘2’ candle object: (a) initial distortion detection with Hough transform peaks, (b) affine geometry recovered image, (c) metric geometry recovered image, and (d) extracted lens lattice lines in the close-up image of an inserted upper left corner in (c).

Fig. 11
Fig. 11

Optically reconstructed 3D images (a) from the distorted elemental image sets and (b) from the elemental image sets rectified by the proposed method.

Tables (3)

Tables Icon

Table 1 Comparison of the transformation matrices between the simulated and extracted results

Tables Icon

Table 2 Transform matrices extracted for the textured box images in the first experiment.

Tables Icon

Table 3 Transform matrices extracted for the letter ‘2’ candle images in the second experiment.

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

x = H x ,
x = ( H ) 1 x = H x .
H = H s H a H p ,
H s = [ s R t 0 T 1 ] , H a = [ 1 / β α / β 0 0 1 0 0 0 1 ] , H p = [ 1 0 0 0 1 0 l 1 l 2 l 3 ] ,
( c α , c β ) = ( d 1 + d 2 2 , 0 ) ,
R = | d 1 d 2 2 | ,
( c α , c β ) = ( Δ x 2 Δ y 2 r 2 Δ x 1 Δ y 1 Δ y 2 2 r 2 Δ y 1 2 , 0 ) ,
R = | r ( Δ x 1 Δ y 2 Δ x 2 Δ y 1 ) Δ y 2 2 r 2 Δ y 1 2 | .

Metrics