Abstract

In most integral image analysis and processing tasks, accurate knowledge of the internal image structure is required. In this paper we present a robust framework for the accurate rectification of perspectively distorted integral images based on multiple line segment detection. The use of multiple line segments increases the overall fault tolerance of our framework providing strong statistical support for the rectification process. The proposed framework is used for the automatic rectification, metric correction, and rotation of distorted integral images. The performance of our framework is assessed over a number of integral images with varying scene complexity and noise levels.

© 2011 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. G. Lippmann, “La Photographie Integràle,” Comptes-Rendus, Acad. Sci. 146, 446–451 (1908).
  2. J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1, 125–140 (2005).
    [CrossRef]
  3. J. Park, Y. Kim, J. Kim, S. Min, and B. Lee, “Three-dimensional display scheme based on integral imaging with three-dimensional information processing,” Opt. Express 12, 6020–6032 (2004).
    [CrossRef]
  4. J. S. Jang and B. Javidi, “Formation of orthoscopic three dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003).
    [CrossRef]
  5. G. Passalis, N. Sgouros, S. Athineos, and T. Theoharis, “Enhanced reconstruction of three-dimensional shape and texture from integral photography images,” Appl. Opt. 46, 5311–5320 (2007).
    [CrossRef]
  6. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University, 2000).
  7. N. Sgouros, S. Athineos, M. Sangriotis, P. Papageorgas, and N. Theofanous, “Accurate lattice extraction in integral images,” Opt. Express 14, 10403–10409 (2006).
    [CrossRef]
  8. K. Hong, J. Hong, J. Jung, J. Park, and B. Lee, “Rectification of elemental image set and extraction of lens lattice by projective image transformation in integral imaging,” Opt. Express 18, 12002–12016 (2010).
    [CrossRef]
  9. R. von Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall, “LSD: A fast line segment detector with a false detection control,” IEEE Trans. Pattern Anal. Machine Intell. 32, 722–732 (2010).
    [CrossRef]
  10. A. Desolneux, L. Moisan, and J.-M. Morel, “Meaningful alignments,” International Journal of Computer Vision 40, 7–23 (2000).
    [CrossRef]
  11. M. Unser and M. Eden, “Multiresolution feature extraction and selection for texture segmentation,” IEEE Trans. Pattern Anal. Machine Intell. 11, 717–728 (1989).
    [CrossRef]
  12. S. Theodoridis and K. Koutroumbas, Pattern Recognition, 3rd ed. (Academic, 2006).
  13. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. (Wiley-Interscience, 2001).
  14. D. Liebowitz and A. Zisserman, “Metric rectification for perspective images of planes,” in Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, 1998), pp. 482–488.
  15. S. S. Athineos, N. P. Sgouros, P. G. Papageorgas, D. E. Maroulis, M. S. Sangriotis, and N. G. Theofanous, “Photorealistic integral photography using a ray-traced model of capturing optics,” J. Electron. Imaging 15, 043007 (2006).
    [CrossRef]

2010

K. Hong, J. Hong, J. Jung, J. Park, and B. Lee, “Rectification of elemental image set and extraction of lens lattice by projective image transformation in integral imaging,” Opt. Express 18, 12002–12016 (2010).
[CrossRef]

R. von Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall, “LSD: A fast line segment detector with a false detection control,” IEEE Trans. Pattern Anal. Machine Intell. 32, 722–732 (2010).
[CrossRef]

2007

2006

N. Sgouros, S. Athineos, M. Sangriotis, P. Papageorgas, and N. Theofanous, “Accurate lattice extraction in integral images,” Opt. Express 14, 10403–10409 (2006).
[CrossRef]

S. S. Athineos, N. P. Sgouros, P. G. Papageorgas, D. E. Maroulis, M. S. Sangriotis, and N. G. Theofanous, “Photorealistic integral photography using a ray-traced model of capturing optics,” J. Electron. Imaging 15, 043007 (2006).
[CrossRef]

2005

J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1, 125–140 (2005).
[CrossRef]

2004

2003

J. S. Jang and B. Javidi, “Formation of orthoscopic three dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003).
[CrossRef]

2000

A. Desolneux, L. Moisan, and J.-M. Morel, “Meaningful alignments,” International Journal of Computer Vision 40, 7–23 (2000).
[CrossRef]

1989

M. Unser and M. Eden, “Multiresolution feature extraction and selection for texture segmentation,” IEEE Trans. Pattern Anal. Machine Intell. 11, 717–728 (1989).
[CrossRef]

1908

G. Lippmann, “La Photographie Integràle,” Comptes-Rendus, Acad. Sci. 146, 446–451 (1908).

Athineos, S.

Athineos, S. S.

S. S. Athineos, N. P. Sgouros, P. G. Papageorgas, D. E. Maroulis, M. S. Sangriotis, and N. G. Theofanous, “Photorealistic integral photography using a ray-traced model of capturing optics,” J. Electron. Imaging 15, 043007 (2006).
[CrossRef]

Desolneux, A.

A. Desolneux, L. Moisan, and J.-M. Morel, “Meaningful alignments,” International Journal of Computer Vision 40, 7–23 (2000).
[CrossRef]

Duda, R. O.

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. (Wiley-Interscience, 2001).

Eden, M.

M. Unser and M. Eden, “Multiresolution feature extraction and selection for texture segmentation,” IEEE Trans. Pattern Anal. Machine Intell. 11, 717–728 (1989).
[CrossRef]

Hart, P. E.

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. (Wiley-Interscience, 2001).

Hartley, R.

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University, 2000).

Hong, J.

Hong, K.

Jakubowicz, J.

R. von Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall, “LSD: A fast line segment detector with a false detection control,” IEEE Trans. Pattern Anal. Machine Intell. 32, 722–732 (2010).
[CrossRef]

Jang, J. S.

J. S. Jang and B. Javidi, “Formation of orthoscopic three dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003).
[CrossRef]

Javidi, B.

J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1, 125–140 (2005).
[CrossRef]

J. S. Jang and B. Javidi, “Formation of orthoscopic three dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003).
[CrossRef]

Jung, J.

Kim, J.

Kim, Y.

Koutroumbas, K.

S. Theodoridis and K. Koutroumbas, Pattern Recognition, 3rd ed. (Academic, 2006).

Lee, B.

Liebowitz, D.

D. Liebowitz and A. Zisserman, “Metric rectification for perspective images of planes,” in Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, 1998), pp. 482–488.

Lippmann, G.

G. Lippmann, “La Photographie Integràle,” Comptes-Rendus, Acad. Sci. 146, 446–451 (1908).

Maroulis, D. E.

S. S. Athineos, N. P. Sgouros, P. G. Papageorgas, D. E. Maroulis, M. S. Sangriotis, and N. G. Theofanous, “Photorealistic integral photography using a ray-traced model of capturing optics,” J. Electron. Imaging 15, 043007 (2006).
[CrossRef]

Min, S.

Moisan, L.

A. Desolneux, L. Moisan, and J.-M. Morel, “Meaningful alignments,” International Journal of Computer Vision 40, 7–23 (2000).
[CrossRef]

Morel, J.-M.

R. von Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall, “LSD: A fast line segment detector with a false detection control,” IEEE Trans. Pattern Anal. Machine Intell. 32, 722–732 (2010).
[CrossRef]

A. Desolneux, L. Moisan, and J.-M. Morel, “Meaningful alignments,” International Journal of Computer Vision 40, 7–23 (2000).
[CrossRef]

Papageorgas, P.

Papageorgas, P. G.

S. S. Athineos, N. P. Sgouros, P. G. Papageorgas, D. E. Maroulis, M. S. Sangriotis, and N. G. Theofanous, “Photorealistic integral photography using a ray-traced model of capturing optics,” J. Electron. Imaging 15, 043007 (2006).
[CrossRef]

Park, J.

Passalis, G.

Randall, G.

R. von Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall, “LSD: A fast line segment detector with a false detection control,” IEEE Trans. Pattern Anal. Machine Intell. 32, 722–732 (2010).
[CrossRef]

Sangriotis, M.

Sangriotis, M. S.

S. S. Athineos, N. P. Sgouros, P. G. Papageorgas, D. E. Maroulis, M. S. Sangriotis, and N. G. Theofanous, “Photorealistic integral photography using a ray-traced model of capturing optics,” J. Electron. Imaging 15, 043007 (2006).
[CrossRef]

Sgouros, N.

Sgouros, N. P.

S. S. Athineos, N. P. Sgouros, P. G. Papageorgas, D. E. Maroulis, M. S. Sangriotis, and N. G. Theofanous, “Photorealistic integral photography using a ray-traced model of capturing optics,” J. Electron. Imaging 15, 043007 (2006).
[CrossRef]

Son, J.-Y.

J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1, 125–140 (2005).
[CrossRef]

Stork, D. G.

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. (Wiley-Interscience, 2001).

Theodoridis, S.

S. Theodoridis and K. Koutroumbas, Pattern Recognition, 3rd ed. (Academic, 2006).

Theofanous, N.

Theofanous, N. G.

S. S. Athineos, N. P. Sgouros, P. G. Papageorgas, D. E. Maroulis, M. S. Sangriotis, and N. G. Theofanous, “Photorealistic integral photography using a ray-traced model of capturing optics,” J. Electron. Imaging 15, 043007 (2006).
[CrossRef]

Theoharis, T.

Unser, M.

M. Unser and M. Eden, “Multiresolution feature extraction and selection for texture segmentation,” IEEE Trans. Pattern Anal. Machine Intell. 11, 717–728 (1989).
[CrossRef]

von Gioi, R.

R. von Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall, “LSD: A fast line segment detector with a false detection control,” IEEE Trans. Pattern Anal. Machine Intell. 32, 722–732 (2010).
[CrossRef]

Zisserman, A.

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University, 2000).

D. Liebowitz and A. Zisserman, “Metric rectification for perspective images of planes,” in Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, 1998), pp. 482–488.

Appl. Opt.

Comptes-Rendus, Acad. Sci.

G. Lippmann, “La Photographie Integràle,” Comptes-Rendus, Acad. Sci. 146, 446–451 (1908).

IEEE Trans. Pattern Anal. Machine Intell.

R. von Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall, “LSD: A fast line segment detector with a false detection control,” IEEE Trans. Pattern Anal. Machine Intell. 32, 722–732 (2010).
[CrossRef]

M. Unser and M. Eden, “Multiresolution feature extraction and selection for texture segmentation,” IEEE Trans. Pattern Anal. Machine Intell. 11, 717–728 (1989).
[CrossRef]

International Journal of Computer Vision

A. Desolneux, L. Moisan, and J.-M. Morel, “Meaningful alignments,” International Journal of Computer Vision 40, 7–23 (2000).
[CrossRef]

J. Disp. Technol.

J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1, 125–140 (2005).
[CrossRef]

J. Electron. Imaging

S. S. Athineos, N. P. Sgouros, P. G. Papageorgas, D. E. Maroulis, M. S. Sangriotis, and N. G. Theofanous, “Photorealistic integral photography using a ray-traced model of capturing optics,” J. Electron. Imaging 15, 043007 (2006).
[CrossRef]

Opt. Eng.

J. S. Jang and B. Javidi, “Formation of orthoscopic three dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003).
[CrossRef]

Opt. Express

Other

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University, 2000).

S. Theodoridis and K. Koutroumbas, Pattern Recognition, 3rd ed. (Academic, 2006).

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. (Wiley-Interscience, 2001).

D. Liebowitz and A. Zisserman, “Metric rectification for perspective images of planes,” in Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, 1998), pp. 482–488.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1.
Fig. 1.

(a) Integral imaging acquisition, (b) InIm display setup.

Fig. 2.
Fig. 2.

(a) 2D rendered image of a dice. (b) An undistorted InIm of the dice, Iid. (c) A distorted InIm of the dice, Iac. The InIm borders are shown for illustration purposes.

Fig. 3.
Fig. 3.

(a) Line segments detected using the LSD. (b) Remaining line segments after rejecting most of the undesired line segments. (c) Line parts fitted on the clustered line segments.

Fig. 4.
Fig. 4.

Histogram of the direction angles (a) of all detected line segments using the LSD algorithm, (b) of the line segments corresponding to horizontal lines, and (c) of the line segments corresponding to vertical lines.

Fig. 5.
Fig. 5.

(a) Geometric representation of the dissimilarity measure dm(li,lj) between two segments. (b) The histogram of {ds(n)}. Values in the first lobe correspond to collinear segments.

Fig. 6.
Fig. 6.

Rectification process.

Fig. 7.
Fig. 7.

Mean affine EI.

Fig. 8.
Fig. 8.

Three characteristic InIms. (a)-(c) 2D rendered images of three 3D objects. (d)-(f) Acquired InIms with the registered grid superimposed. (g)-(i) Rectified InIms along with the reconstructed grid.

Fig. 9.
Fig. 9.

Mean internal angles versus image quality.

Fig. 10.
Fig. 10.

Mean side length ratio versus image quality.

Fig. 11.
Fig. 11.

Optically acquired dice. (a) Photograph of the dice. (b) Acquired InIm with the registered grid superimposed. (c) Rectified InIm along with the reconstructed grid.

Tables (8)

Tables Icon

Table 1. Error Percentages for the Rectification Parameters of the InIm in Fig. 8(g)

Tables Icon

Table 2. Error Percentages for the Rectification Parameters of the InIm in Fig. 8(h)

Tables Icon

Table 3. Error Percentages for the Rectification Parameters of the InIm in Fig. 8(i)

Tables Icon

Table 4. Geometric Consistency Evaluation Parameters for the InIm in Fig. 8(g)

Tables Icon

Table 5. Geometric Consistency Evaluation Parameters for the InIm in Fig. 8(h)

Tables Icon

Table 6. Geometric Consistency Evaluation Parameters for the InIm in Fig. 8(i)

Tables Icon

Table 7. Error Percentages of the Rectification Parameters for the InIm in Fig. 11

Tables Icon

Table 8. Geometric Consistency Parameters for the InIm in Fig. 11

Equations (34)

Equations on this page are rendered with MathJax. Learn more.

dm(li,lj)=max{d(si,lj),d(ei,lj),d(sj,li),d(ej,li)},
dm(Cp,Cq)=min{dm(l,l)},lCp,lCq.
x=Hx˜,
H=HsHaHp,
Hp=(100010l1l2l3),
Ha=(1βαβ0010001),
(cx1,cy1)=(xAByABs12xBCyBCyAB2s12yBC2,0)=(xAByABxBCyBCyAB2yBC2,0),
r1=|s1(xBCyABxAByBC)yAB2s2yBC2|=|(xBCyABxAByBC)yAB2yBC2|,
(cx2,cy2)=(xACyACs22xBDyBDyAC2s22yBD2,0)=(xACyACxBDyBDyAC2yBD2,0)
r2=|s2(xBDyACxACyBD)yAC2s2yBD2|=|(xBDyACxACyBD)yAC2yBD2|,
α=cx2+cx12+(r12r22)2(cx2cx1)
β=±12(cx2cx1)[(r1+r2)2(cx2cx1)2][(cx2cx1)2(r2r1)2].
Hs=(Rx0y0001)(c000c0001),
R=(cosθsinθsinθcosθ).
l1·x+l2·y+l3=0,
AB=dνsinϕandBC=dhsinϕ.
xAB=ABcosϕh=dνsinϕcosϕh
yAB=ABsinϕh=dνsinϕsinϕh
xBC=BCcosϕν=dhsinϕcosϕν
yBC=BCsinϕν=dhsinϕsinϕν.
(cx1,cy1)=(dν2cosϕhsinϕhdh2cosϕνsinϕνdν2sin2ϕhdh2sin2ϕν,0)=(dν2sin2ϕhdh2sin2ϕνdν2(1cos2ϕh)dh2(1cos2ϕν),0)
r1=|dνdhcosϕνsinϕhdhdνcosϕhsinϕν(dhsinϕh)2(dvsinϕv)2|=dνdh|sin(ϕhϕν)||(dh2(1cos2ϕh)dν2(1cos2ϕν)|.
(cx1,cy1)=(dν2sh(1+sν2)dh2sν(1+sh2)dν2sh2(1+sν2)dh2sν2(1+sh2),0)
r1=dhdν(1+sν2)(1+sh2)(sνsh)2|dν2sh2(1+sν2)dh2sν2(1+sh2)|.
(cx2,cy2)=(sν+sh2sνsh,0)
r2=|sνsh|2sνsh.
v^sh=AsBsAsBs=Hαβ(1,sh)T|Hαβ(1,sh)T|=(1αsh,βsh)T(1ash)2+β2sh2
v^sν=(1αsν,βsν)T(1asν)2+β2sν2.
R=(cosθsinθsinθcosθ)=(v^shTv^sνT1)=(v^shTv^sνTT)=(v^shv^sν).
Hs=((v^shv^sν)00001).
as2=AsBs2=HαβAB2,
Hαβ=[1/βα/β01].
as2=dν2(sν2+1)[(1ash)2+β2sh2]β2(sνsh)2.
Hs=((v^shv^sν)00001)(c000c0001).

Metrics