Abstract

A novel method to computationally reconstruct perspective and orthographic view images with full resolution of a recording device from a single integral photograph is proposed. Firstly, a group of image slices that contain full yet redundant information to reconstruct the view image are generated, and the object surface is divided into pieces by the points that correspond to the centers of image slices. Secondly, the image slices that contribute to the pieces are extracted and redundant information embedded in them are figured out by common patches analysis. Finally, the view image is reconstructed by excluding the redundant information and resampling with maximum sampling rate. Each piece of the object surface is represented with 9 patches at most from 4 adjacent elemental images, and view images with high quality are reconstructed. Both simulations and experiments verify the validity of the method.

© 2017 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
View image generation in perspective and orthographic projection geometry based on integral imaging

Jae-Hyeung Park, Ganbat Baasantseren, Nam Kim, Gilbae Park, Jin-Mo Kang, and Byoungho Lee
Opt. Express 16(12) 8800-8813 (2008)

High resolution integral holography using Fourier ptychographic approach

Zhaohui Li, Jianqi Zhang, Xiaorui Wang, and Delian Liu
Opt. Express 22(26) 31935-31947 (2014)

Generic camera model and its calibration for computational integral imaging and 3D reconstruction

Weiming Li and Youfu Li
J. Opt. Soc. Am. A 28(3) 318-326 (2011)

References

  • View by:
  • |
  • |
  • |

  1. G. Lippmann, “La photographic integrale,” CR Acad. Sci. 146, 446–451 (1908).
  2. J.-Y. Son, B. Javidi, S. Yano, and K.-H. Choi, “Recent developments in 3-D imaging technologies,” J. Disp. Technol. 6(10), 394–403 (2010).
    [Crossref]
  3. J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1(1), 125–140 (2005).
    [Crossref]
  4. P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
    [Crossref]
  5. Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Research 1(1), 17–27 (2010).
    [Crossref]
  6. J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
    [Crossref] [PubMed]
  7. M. Levoy, “Light fields and computational imaging,” Computer 39(8), 46–55 (2006).
    [Crossref]
  8. A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
    [Crossref]
  9. H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A 15(8), 2059–2065 (1998).
    [Crossref]
  10. Z. E. Ashari, Z. Kavehvash, and K. Mehrany, “Diffraction influence on the field of view and resolution of three-dimensional integral imaging,” J. Disp. Technol. 10(7), 553–559 (2014).
    [Crossref]
  11. Z. Kavehvash, M. Martinez-Corral, K. Mehrany, S. Bagheri, G. Saavedra, and H. Navarro, “Three-dimensional resolvability in an integral imaging system,” J. Opt. Soc. Am. A 29(4), 525–530 (2012).
    [Crossref] [PubMed]
  12. Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009).
    [Crossref] [PubMed]
  13. J.-H. Park, G. Baasantseren, N. Kim, G. Park, J.-M. Kang, and B. Lee, “View image generation in perspective and orthographic projection geometry based on integral imaging,” Opt. Express 16(12), 8800–8813 (2008).
    [Crossref] [PubMed]
  14. A. Stern and B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003).
    [Crossref] [PubMed]
  15. S. Kishk and B. Javidi, “Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,” Opt. Express 11(26), 3528–3541 (2003).
    [Crossref] [PubMed]
  16. H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26(3), 157–159 (2001).
    [Crossref] [PubMed]
  17. Y. Piao and E.-S. Kim, “Resolution-enhanced reconstruction of far 3-D objects by using a direct pixel mapping method in computational curving-effective integral imaging,” Appl. Opt. 48(34), H222–H230 (2009).
    [Crossref] [PubMed]
  18. L.-Y. Ai, X.-B. Dong, J.-Y. Jang, and E.-S. Kim, “Optical full-depth refocusing of 3-D objects based on subdivided-elemental images and local periodic δ-functions in integral imaging,” Opt. Express 24(10), 10359–10375 (2016).
    [Crossref] [PubMed]
  19. N. Chen, J.-H. Park, and N. Kim, “Parameter analysis of integral Fourier hologram and its resolution enhancement,” Opt. Express 18(3), 2152–2167 (2010).
    [Crossref] [PubMed]
  20. J. Chen, Q. H. Wang, S. L. Li, Z. L. Xiong, and H. Deng, “Multiple elemental image mapping for resolution‐enhanced orthographic view image generation based on integral imaging,” J. Soc. Inf. Disp. 22(9), 487–492 (2014).
    [Crossref]
  21. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27(5), 324–326 (2002).
    [Crossref] [PubMed]
  22. G. Passalis, N. Sgouros, S. Athineos, and T. Theoharis, “Enhanced reconstruction of three-dimensional shape and texture from integral photography images,” Appl. Opt. 46(22), 5311–5320 (2007).
    [Crossref] [PubMed]
  23. J.-S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27(13), 1144–1146 (2002).
    [Crossref] [PubMed]
  24. H. Navarro, R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, “3D integral imaging display by smart pseudoscopic-to-orthoscopic conversion (SPOC),” Opt. Express 18(25), 25573–25583 (2010).
    [Crossref] [PubMed]

2016 (2)

P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
[Crossref]

L.-Y. Ai, X.-B. Dong, J.-Y. Jang, and E.-S. Kim, “Optical full-depth refocusing of 3-D objects based on subdivided-elemental images and local periodic δ-functions in integral imaging,” Opt. Express 24(10), 10359–10375 (2016).
[Crossref] [PubMed]

2014 (2)

Z. E. Ashari, Z. Kavehvash, and K. Mehrany, “Diffraction influence on the field of view and resolution of three-dimensional integral imaging,” J. Disp. Technol. 10(7), 553–559 (2014).
[Crossref]

J. Chen, Q. H. Wang, S. L. Li, Z. L. Xiong, and H. Deng, “Multiple elemental image mapping for resolution‐enhanced orthographic view image generation based on integral imaging,” J. Soc. Inf. Disp. 22(9), 487–492 (2014).
[Crossref]

2012 (1)

2010 (4)

2009 (3)

2008 (1)

2007 (1)

2006 (2)

M. Levoy, “Light fields and computational imaging,” Computer 39(8), 46–55 (2006).
[Crossref]

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
[Crossref]

2005 (1)

J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1(1), 125–140 (2005).
[Crossref]

2003 (2)

2002 (2)

2001 (1)

1998 (1)

1908 (1)

G. Lippmann, “La photographic integrale,” CR Acad. Sci. 146, 446–451 (1908).

Ai, L.-Y.

Arimoto, H.

Ashari, Z. E.

Z. E. Ashari, Z. Kavehvash, and K. Mehrany, “Diffraction influence on the field of view and resolution of three-dimensional integral imaging,” J. Disp. Technol. 10(7), 553–559 (2014).
[Crossref]

Athineos, S.

Baasantseren, G.

Bagheri, S.

Cao, L.

P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
[Crossref]

Cao, W.

P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
[Crossref]

Chen, J.

J. Chen, Q. H. Wang, S. L. Li, Z. L. Xiong, and H. Deng, “Multiple elemental image mapping for resolution‐enhanced orthographic view image generation based on integral imaging,” J. Soc. Inf. Disp. 22(9), 487–492 (2014).
[Crossref]

Chen, N.

Cheng, B.

P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
[Crossref]

Choi, K.-H.

J.-Y. Son, B. Javidi, S. Yano, and K.-H. Choi, “Recent developments in 3-D imaging technologies,” J. Disp. Technol. 6(10), 394–403 (2010).
[Crossref]

Deng, H.

J. Chen, Q. H. Wang, S. L. Li, Z. L. Xiong, and H. Deng, “Multiple elemental image mapping for resolution‐enhanced orthographic view image generation based on integral imaging,” J. Soc. Inf. Disp. 22(9), 487–492 (2014).
[Crossref]

Dong, X.-B.

Hong, K.

Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Research 1(1), 17–27 (2010).
[Crossref]

J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
[Crossref] [PubMed]

Hoshino, H.

Isono, H.

Jang, J.-S.

Jang, J.-Y.

Javidi, B.

J.-Y. Son, B. Javidi, S. Yano, and K.-H. Choi, “Recent developments in 3-D imaging technologies,” J. Disp. Technol. 6(10), 394–403 (2010).
[Crossref]

H. Navarro, R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, “3D integral imaging display by smart pseudoscopic-to-orthoscopic conversion (SPOC),” Opt. Express 18(25), 25573–25583 (2010).
[Crossref] [PubMed]

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
[Crossref]

J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1(1), 125–140 (2005).
[Crossref]

A. Stern and B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003).
[Crossref] [PubMed]

S. Kishk and B. Javidi, “Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,” Opt. Express 11(26), 3528–3541 (2003).
[Crossref] [PubMed]

J.-S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27(13), 1144–1146 (2002).
[Crossref] [PubMed]

J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27(5), 324–326 (2002).
[Crossref] [PubMed]

H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26(3), 157–159 (2001).
[Crossref] [PubMed]

Jin, G.

P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
[Crossref]

Kang, J.-M.

Kavehvash, Z.

Z. E. Ashari, Z. Kavehvash, and K. Mehrany, “Diffraction influence on the field of view and resolution of three-dimensional integral imaging,” J. Disp. Technol. 10(7), 553–559 (2014).
[Crossref]

Z. Kavehvash, M. Martinez-Corral, K. Mehrany, S. Bagheri, G. Saavedra, and H. Navarro, “Three-dimensional resolvability in an integral imaging system,” J. Opt. Soc. Am. A 29(4), 525–530 (2012).
[Crossref] [PubMed]

Kim, E.-S.

Kim, N.

Kim, Y.

Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Research 1(1), 17–27 (2010).
[Crossref]

Kishk, S.

Kwon, K.-C.

Lee, B.

Levoy, M.

M. Levoy, “Light fields and computational imaging,” Computer 39(8), 46–55 (2006).
[Crossref]

Li, S. L.

J. Chen, Q. H. Wang, S. L. Li, Z. L. Xiong, and H. Deng, “Multiple elemental image mapping for resolution‐enhanced orthographic view image generation based on integral imaging,” J. Soc. Inf. Disp. 22(9), 487–492 (2014).
[Crossref]

Liang, X.

P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
[Crossref]

Lim, Y.-T.

Lippmann, G.

G. Lippmann, “La photographic integrale,” CR Acad. Sci. 146, 446–451 (1908).

Ma, J.

P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
[Crossref]

Martinez-Corral, M.

Martínez-Corral, M.

Martínez-Cuenca, R.

Mehrany, K.

Z. E. Ashari, Z. Kavehvash, and K. Mehrany, “Diffraction influence on the field of view and resolution of three-dimensional integral imaging,” J. Disp. Technol. 10(7), 553–559 (2014).
[Crossref]

Z. Kavehvash, M. Martinez-Corral, K. Mehrany, S. Bagheri, G. Saavedra, and H. Navarro, “Three-dimensional resolvability in an integral imaging system,” J. Opt. Soc. Am. A 29(4), 525–530 (2012).
[Crossref] [PubMed]

Navarro, H.

Okano, F.

Park, G.

Park, J.-H.

Passalis, G.

Piao, Y.

Saavedra, G.

Sgouros, N.

Son, J.-Y.

J.-Y. Son, B. Javidi, S. Yano, and K.-H. Choi, “Recent developments in 3-D imaging technologies,” J. Disp. Technol. 6(10), 394–403 (2010).
[Crossref]

J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1(1), 125–140 (2005).
[Crossref]

Stern, A.

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
[Crossref]

A. Stern and B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003).
[Crossref] [PubMed]

Su, P.

P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
[Crossref]

Theoharis, T.

Wang, Q. H.

J. Chen, Q. H. Wang, S. L. Li, Z. L. Xiong, and H. Deng, “Multiple elemental image mapping for resolution‐enhanced orthographic view image generation based on integral imaging,” J. Soc. Inf. Disp. 22(9), 487–492 (2014).
[Crossref]

Xiong, Z. L.

J. Chen, Q. H. Wang, S. L. Li, Z. L. Xiong, and H. Deng, “Multiple elemental image mapping for resolution‐enhanced orthographic view image generation based on integral imaging,” J. Soc. Inf. Disp. 22(9), 487–492 (2014).
[Crossref]

Yano, S.

J.-Y. Son, B. Javidi, S. Yano, and K.-H. Choi, “Recent developments in 3-D imaging technologies,” J. Disp. Technol. 6(10), 394–403 (2010).
[Crossref]

Yuyama, I.

3D Research (1)

Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Research 1(1), 17–27 (2010).
[Crossref]

Appl. Opt. (4)

Computer (1)

M. Levoy, “Light fields and computational imaging,” Computer 39(8), 46–55 (2006).
[Crossref]

CR Acad. Sci. (1)

G. Lippmann, “La photographic integrale,” CR Acad. Sci. 146, 446–451 (1908).

J. Disp. Technol. (4)

J.-Y. Son, B. Javidi, S. Yano, and K.-H. Choi, “Recent developments in 3-D imaging technologies,” J. Disp. Technol. 6(10), 394–403 (2010).
[Crossref]

J.-Y. Son and B. Javidi, “Three-dimensional imaging methods based on multiview images,” J. Disp. Technol. 1(1), 125–140 (2005).
[Crossref]

P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016).
[Crossref]

Z. E. Ashari, Z. Kavehvash, and K. Mehrany, “Diffraction influence on the field of view and resolution of three-dimensional integral imaging,” J. Disp. Technol. 10(7), 553–559 (2014).
[Crossref]

J. Opt. Soc. Am. A (2)

J. Soc. Inf. Disp. (1)

J. Chen, Q. H. Wang, S. L. Li, Z. L. Xiong, and H. Deng, “Multiple elemental image mapping for resolution‐enhanced orthographic view image generation based on integral imaging,” J. Soc. Inf. Disp. 22(9), 487–492 (2014).
[Crossref]

Opt. Express (6)

Opt. Lett. (3)

Proc. IEEE (1)

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
[Crossref]

Supplementary Material (4)

NameDescription
» Visualization 1       The video is an animation of the reconstructed perspective view images of virtual object from various viewpoints.
» Visualization 2       The video is an animation of the reconstructed orthographic view images of virtual object from various view angles.
» Visualization 3       The video is an animation of the reconstructed perspective view images of real object from various viewpoints.
» Visualization 4       The video is an animation of the reconstructed orthographic view images of real object from various view angles.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1
Fig. 1

Schematic diagram of the proposed method of reconstructing perspective view images: the generation of (a) image slices which contains redundant information, and (b) perspective view image with redundancy excluded.

Fig. 2
Fig. 2

Schematic diagram of the proposed method of reconstructing orthographic view images: the generation of (a) image slices which contains redundant information, and (b) orthographic view image with redundancy excluded.

Fig. 3
Fig. 3

Schematic diagram of CPA between adjacent image slices.

Fig. 4
Fig. 4

Merging and resampling strategy when common patches do not exceed the centers of image slices. (a) Compositional structure of image slices in square S m,n , (b) mean filtering of the width of common patches, (c) the merged image, (d) interpolation of merged image to the resolution of H×V.

Fig. 5
Fig. 5

Merging strategy of image slices when common patches exceed the centers of image slices. (a) Compositional structure of image slices in square S m,n , (b) merging strategy, (c) the merged image.

Fig. 6
Fig. 6

Subdivision of object surface into pieces and exactly right description of each piece for (a) perspective projection geometry and (b) orthographic projection geometry.

Fig. 7
Fig. 7

(a) The 3-D model used for simulation, (b) the generated EIA, (c) the view image generated with information redundancy, and aliasing occurs.

Fig. 8
Fig. 8

CPA of horizontal and vertical image slices.

Fig. 9
Fig. 9

The reconstructed perspective view images from (a) different viewpoints and (b) depth, and (c) view images with resolution equal to the number lens of LA. Visualization 1 presents the reconstructed perspective view images from various viewpoints and depth.

Fig. 10
Fig. 10

The reconstructed (a) orthographic view images and (b) sub-images. Visualization 2 presents the reconstructed orthographic view images and sub-images from various view angles.

Fig. 11
Fig. 11

Schematic of capturing EIs using synthetic aperture method.

Fig. 12
Fig. 12

(a) Front view of the scene, and (b) the generated EIA.

Fig. 13
Fig. 13

The reconstructed perspective view images from (a) different viewpoints and (b) depth. Visualization 3 presents the reconstructed perspective view images from various viewpoints and depth.

Fig. 14
Fig. 14

The reconstructed orthographic view images. Visualization 4 presents the reconstructed orthographic view images from various view angles.

Equations (11)

Equations on this page are rendered with MathJax. Learn more.

w 1 = Ll L p l
w 2 = g l w 1
x E n = g L ( x C n x V )+ x C n y E n = g L ( y C n y V )+ y C n
g<l l a
w 2 = g l p l
x E n =gtanθ+ x C n y E n =gtanθ+ y C n
x E ' m,n = l g ( x C m,n x E m,n )+ x C m,n y E ' m,n = l g ( y C m,n y E m,n )+ y C m,n
ssd( t )= sQ [ I m,n ( s, T 0 ) I m,n+1 ( s,t ) ] 2 ,( t N + , t 1 t t 2 )
A m,n =mean( a 1 , a 2 , a 3 , a 4 ) B m,n =mean( b 1 , b 2 , b 3 , b 4 )
H×V=max[ ( w A m,n )×( w B m,n ) ],(1mM1,1nN1)
A m,n =wmean( a 1 , a 2 , a 3 , a 4 )

Metrics