Abstract

In this paper, we present an efficient Computer Generated Integral Imaging (CGII) method, called multiple ray cluster rendering (MRCR). Based on the MRCR, an interactive integral imaging system is realized, which provides accurate 3D image satisfying the changeable observers’ positions in real time. The MRCR method can generate all the elemental image pixels within only one rendering pass by ray reorganization of multiple ray clusters and 3D content duplication. It is compatible with various graphic contents including mesh, point cloud, and medical data. Moreover, multi-sampling method is embedded in MRCR method for acquiring anti-aliased 3D image result. To our best knowledge, the MRCR method outperforms the existing CGII methods in both the speed performance and the display quality. Experimental results show that the proposed CGII method can achieve real-time computational speed for large-scale 3D data with about 50,000 points.

© 2013 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. G. Lippmann, “La photographie integrale,” C.R. Acad. Sci.146, 446–451 (1908).
  2. J.-H. Park, G. Baasantseren, N. Kim, G. Park, J. M. Kang, and B. Lee, “View image generation in perspective and orthographic projection geometry based on integral imaging,” Opt. Express16(12), 8800–8813 (2008), http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-16-12-8800 .
    [CrossRef] [PubMed]
  3. Y. Igarashi, H. Murata, and M. Ueda, “3D display system using a computer generated integral photography,” Jpn. J. Appl. Phys.17(9), 1683–1684 (1978).
    [CrossRef]
  4. M. Halle, “Multiple viewpoint rendering,” SIGGRAPH’98, Proceedings of 25th annual conference on Computer graphics and interactive techniques, 243–254 (1998).
  5. S.-W. Min, J. Kim, and B. Lee, “New characteristics equation of three-dimensional integral imaging system and its applications,” Jpn. J. Appl. Phys.44(2), L71–L74 (2005).
    [CrossRef]
  6. R. Yang, X. Huang, and S. Chen, “Efficient rendering of integral images,” SIGGRAPH’05, Proceedings of 32nd Annual Conference on Computer Graphics and Interactive Techniques, 44 (2005).
  7. S.-W. Min, K. S. Park, B. Lee, Y. Cho, and M. Hahn, “Enhanced image mapping algorithm for computer-generated integral imaging system,” Jpn. J. Appl. Phys.45(28), L744–L747 (2006).
    [CrossRef]
  8. B.-N.-R. Lee, Y. Cho, K. S. Park, S.-W. Min, J.-S. Lim, M. C. Whang, and K. R. Park, “Design and implementation of a fast integral image rendering method,” International Conference on Electronic Commerce, 135–140 (2006).
    [CrossRef]
  9. K. S. Park, S.-W. Min, and Y. Cho, “Viewpoint vector rendering for efficient elemental image generation,” IEICE Trans. Inf. Syst. E 90-D, 231–241 (2007).
  10. F. P. Brooks, “What’s real about virtual reality?” IEEE Comput. Graph. Appl.19(6), 16–27 (1999).
    [CrossRef]
  11. K.-C. Kwon, C. Park, M.-U. Erdenebat, J.-S. Jeong, J.-H. Choi, N. Kim, J.-H. Park, Y.-T. Lim, and K.-H. Yoo, “High speed image space parallel processing for computer-generated integral imaging system,” Opt. Express20(2), 732–740 (2012), http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-20-2-732 .
    [CrossRef] [PubMed]
  12. R. Fukushima, K. Taira, T. Saishu, and Y. Hirayama, “Novel viewing zone control method for computer generated integral 3-D imaging,” Proceedings of SPIE – IS&T Electronic Imaging, SPIE Vol. 5291, 81–92, (2004).
    [CrossRef]
  13. G. Park, J.-H. Jung, K. Hong, Y. Kim, Y.-H. Kim, S.-W. Min, and B. Lee, “Multi-viewer tracking integral imaging system and its viewing zone analysis,” Opt. Express17(20), 17895–17908 (2009), http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-19-5-4129 .
    [CrossRef] [PubMed]
  14. G. Park, J. Hong, Y. Kim, and B. Lee, “Enhancement of viewing angle and viewing distance in integral imaging by head tracking,” Digital Holography and Three-Dimensional Imaging, OSA Technical Digest, DWB27 (1990).
  15. PrimerSense 3D sensor: http://www.primesense.com/solutions/sensor/ .
  16. H. Choi, Y. Kim, J.-H. Park, S. Jung, and B. Lee, “Improved analysis on the viewing angle of integral imaging,” Appl. Opt.44(12), 2311–2317 (2005).
    [CrossRef] [PubMed]
  17. J.-D. Foley, D. Van, Feiner, and Hughes, Computer Graphics: Principles and Practice, 2nd ed. (Addison-Wesley, 1990).
  18. R. Fernando, GPU Gems: Programming Techniques, Tips and Tricks for Real-Time Graphics (Addison-Wesley, 2004).
  19. F. de Sorbier, V. Nozick, and V. Biri, “GPU rendering for autostereoscopic displays,” 4th International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT’ 08), Jun. 2008, (2008).
  20. F. de Sorbier, V. Nozick, and H. Saito, “Multi-view rendering using GPU for 3-D displays,” Computer Games, Multimedia and Allied Technology (CGAT’10), April. 2010, (2010).
    [CrossRef]

2012

2009

2008

2007

K. S. Park, S.-W. Min, and Y. Cho, “Viewpoint vector rendering for efficient elemental image generation,” IEICE Trans. Inf. Syst. E 90-D, 231–241 (2007).

2006

S.-W. Min, K. S. Park, B. Lee, Y. Cho, and M. Hahn, “Enhanced image mapping algorithm for computer-generated integral imaging system,” Jpn. J. Appl. Phys.45(28), L744–L747 (2006).
[CrossRef]

2005

H. Choi, Y. Kim, J.-H. Park, S. Jung, and B. Lee, “Improved analysis on the viewing angle of integral imaging,” Appl. Opt.44(12), 2311–2317 (2005).
[CrossRef] [PubMed]

S.-W. Min, J. Kim, and B. Lee, “New characteristics equation of three-dimensional integral imaging system and its applications,” Jpn. J. Appl. Phys.44(2), L71–L74 (2005).
[CrossRef]

1999

F. P. Brooks, “What’s real about virtual reality?” IEEE Comput. Graph. Appl.19(6), 16–27 (1999).
[CrossRef]

1978

Y. Igarashi, H. Murata, and M. Ueda, “3D display system using a computer generated integral photography,” Jpn. J. Appl. Phys.17(9), 1683–1684 (1978).
[CrossRef]

1908

G. Lippmann, “La photographie integrale,” C.R. Acad. Sci.146, 446–451 (1908).

Baasantseren, G.

Brooks, F. P.

F. P. Brooks, “What’s real about virtual reality?” IEEE Comput. Graph. Appl.19(6), 16–27 (1999).
[CrossRef]

Cho, Y.

K. S. Park, S.-W. Min, and Y. Cho, “Viewpoint vector rendering for efficient elemental image generation,” IEICE Trans. Inf. Syst. E 90-D, 231–241 (2007).

S.-W. Min, K. S. Park, B. Lee, Y. Cho, and M. Hahn, “Enhanced image mapping algorithm for computer-generated integral imaging system,” Jpn. J. Appl. Phys.45(28), L744–L747 (2006).
[CrossRef]

B.-N.-R. Lee, Y. Cho, K. S. Park, S.-W. Min, J.-S. Lim, M. C. Whang, and K. R. Park, “Design and implementation of a fast integral image rendering method,” International Conference on Electronic Commerce, 135–140 (2006).
[CrossRef]

Choi, H.

Choi, J.-H.

Erdenebat, M.-U.

Hahn, M.

S.-W. Min, K. S. Park, B. Lee, Y. Cho, and M. Hahn, “Enhanced image mapping algorithm for computer-generated integral imaging system,” Jpn. J. Appl. Phys.45(28), L744–L747 (2006).
[CrossRef]

Hong, K.

Igarashi, Y.

Y. Igarashi, H. Murata, and M. Ueda, “3D display system using a computer generated integral photography,” Jpn. J. Appl. Phys.17(9), 1683–1684 (1978).
[CrossRef]

Jeong, J.-S.

Jung, J.-H.

Jung, S.

Kang, J. M.

Kim, J.

S.-W. Min, J. Kim, and B. Lee, “New characteristics equation of three-dimensional integral imaging system and its applications,” Jpn. J. Appl. Phys.44(2), L71–L74 (2005).
[CrossRef]

Kim, N.

Kim, Y.

Kim, Y.-H.

Kwon, K.-C.

Lee, B.

Lee, B.-N.-R.

B.-N.-R. Lee, Y. Cho, K. S. Park, S.-W. Min, J.-S. Lim, M. C. Whang, and K. R. Park, “Design and implementation of a fast integral image rendering method,” International Conference on Electronic Commerce, 135–140 (2006).
[CrossRef]

Lim, J.-S.

B.-N.-R. Lee, Y. Cho, K. S. Park, S.-W. Min, J.-S. Lim, M. C. Whang, and K. R. Park, “Design and implementation of a fast integral image rendering method,” International Conference on Electronic Commerce, 135–140 (2006).
[CrossRef]

Lim, Y.-T.

Lippmann, G.

G. Lippmann, “La photographie integrale,” C.R. Acad. Sci.146, 446–451 (1908).

Min, S.-W.

G. Park, J.-H. Jung, K. Hong, Y. Kim, Y.-H. Kim, S.-W. Min, and B. Lee, “Multi-viewer tracking integral imaging system and its viewing zone analysis,” Opt. Express17(20), 17895–17908 (2009), http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-19-5-4129 .
[CrossRef] [PubMed]

K. S. Park, S.-W. Min, and Y. Cho, “Viewpoint vector rendering for efficient elemental image generation,” IEICE Trans. Inf. Syst. E 90-D, 231–241 (2007).

S.-W. Min, K. S. Park, B. Lee, Y. Cho, and M. Hahn, “Enhanced image mapping algorithm for computer-generated integral imaging system,” Jpn. J. Appl. Phys.45(28), L744–L747 (2006).
[CrossRef]

S.-W. Min, J. Kim, and B. Lee, “New characteristics equation of three-dimensional integral imaging system and its applications,” Jpn. J. Appl. Phys.44(2), L71–L74 (2005).
[CrossRef]

B.-N.-R. Lee, Y. Cho, K. S. Park, S.-W. Min, J.-S. Lim, M. C. Whang, and K. R. Park, “Design and implementation of a fast integral image rendering method,” International Conference on Electronic Commerce, 135–140 (2006).
[CrossRef]

Murata, H.

Y. Igarashi, H. Murata, and M. Ueda, “3D display system using a computer generated integral photography,” Jpn. J. Appl. Phys.17(9), 1683–1684 (1978).
[CrossRef]

Park, C.

Park, G.

Park, J.-H.

Park, K. R.

B.-N.-R. Lee, Y. Cho, K. S. Park, S.-W. Min, J.-S. Lim, M. C. Whang, and K. R. Park, “Design and implementation of a fast integral image rendering method,” International Conference on Electronic Commerce, 135–140 (2006).
[CrossRef]

Park, K. S.

K. S. Park, S.-W. Min, and Y. Cho, “Viewpoint vector rendering for efficient elemental image generation,” IEICE Trans. Inf. Syst. E 90-D, 231–241 (2007).

S.-W. Min, K. S. Park, B. Lee, Y. Cho, and M. Hahn, “Enhanced image mapping algorithm for computer-generated integral imaging system,” Jpn. J. Appl. Phys.45(28), L744–L747 (2006).
[CrossRef]

B.-N.-R. Lee, Y. Cho, K. S. Park, S.-W. Min, J.-S. Lim, M. C. Whang, and K. R. Park, “Design and implementation of a fast integral image rendering method,” International Conference on Electronic Commerce, 135–140 (2006).
[CrossRef]

Ueda, M.

Y. Igarashi, H. Murata, and M. Ueda, “3D display system using a computer generated integral photography,” Jpn. J. Appl. Phys.17(9), 1683–1684 (1978).
[CrossRef]

Whang, M. C.

B.-N.-R. Lee, Y. Cho, K. S. Park, S.-W. Min, J.-S. Lim, M. C. Whang, and K. R. Park, “Design and implementation of a fast integral image rendering method,” International Conference on Electronic Commerce, 135–140 (2006).
[CrossRef]

Yoo, K.-H.

Appl. Opt.

C.R. Acad. Sci.

G. Lippmann, “La photographie integrale,” C.R. Acad. Sci.146, 446–451 (1908).

IEEE Comput. Graph. Appl.

F. P. Brooks, “What’s real about virtual reality?” IEEE Comput. Graph. Appl.19(6), 16–27 (1999).
[CrossRef]

IEICE Trans. Inf. Syst.

K. S. Park, S.-W. Min, and Y. Cho, “Viewpoint vector rendering for efficient elemental image generation,” IEICE Trans. Inf. Syst. E 90-D, 231–241 (2007).

Jpn. J. Appl. Phys.

S.-W. Min, K. S. Park, B. Lee, Y. Cho, and M. Hahn, “Enhanced image mapping algorithm for computer-generated integral imaging system,” Jpn. J. Appl. Phys.45(28), L744–L747 (2006).
[CrossRef]

Y. Igarashi, H. Murata, and M. Ueda, “3D display system using a computer generated integral photography,” Jpn. J. Appl. Phys.17(9), 1683–1684 (1978).
[CrossRef]

S.-W. Min, J. Kim, and B. Lee, “New characteristics equation of three-dimensional integral imaging system and its applications,” Jpn. J. Appl. Phys.44(2), L71–L74 (2005).
[CrossRef]

Opt. Express

Other

R. Yang, X. Huang, and S. Chen, “Efficient rendering of integral images,” SIGGRAPH’05, Proceedings of 32nd Annual Conference on Computer Graphics and Interactive Techniques, 44 (2005).

M. Halle, “Multiple viewpoint rendering,” SIGGRAPH’98, Proceedings of 25th annual conference on Computer graphics and interactive techniques, 243–254 (1998).

B.-N.-R. Lee, Y. Cho, K. S. Park, S.-W. Min, J.-S. Lim, M. C. Whang, and K. R. Park, “Design and implementation of a fast integral image rendering method,” International Conference on Electronic Commerce, 135–140 (2006).
[CrossRef]

R. Fukushima, K. Taira, T. Saishu, and Y. Hirayama, “Novel viewing zone control method for computer generated integral 3-D imaging,” Proceedings of SPIE – IS&T Electronic Imaging, SPIE Vol. 5291, 81–92, (2004).
[CrossRef]

J.-D. Foley, D. Van, Feiner, and Hughes, Computer Graphics: Principles and Practice, 2nd ed. (Addison-Wesley, 1990).

R. Fernando, GPU Gems: Programming Techniques, Tips and Tricks for Real-Time Graphics (Addison-Wesley, 2004).

F. de Sorbier, V. Nozick, and V. Biri, “GPU rendering for autostereoscopic displays,” 4th International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT’ 08), Jun. 2008, (2008).

F. de Sorbier, V. Nozick, and H. Saito, “Multi-view rendering using GPU for 3-D displays,” Computer Games, Multimedia and Allied Technology (CGAT’10), April. 2010, (2010).
[CrossRef]

G. Park, J. Hong, Y. Kim, and B. Lee, “Enhancement of viewing angle and viewing distance in integral imaging by head tracking,” Digital Holography and Three-Dimensional Imaging, OSA Technical Digest, DWB27 (1990).

PrimerSense 3D sensor: http://www.primesense.com/solutions/sensor/ .

Supplementary Material (3)

» Media 1: MOV (2760 KB)     
» Media 2: MOV (2234 KB)     
» Media 3: MOV (703 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (17)

Fig. 1
Fig. 1

An illustration for our interactive integral imaging system.

Fig. 2
Fig. 2

Viewing zone of integral imaging system with light field rays (a) The proposed integral imaging system (b) Conventional integral imaging system.

Fig. 3
Fig. 3

Illustration of multiple ray cluster (a) Ray clusters in integral imaging system (b) Multiple perspective view frustums for rendering.

Fig. 4
Fig. 4

MRC calculation in one perspective view frustum.

Fig. 5
Fig. 5

Flowchart of EIA computation process on GPU.

Fig. 6
Fig. 6

Description of the geometry duplication used in the MRC rendering.

Fig. 7
Fig. 7

Illustration of super sampling algorithm and the comparison of rendered ray cluster between without and with 32xMSAA.

Fig. 8
Fig. 8

Description of (a) the pixel translation calculation, (b) the render result of MRC, and (c) the rectified result of MRC.

Fig. 9
Fig. 9

Illustration of the pixel re-arrangement method.

Fig. 10
Fig. 10

Example of (a) 3D data (mesh with 4798 vertices and texture), (b) elemental images set generated by the proposed method (Media 1) and (c) 3D image optically reconstructed using interactive integral imaging display system.

Fig. 11
Fig. 11

Interactive integral imaging system with (a) optical experimental setup and (b) user controlled 3D image (Media 2). (The tiled lens array consists of 2x2 small lens arrays)

Fig. 12
Fig. 12

Displayed EIA accompanied with the reconstructed view images and the 3D image. (Media 3)

Fig. 13
Fig. 13

3D objects and displayed 3D images in experiments; (a) Dragon: 50,000 vertices, (b) Bunny: 2503 vertices, (c) MRI: 128x128x40, (d) Buddha: 49,990 point, (e) CT: 128x128x256.

Fig. 14
Fig. 14

The measurement result of proposed CGII and previous methods.

Fig. 15
Fig. 15

The measurement result of speed performance with different 3D data size.

Fig. 16
Fig. 16

Viewing zone results in our interactive integral imaging system at (a) view distance is 0.6m, (b) view distance is 0.8m, (c) view distance is 1.0m, (d) view distance is 2.0m, and (f) viewing zone in conventional integral imaging system.

Fig. 17
Fig. 17

The reconstructed 3D images from (a) traditional EIA rendering method, and (b) our MRC method.

Tables (3)

Tables Icon

Table 1 Experiment Environment and Computing Parameters

Tables Icon

Table 2 Integral imaging system characteristics

Tables Icon

Table 3 Experimental parameters for interactive integral imaging system

Equations (15)

Equations on this page are rendered with MathJax. Learn more.

W=p× (D+g) g ,
E=W× g D ,
n x E p d , n x N,
V i =- W 2 + W n x 1 ×i,
θ i =arctan( L/2p/2 V i D )arctan( L/2+p/2 V i D ),
V=(0,0),
θ=2arctan( (Lp) n x D ).
s= E p d × n x .
T i',j' =[ 1/ n x 0 0 0 0 1/ n y 0 0 0 0 1 0 W 2 +( W n x 1 )·i' W 2 +( W n y 1 )·j' 0 1 ],
M={ v 1 , v 2 ,, v m }, v k M,
v k =[ x k y k z k 1 ].
v i',j',k = v k · T i',j' .
R i = V i
S i = (Lp) n x 2 +(Lp)( n x i)+ (Lp) 2
O i = S i R i = V i + (Lp)( n x +1) 2 -(Lp)i

Metrics