Abstract

In three-dimensional television (3D TV) broadcasting, we find the effect of fundamental depth resolution and the cardboard effect to the perceived depth resolution on multi-view display is important. The observer distance and the specification of multi-view display quantize the expressible depth range, which affect the perception of depth resolution of the observer. In addition, the multi-view 3D TV needs the view synthesis process using depth image-based rendering which induces the cardboard effect from the relation among the stereo pickup, the multi-view synthesis and the multi-view display. In this paper, we analyze the fundamental depth resolution and the cardboard effect from the synthesis process in the multi-view 3D TV broadcasting. After the analysis, the numerical comparison and subjective tests with 20 participants are performed to find the effect of fundamental depth resolution and the cardboard effect to the perceived depth resolution.

© 2011 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
    [CrossRef]
  2. M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
    [CrossRef]
  3. B. Lee, J.-H. Park, and S.-W. Min, Digital Holography and Three-Dimensional Display, T.-C. Poon, ed. (Springer US, 2006), Chap. 12.
  4. Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Research 1(1), 17–27 (2010).
    [CrossRef]
  5. D. Minoli, 3DTV Content Capture, Encoding and Transmission: Building the Transport Infrastructure for Commercial Services (John Wiley and Sons, 2010), Chap. 3.
  6. J.-H. Jung, J. Hong, G. Park, K. Hong, S.-W. Min, and B. Lee, “Evaluation of perceived depth resolution in multi-view three-dimensional display using depth image-based rendering,” in Proceedings of IEEE Conference on 3DTV Conference 2011 (Antalya, Turkey, 2011), pp. 1–4.
  7. C. van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
    [CrossRef]
  8. C. van Berkel, “Image preparation for 3D-LCD,” Proc. SPIE 3639, 84–91 (1999).
    [CrossRef]
  9. Y.-G. Lee and J. B. Ra, “New image multiplexing scheme for compensating lens mismatch and viewing zone shifts in three-dimensional lenticular displays,” Opt. Eng. 48(4), 044001 (2009).
    [CrossRef]
  10. H. Kim, J. Hahn, and H.-J. Choi, “Numerical investigation on the viewing angle of a lenticular three-dimensional display with a triplet lens array,” Appl. Opt. 50(11), 1534–1540 (2011).
    [CrossRef] [PubMed]
  11. J.-C. Liou and F.-H. Chen, “Design and fabrication of optical system for time-multiplex autostereoscopic display,” Opt. Express 19(12), 11007–11017 (2011).
    [CrossRef] [PubMed]
  12. C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV,” Proc. SPIE 5291, 93–104 (2004).
    [CrossRef]
  13. K.-J. Oh, A. Vetro, and Y.-S. Ho, “Depth coding using a boundary reconstruction filter for 3-D video systems,” IEEE Trans. Circ. Syst. Video Tech. 21(3), 350–359 (2011).
    [CrossRef]
  14. Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, “Boundary artifact reduction in view synthesis of 3D video: from perspective of texture-depth alignment,” IEEE Trans. Broadcast 57(2), 510–522 (2011).
    [CrossRef]
  15. J.-H. Jung, K. Hong, G. Park, I. Chung, J.-H. Park, and B. Lee, “Reconstruction of three-dimensional occluded object using optical flow and triangular mesh reconstruction in integral imaging,” Opt. Express 18(25), 26373–26387 (2010).
    [CrossRef] [PubMed]
  16. M. Tanimoto, T. Fujii, and K. Suzuki, “View synthesis algorithm in view synthesis reference software 2.0 (VSRS2.0),” ISO/IEC JTC1/SC29/WG11 Doc. M16090, Feb. 2009.
  17. A. Woods, T. Docherty, and R. Koch, “Image distortions in stereoscopic video systems,” Proc. SPIE 1915, 36–48 (1993).
    [CrossRef]
  18. T. Koike, A. Yuuki, S. Uehara, K. Taira, G. Hamagishi, K. Izumi, T. Nomura, K. Mashitani, A. Miyazawa, T. Horikoshi, and H. Ujike, “Measurement of multi-view and integral photography displays based on sampling in ray space,” in Proceedings of IDW ’08 Technical Digest (Niigata Convention Center, Japan, 2008), pp. 1115–1118.
  19. H. Yamanoue, M. Okui, and I. Yuyama, “A Study on the relationship between shooting conditions and cardboard effect of stereoscopic images,” IEEE Trans. Circ. Syst. Video Tech. 10(3), 411–416 (2000).
    [CrossRef]
  20. H. Yamanoue, M. Okui, and F. Okano, “Geometrical analysis of puppet-theater and cardboard effects in stereoscopic HDTV images,” IEEE Trans. Circ. Syst. Video Tech. 16(6), 744–752 (2006).
    [CrossRef]
  21. J. Cutting and P. Vishton, Perception of Space and Motion, W. Epstein, ed. (Academic Press, 1995), Chap. 3.
  22. Philips (in Coop with 3D4YOU), “Response to New Call for 3DV Test Material: Beergarden,” ISO/IEC JTC1/SC29/WG11 Doc. M16421, Apr. 2009.

2011

K.-J. Oh, A. Vetro, and Y.-S. Ho, “Depth coding using a boundary reconstruction filter for 3-D video systems,” IEEE Trans. Circ. Syst. Video Tech. 21(3), 350–359 (2011).
[CrossRef]

Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, “Boundary artifact reduction in view synthesis of 3D video: from perspective of texture-depth alignment,” IEEE Trans. Broadcast 57(2), 510–522 (2011).
[CrossRef]

H. Kim, J. Hahn, and H.-J. Choi, “Numerical investigation on the viewing angle of a lenticular three-dimensional display with a triplet lens array,” Appl. Opt. 50(11), 1534–1540 (2011).
[CrossRef] [PubMed]

J.-C. Liou and F.-H. Chen, “Design and fabrication of optical system for time-multiplex autostereoscopic display,” Opt. Express 19(12), 11007–11017 (2011).
[CrossRef] [PubMed]

2010

2009

Y.-G. Lee and J. B. Ra, “New image multiplexing scheme for compensating lens mismatch and viewing zone shifts in three-dimensional lenticular displays,” Opt. Eng. 48(4), 044001 (2009).
[CrossRef]

2007

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
[CrossRef]

2006

H. Yamanoue, M. Okui, and F. Okano, “Geometrical analysis of puppet-theater and cardboard effects in stereoscopic HDTV images,” IEEE Trans. Circ. Syst. Video Tech. 16(6), 744–752 (2006).
[CrossRef]

2004

C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV,” Proc. SPIE 5291, 93–104 (2004).
[CrossRef]

2000

H. Yamanoue, M. Okui, and I. Yuyama, “A Study on the relationship between shooting conditions and cardboard effect of stereoscopic images,” IEEE Trans. Circ. Syst. Video Tech. 10(3), 411–416 (2000).
[CrossRef]

1999

C. van Berkel, “Image preparation for 3D-LCD,” Proc. SPIE 3639, 84–91 (1999).
[CrossRef]

1997

C. van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
[CrossRef]

1993

M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
[CrossRef]

A. Woods, T. Docherty, and R. Koch, “Image distortions in stereoscopic video systems,” Proc. SPIE 1915, 36–48 (1993).
[CrossRef]

Chen, F.-H.

Chen, T.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
[CrossRef]

Chen, Z.

Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, “Boundary artifact reduction in view synthesis of 3D video: from perspective of texture-depth alignment,” IEEE Trans. Broadcast 57(2), 510–522 (2011).
[CrossRef]

Choi, H.-J.

Chung, I.

Clarke, J. A.

C. van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
[CrossRef]

Docherty, T.

A. Woods, T. Docherty, and R. Koch, “Image distortions in stereoscopic video systems,” Proc. SPIE 1915, 36–48 (1993).
[CrossRef]

Fehn, C.

C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV,” Proc. SPIE 5291, 93–104 (2004).
[CrossRef]

Hahn, J.

Ho, Y.-S.

K.-J. Oh, A. Vetro, and Y.-S. Ho, “Depth coding using a boundary reconstruction filter for 3-D video systems,” IEEE Trans. Circ. Syst. Video Tech. 21(3), 350–359 (2011).
[CrossRef]

Hong, K.

Jung, J.-H.

Kanade, T.

M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
[CrossRef]

Kim, H.

Kim, Y.

Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Research 1(1), 17–27 (2010).
[CrossRef]

Koch, R.

A. Woods, T. Docherty, and R. Koch, “Image distortions in stereoscopic video systems,” Proc. SPIE 1915, 36–48 (1993).
[CrossRef]

Kubota, A.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
[CrossRef]

Lee, B.

Lee, Y.-G.

Y.-G. Lee and J. B. Ra, “New image multiplexing scheme for compensating lens mismatch and viewing zone shifts in three-dimensional lenticular displays,” Opt. Eng. 48(4), 044001 (2009).
[CrossRef]

Liou, J.-C.

Magnor, M.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
[CrossRef]

Oh, K.-J.

K.-J. Oh, A. Vetro, and Y.-S. Ho, “Depth coding using a boundary reconstruction filter for 3-D video systems,” IEEE Trans. Circ. Syst. Video Tech. 21(3), 350–359 (2011).
[CrossRef]

Okano, F.

H. Yamanoue, M. Okui, and F. Okano, “Geometrical analysis of puppet-theater and cardboard effects in stereoscopic HDTV images,” IEEE Trans. Circ. Syst. Video Tech. 16(6), 744–752 (2006).
[CrossRef]

Okui, M.

H. Yamanoue, M. Okui, and F. Okano, “Geometrical analysis of puppet-theater and cardboard effects in stereoscopic HDTV images,” IEEE Trans. Circ. Syst. Video Tech. 16(6), 744–752 (2006).
[CrossRef]

H. Yamanoue, M. Okui, and I. Yuyama, “A Study on the relationship between shooting conditions and cardboard effect of stereoscopic images,” IEEE Trans. Circ. Syst. Video Tech. 10(3), 411–416 (2000).
[CrossRef]

Okutomi, M.

M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
[CrossRef]

Park, G.

Park, J.-H.

Ra, J. B.

Y.-G. Lee and J. B. Ra, “New image multiplexing scheme for compensating lens mismatch and viewing zone shifts in three-dimensional lenticular displays,” Opt. Eng. 48(4), 044001 (2009).
[CrossRef]

Smolic, A.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
[CrossRef]

Tanimoto, M.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
[CrossRef]

Tian, D.

Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, “Boundary artifact reduction in view synthesis of 3D video: from perspective of texture-depth alignment,” IEEE Trans. Broadcast 57(2), 510–522 (2011).
[CrossRef]

van Berkel, C.

C. van Berkel, “Image preparation for 3D-LCD,” Proc. SPIE 3639, 84–91 (1999).
[CrossRef]

C. van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
[CrossRef]

Vetro, A.

K.-J. Oh, A. Vetro, and Y.-S. Ho, “Depth coding using a boundary reconstruction filter for 3-D video systems,” IEEE Trans. Circ. Syst. Video Tech. 21(3), 350–359 (2011).
[CrossRef]

Woods, A.

A. Woods, T. Docherty, and R. Koch, “Image distortions in stereoscopic video systems,” Proc. SPIE 1915, 36–48 (1993).
[CrossRef]

Yamanoue, H.

H. Yamanoue, M. Okui, and F. Okano, “Geometrical analysis of puppet-theater and cardboard effects in stereoscopic HDTV images,” IEEE Trans. Circ. Syst. Video Tech. 16(6), 744–752 (2006).
[CrossRef]

H. Yamanoue, M. Okui, and I. Yuyama, “A Study on the relationship between shooting conditions and cardboard effect of stereoscopic images,” IEEE Trans. Circ. Syst. Video Tech. 10(3), 411–416 (2000).
[CrossRef]

Yu, L.

Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, “Boundary artifact reduction in view synthesis of 3D video: from perspective of texture-depth alignment,” IEEE Trans. Broadcast 57(2), 510–522 (2011).
[CrossRef]

Yuyama, I.

H. Yamanoue, M. Okui, and I. Yuyama, “A Study on the relationship between shooting conditions and cardboard effect of stereoscopic images,” IEEE Trans. Circ. Syst. Video Tech. 10(3), 411–416 (2000).
[CrossRef]

Zhang, C.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
[CrossRef]

Zhao, Y.

Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, “Boundary artifact reduction in view synthesis of 3D video: from perspective of texture-depth alignment,” IEEE Trans. Broadcast 57(2), 510–522 (2011).
[CrossRef]

Zhu, C.

Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, “Boundary artifact reduction in view synthesis of 3D video: from perspective of texture-depth alignment,” IEEE Trans. Broadcast 57(2), 510–522 (2011).
[CrossRef]

3D Research

Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Research 1(1), 17–27 (2010).
[CrossRef]

Appl. Opt.

IEEE Trans. Broadcast

Y. Zhao, C. Zhu, Z. Chen, D. Tian, and L. Yu, “Boundary artifact reduction in view synthesis of 3D video: from perspective of texture-depth alignment,” IEEE Trans. Broadcast 57(2), 510–522 (2011).
[CrossRef]

IEEE Signal Process. Mag.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
[CrossRef]

IEEE Trans. Circ. Syst. Video Tech.

H. Yamanoue, M. Okui, and I. Yuyama, “A Study on the relationship between shooting conditions and cardboard effect of stereoscopic images,” IEEE Trans. Circ. Syst. Video Tech. 10(3), 411–416 (2000).
[CrossRef]

IEEE Trans. Circ. Syst. Video Tech.

H. Yamanoue, M. Okui, and F. Okano, “Geometrical analysis of puppet-theater and cardboard effects in stereoscopic HDTV images,” IEEE Trans. Circ. Syst. Video Tech. 16(6), 744–752 (2006).
[CrossRef]

IEEE Trans. Circ. Syst. Video Tech.

K.-J. Oh, A. Vetro, and Y.-S. Ho, “Depth coding using a boundary reconstruction filter for 3-D video systems,” IEEE Trans. Circ. Syst. Video Tech. 21(3), 350–359 (2011).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell.

M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
[CrossRef]

Opt. Eng.

Y.-G. Lee and J. B. Ra, “New image multiplexing scheme for compensating lens mismatch and viewing zone shifts in three-dimensional lenticular displays,” Opt. Eng. 48(4), 044001 (2009).
[CrossRef]

Opt. Express

Proc. SPIE

C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV,” Proc. SPIE 5291, 93–104 (2004).
[CrossRef]

C. van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
[CrossRef]

C. van Berkel, “Image preparation for 3D-LCD,” Proc. SPIE 3639, 84–91 (1999).
[CrossRef]

A. Woods, T. Docherty, and R. Koch, “Image distortions in stereoscopic video systems,” Proc. SPIE 1915, 36–48 (1993).
[CrossRef]

Other

T. Koike, A. Yuuki, S. Uehara, K. Taira, G. Hamagishi, K. Izumi, T. Nomura, K. Mashitani, A. Miyazawa, T. Horikoshi, and H. Ujike, “Measurement of multi-view and integral photography displays based on sampling in ray space,” in Proceedings of IDW ’08 Technical Digest (Niigata Convention Center, Japan, 2008), pp. 1115–1118.

M. Tanimoto, T. Fujii, and K. Suzuki, “View synthesis algorithm in view synthesis reference software 2.0 (VSRS2.0),” ISO/IEC JTC1/SC29/WG11 Doc. M16090, Feb. 2009.

J. Cutting and P. Vishton, Perception of Space and Motion, W. Epstein, ed. (Academic Press, 1995), Chap. 3.

Philips (in Coop with 3D4YOU), “Response to New Call for 3DV Test Material: Beergarden,” ISO/IEC JTC1/SC29/WG11 Doc. M16421, Apr. 2009.

B. Lee, J.-H. Park, and S.-W. Min, Digital Holography and Three-Dimensional Display, T.-C. Poon, ed. (Springer US, 2006), Chap. 12.

D. Minoli, 3DTV Content Capture, Encoding and Transmission: Building the Transport Infrastructure for Commercial Services (John Wiley and Sons, 2010), Chap. 3.

J.-H. Jung, J. Hong, G. Park, K. Hong, S.-W. Min, and B. Lee, “Evaluation of perceived depth resolution in multi-view three-dimensional display using depth image-based rendering,” in Proceedings of IEEE Conference on 3DTV Conference 2011 (Antalya, Turkey, 2011), pp. 1–4.

Supplementary Material (8)

» Media 1: MOV (10 KB)     
» Media 2: MOV (11 KB)     
» Media 3: MOV (19 KB)     
» Media 4: MOV (107 KB)     
» Media 5: MOV (2681 KB)     
» Media 6: MOV (3809 KB)     
» Media 7: MOV (3477 KB)     
» Media 8: MOV (5186 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1

Evaluation process of perceived depth resolution in multi-view display.

Fig. 2
Fig. 2

Multi-view 3D display based on slanted lenticular lens: parameters of slanted lenticular system (a) in front view and (b) in upper view.

Fig. 3
Fig. 3

Expressible depth planes in (a) real and (b) virtual mode of multi-view display (k = 3).

Fig. 4
Fig. 4

Parameters of stereo to multi-view camera configuration and display: (a) stereo pickup, (b) multi-view pickup, and (c) multi-view display.

Fig. 5
Fig. 5

Contents for evaluation process of perceived depth resolution: (a) pyramid, (b) car, (c) cow, and (d) beergarden.

Fig. 6
Fig. 6

Acquired depth maps of 4 contents with varying depth resolution from 1 bit to 12 bits: (a) pyramid (Media 1), (b) car (Media 2), (c) cow (Media 3), and (d) beergarden (Media 4).

Fig. 7
Fig. 7

Numerical comparison of synthesized view image and ground truth image in PSNR and NCC with varying depth resolution: (a) pyramid, (b) car, (c) cow, (d) beergarden.

Fig. 8
Fig. 8

Experimental setup of subjective test for perceived depth resolution.

Fig. 9
Fig. 9

Represented 3D objects in 4 contents using 9-view slanted lenticular monitor: (a) pyramid (Media 5), (b) car (Media 6), (c) cow (Media 7), and (d) beergarden (Media 8).

Fig. 10
Fig. 10

Experimental result of subjective test with varying depth resolution: (a) pyramid, (b) car, (c) cow, and (d) beergarden.

Fig. 11
Fig. 11

Experimental result of subjective test with average values in different modes.

Tables (3)

Tables Icon

Table 1 Stereo pickup specification of contents for evaluation of perceived depth resolution

Tables Icon

Table 2 Multi-view pickup specification of beergarden contents

Tables Icon

Table 3 Specification of the experimental setup

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

X h = N X v = t p l f p s p cos α ,
D = t d e k p e f f = t p l N p e f f cos α p l .
p l = N p e f f d e cos α k p e f f + d e ,
t = N f p e f f cos α p l .
d n = n p l D k g cos α + n p l .
n r = D N k g cos α ( D D N ) p l , n v = D F k g cos α ( D D F ) p l .
n d = n r n v = k g cos α p l ( D N D D N D F D D F ) .
g c = { g D N D D N D r D N S r D N S , D N S D F S g D F D D F D r D F S r D F S , D N S < D F S .
E c = D g g c D c .
I = ( 2 k 1 ) [ D N S ( D F S D ) D ( D F S D N S ) ] ,

Metrics