Abstract

We propose a computer-generated hologram technique that reduces latency caused by hologram calculations in holographic near-to-eye displays. The proposed method applies a foveated rendering technique to a triangular mesh-based computer-generated hologram to reduce the computational load while maintaining the perceived image quality. Progressive update from low resolution to high resolution is achieved with minimal computational load by overlaying a new high-resolution occluding mesh patch on a low-resolution mesh model of the scene. The reduced latency for the first hologram generation with low reconstruction resolution and its smooth update to the high-resolution reconstructions using the proposed method was verified by numerical simulations and optical experiments.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Continuous shading and its fast update in fully analytic triangular-mesh-based computer generated hologram

Jae-Hyeung Park, Seong-Bok Kim, Han-Ju Yeom, Hee-Jae Kim, HuiJun Zhang, BoNi Li, Yeong-Min Ji, Sang-Hoo Kim, and Seok-Bum Ko
Opt. Express 23(26) 33893-33901 (2015)

Speckle reduction using angular spectrum interleaving for triangular mesh based computer generated hologram

Seok-Beom Ko and Jae-Hyeung Park
Opt. Express 25(24) 29788-29797 (2017)

References

  • View by:
  • |
  • |
  • |

  1. E. Moon, M. Kim, J. Roh, H. Kim, and J. Hahn, “Holographic head-mounted display with RGB light emitting diode light source,” Opt. Express 22(6), 6526–6534 (2014).
    [Crossref] [PubMed]
  2. H.-J. Yeom, H.-J. Kim, S.-B. Kim, H. Zhang, B. Li, Y.-M. Ji, S.-H. Kim, and J.-H. Park, “3D holographic head mounted display using holographic optical elements with astigmatism aberration compensation,” Opt. Express 23(25), 32025–32034 (2015).
    [Crossref] [PubMed]
  3. A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 85 (2017).
    [Crossref]
  4. J.-H. Park and S.-B. Kim, “Optical see-through holographic near-eye-display with eyebox steering and depth of field control,” Opt. Express 26(21), 27076–27088 (2018).
    [Crossref] [PubMed]
  5. T. J. Buker, D. A. Vincenzi, and J. E. Deaton, “The effect of apparent latency on simulator sickness while using a see-through helmet-mounted display: reducing apparent latency with predictive compensation,” Hum. Factors 54(2), 235–249 (2012).
    [Crossref] [PubMed]
  6. R. Albert, A. Patney, D. Luebke, and J. Kim, “Latency requirements for foveated rendering in virtual reality,” ACM Trans. Appl. Percept. 14(4), 25 (2017).
    [Crossref]
  7. B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder, “Foveated 3D graphics,” ACM Trans. Graph. 31(6), 156 (2012).
    [Crossref]
  8. A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
    [Crossref]
  9. E. Arabadzhiyska, O. T. Tursun, K. Myszkowski, H.-P. Seidel, and P. Didyk, “Saccade landing position prediction for gaze-contingent rendering,” ACM Trans. Graph. 36(4), 50 (2017).
    [Crossref]
  10. J. S. Hong, Y. M. Kim, S. H. Hong, C. S. Shin, and H. J. Kang, “Gaze contingent hologram synthesis for holographic head-mounted-display,” Proc. SPIE 9771, 97710K (2016).
    [Crossref]
  11. L. Wei and Y. Sakamoto, “Fast calculation method with foveated rendering for computer-generated holograms using an angle-changeable ray-tracing method,” Appl. Opt. 58(5), A258–A266 (2019).
    [Crossref] [PubMed]
  12. M. Askari, S. B. Kim, K. S. Shin, S. B. Ko, S. H. Kim, D. Y. Park, Y. G. Ju, and J. H. Park, “Occlusion handling using angular spectrum convolution in fully analytical mesh based computer generated hologram,” Opt. Express 25(21), 25867–25878 (2017).
    [Crossref] [PubMed]
  13. J. H. Park, “Recent progresses in computer generated holography for three-dimensional scene,” J. Inform. Disp. 18(1), 1–12 (2017).
    [Crossref]
  14. J. H. Park, S. B. Kim, H. J. Yeom, H. J. Kim, H. Zhang, B. Li, Y. M. Ji, S. H. Kim, and S. B. Ko, “Continuous shading and its fast update in fully analytic triangular-mesh-based computer generated hologram,” Opt. Express 23(26), 33893–33901 (2015).
    [Crossref] [PubMed]
  15. H. Hoppe, “Progressive meshes,” in Proceedings of SIGGRAPH (ACM SIGGRAPH, 1996), pp. 99–108.
    [Crossref]
  16. H. Hoppe, “View-dependent refinement of progressive meshes,” in Proceedings of SIGGRAPH (ACM SIGGRAPH, 1997), pp. 189–198.
    [Crossref]

2019 (1)

2018 (1)

2017 (5)

R. Albert, A. Patney, D. Luebke, and J. Kim, “Latency requirements for foveated rendering in virtual reality,” ACM Trans. Appl. Percept. 14(4), 25 (2017).
[Crossref]

A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 85 (2017).
[Crossref]

M. Askari, S. B. Kim, K. S. Shin, S. B. Ko, S. H. Kim, D. Y. Park, Y. G. Ju, and J. H. Park, “Occlusion handling using angular spectrum convolution in fully analytical mesh based computer generated hologram,” Opt. Express 25(21), 25867–25878 (2017).
[Crossref] [PubMed]

J. H. Park, “Recent progresses in computer generated holography for three-dimensional scene,” J. Inform. Disp. 18(1), 1–12 (2017).
[Crossref]

E. Arabadzhiyska, O. T. Tursun, K. Myszkowski, H.-P. Seidel, and P. Didyk, “Saccade landing position prediction for gaze-contingent rendering,” ACM Trans. Graph. 36(4), 50 (2017).
[Crossref]

2016 (2)

J. S. Hong, Y. M. Kim, S. H. Hong, C. S. Shin, and H. J. Kang, “Gaze contingent hologram synthesis for holographic head-mounted-display,” Proc. SPIE 9771, 97710K (2016).
[Crossref]

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

2015 (2)

2014 (1)

2012 (2)

T. J. Buker, D. A. Vincenzi, and J. E. Deaton, “The effect of apparent latency on simulator sickness while using a see-through helmet-mounted display: reducing apparent latency with predictive compensation,” Hum. Factors 54(2), 235–249 (2012).
[Crossref] [PubMed]

B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder, “Foveated 3D graphics,” ACM Trans. Graph. 31(6), 156 (2012).
[Crossref]

Albert, R.

R. Albert, A. Patney, D. Luebke, and J. Kim, “Latency requirements for foveated rendering in virtual reality,” ACM Trans. Appl. Percept. 14(4), 25 (2017).
[Crossref]

Arabadzhiyska, E.

E. Arabadzhiyska, O. T. Tursun, K. Myszkowski, H.-P. Seidel, and P. Didyk, “Saccade landing position prediction for gaze-contingent rendering,” ACM Trans. Graph. 36(4), 50 (2017).
[Crossref]

Askari, M.

Benty, N.

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

Buker, T. J.

T. J. Buker, D. A. Vincenzi, and J. E. Deaton, “The effect of apparent latency on simulator sickness while using a see-through helmet-mounted display: reducing apparent latency with predictive compensation,” Hum. Factors 54(2), 235–249 (2012).
[Crossref] [PubMed]

Deaton, J. E.

T. J. Buker, D. A. Vincenzi, and J. E. Deaton, “The effect of apparent latency on simulator sickness while using a see-through helmet-mounted display: reducing apparent latency with predictive compensation,” Hum. Factors 54(2), 235–249 (2012).
[Crossref] [PubMed]

Didyk, P.

E. Arabadzhiyska, O. T. Tursun, K. Myszkowski, H.-P. Seidel, and P. Didyk, “Saccade landing position prediction for gaze-contingent rendering,” ACM Trans. Graph. 36(4), 50 (2017).
[Crossref]

Drucker, S.

B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder, “Foveated 3D graphics,” ACM Trans. Graph. 31(6), 156 (2012).
[Crossref]

Finch, M.

B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder, “Foveated 3D graphics,” ACM Trans. Graph. 31(6), 156 (2012).
[Crossref]

Georgiou, A.

A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 85 (2017).
[Crossref]

Guenter, B.

B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder, “Foveated 3D graphics,” ACM Trans. Graph. 31(6), 156 (2012).
[Crossref]

Hahn, J.

Hong, J. S.

J. S. Hong, Y. M. Kim, S. H. Hong, C. S. Shin, and H. J. Kang, “Gaze contingent hologram synthesis for holographic head-mounted-display,” Proc. SPIE 9771, 97710K (2016).
[Crossref]

Hong, S. H.

J. S. Hong, Y. M. Kim, S. H. Hong, C. S. Shin, and H. J. Kang, “Gaze contingent hologram synthesis for holographic head-mounted-display,” Proc. SPIE 9771, 97710K (2016).
[Crossref]

Hoppe, H.

H. Hoppe, “Progressive meshes,” in Proceedings of SIGGRAPH (ACM SIGGRAPH, 1996), pp. 99–108.
[Crossref]

H. Hoppe, “View-dependent refinement of progressive meshes,” in Proceedings of SIGGRAPH (ACM SIGGRAPH, 1997), pp. 189–198.
[Crossref]

Ji, Y. M.

Ji, Y.-M.

Ju, Y. G.

Kang, H. J.

J. S. Hong, Y. M. Kim, S. H. Hong, C. S. Shin, and H. J. Kang, “Gaze contingent hologram synthesis for holographic head-mounted-display,” Proc. SPIE 9771, 97710K (2016).
[Crossref]

Kaplanyan, A.

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

Kim, H.

Kim, H. J.

Kim, H.-J.

Kim, J.

R. Albert, A. Patney, D. Luebke, and J. Kim, “Latency requirements for foveated rendering in virtual reality,” ACM Trans. Appl. Percept. 14(4), 25 (2017).
[Crossref]

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

Kim, M.

Kim, S. B.

Kim, S. H.

Kim, S.-B.

Kim, S.-H.

Kim, Y. M.

J. S. Hong, Y. M. Kim, S. H. Hong, C. S. Shin, and H. J. Kang, “Gaze contingent hologram synthesis for holographic head-mounted-display,” Proc. SPIE 9771, 97710K (2016).
[Crossref]

Ko, S. B.

Kollin, J. S.

A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 85 (2017).
[Crossref]

Lefohn, A.

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

Li, B.

Luebke, D.

R. Albert, A. Patney, D. Luebke, and J. Kim, “Latency requirements for foveated rendering in virtual reality,” ACM Trans. Appl. Percept. 14(4), 25 (2017).
[Crossref]

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

Maimone, A.

A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 85 (2017).
[Crossref]

Moon, E.

Myszkowski, K.

E. Arabadzhiyska, O. T. Tursun, K. Myszkowski, H.-P. Seidel, and P. Didyk, “Saccade landing position prediction for gaze-contingent rendering,” ACM Trans. Graph. 36(4), 50 (2017).
[Crossref]

Park, D. Y.

Park, J. H.

Park, J.-H.

Patney, A.

R. Albert, A. Patney, D. Luebke, and J. Kim, “Latency requirements for foveated rendering in virtual reality,” ACM Trans. Appl. Percept. 14(4), 25 (2017).
[Crossref]

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

Roh, J.

Sakamoto, Y.

Salvi, M.

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

Seidel, H.-P.

E. Arabadzhiyska, O. T. Tursun, K. Myszkowski, H.-P. Seidel, and P. Didyk, “Saccade landing position prediction for gaze-contingent rendering,” ACM Trans. Graph. 36(4), 50 (2017).
[Crossref]

Shin, C. S.

J. S. Hong, Y. M. Kim, S. H. Hong, C. S. Shin, and H. J. Kang, “Gaze contingent hologram synthesis for holographic head-mounted-display,” Proc. SPIE 9771, 97710K (2016).
[Crossref]

Shin, K. S.

Snyder, J.

B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder, “Foveated 3D graphics,” ACM Trans. Graph. 31(6), 156 (2012).
[Crossref]

Tan, D.

B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder, “Foveated 3D graphics,” ACM Trans. Graph. 31(6), 156 (2012).
[Crossref]

Tursun, O. T.

E. Arabadzhiyska, O. T. Tursun, K. Myszkowski, H.-P. Seidel, and P. Didyk, “Saccade landing position prediction for gaze-contingent rendering,” ACM Trans. Graph. 36(4), 50 (2017).
[Crossref]

Vincenzi, D. A.

T. J. Buker, D. A. Vincenzi, and J. E. Deaton, “The effect of apparent latency on simulator sickness while using a see-through helmet-mounted display: reducing apparent latency with predictive compensation,” Hum. Factors 54(2), 235–249 (2012).
[Crossref] [PubMed]

Wei, L.

Wyman, C.

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

Yeom, H. J.

Yeom, H.-J.

Zhang, H.

ACM Trans. Appl. Percept. (1)

R. Albert, A. Patney, D. Luebke, and J. Kim, “Latency requirements for foveated rendering in virtual reality,” ACM Trans. Appl. Percept. 14(4), 25 (2017).
[Crossref]

ACM Trans. Graph. (4)

B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder, “Foveated 3D graphics,” ACM Trans. Graph. 31(6), 156 (2012).
[Crossref]

A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 179 (2016).
[Crossref]

E. Arabadzhiyska, O. T. Tursun, K. Myszkowski, H.-P. Seidel, and P. Didyk, “Saccade landing position prediction for gaze-contingent rendering,” ACM Trans. Graph. 36(4), 50 (2017).
[Crossref]

A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 85 (2017).
[Crossref]

Appl. Opt. (1)

Hum. Factors (1)

T. J. Buker, D. A. Vincenzi, and J. E. Deaton, “The effect of apparent latency on simulator sickness while using a see-through helmet-mounted display: reducing apparent latency with predictive compensation,” Hum. Factors 54(2), 235–249 (2012).
[Crossref] [PubMed]

J. Inform. Disp. (1)

J. H. Park, “Recent progresses in computer generated holography for three-dimensional scene,” J. Inform. Disp. 18(1), 1–12 (2017).
[Crossref]

Opt. Express (5)

Proc. SPIE (1)

J. S. Hong, Y. M. Kim, S. H. Hong, C. S. Shin, and H. J. Kang, “Gaze contingent hologram synthesis for holographic head-mounted-display,” Proc. SPIE 9771, 97710K (2016).
[Crossref]

Other (2)

H. Hoppe, “Progressive meshes,” in Proceedings of SIGGRAPH (ACM SIGGRAPH, 1996), pp. 99–108.
[Crossref]

H. Hoppe, “View-dependent refinement of progressive meshes,” in Proceedings of SIGGRAPH (ACM SIGGRAPH, 1997), pp. 189–198.
[Crossref]

Supplementary Material (1)

NameDescription
» Visualization 1       Interactive demonstration of the progressive update of the foveated computer generated hologram

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1 Foveated hologram concept.
Fig. 2
Fig. 2 Comparison between point density control and mesh density control.
Fig. 3
Fig. 3 Proposed method. (a) Desired reconstructions of the foveated and progressive update process, (b) update operation of the proposed method illustrated in reconstruction space, (c) update operation of the proposed method illustrated in hologram plane.
Fig. 4
Fig. 4 Hierarchical mesh vertex representation. (a) vertices hierarchy (b) local resolution control according to the eye gaze point.
Fig. 5
Fig. 5 Holograms and corresponding reconstructions for a single object (a) with a normal plane carrier wave, and (b) with a random phase carrier wave.
Fig. 6
Fig. 6 Holograms and corresponding reconstructions for two objects (a) holograms and (b),(c) corresponding reconstructions of two objects (stars and dragon).
Fig. 7
Fig. 7 Optical experiment setup.
Fig. 8
Fig. 8 Optical reconstructions captured when the camera focus is at (a) dragon, and (b) stars.
Fig. 9
Fig. 9 Result of the interactive update experiment (see Visualization 1).
Fig. 10
Fig. 10 Geometry for mesh angular spectrum calculation.

Tables (3)

Tables Icon

Table 1 Computation time comparison between the proposed progressive method and conventional one-step method in Fig. 5 simulation.

Tables Icon

Table 2 Computation time comparison between the progressive method and conventional one-step method in Fig. 6 simulation.

Tables Icon

Table 3 Computation time in the interactive update experiment.

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

A S updated ( f x,y )=A S prev ( f x,y )+ n=1 N A S n ( f x,y )A S prev,n ( f x,y ) ,
A S n ( f x,y )A S prev,n ( f x,y )=[ { D n ( f x,y )A S prev ( f x,y ) P n ( f x,y ) } B n ( f x,y ) ] E n ( f x,y ),
r xl,yl,zl =R( r x,y,z r x,y,z o ),
P n ( f x,y )=exp[ j2π f x,y,z T r x,y,z o ],
B n ( f x,y )=A S r ( A T [ 1 0 0 0 1 0 ]R f x,y ),
E n ( f x,y )= exp[ j2π f x,y,z T r x,y,z o ] det( A ) f zl f z ,

Metrics