Abstract

The human visual system can perceive a much wider range of contrast than the addressable dynamic range afforded by the state-of-art head mounted displays (HMD). Increasing the dynamic range of HMDs becomes critical especially for augmented reality applications where the dynamic range of outdoor scenes can be as large as 14 orders of magnitude. In this paper, we present the integrated work of the design, implementation, calibration, and image-rendering algorithm of a high dynamic range HMD system. By using a pair of LCoS microdisplays as the spatial light modulators, accompanied with the relay optics to optically overlay the modulation layers, we demonstrate the reconstruction of high dynamic range images with high accuracy.

© 2017 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Systematic characterization and optimization of 3D light field displays

Hekun Huang and Hong Hua
Opt. Express 25(16) 18508-18525 (2017)

Development of an immersive virtual reality head-mounted display with high performance

Yunqi Wang, Weiqi Liu, Xiangxiang Meng, Hanyi Fu, Daliang Zhang, Yusi Kang, Rui Feng, Zhonglun Wei, Xiuqing Zhu, and Guohua Jiang
Appl. Opt. 55(25) 6969-6977 (2016)

Design and fabrication of an off-axis see-through head-mounted display with an x–y polynomial surface

Zhenrong Zheng, Xu Liu, Haifeng Li, and Liang Xu
Appl. Opt. 49(19) 3661-3668 (2010)

References

  • View by:
  • |
  • |
  • |

  1. D. Gerwin, H. Seetzen, G. Ward, W. Heidrich, and L. Whitehead, “3.2: High dynamic range projection systems,” SID Symposium Digest of Technical Papers38(1), 4–7 (2007).
  2. B. Hoefflinger, High-Dynamic-Range (HDR) Vision (Springer 2007).
  3. H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
    [Crossref]
  4. G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
    [Crossref]
  5. G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
    [Crossref]
  6. M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
    [Crossref]
  7. S. H. Lu and H. Hua, “Imaging properties of extended depth of field microscopy through single-shot focus scanning,” Opt. Express 23(8), 10714–10731 (2015).
    [Crossref] [PubMed]
  8. J. B. Sibarita, “Deconvolution microscopy,” Adv. Biochem. Eng. Biotechnol. 95, 201–243 (2005).
    [Crossref] [PubMed]
  9. P. Nema, “Digital imaging and communications in medicine (DICOM) Part 14: Grayscale standard display function” National Electrical Manufacturers Association, Rosslyn, VA (2000).
  10. R. Hainich and O. Bimber, Displays: Fundamentals and Applications (CRC, 2016).
  11. CITIZEN FINEDEVICE Co, LTD., “QuadVGA −1280×960, 0.40″ diagonal, single chip FLCoS display,” https://www.miyotadca.com/mdca_product/quadvga .
  12. M. Xu and H. Hua, “46‐1: Dual‐layer High Dynamic Range Head Mounted Display,” SID Symposium Digest of Technical Papers48(1), 668–671 (2017).
    [Crossref]
  13. Z. Zhang, “Flexible camera calibration by viewing a plane from unknown orientations,” in The Proceedings of the Seventh IEEE International Conference on Computer Vision (IEEE, 1999) 1, pp. 666–673.
    [Crossref]
  14. S. Lee and H. Hua, “A robust camera-based method for optical distortion calibration of head-mounted displays,” J. Disp. Technol. 11(10), 845–853 (2015).
    [Crossref]
  15. O. Faugeras, Three-dimensional Computer Vision: A Geometric Viewpoint (MIT, 1993).

2015 (2)

S. Lee and H. Hua, “A robust camera-based method for optical distortion calibration of head-mounted displays,” J. Disp. Technol. 11(10), 845–853 (2015).
[Crossref]

S. H. Lu and H. Hua, “Imaging properties of extended depth of field microscopy through single-shot focus scanning,” Opt. Express 23(8), 10714–10731 (2015).
[Crossref] [PubMed]

2014 (1)

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

2012 (1)

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

2011 (1)

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

2005 (1)

J. B. Sibarita, “Deconvolution microscopy,” Adv. Biochem. Eng. Biotechnol. 95, 201–243 (2005).
[Crossref] [PubMed]

2004 (1)

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

Gerwin, D.

D. Gerwin, H. Seetzen, G. Ward, W. Heidrich, and L. Whitehead, “3.2: High dynamic range projection systems,” SID Symposium Digest of Technical Papers38(1), 4–7 (2007).

Ghosh, A.

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

Heidrich, W.

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

D. Gerwin, H. Seetzen, G. Ward, W. Heidrich, and L. Whitehead, “3.2: High dynamic range projection systems,” SID Symposium Digest of Technical Papers38(1), 4–7 (2007).

Hirsch, M.

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

Hua, H.

S. Lee and H. Hua, “A robust camera-based method for optical distortion calibration of head-mounted displays,” J. Disp. Technol. 11(10), 845–853 (2015).
[Crossref]

S. H. Lu and H. Hua, “Imaging properties of extended depth of field microscopy through single-shot focus scanning,” Opt. Express 23(8), 10714–10731 (2015).
[Crossref] [PubMed]

M. Xu and H. Hua, “46‐1: Dual‐layer High Dynamic Range Head Mounted Display,” SID Symposium Digest of Technical Papers48(1), 668–671 (2017).
[Crossref]

Lanman, D.

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

Lee, S.

S. Lee and H. Hua, “A robust camera-based method for optical distortion calibration of head-mounted displays,” J. Disp. Technol. 11(10), 845–853 (2015).
[Crossref]

Lu, S. H.

Raskar, R.

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

Seetzen, H.

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

D. Gerwin, H. Seetzen, G. Ward, W. Heidrich, and L. Whitehead, “3.2: High dynamic range projection systems,” SID Symposium Digest of Technical Papers38(1), 4–7 (2007).

Sibarita, J. B.

J. B. Sibarita, “Deconvolution microscopy,” Adv. Biochem. Eng. Biotechnol. 95, 201–243 (2005).
[Crossref] [PubMed]

Stuerzlinger, W.

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

Trentacoste, M.

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

Vorozcovs, A.

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

Ward, G.

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

D. Gerwin, H. Seetzen, G. Ward, W. Heidrich, and L. Whitehead, “3.2: High dynamic range projection systems,” SID Symposium Digest of Technical Papers38(1), 4–7 (2007).

Wetzstein, G.

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

Whitehead, L.

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

D. Gerwin, H. Seetzen, G. Ward, W. Heidrich, and L. Whitehead, “3.2: High dynamic range projection systems,” SID Symposium Digest of Technical Papers38(1), 4–7 (2007).

Xu, M.

M. Xu and H. Hua, “46‐1: Dual‐layer High Dynamic Range Head Mounted Display,” SID Symposium Digest of Technical Papers48(1), 668–671 (2017).
[Crossref]

ACM Trans. Graph. (4)

H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, “High dynamic range display systems,” ACM Trans. Graph. 23(3), 760–768 (2004).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

Adv. Biochem. Eng. Biotechnol. (1)

J. B. Sibarita, “Deconvolution microscopy,” Adv. Biochem. Eng. Biotechnol. 95, 201–243 (2005).
[Crossref] [PubMed]

J. Disp. Technol. (1)

S. Lee and H. Hua, “A robust camera-based method for optical distortion calibration of head-mounted displays,” J. Disp. Technol. 11(10), 845–853 (2015).
[Crossref]

Opt. Express (1)

Other (8)

O. Faugeras, Three-dimensional Computer Vision: A Geometric Viewpoint (MIT, 1993).

D. Gerwin, H. Seetzen, G. Ward, W. Heidrich, and L. Whitehead, “3.2: High dynamic range projection systems,” SID Symposium Digest of Technical Papers38(1), 4–7 (2007).

B. Hoefflinger, High-Dynamic-Range (HDR) Vision (Springer 2007).

P. Nema, “Digital imaging and communications in medicine (DICOM) Part 14: Grayscale standard display function” National Electrical Manufacturers Association, Rosslyn, VA (2000).

R. Hainich and O. Bimber, Displays: Fundamentals and Applications (CRC, 2016).

CITIZEN FINEDEVICE Co, LTD., “QuadVGA −1280×960, 0.40″ diagonal, single chip FLCoS display,” https://www.miyotadca.com/mdca_product/quadvga .

M. Xu and H. Hua, “46‐1: Dual‐layer High Dynamic Range Head Mounted Display,” SID Symposium Digest of Technical Papers48(1), 668–671 (2017).
[Crossref]

Z. Zhang, “Flexible camera calibration by viewing a plane from unknown orientations,” in The Proceedings of the Seventh IEEE International Conference on Computer Vision (IEEE, 1999) 1, pp. 666–673.
[Crossref]

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

Schematic layout of multi-layer modulating scheme of 2D HDR display with two different cases. (a) two SLM layers perfectly overlay as they have same spatial resolution. The spatial frequency contents of the image are distributed equally in both layers and (b) two SLM layers are separated with a gap as they have different spatial resolution. The spatial frequency contents of the image are distributed unevenly in two layers.

Fig. 2
Fig. 2

Image reconstruction results of a dual-layer HDR-HMD and reconstructed image performance evaluation with the physical separation of the two SLMs set at 0.01mm and 2mm, respectively. (a) tone-mapped HDR target image, along with the intensity distribution along the dashed line; (b) the tone-mapped reconstructed HDR image; (c) binary noticeable difference map, where white areas denote the pixels with errors beyond the JND threshold; (d) PSF cross section at the display layer.

Fig. 3
Fig. 3

Schematic layout of a monocular HDR-HMD based on dual-layer modulation scheme.

Fig. 4
Fig. 4

Optical design of an HDR image generator: (a) optical layout; (b) polychromatic modulation transfer function; and (c) distortion.

Fig. 5
Fig. 5

Bench prototype of an HDR-HMD system based on the optical design in Fig. 4.

Fig. 6
Fig. 6

Illustration of the projection process of points displayed in the virtual images of the SLMs onto the camera image plane (black vectors and coordinates are denoted in the camera global coordinate OXYZ).

Fig. 7
Fig. 7

Geometric calibration procedure and alignment performance of a dual-layer HDR display. (a) geometric rendering procedure to create pre-warped images for LCoS1 and LCoS2 using a repetitive dot pattern as an input image; (b) error analysis after alignment procedure, Asterisk and circular dots stand for sampled coordinates on each LCoS. Arrow directions and magnitudes stand for the directions and relative magnitudes of the residual alignment errors; (c.1) and (c.2) photographs captured through the HDR system before applying the geometric pre-warping rendering and one after applying the correction, respectively.

Fig. 8
Fig. 8

The results of radiance response calibration and compensation. (a) red, green and blue channel response curves for LCoS1, (b) field normalization map obtained from radiance calibration, (c) tone response curves of LCoS1 green channel at the central field and four corner fields after radiance correction.

Fig. 9
Fig. 9

HDR image generation: (a)-(k) raw LDR images of an HDR scene captured with different camera exposure settings with the exposure time shown on the top-left corner of each image, (l) tone-mapped HDR image synthesized from the raw LDR images.

Fig. 10
Fig. 10

Performance comparison of an HDR-HMD and a LDR-HMD. (a)-(c) images captured by a camera of different exposures through an HDR-HMD using an HDR image source, a LDR-HMD using a tone-mapped LDR image source, and a LDR-HMD using an LDR image under moderate exposure. (d)-(f) images synthesized and tone-mapped from the raw images in (a), (b) and (c), respectively.

Equations (2)

Equations on this page are rendered with MathJax. Learn more.

PSF(r,Δz)= | 2 0 1 J 0 ( 2π λ NArρ)exp( i 2π λ σ ρ 2 )ρdρ | 2 with σ=2Δz sin 2 ( α 2 )
[ x L1 y L1 0 1 ]= T camL1 1 αt T camL2 [ x L2 y L2 0 1 ] where t= A 2 + B 2 + C 2 Al+Bn+Cp

Metrics