Abstract

While the augmentation of head-mounted displays (HMDs) with eye-tracking (ET) capabilities adds challenges to designing compact and portable displays, a systematic approach to integration offers opportunities to improve overall performance and robustness. To investigate the design and optimization of illumination schemes in an ET-HMD system, we present a simulated eye illumination and imaging system, which allows us to explore the critical parameters that affect the quality of the eye illumination. We present details on the modeling process and simulation results.

© 2007 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |

  1. W. Barfield and T. Caudell, eds., Fundamentals of Wearable Computers and Augmented Reality (Lawrence Erlbaum Associates, 2001).
  2. O. Bimber and R. Raskar, Spatial Augmented Reality: Merging Real and Virtual Worlds (A. K. Peters, 2005).
  3. O. Cakmakci and J. P. Rolland, "Head-worn displays: a review," J. Display Technol. 2, 199-216 (2006).
    [CrossRef]
  4. R. J. K. Jacob and K. S. Karn, "Eye tracking in human-computer interaction and usability research: ready to deliver the promises (Section Commentary)," in The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research, J. Hyona, R. Radach, and H. Deubel, eds. (Elsevier Science, 2003), pp. 573-605.
  5. M. Hayhoe, D. Ballard, J. Triesch, and H. Shinoda, "Vision in natural and virtual environments," in Proceedings of ACM 2002 Symposium of Eye Tracking Research and Applications (ACM, 2002), pp. 7-13.
    [CrossRef]
  6. J. Rolland, Y. Ha, and C. Fidopiastis, "Albertian errors in head-mounted displays: I. Choice of eye-point location for a near- or far-field task visualization," J. Opt. Soc. Am. A 21, 901-912 (2004).
    [CrossRef]
  7. K. Iwamoto, K. Komoriya, and K. Tanie, "Eye movement tracking type image display system for wide view image presentation with high-resolution: evaluation of high-resolution image presentation," in International Conference on Intelligent Robots and Systems (IEEE, 2002), pp. 1190-1195.
  8. A. T. Duchowski, "Incorporating the viewer's point-of-regard (POR) in gaze-contingent virtual environments," Proc. SPIE 3295, 332-343 (1998).
    [CrossRef]
  9. L. Vaissie and J. Rolland, "Head mounted display with eye tracking capability," U.S. patent 6,433,760 (13 August 2002).
  10. H. Hua, "Integration of eye tracking capability into optical see-through head mounted displays," Proc. SPIE 4297, 496-503 (2001).
    [CrossRef]
  11. C. Curatu, H. Hua, and J. Rolland, "Projection-based head mounted display with eye tracking capabilities," Proc. SPIE 5875, 128-140 (2005).
  12. A. T. Duchowski, Eye Tracking Methodology: Theory and Practice (Springer-Verlag, 2003).
  13. M. Eizenman, R. C. Frecker, and P. E. Hallett, "Precise non-contacting measurement of eye movements using the corneal reflex," Vision Res. 24, 167-174 (1984).
    [CrossRef] [PubMed]
  14. T. N. Cornsweet and H. D. Crane, "Accurate two-dimensional eye tracker using first and fourth Purkinje images," J. Opt. Soc. Am. 63, 921-928 (1973).
    [CrossRef] [PubMed]
  15. H. Hua, P. Krishnaswamy, and J. P. Rolland, "Video-based eyetracking methods and algorithms in head-mounted displays," Opt. Express 14, 4328-4350 (2006).
    [CrossRef] [PubMed]
  16. P. Krishnaswamy, "Design and assessment of improved feature-based eye tracking methods for head-mounted displays," M.S. thesis (Department of Electrical and Computer Engineering, University of Arizona, 2005).
  17. C. W. Pansing, H. Hua, and J. P. Rolland, "Optimization of illumination schemes in a head-mounted display integrated with eye tracking capabilities," Proc. SPIE 5875, 587501 (2005).
  18. C. Pansing, "Optimization of illumination schemes for an eye-tracked head mounted display," M.S. thesis (College of Optical Sciences, University of Arizona, 2006).
  19. L. G. Farkas, Anthropometry of the Head and Face, 2nd ed. (Raven, 1994).
  20. C. H. Morimoto, D. Koons, A. Amir, and M. Flickner, "Pupil detection and tracking using multiple light sources," Image Vision Comput. 18, 331-335 (2000).
    [CrossRef]
  21. J. Schwiegerling, Field Guide to Visual and Opthalmic Optics (SPIE, 2004).
    [CrossRef]
  22. T. W. Olsen, S. Y. Aaberg, D. H. Geroski, and H. F. Edelhauser, "Human sclera: thickness and surface area," Am. J. Opthamol. 125, 237-241 (1998).
    [CrossRef]
  23. D. A. Atchison and G. Smith, Optics of the Human Eye (Reed Educational and Professional, 2000).
  24. J. P. Craig, P. A. Simmons, S. Patel, and A. Tomlinson, "Refractive index and osmolality of human tears," Optom. Vision Sci. 72, 718-724 (1995).
    [CrossRef]
  25. G. V. G. Baranoski and A. Krishnaswamy, "An introduction to light interaction with human skin," RITA 9, 33-62 (2004).
  26. R. R. Anderson, J. Hu, and J. A. Parrish, "Optical radiation transfer in the human skin and applications in vivo remittance spectroscopy," in Bioengineering and the Skin, R. Marks and P. A. Payne, eds. (MTP Press Ltd., 1981).
  27. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).
  28. V. Tuchin, Tissue Optics (SPIE, 2000).
  29. J. G. Baker, "Highly corrected objective having two inner divergent meniscus components between collective components," U.S. patent 2,532,751 (5 December 1950).
  30. A. Rose, Vision: Human and Electronic (Kluwer Academic, 1973).
  31. R. J. Koshel, "Aspects of illumination system optimization," Proc. SPIE 5529, 206-217 (2004).
    [CrossRef]
  32. G. L. Perterson, "How many rays do I need to trace? Applying the Rose model to computer analysis of illumination systems," Breault Research Organization White Paper (3 January 2005).

2006

2005

C. Curatu, H. Hua, and J. Rolland, "Projection-based head mounted display with eye tracking capabilities," Proc. SPIE 5875, 128-140 (2005).

C. W. Pansing, H. Hua, and J. P. Rolland, "Optimization of illumination schemes in a head-mounted display integrated with eye tracking capabilities," Proc. SPIE 5875, 587501 (2005).

G. L. Perterson, "How many rays do I need to trace? Applying the Rose model to computer analysis of illumination systems," Breault Research Organization White Paper (3 January 2005).

2004

G. V. G. Baranoski and A. Krishnaswamy, "An introduction to light interaction with human skin," RITA 9, 33-62 (2004).

R. J. Koshel, "Aspects of illumination system optimization," Proc. SPIE 5529, 206-217 (2004).
[CrossRef]

J. Rolland, Y. Ha, and C. Fidopiastis, "Albertian errors in head-mounted displays: I. Choice of eye-point location for a near- or far-field task visualization," J. Opt. Soc. Am. A 21, 901-912 (2004).
[CrossRef]

2002

L. Vaissie and J. Rolland, "Head mounted display with eye tracking capability," U.S. patent 6,433,760 (13 August 2002).

2001

H. Hua, "Integration of eye tracking capability into optical see-through head mounted displays," Proc. SPIE 4297, 496-503 (2001).
[CrossRef]

2000

C. H. Morimoto, D. Koons, A. Amir, and M. Flickner, "Pupil detection and tracking using multiple light sources," Image Vision Comput. 18, 331-335 (2000).
[CrossRef]

1998

T. W. Olsen, S. Y. Aaberg, D. H. Geroski, and H. F. Edelhauser, "Human sclera: thickness and surface area," Am. J. Opthamol. 125, 237-241 (1998).
[CrossRef]

A. T. Duchowski, "Incorporating the viewer's point-of-regard (POR) in gaze-contingent virtual environments," Proc. SPIE 3295, 332-343 (1998).
[CrossRef]

1995

J. P. Craig, P. A. Simmons, S. Patel, and A. Tomlinson, "Refractive index and osmolality of human tears," Optom. Vision Sci. 72, 718-724 (1995).
[CrossRef]

1984

M. Eizenman, R. C. Frecker, and P. E. Hallett, "Precise non-contacting measurement of eye movements using the corneal reflex," Vision Res. 24, 167-174 (1984).
[CrossRef] [PubMed]

1973

1950

J. G. Baker, "Highly corrected objective having two inner divergent meniscus components between collective components," U.S. patent 2,532,751 (5 December 1950).

Am. J. Opthamol.

T. W. Olsen, S. Y. Aaberg, D. H. Geroski, and H. F. Edelhauser, "Human sclera: thickness and surface area," Am. J. Opthamol. 125, 237-241 (1998).
[CrossRef]

Image Vision Comput.

C. H. Morimoto, D. Koons, A. Amir, and M. Flickner, "Pupil detection and tracking using multiple light sources," Image Vision Comput. 18, 331-335 (2000).
[CrossRef]

J. Display Technol.

J. Opt. Soc. Am.

J. Opt. Soc. Am. A

Opt. Express

Optom. Vision Sci.

J. P. Craig, P. A. Simmons, S. Patel, and A. Tomlinson, "Refractive index and osmolality of human tears," Optom. Vision Sci. 72, 718-724 (1995).
[CrossRef]

Proc. SPIE

C. W. Pansing, H. Hua, and J. P. Rolland, "Optimization of illumination schemes in a head-mounted display integrated with eye tracking capabilities," Proc. SPIE 5875, 587501 (2005).

R. J. Koshel, "Aspects of illumination system optimization," Proc. SPIE 5529, 206-217 (2004).
[CrossRef]

A. T. Duchowski, "Incorporating the viewer's point-of-regard (POR) in gaze-contingent virtual environments," Proc. SPIE 3295, 332-343 (1998).
[CrossRef]

H. Hua, "Integration of eye tracking capability into optical see-through head mounted displays," Proc. SPIE 4297, 496-503 (2001).
[CrossRef]

C. Curatu, H. Hua, and J. Rolland, "Projection-based head mounted display with eye tracking capabilities," Proc. SPIE 5875, 128-140 (2005).

RITA

G. V. G. Baranoski and A. Krishnaswamy, "An introduction to light interaction with human skin," RITA 9, 33-62 (2004).

Vision Res.

M. Eizenman, R. C. Frecker, and P. E. Hallett, "Precise non-contacting measurement of eye movements using the corneal reflex," Vision Res. 24, 167-174 (1984).
[CrossRef] [PubMed]

Other

W. Barfield and T. Caudell, eds., Fundamentals of Wearable Computers and Augmented Reality (Lawrence Erlbaum Associates, 2001).

O. Bimber and R. Raskar, Spatial Augmented Reality: Merging Real and Virtual Worlds (A. K. Peters, 2005).

R. J. K. Jacob and K. S. Karn, "Eye tracking in human-computer interaction and usability research: ready to deliver the promises (Section Commentary)," in The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research, J. Hyona, R. Radach, and H. Deubel, eds. (Elsevier Science, 2003), pp. 573-605.

M. Hayhoe, D. Ballard, J. Triesch, and H. Shinoda, "Vision in natural and virtual environments," in Proceedings of ACM 2002 Symposium of Eye Tracking Research and Applications (ACM, 2002), pp. 7-13.
[CrossRef]

P. Krishnaswamy, "Design and assessment of improved feature-based eye tracking methods for head-mounted displays," M.S. thesis (Department of Electrical and Computer Engineering, University of Arizona, 2005).

R. R. Anderson, J. Hu, and J. A. Parrish, "Optical radiation transfer in the human skin and applications in vivo remittance spectroscopy," in Bioengineering and the Skin, R. Marks and P. A. Payne, eds. (MTP Press Ltd., 1981).

G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

V. Tuchin, Tissue Optics (SPIE, 2000).

J. G. Baker, "Highly corrected objective having two inner divergent meniscus components between collective components," U.S. patent 2,532,751 (5 December 1950).

A. Rose, Vision: Human and Electronic (Kluwer Academic, 1973).

C. Pansing, "Optimization of illumination schemes for an eye-tracked head mounted display," M.S. thesis (College of Optical Sciences, University of Arizona, 2006).

L. G. Farkas, Anthropometry of the Head and Face, 2nd ed. (Raven, 1994).

J. Schwiegerling, Field Guide to Visual and Opthalmic Optics (SPIE, 2004).
[CrossRef]

A. T. Duchowski, Eye Tracking Methodology: Theory and Practice (Springer-Verlag, 2003).

L. Vaissie and J. Rolland, "Head mounted display with eye tracking capability," U.S. patent 6,433,760 (13 August 2002).

G. L. Perterson, "How many rays do I need to trace? Applying the Rose model to computer analysis of illumination systems," Breault Research Organization White Paper (3 January 2005).

K. Iwamoto, K. Komoriya, and K. Tanie, "Eye movement tracking type image display system for wide view image presentation with high-resolution: evaluation of high-resolution image presentation," in International Conference on Intelligent Robots and Systems (IEEE, 2002), pp. 1190-1195.

D. A. Atchison and G. Smith, Optics of the Human Eye (Reed Educational and Professional, 2000).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

(Color online) Schematic of an ET-HMD integrated system.

Fig. 2
Fig. 2

(Color online) Example of a well-illuminated eye image: (a) original image; (b) intensity profile along a horizontal line crossing two of the glints.

Fig. 3
Fig. 3

(Color online) Example of an ill-illuminated eye image: (a) original image; (b) intensity profile along a horizontal line crossing two glints.

Fig. 4
Fig. 4

(Color online) Flow chart of the framework for eye illumination-imaging simulation and optimization.

Fig. 5
Fig. 5

(Color online) Subset of input irradiance distribution examples using four Gaussian distributions: The peak offset distance r was varied from 0 to 50 mm, and the fall-off factor of the Gaussian profile δ was varied from 0 to 0.95.

Fig. 6
Fig. 6

Parameters characterizing IRED placements.

Fig. 7
Fig. 7

(Color online) Combined facial and eye structures.

Fig. 8
Fig. 8

(Color online) Simulated eye imaging system.

Fig. 9
Fig. 9

Example of simulated results: (a) Simulated input irradiance with δ equal to 0.8 and r equal to 0   mm ; (b), (c) simulated eye images from input irradiance in (a), for (b) zero eye rotation along with Caucasian skin, and (c) 10° horizontal eye rotation combined with black skin; (d) simulated input irradiance with δ equal to 0.8 and r equal to 20   mm ; (e), (f) simulated eye images from input irradiance in (d), for (e) 10° vertical eye rotation along with Caucasian skin, and (f) 10° diagonal eye rotation combined with black skin.

Fig. 10
Fig. 10

Example of mask generation: (a) Simulated eye image; (b) masked image revealing the skin area; (c) masked image revealing the sclera; (d) masked image revealing the iris-pupil area with glints removed.

Tables (2)

Tables Icon

Table 1 Summary of Variables for the Illumination Generator

Tables Icon

Table 2 Minimally Traced Rays and Computation Time Vary with Desired Pixel Accuracy and Feature Contrast a

Equations (43)

Equations on this page are rendered with MathJax. Learn more.

40   mm × 3 0   mm
r i
r i = ( x i 2 + y i 2 ) 1 / 2
( x i , y i )
δ i = 1 exp ( d 2 / 8 σ i 2 )
σ i
50   mm
2.5   mm
( 0.6 , 30 )
( 0.3 , 50 )
850   nm
5   mm
20 40   mm
30   mm
40   mm × 30   mm
24 × 18
40 m m × 30 m m
87 × 65
3.5 × 10 6
2 h
3.2   GHz
1   GByte
20   mm
A o
M min = A o A k 2 C 2 ,
C = Δ E E ,
P N ( k σ )
Δ E k σ
A o
P f a l s e
P f a l s e = 2 A o A P N ( k σ ) ,
P N ( k σ )
87 × 65
A o / A = 1
M min = S T k 2 e C 2 ,
87 × 65
3.2   GHz
1   GByte
3.5 × 10 6
90 %
99 %
0   mm
20   mm

Metrics