Binocular disparity and monocular depth information are the principal functions of ideal 3D displays. 3D display systems such as stereoscopic or multi-view, super multi-view (SMV), and multi-focus (MF) displays were considered for the testing of the satisfaction level with the monocular accommodation of three different depths of 3D object points. The numerical simulation and experimental results show that the MF 3D display gives a monocular depth cue. In addition, the experimental results of the monocular MF 3D display show clear monocular focus on four different depths. Therefore, we can apply the MF 3D display to monocular 3D displays.
©2008 Optical Society of America
Human perception of 3D images is basically composed of two categories: the optical perception through the eyes, and the brain operation. The optical perception includes the binocular disparity with the convergence and the monocular accommodation via the focus control of each eye. The brain operation involves complex reasoning through the motion parallax caused by the movement of the viewer position, perspective, texture gradient, and shadow, which are related to the experience. Therefore, binocular disparity and monocular accommodation are the principal optical functions of 3D displays that can enable the production of real-life 3D images, and the motion parallax is important for the viewer’s freedom of position. Every 3D display gives binocular disparity, and the motion parallax is provided in the multi-view display, the SMV 3D display, and the VR display using a tracking system [1–6]. Satisfaction with the monocular accommodation is not yet well considered in 3D displays, though. There are researches concerning the monocular depth cue in Integral Imaging [7, 8]. In the case of the near-view 3D display condition, the effect of the monocular accommodation increases with the blurring effect [9, 10], so viewers can see unclear 3D images. Therefore, this is an inherent problem with good 3D image quality at the near-view condition. The monocular accommodation can be perfectly given with the volumetric and the full parallax holographic 3D displays. The applications of the volumetric 3D display  are limited, however, because the displayable 3D volume is confined within the volumetric 3D display volume. Moreover, the high performance level of the required spatial light modulator, the computational power, and the technical difficulty of the diffractive optical property of the electro-holographic display  make it difficult to realize a reasonable holographic 3D display size and holographic 3D image quality in the near future.
There are two additional 3D display types that have the possibility of giving a monocular depth cue. These are the SMV and the MF 3D displays [4, 5, 13, 14]. The SMV 3D display can be a good 3D personal computer monitor or 3D television because of its smooth motion parallax. The MF 3D display is a good candidate for application to the HMD-type monocular 3D display because of its monocular focus control feature. Detailed calculations and additional experimental verification of the monocular accommodation of stereoscopic, multi-view, two SMV types of, and MF-type 3D displays were studied and compared. In addition, the stereoscopic view and the multi-view methods were simply considered the stereoscopic view method hereafter because stereoscopic and multi-view images are the same as 2D images in the monocular view condition. In this paper, the monocular accommodation is considered at the horizontal parallax condition, and the methodology used at the horizontal parallax condition can be applied to the case of the full parallax condition. In addition, the experimental results of the reconstructed monocular MF 3D display give clear evidences of the monocular focus on four different depths.
2. Evaluation of the depth of focus (DOF) at the 3D display conditions
Each unit pixel of a viewpoint image captured as a 2D image of a 3D scene at a defined position and direction of stereoscopic, multi-view, two SMV types of, and MF-type 3D displays can be used to express a 3D point at an arbitrary depth. The experimental and numerical simulation results pertaining to the differences in the diameters of the unit pixels of a viewpoint image on the retina at three different depths via the focus adjustment of the eyeball lens were tested and compared. The differences in the maximum image diameters on the retina of the unit pixel of a viewpoint image at two different depths among infinity, 600 mm, and 250 mm were calculated for all the systems via geometrical optics. A CCD camera with a unit size of 7.4µm in the horizontal and vertical directions, which corresponds to the retina, and a lens with a 100mm focal length (L100mm), which corresponds to the eyeball lens, were used to record the image of the unit pixels for the experimental verification of the numerical simulation. The distance between the center of the eyeball lens and the retina was almost constant at 22.8 mm. The image diameter of the unit pixels that focused on different depths was captured by changing the distance between L100mm and the CCD camera to simulate the differences in the image diameter due to the changes in the thickness of the eyeball lens for the corresponding focus depth. The experimental results that pertained to the diameter differences were converted to values that corresponded to the eye via the ratio (mD) of the distance between L100mm and the CCD surface and the distance between the eyeball lens and the retina. When the focus was on distance D, the lens equation for a simplified eyeball lens was written as in Eq. (1). Equation (2) was formulated to simulate the eyeball lens via the lens L100mm. F in Eq. (2) was FD•mD, and the value was 100 mm.
Here, OD is the distance between the center of the eyeball lens and the target depth of focus among infinity, 600 mm, or 250 mm, and RD is 22.8 mm. The fixed focal length that corresponds to the focal length of the eyeball lens is used to construct an equivalent optical system, as in Eq. (1). From the distance between the lens and the CCD surface, m ∞ is 4.386 for an infinite object distance, m 600mm is 4.553 for a 600mm object distance, and m 250mm is 4.768 for a 250mm object distance using Eqs. (1) and (2). The image diameter of the unit pixel captured by the CCD was converted to the image diameter at the retina situation by multiplying it with 1/mD.
The experimental setups for the capture of the unit pixel of a viewpoint image in the stereoscopic, SMV, and MF conditions are shown in Fig. 1. The stereoscopic 3D display gave two different viewpoint images to two eyes. The schematic diagrams of the stereoscopic 3D display are shown in Figs. 1(a) and 1(b) of Ref. . The SMV condition means more than two unit pixels of viewpoint images were passing through the pupil of each of the viewer’s eyes. The diagram of the SMV condition is shown in Fig. 2 of Ref. . In the stereoscopic imaging condition shown in Fig. 1(a), the light rays from the unit pixel of the viewpoint image always cover the entire pupil area. The distance between the display surface and the viewer pupil is 600 mm, which is the usual viewing distance of the computer monitor in the stereoscopic and SMV imaging systems. In addition, the pupil diameter is 3 mm. Therefore, the aperture diameter (Ap) in the stereoscopic display condition is 3 mm x mD. From the definition of SMV , at least two unit pixels of two viewpoint images pass through the pupil. The experimental setups for the SMV conditions are shown in Figs. 1(a) and 1(b). They correspond to the Focused Light Array (FLA)  and the directional image  SMVs, respectively. Therefore, two unit pixels of two viewpoint images from the surface of the FLA display system and two parallel light beams from the surface of the directional image system go into the pupil diameter in the case of the minimum condition. These situations are shown in Fig. 2 of Ref.  in the case of the FLA SMV, and in Figs. 1 and 2 of Ref.  in the case of the directional image SMV. Therefore, the diameter of the unit pixel on the Ap of the FLA SMV condition is half the size of the Ap in Fig. 1(a). In addition, the diameter of the unit pixel on the Ap in the experimental setup of the FLA and directional image SMV conditions in Figs. 1(a) and 1(b) is 3 mm x mD/2. The parallel beam shown in Fig. 1(b) comes from the lens that forms the parallel beam, and the lens is 600 mm from the viewer’s pupil. The MF 3D display needs a special condition of the light rays from the unit pixel of the viewpoint image. This special condition means that the light rays at the pupil from each unit pixel should have much smaller diameters than the diameter of the pupil. The schematic diagram of the MF 3D display is shown in Fig. 4 of reference . In the case of the MF condition shown in Fig. 1(c), the field of view is matched to the situation of the 17-inch display with a 600mm viewing distance, and the horizontal resolution is 1,024 pixels. Each light beam that corresponds to the unit pixel of a viewpoint image converges 10 mm ahead of the pupil position with a converging angle of 0.0325°. The light source of all the situations in Fig. 1 is the laser light with a 670nm wavelength.
All the measurements of the point spread in Figs. 1(a), 1(b), and 1(c) corresponded to the focus on infinity, 600 mm, and 250 mm via the artificial eye composed of the lens (L100mm) and the CCD when the light source conditions were fixed during the measurements. Therefore, the focus planes were changed to the three different depths when the point spreads were measured.
The experimental results are shown in Fig. 2. Two depths were selected to find the maximum image diameter difference among the three different focus depths. There was a large diameter difference between the two selected depths shown in Figs. 2(a), 2(b), and 2(c). These results mean that a large defocus can occur at those conditions. There was a very small diameter difference, however, between the two selected depths shown in Fig. 2(d). This means that there can be a very small defocus at the MF condition. For a detailed numerical comparison of all the conditions, the experimental results that were converted to the eye situation using the magnification of 1/mD and the numerical simulation through geometrical optics are shown in Table 1.
All the unit pixels of the viewpoint image from the stereoscopic display, FLA, and directional image SMVs, and the MF 3D display conditions had the same optical characteristics regardless of the depths of the representative 3D point.
In the experimental results shown in Table 1, there were 139.14µm and 48.69 µm differences between the 250mm and 600mm focuses at the stereoscopic and FLA SMV display conditions, a 98.35µm difference between the infinity and 250mm focuses at the directional image SMV condition, and a 0.598 µm difference between the infinity and 250mm focuses at the MF 3D display condition. The minimal diameter of the cone cell near the fovea is between 1.0 and 1.5 µm , and the result at the MF 3D display condition was even less than the minimal diameter of the cone cell. These results mean that the condition of the MF 3D display can give the proper monocular depth cue to the eye because the diameter differences of the unit pixels of other types of 3D display conditions are more than several tens of the minimum diameter of the cone cell. Therefore, the MF has the ability to give a monocular depth cue at a wide depth range. By achieving the MF condition, a monocular 3D display can be made as well as a see-through type of monocular MR 3D display, which gives the monocular accommodation to the displayed virtual objects in various depths via only the monocular view.
2. Monocular depth cue at the MF 3D display
The photos in Fig. 3 were captured with a digital video camera, but the observer saw almost the same results with his/her own eyes. The focused characters “K” [Fig. 3(a)], “I” [Fig. 3(b)], “S” [Fig. 3(c)], and “T” [Fig. 3(d)] were clear and the defocuses of the unfocused characters at the different depths were seen with the naked eye and with the digital video camera. Four viewpoint images with 3mm horizontal widths were used to represent the four different depth characters. Therefore, each character split into four same characters when the focus depth was far from the depth position of the concerned character. As a result, the unfocused characters shown in Figs. 3(a)–3(d) seemed to have been blurred, although the unfocused characters were split into four same characters depending on the depth difference between the focused character and the unfocused character. These results are similar to the action of accommodation in the real world. Therefore, the diameter of the unit pixel of a viewpoint image on the retina should not change when the focal length of the eyeball lens changes to focus on different depths. Moreover, the focus effect on the retina that corresponds to the depth of the 3D point should be achieved at only a relative distance between the unit pixel images of the viewpoint image to represent the 3D point, and not by blurring the unit pixel image of the viewpoint image in the considered 3D display.
Figure 3 indirectly shows that the MF 3D display can express four different depths with monocular accommodation. Therefore, a more accurate numerical evidence of the monocular depth cue is necessary, and the long DOF range of each viewpoint image is important in the MF 3D display. The DOF range of one viewpoint image among four viewpoint images of the 3D MF image shown in Fig. 3 was tested, and the experimental DOF condition was found to have been between 0.25 m and 2.0 m. The four different focus images on one viewpoint image are shown in Fig. 4. The width of the vertical part of the character “T” in Fig. 4 was measured 10 times at each focus depth, and the average and standard deviations of the measures are shown in Table 2. The maximum difference between the two extreme average values was 0.07 pixel of the CCD camera. The horizontal pixel size of the CCD camera (Sony DCR-PC115) was s 7.338µm. Therefore, the maximum average difference was 0.514 µm, and the average standard deviation was 2.09 µm. When these values were compared with the values shown in Table 1, they showed a reasonable long DOF between 0.25 m and 2.0 m from the viewpoint of the minimum cone cell size.
The long DOF range of the MF 3D display condition shown in Fig. 1(c) was verified. Even though the monocular depth cue given by the MF 3D display system was not the same as the blurring in the real world, the monocular depth cue given by the relative movement of the unit pixels of the viewpoint images was effective when the evidences in Figs. 2, 3, and 4 and in Tables 1 and 2 are considered. Therefore, it is possible to achieve the monocular 3D depth cue at the MF 3D display condition, but it is almost impossible to give the monocular depth cue at the stereoscopic or SMV display conditions. Moreover, the MF displayed the ability to give a monocular depth cue at various depth levels. By achieving the MF function, a monocular 3D display system can be developed that gives monocular accommodation to the displayed virtual objects at various depths.
The results of the numerical simulation and experimental verification of the monocular accommodation of the stereoscopic, SMV, and MF-type 3D displays were compared. The numerical simulation and experimental results showed that only the MF 3D display condition gave a monocular depth cue. In addition, the experimental results for the monocular MF 3D display showed a clear monocular focus at four different depths.
The diffraction effect for the calculation of the diameter difference between the unit pixel images on the retina and the situation when more than two unit pixels of the viewpoint images represented one 3D point at the SMV display condition were not yet considered. Moreover, only a specific convergence distance and one angle of the MF 3D display condition were considered in this study because the main purpose of this study was to test and verify the condition of eye accommodation at defined 3D depths in various 3D display conditions. Therefore, a more detailed and general eye accommodation condition at the MF 3D display and at the full parallax MF 3D display will be the next subject of the authors’ research.
This work was supported by Korea Institute of Science and Technology (KIST) through the Tangible Web Project.
References and links
1. C. Moller and A. Travis, “Time multiplexed Autostereoscopic Flat Panel Display using an Optical Wedge,” Proc. SPIE 5664, 150–157 (2005). [CrossRef]
2. H. Choi, J. H. ark, J. H. Kim, S. W. Cho, and B. Lee, “Wide-viewing-angle 3D/D convertible display system suing two display devices and a lens array,” Opt. Express 13, 424–8432 (2005). [CrossRef]
3. J. H. Park, H. R. Kim, Y. Kim, J. Kim, J. Hong, S. D. Lee, and B. Lee, “Depth-enhanced three-dimensional - two dimensional convertible display based on modified integral imaging,” Opt. Lett. 29. 2734–2736 (2004). [CrossRef] [PubMed]
4. Y. Kajiki, H. Yoshikawa, and T. Honda, “Hologram-Like Video Images by 45-View Stereoscopic Display,” Proc. SPIE 3012, 154–166 (1997). [CrossRef]
5. H. Nakanuma, H. Kamei, and Y. Takaki, “Natural 3D display with 128 directional images used for human engineering evaluation,” Proc. SPIE 5664, 56–63 (2005).
6. D. Sandin, T. Margolis, J. Ge, J. Girado, T. Peterka, and T. DeFanti, “The VarrierTM autostereoscopic virtual reality, Proc. ACM SIGGRAPH 24, 894–903 (2005). [CrossRef]
8. R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Enhanced depth field integral imaging with sensor resolution constraints,” Opt. Express 12, 5237–5242 (2004). [CrossRef] [PubMed]
9. S. Yano, S. Ide, T. Mitsuhashi, and H. Thwaites, “A study of visual fatigue and visual comfort for 3D HDTV/HDTV images,” Displays 23, 191–201 (2002). [CrossRef]
10. N. Hiruma and T. Fukuda, “Accommodation response to binocular stereoscopic TV images and their viewing condtions,” SMPTE J. 102, 1137–1144 (1993). [CrossRef]
11. W. S. Chun, J. Napoli, O. S. Cossairt, R. K. Dorval, D. M. Hall, T. J. Purtell II, J. F. Schooler, Y. Baker, and G. E. Favalra, “Spatial 3-D Infrastructure: Display-Independent Software Framework, High-Speed Rendering Elecronics, and Several New Displays,” Proc. SPIE 5664, 302–312 (2005). [CrossRef]
12. St. Hilaire, S. A. Benton, M. Lucente, M. L. Jepsen, J. Kollin, H. Yoshikawa, and J. Underkoffler, “Electronic Display System for omputational Holography,” Proc. SPIE 1212, 1212–1220 (1990).
13. S. K. Kim, D. W. Kim, M. C. Park, J. Y. Son, and T. Honda, “Development of the 2nd generation of HMD type multi-focus 3D display system,” Proc. SPIE 6016, 60160P (2005). [CrossRef]
14. S. K. Kim, D. W. Kim, M. C. Park, Y. M Kwon, and J. Y. Son, “Development of a HMD-type multifocus 3D display system using LEDs,” Proc. SPIE 6392, 63920B (2006) [CrossRef]
15. D. Atchison and G. Smith, Optics of the Human Eye, (Butterworth-Heinemann, 2000), Section. 1.