We propose a 360-degree three-dimensional display with a table screen, which consists of a small number of high-speed projectors and a rotating screen. Because each high-speed projector is located outside the rotating axis of the screen, multiple projectors can be aligned. The lens shift technique is used to superimpose multiple images generated by all projectors on the rotating screen. The screen has an off-axis lens function such that the rotation of the screen generates numerous viewing points on a circle. The use of multiple projectors enables an increase in the number of colors, an increase in the number of viewing points, and a reduction in the screen rotation speed. We develop an image synthesis technique for the proposed display system. A color display system employing three high-speed projectors is demonstrated.
© 2012 OSA
Most two-dimensional (2D) displays are provided with a vertical flat display screen that allows viewers to see 2D images from the direction normal to the screen. Most three-dimensional (3D) displays also have a vertical flat screen that can be viewed from the normal direction. Because 3D displays can produce images outside the display screen, various types of screen surfaces can be used and different observation styles can be offered to viewers. In the present study, we develop a 3D display that is provided with a table screen that can be viewed from all 360-degree directions (360-degree display).
Several display systems have been proposed to construct 360-degree 3D displays. In particular, systems consisting of a high-speed projector and a rotating screen [1–7] and those that employ many projectors surrounding a display screen [8–10] have been reported. Systems employing rotating LED arrays have also been proposed [11–15].
360-degree 3D displays with a table screen have been constructed using the former two types of 360-degree 3D display systems: the high-speed projector system [6, 7] and the multiple projector system . Figure 1(a) depicts the high-speed projector system. A high-speed projector is located below a rotating screen. It employs a digital micromirror device (DMD) to generate images at a high frame rate. A holographic screen is used as a projection screen that converts the diverging rays emitted from the projector into parallel rays. The holographic screen is rotated by a servo motor. The system described in Ref. 7 projects 360 images in different horizontal directions at a frame rate of 30 fps. This high-speed projector system has a simple structure. However, the high frame rate operation of the projector usually reduces the number of colors represented by the projector, and stable high-speed rotation of a large screen is difficult. Figure 1(b) depicts the multiple projector system. A large number of projectors are placed around a conical screen. In Ref. 10, a system using 103 small projectors arranged on one-third of the table screen circumference is described. Because there is a large gap between the conical screen and a table screen, blurred 3D images are obtained. The multiple projector system does not require mechanical parts and has good color representation. However, the system is complex, costs high, and requires a large space.
A 360-degree display allows multiple viewers to be located at any position around the display. The introduction of a table screen to 360-degree displays enables face-to-face discussions among the viewers. The 3D image generation on a table screen is suitable for displaying landscape views of 3D objects and scenes. Therefore, it has potential applications in various areas such as industrial design, city planning, and air traffic control system. Because the observed 3D images depend on the viewers’ positions, 360-degree display systems can be used in a table game for multiple players: each player can see his own cards or tiles, but not of his opponents. Moreover, 360-degree systems can display different 3D contents to different viewers.
In order to solve the problems of the previous display systems, the present study proposes a new 360-degree 3D display with a table screen, which combines the high-speed projector system and the multiple projector system; we call this the hybrid 360-degree 3D display. The hybrid display provides better color representation of 3D images, requires a lower rotation speed of the table screen than that required by high-speed projector systems, and needs a much smaller number of projectors than that needed by multiple projector systems. We developed an image synthesis technique to allow the use of multiple projectors simultaneously. Preliminary reporting of the work presented herein was limited to early experimental results and only provided a brief description of the system configuration . It contained only one experimental result showing the superposition of monochromatic images projected by two projectors and did not present any discussion. The work presented here culminates with additional experimental results and expands into 3D color image generation with R, G, and B projectors.
2. Display system
The proposed hybrid 360-degree 3D display system is illustrated in Fig. 2 . The system consists of a small number of high-speed projectors and a rotating screen. Figures 2(a) and 2(b) show the display systems with a transmissive screen and a reflective screen, respectively. Each high-speed projector is located outside the rotation axis of the screen so that multiple projectors can be aligned below or above the rotating screen. The lens shift technique, which shifts an image position laterally by shifting a projection lens position laterally, is used to superimpose all images generated by the multiple off-axis projectors on the rotating screen.
First, we describe the generation of viewing points for the transmissive screen system because the explanation is simpler. As shown in Fig. 3(a) , the table screen has a lens function and the lens axis is not coaxial with the rotation axis of the screen. Thus, an image of the projection lens of the projector is formed outside the rotation axis. This image generates a viewing point because an image displayed by the projector can be seen at this position. The viewing point rotates with the screen, as shown in Fig. 3(b). When the high-speed projector displays several images during one rotation of the screen, several viewing points are generated on a circle, i.e., a circular multiview display is achieved. When the distance between the rotating screen and the projector is denoted by l, that between the rotating screen and viewing points is denoted by l’, and the focal length of the screen lens is denoted by f, as shown in Fig. 3(a), the lens maker’s formula requires 1/l + 1/l’ = 1/f. When the distance between the rotation axis and the lens axis is denoted by r, the radius of the circle on which the viewing points are generated is given by (1 + l’/l)r. When the rotation axis of the screen is placed at the origin of the coordinate system and the position of the projection lens is denoted by (xp, yp), the circle is represented by the following equation:17].
Next, we explain the generation of viewing points by multiple off-axis projectors. As shown in Fig. 4(a) , because the offset is different for all projectors, different projectors generate viewing points at different positions. When the screen rotates, each projector generates a number of viewing points on a different circle, as shown in Fig. 4(b). The parallax images displayed by the projectors need to be calculated by referring to the positions of the viewing points. The image synthesis technique is explained in detail in Section 3.
In Figs. 3 and 4, the screen lens transforms diverging rays emitted from the projectors into rays that converge to a viewing point. Thus, the screen emits rays that converge to the viewing point, as shown in Fig. 5(a) . In this case, the distance between the screen and projectors is greater than the focal length of the screen lens, i.e., l > f. This image projection method was previously used in the 360-degree display system having a vertical screen . Other ray transformation schemes can also be used, as shown in Figs. 5(b) and 5(c). When the screen emits diverging rays, as shown in Fig. 5(b), virtual viewing points are formed behind the screen. In this case, the distance between the screen and projectors is less than the focal length of the screen lens, i.e., l < f. Image projection with diverging rays was previously employed in the high-speed projector system  and the multiple projector system . When the screen emits parallel rays, as shown in Fig. 5(c), viewing points are formed at infinity. In this case, the distance between the screen and projectors is equal to the focal length of the screen lens, i.e., l = f. This ray transformation scheme was used in the high-speed projector system with a table screen [6, 7].
The proposed hybrid 360-degree display system enables the use of multiple high-speed projectors. The number of colors generated by the high-speed projector often decreases with an increase in the frame rate of the projector. In the case of monochromatic image generation, when the number of projectors is N and each projector generates binary images, the use of multiple projectors can increase the number of graylevels of 3D images to 2N by determining the illumination light intensity of the projectors as a power of two. In the case of color image generation, when the numbers of red, green, and blue projectors are R, G, and B, respectively, the number of colors can be increased to 2R + G + B by determining the illumination light intensity of the projectors as a power of two in each color. The use of multiple projectors also enables increasing the number of viewing points. The illumination light of the projectors should be temporally modulated to avoid superposition of viewing points generated by different projectors. The rotation speed of the screen might also be decreased because multiple viewing points rotate during one screen rotation. Thus, the hybrid system addresses the drawbacks of the high-speed projector system. Moreover, it requires fewer projectors than the multiple projector system.
Until now, we explained a display system that employs a transmissive screen. A reflective screen can also be used to construct the proposed display system. The transmissive screen is required to have both lens and vertical light diffusion functions, whereas the reflective screen is required to have a reflection function in addition to the two functions. It is easier to rotate the reflective screen because a rotation shaft can be attached to the center of the back side of the screen. The rotation of the transmissive screen is more difficult because the screen has to be rotated along its perimeter. However, the use of the transmissive screen allows the installation of an all optical system below the screen. The prototype display system shown in this paper employs the reflective screen.
3. Image synthesis
Now, we explain the image synthesis method developed for the proposed system. The display system provides only horizontal parallax so that the observed 3D image distorts vertically depending on the viewing position. Because each projector generates viewing points on a different circle, the 3D image generated by each projector distorts differently. Therefore, if parallax images are independently calculated for each projector, multiple images with different distortions are observed simultaneously. In this study, we develop an image synthesis method to eliminate the differences among the vertical distortions in 3D images generated by the multiple projectors.
The developed image synthesis method is depicted in Fig. 6 . Rays converging to an actual viewing point from the rotating screen are considered. Common viewing points are densely generated on a circle that is coaxial to the rotation axis of the screen. The parallax images viewed from the common viewing points are rendered using computer graphics programs. The color and intensity of the rays converging to the actual viewing point are extracted from these parallax images rendered for the common viewing points. Then, the image projected to the actual viewing point is synthesized. The vertical parallax of the synthesized image is the same as the parallax images for the common viewing points because image synthesis is performed in the horizontal direction. The radius of the circle on which common viewing points are generated is set to (1 + l’/l)r, which is equal to that of the circles on which the actual viewing points generated by the projectors are located. Because the images projected to the actual viewing points generated by the different projectors are synthesized from parallax images for common viewing points, the synthesized images have a common vertical parallax.
Figure 7 shows how the image data at the position (xi, yi) on the screen projected by the projector located at the position (xp, yp) are calculated. The position of the viewing point, denoted by (xv, yv), is given by xv = (r cosθ − xp)l’/l + r cosθ and yv = (r sinθ − yp)l’/l + r sinθ, where θ is the rotation angle of the screen. The line connecting the image point (xi, yi) and the viewing point (xv, yv) is considered. The circle on which the common viewing points are located is expressed by x2 + y2 = (1 + l’/l)2r2. The intersection between this line and circle is calculated. The nearest and second nearest common viewing points to the intersection are determined. The image data at the position (xi, yi) of two parallax images generated for the nearest and the second nearest common viewing points are linearly interpolated by considering the distances s1 and s2 shown in Fig. 7, which are the distances from the two common viewing points to the line connecting the image point and the actual viewing point. Before the interpolation, the parallax images for the common viewing points are appropriately rotated depending on their horizontal viewing angles. After the interpolation, the generated image is rotated depending on the horizontal viewing angle of the actual viewing point.
4. Experimental system
The generation of 3D images by the proposed hybrid 360-degree 3D display system and the image synthesis method were experimentally verified. The display system was constructed using a reflective screen.
The structure of the reflective screen is shown in Fig. 8(a) . It consists of an off-axis Fresnel lens, a lenticular lens, and an aluminum-coated mirror. The Fresnel lens was placed with its saw-tooth structure pointing downwards, and the lenticular lens was used as a vertical diffuser. Figure 8(b) shows the off-axis Fresnel lens; its groove pitch, diameter, and focal length were 0.30 mm, 300 mm, and 800 mm, respectively. The separation between the lens axis and the screen center (rotation axis) was r = 200 mm. Because the rays from the screen passed through the Fresnel lens twice, the effective focal length of the reflective screen was 400 mm. The lens plane of the lenticular lens should be attached to the reflection plane of the mirror. However, a subtle change in the gap between the two planes resulted in a non-uniform intensity distribution of the reflected light so that the lens plane faced the direction opposite to the mirror plane. The pitch of the lenticular lens was 0.18 mm. The projection lenses of the projectors were placed at a height of 800 mm above the screen so that the viewing points were generated at the same height of 800 mm above the screen. The viewing points were generated on a circle with a radius of 400 mm. The outer diameter of the reflective screen holder was 330 mm. A servo motor was used to rotate the screen. A photograph of the screen with the rotation mechanism is shown in Fig. 8(c). The use of an off-axis Fresnel lens for a high-speed projector system was described in Ref. 6.
A DMD and a projection optics were combined to construct the high-speed projectors. The DMD used was the DLP DiscoveryTM 4100 (Texas Instruments, Inc.) with the ALP-4.1 high-speed accessory software package that enables generating binary images with a maximum frame rate of 22,727 Hz. The image resolution was 1,024 × 768 pixels and the size of the image area was 0.55″. To synchronize multiple DMDs, the frame rate was reduced to 22,222 Hz. The focal length of the projection lens was 29.6 mm. The central 768 × 768 pixels were used to display images, which were magnified to a 300 × 300 mm2 area on the screen by the projection lens. The image update signal from the DMD controllers was used to control the servo motor.
In the experiments, each high-speed projector generated 800 viewing points on a circle so that the frame rate of displaying 3D images was 27.8 fps. Thus, the rotation speed of the servo motor was set to 1,667 rpm. The pitch of the viewing points on the circle was 3.1 mm.
First, the proposed image synthesis method described in Section 3 was validated using two high-speed projectors. Each projector employed a white LED as an illumination light source. A photograph of the two projectors is shown in Fig. 9(a) . We used 1,600 common viewing points on a circle with a radius of 400 mm. The images displayed by both projectors were synthesized from parallax images generated for the common viewing points. The coincidence of superimposing two 3D images generated by the two projectors was examined from different horizontal directions. First, the two projectors were placed with a separation of 120 mm. The generated 3D images are shown in Fig. 10 . The 3D image consisted of five squares at heights of 0, 25, 50, 75, and 100 mm above the screen. Figure 10(a) shows the 3D images captured from four different horizontal directions around the screen by using only one projector. Figure 10(b) shows the images obtained when the other projector was used, and Fig. 10(c) shows those obtained when both projectors were used. No difference was observed among the images shown in Figs. 10(a), 10(b), and 10(c). The coincidence of the two 360-degree 3D images was verified at all distances from the screen. Figure 11 shows the results when the separation of the two projectors was increased to 250 mm, which is the maximum separation that the experimental system can handle. In this case, the separation of the two 3D images was observable. The image separation was caused by the difference in the image distortions due to the difference in the lens shifts between the two projectors.
Next, color 3D images were generated using three projectors. Each projector was fitted with an R, G, or B LED as the illumination light source; their center wavelengths were 624, 528, and 464 nm, respectively. Figure 9(b) shows the RGB projector array. The lenses of the three projectors were located on the three vertices of an isosceles triangle. The base length and height of the triangle were 164 mm and 82 mm, respectively. The number of common viewing points was 24,000. The image synthesis method described in Sec. 3 was used to generate R, G, and B images displayed by the R, G, and B projectors. Figures 12(a) , 12(b), and 12(c) show the 3D images obtained with each projector (R, G, and B) used alone. Figure 12(d) shows the color 3D images generated when all projectors were used. The figure shows that the three images were successfully superimposed.
Figure 13 shows other 360-degree 3D images generated by the display system with the RGB projector array. The number of colors represented by the three projector system was eight because each projector could generate binary images. We used the error diffusion dither algorithm  to represent halftone images.
The resolution of the generated 3D images was less than 768 × 768 pixels. This is because the individual pixels were not resolved. The 3D images were blurred on the rotating screen owing to the separation of the reflection plane of the mirror and the lens plane of the lenticular lens. The groove structure of the Fresnel lens also contributed to the blur. The pixel pitch of the images projected on the screen was 0.40 mm so that the groove pitch should be smaller than 0.20 mm from the sampling theorem.
Weak ghost images were observed in the 3D images. They were caused by light reflected from the surfaces of the Fresnel lens. The transmissive screen system is preferable over the reflective screen system because such a reflection does not affect the 3D images obtained with a transmissive screen.
We used the reflective screen system because it was easier to construct. When observers brought their fingers close to the 3D images, light from the projectors was blocked by the fingers. On the other hand, the transmissive screen is more suitable for enabling interaction between fingers and the 3D images. The transmissive screen was used in the system described in [6, 7]. Because the rotating screen should be thin and lightweight, the development of a thin screen that provides both lens and one-dimensional light diffusion functions is required.
In the proposed hybrid system, the number of projectors that can be used is limited by image degradation including image distortion and image blur, which are caused by the lens shift of the projectors. For the results shown in Fig. 10 using two projectors, the lens shift angle was 4.3°, image distortion was not observable, and the superposition of the two images was successful. For those shown in Fig. 11 using two projectors, the lens shift angle was 8.9° and image distortion caused the failure of the image superposition. For those shown in Figs. 12 and 13 using R, G, and B projectors, the lens shift angle was 5.9° and the image superposition was successful. The image blur caused by lens shift was not observable in any of the experiments. The projector lenses used in the experiments were ready-made products with maximum lens shift angle of 12.5°. However, the determination of this maximum lens shift angle did not consider a setup for the superposition of multiple images. The use of high-quality projection lenses enables the increase in the lens shift. The image distortion might be reduced by using an image processing technique that corrects the image distortion by electronically anti-distorting images.
Because the frame rate of the experimental system was 27.8 fps, flicker was observed in the produced 3D images. We reduced the number of circular viewing points to 400 and increased the rotating speed of the servo motor to 3,333 rpm in order to obtain a frame rate of 55.6 fps. In this case, flicker was rarely observed. However, the sound of the motor was very loud at this high-speed rotation. From the experimental results shown in Figs. 10 and 11, when the frame rate was 27.8 fps (the motor rotation speed was 1,667 rpm), the flicker with two projectors was less observable than that with one projector.
The proposed system generates 3D images having only a horizontal parallax. The lack of a vertical parallax caused a distortion in the 3D images depending on the observation position . When observed near the viewing points, i.e., at a horizontal distance of about 400 mm from the center of the screen and at a height of about 800 mm above the screen, the 3D images showed little distortion. However, when observers moved away from the viewing points, the distortion became evident and the 3D images appeared slanted toward viewers. Therefore, the height of the 3D images should be limited. The use of face or eye detection systems can eliminate such image distortions by rendering the parallax images for the common viewing points through incorporating the height of the observer’s viewing position. The complexity of the image synthesis method thus increases to eliminate the differences among vertical distortions in 3D images.
In the experimental system, the viewing points were generated at a pitch of 3.1 mm. The light intensity distribution width of the viewing points determines the effective pitch of the viewing points. The exit pupil of the projection lens, the imaging property of the screen lens, and the rotation speed of the screen lens determine the width of the light distribution. The measured width of the distribution was 10–12 mm. The width of the distribution decreased with the diameter of the exit pupil. In the experiments performed in this study, the diameter of the pupil was set to a maximum of 12 mm in order to increase the light intensity of 3D images. When the width of the viewing points is three times the pitch of the viewing points, color 3D images can be generated using only one projector by sequentially changing the color of the illumination light among R, G, and B. When the width of the viewing points is less than 5 mm (average diameter of human pupils), the super multi-view (SMV) condition [20, 21] can be achieved so that the visual fatigue of the conventional 3D displays might be reduced. The construction of a 360-degree SMV display is much simpler with the proposed system than with the multiple projector system.
The experimental results showed that the proposed image synthesis method worked well for the 360-degree 3D display system consisting of multiple projectors with different lens shifts and a rotating off-axis screen lens. The use of multiple projectors can increase the number of colors for representing the 3D images. Such a system can also be used to increase the number of viewing points. Furthermore, it might be used to decrease the rotating speed of the screen. As mentioned above, increasing the number of projectors reduced the flicker. Thus, the rotation speed of the screen would be reduced by using multiple projectors. Flicker perception also depends on the ray transformation scheme of the screen lens shown in Fig. 5 and the spatial arrangement of the multiple projectors.
The hybrid system proposed in this study is compared with the conventional methods. The high-speed projection system described in Refs. 6 and 7 can generate 3D images with only one high-speed projector. The use of the lens shift technique and the image synthesis method allows the hybrid system to use multiple projectors so that the number of colors and viewing points can be increased and the screen rotation speed can be decreased, while image degradation caused by lens shift can result. The lens shift technique is well developed because it is widely used in commercial video projectors and PC projectors. Because the number of projectors required for the hybrid system is small, the use of high-quality and expensive projection lenses might be acceptable. However, the cost of the hybrid system is higher than that of conventional high-speed projector system. Systems that use multiple projectors can generate 3D full-color images at the video rate, although such systems need a large number of projectors. The system described in Ref. 10 arranged 103 projectors to generate 103 virtual viewing points with an angle pitch of 1°, while the experimental system developed in this study generated 800 viewing points with an angle pitch of 0.45° at a frame rate of 30 fps. To generate 3D full-color images without dither, the hybrid system needs to use 24 projectors.
A 360-degree 3D display with a table screen was constructed by combining a rotating off-axis lens screen and a small array of off-axis high-speed projectors. An image synthesis method for the proposed system was developed. The proposed image synthesis method was verified using two projectors. Color 3D images were generated using an RGB projector array with a frame rate of 27.8 fps. The number of viewing points was 800 for each color.
This study was supported by a Grant-in-Aid for Scientific Research, No. (B) 23360148 from the Japan Society for the Promotion of Science (JSPS).
References and links
1. G. E. Favalora, J. Napoli, D. M. Hall, R. K. Dorval, M. G. Giovinco, and M. J. Richmond, andW. S. Chun, “100 million voxel volumetric display,” Proc. SPIE 4712, 300–312 (2002). [CrossRef]
3. A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007). [CrossRef]
4. A. Jones, M. Lang, G. Fyffe, X. Yu, J. Busch, I. McDowall, M. Bolas, and P. Debevec, “Achieving eye contact in a one-to-many 3D video teleconferencing system,” ACM Trans. Graph. 28(3), 1–64 (2009). [CrossRef]
5. C. Yan, X. Liu, H. Li, X. Xia, H. Lu, and W. Zheng, “Color three-dimensional display with omnidirectional view based on a light-emitting diode projector,” Appl. Opt. 48(22), 4490–4495 (2009). [CrossRef]
6. G. E. Favalora and O. S. Cossairt, “Theta-parallax-only (TPO) displays,” US Pat. 7,364,300.
7. H. Horimai, D. Horimai, T. Kouketsu, P. Lim, and M. Inoue, “Full-color 3D display system with 360 degree horizontal viewing angle,” Proc. Int. Symposium of 3D and Contents, 7–10 (2010).
8. R. Otsuka, T. Hoshino, and Y. Horry, “Transpost: all-around three-dimensional display system,” Proc. SPIE 5599, 56–65 (2004). [CrossRef]
9. R. Otsuka, T. Hoshino, and Y. Horry, “Transpost: A novel approach to the display and transmission of 360 degrees-viewable 3D solid images,” IEEE Trans. Vis. Comput. Graph. 12(2), 178–185 (2006). [CrossRef]
10. S. Yoshida, “fVisiOn: glasses-free tabletop 3-D display,” in Proceedings of Digital Holography and 3-D Imaging (Tokyo, 2011), DTuA1.
11. T. Honda, Y. Kajiki, K. Susami, T. Hamaguchi, T. Endo, T. Hatada, and T. Fujii, “Three-dimensional display technologies satisfying super multiview condition,” SPIE Crtical Reviews CR76, 218–249.
12. T. Yendo, T. Fujii, M. Tanimoto, and M. P. Tehrani, “The Seelinder: cylindrical 3D display viewable from 360 degrees,” J. Visual Commun. Image Represent 21(5-6), 586–594 (2010). [CrossRef]
13. Y. Sakamoto, S. Maruyama, and I. Fukuda, “Turn type three-dimensional display system using arrayed light emitting diodes,” Proc. of the 9th Int. Display Workshops (IDW’02), 1257–1260 (2002).
14. T. Yamaguchi, A. Ito, Y. Sakamoto, and I. Fukuda, “A turn-type high-resolution 3-D display using LEDs,” Proc. of the 13th Int. Display Workshops (IDW ’06), 1405–1406 (2006).
15. M. Gately, Y. Zhai, M. Yeary, E. Petrich, and L. Sawalha, “A three-dimensional swept volume display based on LED arrays,” J. Disp. Technol. 7(9), 503–514 (2011). [CrossRef]
16. S. Uchida and Y. Takaki, “360-degree three-dimensional table-screen display using small array of high-speed projectors,” Proc. SPIE 8288, 82880D, 82880D-7 (2012). [CrossRef]
17. P. St. Hilaire, S. A. Benton, and M. Lucente, “Synthetic aperture holography: a novel approach to three-dimensional displays,” J. Opt. Soc. Am. A 9(11), 1969–1977 (1992). [CrossRef]
18. J. F. Jarvis, C. N. Judice, and W. H. Ninke, “A survey of techniques for the display of continuous tone pictures on bilevel displays,” Comput. Graph. Image Proc. 5(1), 13–40 (1976). [CrossRef]
19. M. W. Halle, S. A. Benton, M. A. Klug, and J. S. Underkoffler, “The Ultragram: a generalized holographic stereogram,” Proc. SPIE 1461, 142–155 (1991). [CrossRef]