Open Access
Add to BibSonomy
Abstract
We present in details the development of a 360° volumetric display based on the use of a transparent projection surface. The optical scheme derives from the Pepper’s ghost configuration, known as holographic display. Our optical system requires high angular diffusion efficiency and an innovative transparent retro-reflective surface has been developed for that purpose. This surface is made of sparse cube corner distribution and we give some elements of its design and characterization. We describe also the optical design of the 360° display and gave the feedback of its presentation to the public during a symposium.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
With the recent progress in computer graphics and display technologies there have been a renewing interest for 3D displays and more generally for volumetric displays during the last decades [1]. This research for efficient ways of representation of reality is an old quest of human being as recent studies in cave painting suggest that from paleolithic ages, 20, 000 years ago, humans were using the volume of the cave to give the illusion of volume and movement to the animals they were painting [2,3]. The project to give life to the abstraction was already in the mind of the people of these early ages and recent public demonstrations of deceased famous artist playing on the scene as holograms are maybe an extension of such shared impressive experience.
We have presented in 2019 our work on a volumetric display developed for a theatrical performance [4]. The device was used to promote a show from a group of live performing artists, Théâtre Nouvelle Génération (TNG), based in Lyon, France. The futuristic show drawn from the literary work of Philip K. Dick [5] was relayed in our display by a virtual robot introducing it to the public. This display based on the use of transparent retro-reflective surfaces is described into more details in this paper. In the first section we give some generalities on 3D perception and volumetric displays that highlight our approach on the development of a 360° multi-user device. We then present in the second section the transparent retro-reflective surface that is the core of our development. Section three is dedicated to the description of our optical system. We give in section four details on the development of the device and on the content generation. The results of the show and the perspective for future works are presented in the last section.
2. 3D perception and volumetric displays
2.1 Generality on 3D perception
When it comes to represent virtual objects as real, various senses can be involved, but one of the most impressive is certainly related to the perception of volume. Numerous studies has been dedicated to the 3 dimensions perception and to the technological solutions developed for that purpose. We can cite the work of reference of Jason Geng published in 2013 [6]. The author describes the fundamental physical and physiological depth cues involved in our 3D perception. He also underlines their respective role depending on the viewing distance, showing that after few meters, motion parallax is the dominant factor before that physiological cues take the lead.
We don’t develop these notion here but to highlight the complexity of the problem we can underline some paradox of our volume perception. The first paradox, humorously described in Fig. 1 is that our vision doesn’t allows us to see in 3D. Solving a problem with three unknowns requires three uncorrelated information so that we should require a third eye to have a full 3D vision. This paradox is certainly due to the fact that the world for a human being is mainly bidimensional as we evolve on a surface where vertical structures are more prominent. Another paradox is that our perception of volume is still possible with a single eye. This underlines that the stereoscopic behavior, often operated in 3D display, doesn’t play a significant role in volume perception and that motion parallax and occlusion are of primary importance.
Fig. 1. Description of the stereoscopic cue paradox, (a) three eyes should be required to have a full 3D vision, (b) our environment is mainly a 2D surface and (c) volume perception is still possible with a single eye.
We want to highlight in this work the role of occlusion as a major vector of 3D perception. In Fig. 2(a) we have drawn a series of numerated objects we can associate to stones. We guess that stones 1, 3 and 2 are in front of the 5 and the 4 as they are occluding themselves. To differentiate the position of stone 4 and 5 we need more information as the perspective information shown in Fig. 2(b). These figures involve nearly all the physiological cues described in [6]: occlusion, perspective, texture, shading and prior knowledge. If these cues are used in an incoherent way, it results in famous troubling optical illusions as shown in Fig. 2(c).
Fig. 2. Illustration of some prominent physiological cues, (a) occlusion, (b) perspective and (c) example of optical illusion mixing both
2.2 Volumetric displays
3D vision is generally associated to the stereoscopic vision related to binocular disparity and this physical cue has been fully explored in a large number of display solutions. Of particular interest are the autostereoscopic solutions that don’t require the use of dedicated glasses and that are generally based on the use of lenticular optical systems. For these devices the number of view is a major concern to provide the best 3D experience through a smooth motion parallax. A particular drawback of these systems is the vergence accommodation conflict (VAC) related to the gap between accommodation on a 2D display and 3D vergence sensation generated by the binocular disparity.
To overcome this focus/vergence mismatch, multi-view light-field displays have been developed that allow the viewer to focus on the vergence location. However, the tradeoff between the image resolution and the number of views still require very complex system [7], moreover the content generation that allows the accommodation in front or behind the display require to know with precision the location of the viewer [8].
Scientist developing such complex and costly solutions are often distraught in front of very simple and cheap systems that produce impressive effect on the public. An example of particular interest is related to the displays based on the Pepper’s ghost effect. This elementary system is described in Fig. 3(a), it produces a virtual image from a display by reflection on a semi-reflective surface. As the displayed image is surrounded by a black background, it seems as a floating object that occludes the real background behind the display and no VAC occur between the virtual object and the real background. Occlusion plays a significant role here to introduce the volume perception. Moreover, if the image shows a video with a moving scene, a multi-view perception is drawn in a temporal way. Several companies proposes commercial products based on this principle like Holusion in France or Realfiction in Denmark. In Fig. 3(b) we show a typical rendering from a Holusion display.
Fig. 3. (a) Principle of the Pepper’s ghost display, and (b) example of a commercial holographic display from Holusion.
On this display solution, the image can be displayed on the four faces of the semi-reflective pyramid and it can be related to another family of volumetric display that can render virtual images on a 360 degrees angular distribution.
2.3 360-degree displays
360 degrees display is a class of visual systems that allows the user to turn around the device offering a view of an object on various angles. If the aspect of the floating object changes in a coherent way with the movement of the viewer it gives the illusion of the presence of a volumetric object. This system is particularly efficient for displaying animated objects or figures. Another interest of this display is the ability to share the volumetric rendering experience with a group of various viewers.
An efficient way to display a scene in a 360° configuration is to use a moving projection surface in association with a synchronized projection system. An example is given with the commercial product of the company Voxon that uses a vertically oscillating diffuser synchronized with a projection system. Through retinal persistence, the viewer sees the image as a full object and the high oscillating frequency preclude the diffuser to be seen leading to a general transparency of the system. As the image is diffused in a large field of view, the number of viewers is not limited. On the contrary occlusion from the different parts of the image is not possible limiting this kind of display to ghosted content.
To allow occlusion Jones et al. proposed a device using a holographic rotating diffuser with specific diffusing angular characteristics. This light-field display generates 288 images on a 360° horizontal field of view. Multi-user can share the visual experience at a given height and distance from the device [9].
Another interesting research proposes a simple 360° display based on a “Pepper’s cone” configuration. A 360° symmetrical conical semi-reflective surface reflects an anamorphic image into a real undistorted aerial image [10]. As the viewer is turning the display device, the image is changed in order to offer a 360° interactive view of the figure. This low cost and efficient solution with no moving parts is however constrained to a limited number of viewers as only one point of view is displayed at a time.
Our concept of 360° display is also based on a Pepper’s ghost configuration with no moving parts and reduced cost but intent to promote a volumetric viewing experience for a large number of users. The key element for this development is a high angular efficiency transparent projection surface that allows to display bright images.
3. Transparent cube corner film
3.1 Projection on transparent surfaces
The concept of projection on transparent surface is paradoxical as a transparent surface is by definition unable to fix the light. It has been studied recently with the development of Augmented Reality as it allows to see digital information in front of a surrounding scene [11]. The major challenge in developing such surface functionality is to manage the compromise between the diffusion and the transparency of the surface.
We give in Fig. 4(a) an example of volumetric display based on the use of transparent projection surface. This simple device developed in our laboratory is directly inspired by the Pepper’s ghost configuration. The semi-reflective surface is not used to reflect a virtual image but to project a real image on the transparent projection surface. The image seen through the semi-reflective surface seems to float if the figure is surrounded by a black background on the projection system. The occlusion between the bright image and the real background plays its role and if the figure is moving with a qualitative content, it seems to be volumetric. This very simple configuration is based on our transparent retro-reflective surface presented in the next sessions. This solution is efficient with the advantage, as compared to the classical Pepper’s ghost design, to avoid the presence of a display panel above the device and to present a bright image. Figure 4(b) shows the rendering of the projection of a cartoon figure.
Fig. 4. (a) Principle of an aerial display developed in the laboratory based on the use of transparent projection surface and (b) visual rendering of the prototype with a cartoon figure projection.
A first way to develop a transparent projection surface is to use a partial diffuser related to a sparse diffuser distribution or to a semi-reflective diffuser [12]. This solution is however hardly efficient if the diffuser is not designed with restricted angular diffusion and other solutions more efficient in terms of angular response has been developed to address the brightness issue [11,13,14].
3.2 Brightness issue
The brightness of a display, also referred to the Luminance L, is defined as the amount of visual optical power emitted by units of display surface and viewing solid angle. It characterizes for the device its ability to display bright images.
For a given display radiant exitance M, the brightness strongly depend on the emissive angle γ of each pixel. If we suppose the brightness uniform in an emissive angular cone γ, it can be expressed by the simple formula:
In the case of the projection on transparent surface, only a small amount of the projected optical power is to be reflected by the transparent surface. The only way to improve the perceived luminosity of the projected image is to reduce the angle of emission γ of the perceived optical power. This is done by controlling the diffusion angle of the projection screen. However, in a classical diffuser scheme, the diffusion angle is applied to the specular reflection of the optical beam incident on the transparent screen. If the angular field of projection is large and not reflection-oriented toward the viewer, reducing the diffusion angle makes no sense and the brightness issue cannot be resolved. A solution is to use a holographic diffuser with a reflection characteristic not correlated to the specular reflection [11], however the development of such solution is complex and cost effective, in particular for color projection.
A typical example for such a brightness issue is encountered in the road signs development. The limited projection power of the car headlight and the absence of specular reflection between the car and the road sign require very specific projection surface using retro-reflective properties.
3.3 Principle of retro-reflection
The principle of retro-reflection is described in Fig. 5(a). An optical ray incident on the surface with a direction (α,β) is reflected on the same direction inside a small diffusion cone (ψ,φ). Two main solutions exist to produce the particular geometrical effect. First one is the use of microbeads where the optical ray is focus and reflected on the back side of a spherical lens [14]. This solution is commonly encountered on the safety jacket but has the drawback of a small efficiency and cannot be encapsulated. The other solution massively used in the road signs is the Cube Corner (CC) geometry.
Fig. 5. (a) Principle of a retro-reflective surface, and (b) principle of a cube corner retro-reflector.
Figure 5(b) describes the principle of this optical element. It is formed by three mirrors oriented at 90° with each other. An optical ray that enters the element and is reflected by each of the mirrors is coming back with the same direction by geometrical construction. This optical element is known and manufactured for a long time at macroscopic scale [15], but its manufacturing process at microscopic scale for the development of thin retro-reflective film is still subject of active research [16]. In order to use the specific angular directionality and the low cost manufacturing process of this optical element we have proposed recently the development of transparent retro-reflective surface based on sparse cube corner retro-reflectors.
3.4 Sparse cube corner film
The concept of our transparent CC film is described in Fig. 6. We show in Fig. 6(a) a typical visual representation of a classical CC film as developed for the road signs. It is composed by a succession of pyramidal structures with a triangular aperture (nine CC patterns on the figure). In order to fill the surface the triangles are tiled up-side and down-side. Our concept is to introduce a flat surface between each CC in order to mix both transparency of the plastic material and retro-reflectivity of the CC. We show in Figs. 6(b) and 6(c) two configurations with the same periodic structure than Fig. 6(a) but with different height of the cube corner leading to different ratio between transparency and retro-reflection efficiency.
Fig. 6. a) Classical distribution of triangular aperture CC on a retro-reflective surface, (b) and (c) two cases of transparent retro-reflective surface with flat surface between the CC.
The manufacturing process used to produce the transparent CC film is based on the grinding and polishing of the female master used to generate the structure of Fig. 6(a). It allows a rapid prototyping and a cost-effective production of the CC transparent retro-reflective surface. The film used for the development of our 360° display have been done in collaboration with Orafol Fresnel Optics in Germany.
Figure 7(a) shows a photography of a typical CC structure commercialized by this company and Fig. 7(b) shows an example of structure developed for our application. The period of the CC structure (base of the triangular CC aperture) is about 300 µm, the size of the remaining CC is about 150 µm.
Fig. 7. (a) Photography of a full CC structure molded in PMMA, and (b) photography of a transparent sparse CC structure (above a blue graph paper).
Figure 8 shows the angular characteristics of the CC film in diffusion. Figure 8(a) presents the case of the full CC structure of Fig. 7(a), we measure a diffusion of about 0.7 degrees in (ψ,φ). Figures 8(b) and 8(c) show the angular characteristics of the sparse CC film of Fig. 7(b), we measure an angular distribution with a FWHM about 4 degrees.
Fig. 8. (a) Diffusion response of the full CC film at an incidence of about 2°, (b) diffusion response of the sparse CC film at an incidence of about 1.5°, and (c) section of the diffusion response of the sparse CC film.
We show in Fig. 9 the details of the CC geometry with the angle δ = 54.7° and γ = 35.3°. Figure 9(b) presents the results of geometrical simulation for the efficiency of the CC geometry as a function of the incidence angle. On the figure of the top is represented the response of a single CC, we notice a specific shape that limit the incidence efficiency depending on the orientation of the incoming ray. The figure below gives the response of the CC structure tiled up-side and down-side. Figure 9(c) shows the measured angular efficiency of the CC structures. The full CC film has an efficiency about 60% and an angular incidence domain +/- 25°, the sparse CC film has an efficiency about 12% and an angular incidence domain +/- 30°.
Fig. 9. (a) geometrical description of the CC with triangular aperture, (b) result of efficiency simulation for single and tiled CC structure, and (c) experimental result on the incidence efficiency of the structure of Figs. 7(a) and 7(b).
The low efficiency of the full CC structure is due to geometrical effects related to the triangular aperture of the CC design [17]. Due to the sparse geometric configuration the transparency of our transparent CC film is estimated at about 75%. This good transparency and the limited diffusion angle has shown very efficient behavior for the projection of aerial images as shown before on Fig. 4(b), these good results lead us to consider an innovative optical configuration for a potential display working on a 360° configuration.
4. Presentation of the 360-degree display
4.1 General concept
Our concept of display leverages the transparent retro-reflective effect not directly in a real image configuration as shown in Fig. 4 but in a virtual configuration related to the Pepper’s ghost construction. The principle of this concept is described in Fig. 10. The transparent projection surface is not used to superimpose the real projected image to the background but to see through its surface the reflection of an image projected on its own surface. We use a projection system to project an aerial image on our transparent surface disposed in the periphery of the display. The image is going through a semi-reflective surface before being incident on the CC film. The image is retro-reflected in direction of the projection system and is then reflected by the semi-reflective surface towards the outside of the device. If the viewer is located on the right position, he can see a virtual floating image. This concept can work with any transparent projection surface but the use of a retro-reflective film allows to concentrate the reflected virtual image on a single angular direction and limit theoretically the vision of the real image from outside of the device.
The interest of the use of a virtual image as compared to the design of Fig. 4 is the ability to superimpose various different images on the same location, free of projection surface, or to superimposed the virtual image with a real object. This advantage is described in the Fig. 11 where the basic optical module of Fig. 10 is replicated on another location around a central axis. In this figure we show the projection of the image of characters R and L, depending on the location of the viewer, the directivity of the CC structure allows to see only one image at a time. By considering the replication of the optical module on a 360° scheme we can show on each viewer location a different viewing angle of the floating image leading to a volumetric display experience.
4.2 Angular design
The angular design of the display is described in Fig. 12 that shows a side view of the geometric configuration of the projection module. The center of the exit pupil of the projection system is J0, J1 is the intersection of one ray with the first mirror that sends light toward the retro-reflective surface on the point J3. The ray is reflected on the semi-reflective surface on the point J2 and form the virtual image J3’ at the distance Dx from the central axe of symmetry of the whole device. In order to display a vertical virtual image, the semi-reflective surface is oriented at an angle α that is the half of the angle of the transparent retro-reflective film. Of particular interest are the angle θ that defines the direction of optimal vision and the angle Δθ that defines the shift between the vision angle and the direct light from the projection system.
We consider for the drawing and the system design the central ray of projection determined by the angle β. The choice of β, θ and Δθ determines the angle φ of the mirror and the angle α of the semi-reflective surface:
The number of viewing directions depends on the ability to maintain the vision of one image in its dedicated angular portion. Each pixel of the image emits an angular cone γ defined by the sum of the optical aperture of the projection system and of the diffusion angle of the retro-reflective surface.
We describe in Fig. 13 the angular construction of the module on the top view. The location of the virtual projector exit pupil, reflected by the mirror and the semi-reflective surface, is J0’. The angular aperture σ of the optical module is given by the number of views N in a simple way:
To optimize the image rendering, the angle σ is higher than, and ideally as close as possible from the angular emissivity γ of the pixels. It fixes the size of the semi-reflective and transparent retro-reflective surfaces.
The projection distance ZP and the distance D between the virtual image and the semi-reflective surface fixe the half size limit t of the image that can be seen on the display:
Equations (4) and (5) show that the size of the image displayed is inversely proportional to the number of views.
Despite the sparse CC pattern are not distributed in a square grid we can estimate the resolution limit as the ratio between the virtual image width 2t and the period of the pattern.
4.3 Brightness analysis
The brightness of the image seen on the device depends on the radiant exitance MV of the virtual image and on the emissive angle γ of the screen as expressed in Eq. (1).
The illuminance IP on the transparent retro-reflective film is given by the projector optical power Φ, the projection distance ZP, the throw and aspect ratio rt and ra and by the characteristics of the mirror and the semi-reflective surface. It is expressed as follows:
We use Rm as the reflection of the first mirror and T1 as the transmission of the semi-reflective surface. A correction factor ρ accounts for the effective properties of the projector (brightness settings, keystone adjustment, ageing … etc.).
The virtual image radiant exitance MV is expressed from the projected image illuminance by the following equation:
Where τCC is the fill factor of the cube corner pattern, τRR is the retro-reflectivity efficiency of the cube corner, R1 is the reflection of the semi-reflective surface and T2 is the transmission of the transparent projection surface.
Our concept is evaluated with a single view prototype using a DLP projector ML750e from Optoma. The prototype is shown in Fig. 14(a). We can see a calibration target projected on the transparent surface and the virtual retro-reflected target seen at a viewing angle about 10 degrees.
Fig. 14. (a) Photography of the single view prototype used to characterize the brightness of our concept (retro-reflective transparent surface is underlined in dotted blue, semi-reflective surface in dotted red), and (b) angular brightness distribution for the measured target image and for the uniform brightness angular cone model.
We use for the projection a sheet of PMMA with the CC structure shown in Fig. 7(b) molded on one side and an antireflective moth-eye structure molded on the other side. The semi-reflective surface is a simple sheet of PMMA. To avoid dual image reflection, one side of the sheet is also molded with a moth-eye antireflective surface.
The parameters of the projection system are given from the projector datasheet and from measurements on the optical elements, they are listed in Table 1.
Table 1. Parameters of the projector and of the optical system.
To measure the brightness of the real projected target image on the transparent retro-reflective surface, we replace this surface with a Lambertian diffuser. The brightness is measured with a colorimeter (Konika Minolta Chroma Meter CS-150). The measured value is 1450 Cd/m2 that corresponds to an illuminance IP = 4555 Lux.
With the transparent retro-reflective surface in place we measure the brightness of the virtual target image for various viewing angles. The results are given in Fig. 14(b) and show a maximum brightness about 6000 Cd/m2 with a Gaussian shape about 3.6° FWHM. This shape differs in form but not in size as compared to our local measurement of Fig. 8(c).
The brightness of the real projected image as measured from the viewer point of view is nearly uniform with a value about 50 Cd/m2. This unwanted ghost image might be due to manufacturing defects generated during the master grinding and by diffraction on the borders of the cube corners.
The parameters of Table 1 applied to Eq. (6) give a value consistent with our illuminance measurement for a correction factor ρ about 39%. This factor is low but seems realistic due to our projection configuration and settings. We use the measured illuminance and Eqs. (1) and (7) to estimate the theoretical brightness of the virtual image. We calculate a brightness value LV about 5400 Cd/m2 for the 1.8° angular cone. This uniform brightness distribution is plotted in Fig. 14(b) and is consistent with the measured angular brightness.
We use the measured brightness of Fig. 14(b) to calculate the mean brightness seen by the user. The plotted data are duplicated and shifted with an angle that corresponds to the angular separation of each viewer eyes. Both data are then averaged and plotted in logarithmic scale in Fig. 15. We consider an inter pupillary distance about 6 cm and a viewing distance about 70 cm that correspond to an angular separation about 5 degrees.
Fig. 15. Averaged brightness distribution for the two viewer eyes in the case of a binocular angular separation of 5°.
The perceived brightness is above 1000 Cd/m2 over +/-5°. As it can be seen on Fig. 14(a), at a viewing angle about 8° the virtual image brightness is low and similar to the ghost projected image brightness.
The non-uniformity of the brightness angular distribution described in Fig. 15, is a drawback of our concept. It can be compensated by increasing the number of view as shown in the next section or by increasing the diffusion angle γ of the retro-reflection surface.
Our theoretical analysis on the brightness is consistent with our measurements and allows us to evaluate the potential of this concept. Our current design is focused on the overall transparency of the device with a simple semi-reflective surface about 4% in reflection. However, in terms of brightness improvement, Eqs. (6) and (7) show that the use of a 50% semi-reflective surface can increase the brightness of the virtual image to a maximum value. With T1 = R1 = 50% we calculate a uniform brightness about 35.103 Cd/m2 that could allow the device to be used in outdoor conditions.
4.4 Comparison with the Pepper’s ghost design
Brightness limitation generally constrains the use of the standard aerial display to indoor conditions. This is particularly the case for the Pepper’s ghost devices. As explained in a former section, these devices are efficient and low cost but they suffer from some drawbacks we have tried to solve with our concept.
A first drawback is related to the geometrical configuration of the Pepper’s ghost design. As shown in Fig. 3, in order to present the semi-reflective surface in an aesthetic configuration, similar to a vitrine, the display is generally located as a cover in the upper part of the device. This cover can be suppressed by locating the display in the bottom part of the device. However, in this case the semi-reflective part is in an inverted pyramid configuration that looks unnatural. Moreover, this inverted configuration prevents the user to locate a real object to be mixed with the virtual image as in a standard vitrine.
One advantage of our configuration is to release the cover of the aerial display for more transparency and keep the possibility to show a virtual object as in a usual vitrine configuration.
The other drawback of the Pepper’s ghost design is the lack of brightness and is easily expressed by calculating the virtual image brightness LPG. It is defined in a simple way as a function of the display brightness LD and the reflection of the semi-reflective surface R1:
The brightness of a standard display, with quasi Lambertian characteristics, is about few hundreds of Cd/m2. Displaying a virtual image of few thousands of Cd/m2 requires high luminance displays that are expensive and power consuming.
The implementation of the Pepper’s ghost concept in a multi-view configuration also presents some problems. In particular, as the number of view increases, the effective surface of the display used to project the different views of the virtual image is reduced. This surface reduction comes with a reduction of the image resolution and of the image size. If the concept with a single display is commonly developed in a four views configuration, its use with more than a dozen of views is more complicated and other concept must be implemented like the Pepper’s cone [10] or the use of various display panels.
5. Prototype realization
5.1 Device design
The design of our multi-view device is mainly governed by the choice of the projection system. We have chosen to display our volumetric image according to N = 18 views with 6 projection systems as described in Fig. 16. Projection distance ZP is about 40 cm and the distance D is chosen about 15 cm, leading to a maximum image width of about 8.5 cm. Taking into account the period of the CC pattern about 300 µm it gives a resolution of the image in width of about 280 pixels. As three views are projected with a single projector image, the projector resolution might be above 840 pixels. We have chosen a lateral resolution of 1280 pixels for our projection system that allows a good match with the CC pattern period.
We have chosen to display the image in low-angle shot with θ = -14° and a separation between the projection axis and the viewing axis Δθ = 45°. It results in a tilt of the semi-reflective surface α = 9°. The geometrical configuration of the optical module is described in Fig. 17(a) with the dimension normalized to the height of the device.
Fig. 17. (a) Geometrical configuration of the projection system module, (b) view of the module with the projection of a virtual retro-reflected image at a viewing angle near 0°, and (c) side view of the optical module.
We show in Fig. 17(b) the result of the image projection on the module. We can see the real ghost image projected on the CC structure on the top of the device. The virtual retro-reflected image is clearly seen as a floating image and we notice the saturation of the image sensor due to the high brightness of the image. The side view of a three views module is shown in Fig. 17(c). We use as projection system a short throw DLP projector ML750ST from Optoma, its characteristics are given in Table 2. It is chosen for its short projection distance and for its angle of projection of about 52° that allows to cover nearly uniformly the 360° configuration with 6 projection systems.
Table 2. Parameters of the projector used in the 18 views device
Using our brightness model with the optical parameter of Table 1 and an improved correction factor of about 50% the brightness of the virtual image is estimated at about 1300 Cd/m2.
5.2 Device manufacturing
The complete display device is made with an assembly of six optical modules as shown in Fig. 18(a). The retro-reflective peripheral surface is made of 18 PMMA sheets located in a conical configuration. The semi-reflective surface is made of 18 PMMA sheets with a particular attention to maintain the flatness of the reflecting surfaces. The mechanical elements are manufactured by 3D printing (orange parts on figure) or by laser cutting (green parts on figure).
Fig. 18. (a) CAD of the display device, (b) photography of the six projection systems assembly, and (c) photography of the assembly of the retro-reflective PMMA sheets.
We show on Figs. 18(b) and 18(c) the assembly of the projection systems and of the PMMA sheets. Each projector is addressed by a small single board computer and each computer is monitored by a network switch to synchronize the videos.
During the test of the 360° configuration we notice disturbing multiple reflections on the virtual image so that with have to add a black cylinder at the center of the device to improve the virtual image contrast.
5.3 Content generation
A particular complex issue for volumetric display is the management of content generation. In our case we choose to generate the image of the display by the recording of real objects. The optical setup used to record the images is shown in Fig. 19(a). It consists in a camera that takes single pictures of an object according to the vertical view angle and a rotation stage to change the horizontal angle. A computer commands the camera and the stage to take 18 photos of the object under 18 views separated by 20°. To allow aerial images projection of the object, a black background is formed with a light trap.
Fig. 19. (a) Description of the optical set-up used to record the volumetric content, and (b) view of the HitFilm window during the video editing.
Between each series of 18 views, the object is modified in order to produce an animation in a stop motion mode. We use the software HitFilm to generate from the bibliotheca of images a series of 18 synchronized video showing the animated object. The videos are then integrated by groups of 3 in 6 videos that integrate the optical deformation of the projection. Figure 19(b) shows the video software with the composition of three images. The original object image is sampled with 400 × 400 pixels. The ML750ST projector has a resolution of 1280 × 800 pixels and the composition of the three images of the object is made to compensate for the geometric distortion of the projection. Vertical and horizontal spatial shifts are given in pixels with respect to the center of the projected image, size ratio with respect to the original object image size:
- • Central image : shift (0:-176), ratio 77.5%, rotation 0°
- • Left image: shift (-352:-134), ratio 84.5%, rotation 10°
- • Right image: shift (352:-134), ratio 84.5%, rotation -10°
The final result is a looping 5 minutes long 360° video showing moving objects that introduces, with a TNG original story-telling, the Artefact production.
6. Results and perspectives
6.1 Experimenta show
The 360° display was presented during the Art/Science symposium Experimenta held in Grenoble in 2018. It has received the visit of hundreds of people. The use of earphone has limited the number of users to about 20 persons per session but the short video duration time session has allowed us to test the display on a large number of users. It was then presented during several month in the Maif Social Club in Paris.
We present in Fig. 20 a photography of a session that shows people around the device located on top of a pedestal. The limited viewing angle obliges people to adopt a specific position depending on their height.
We present in Fig. 21 a series of images of the aerial figure when the viewer is turning around the device. The image is changing from one position to the other as we display a video but we have identified the frontier between two successive angular views by a green arrow. A zoom on the right bottom side of the image gives a specific view of the virtual image. We notice a good visual transition between the two neighboring angular views. This visual transition correspond to a viewing angle about +/- 10°. At this viewing angle the brightness of the virtual image is low.
Fig. 21. (a), (b), (c), and (d) views of the display near the viewing angle when the viewer is moving from one angular view to the other. On the bottom right is a zoom of the virtual image.
We remark the presence of the various ghost images from the real image that disturbs the visual experience. As explained before they are due to the diffusion and diffraction of the projected image on the sparse CC film.
6.2 Analysis of the public feedback
The feedback of the public was very positive but the 360° display appeared more like a visual attraction than like a casual display solution. The major drawback is the presence of the diffused real image on top of the device and the need of the black cylinder that limits the occlusion perception with the background.
Of particular interest was the attitude of the public in relation to the 360° configuration. We expected the users to turn around the display to experience the volumetric rendering but certainly due to the inertia of the crowd and to the attention paid to the content, the viewers generally remain in a static position. The interest of the display was then more to propose a shared visual experience with individual point of view, only possible with a 360° multi-view display solution or during a theater production.
6.3 Perspectives for the display evolution
In order to improve the visual result we need to reduce the visual rendering of the real diffused image. This could be done with a better CC structure free of the diffusing defect generated by the master grinding process. As explained before, increasing the reflectivity of the semi-reflective surface increases the brightness of the virtual image and reduce the brightness of the ghost image. This could be an interesting evolution at the cost of a device less transparent.
Another improvement could be to manage in a better way the retro-diffusion of the sparse CC structure by introducing a diffuser inside the CC pattern to give a more uniform angular viewing experience. This technological research is currently held in our laboratory and results will be published in a next future [18].
The display performance can also be improved by the use of polarized light and polarizer on the periphery of the device, the specific polarization behavior of the CC pattern could be used for that purpose [19].
In order to improve the brightness of the virtual image and reduce the visual aspect of the diffused real image, the use of anamorphic projection in association with a conical semi-reflective surface is also a way to improve the device.
7. Conclusion
The consequences of the Covid-19 pandemic have been for a large number of people the obligation to work at home in teleworking. The organization of the work is then complicated by the difficulties associated to the virtual meeting: intelligibility of multi-speaker, lack of body language, eye gaze contact, impromptu meeting organization … For some of these problems, volumetric display could be an interesting solution to allow various users to share face to face meeting between different groups of peoples. For this application the image quality and the cost and compactness of the device are important factors. For this purpose hardware technology is not the only answer and the physiological cues of 3D perception have to be considered together with the content generation and staging.
We have introduce an original concept of 360° volumetric display. The basic idea of this device is the ability to see the reflection of an image through the image itself by the use of transparent projection surface. This solution is only possible with a strict control of the angular diffusion characteristics of the surface and we have introduced for that purpose an interesting optical component based on sparse cube corner patterns. The device we have developed is functional and has allowed to promote the work of a group of artists but it is still perfectible for a general use as a consumer product. We have drawn some possible improvements to allow a better visual experience.
Among the result of this studies is the importance of the very specific multi-user configuration offered by a 360° display. With this family of display, a large number of people can share a visual experience while having each one its own point of view. In the case of a meeting, such device at the center of the meeting room should allow people to share a discussion with a virtual speaker as an actor face-to-face or as a spectator looking at two people facing each other.
More generally the transparent cube corner film we introduced here can be seen as an innovative low cost optical component with a lot of potential applications.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References
1. D. Poon, T.-C. Smalley, H. Gao, J. Kvavle, and K. Qaderi, “Volumetric Displays: Turning 3-D Inside-Out,” Opt. Photonics News 29(6), 26–33 (2018). [CrossRef]
2. R. Pigeaud, “Dialogue avec la paroi: cas des représentations paléolithiques de la grotte ornée Mayenne-Science’, Actes du colloque UISPP, Liège (2001), pp. 21–43.
3. E. Robert, “The role of the cave in the expression of prehistoric societies,” Quatern Int. 432(Part B), 59–65 (2017). [CrossRef]
4. C. Martinez, Y. Lee, N. Clement, F. Sermet, D. Sarrasin, J. Mathieu, and N. Boudier, “Development of a 360° display based on transparent projection surface: application to a theatrical performance,” Proc. SPIE 10942, 17 (2019). [CrossRef]
5. http://www.tng-lyon.fr/wp-content/uploads/2017/01/Artefact-Dossier-Spectacle-WEB-EN4.pdf
6. J. Geng, “Three-dimensional display technologies,” Adv. Opt. Photonics 5(4), 456–535 (2013). [CrossRef]
7. H. Okaichi, N. Watanabe, T. Omura, M. Kano, H. Sasaki, and M. Kawakita, “Aktina Vision: Full-parallax three-dimensional display with 100 million light rays,” Sci. Rep. 9(1), 17688 (2019). [CrossRef]
8. L. Yang, X. Sang, X. Yu, B. L. B. Yan, K. Wang, and C. Yu, “A crosstalk-suppressed dense multi-view light-field display based on real-time light-field pickup and reconstruction,” Opt. Express 26(26), 34412–34427 (2018). [CrossRef]
9. A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Transactions on Graphics (TOG) –Proceedings of ACM SIGGRAPH 200726(3), (2007)
10. X. Luo, J. Lawrence, and S. M. Seitz, “Pepper's Cone: An Inexpensive Do-It-Yourself 3D Display.” UIST, 2017, roxanneluo.github.io/papers/PeppersCone/Pepper-s_ConeUIST_2017.pdf.
11. C. Martinez, U. Rossini, D. Sarrasin, M. Lechevin, G. Dausmann, Z. Yang, and M. Krug, “Holographic Display for Extended Informative Windshield,” in Imaging and Applied Optics 2015, OSA Technical Digest (online) (Optical Society of America, 2015), paper AIW4C.2., (2015).
12. M. Kivanc Hedili, M. O. Freeman, and H. Urey, “Transmission characteristics of a bidirectional transparent screen based on reflective microlenses,” Opt. Express 21(21), 24636–24646 (2013). [CrossRef]
13. M. Bolas and D.M. Krum, “Augmented reality applications and user interfaces using head-coupled near-axis personal projectors with novel retroreflective props and surfaces,”. In: Pervasive 2010 Ubiprojection workshop, (2010)
14. S. R. Soomro and H. Urey, “Design, fabrication and characterization of transparent retro-reflective screen,” Opt. Express 24(21), 24232–24241 (2016). [CrossRef]
15. J. C. Stimson, “Process and apparatus for making central triple reflectors,” US patent 1,591,572 (1926)
16. E. Schönemann and L. Brinksmeier, “Generation of discontinuous microstructures by Diamond Micro Chiseling,” CIRP Ann. 63(1), 49–52 (2014). [CrossRef]
17. H. D. Eckhardt, “Simple Model of Corner Reflector Phenomena,” Appl. Opt. 10(7), 1559–1566 (1971). [CrossRef]
18. C. Martinez, Y. Lee, F. Sermet, M. Consonni, and B. Aventurier, “Transparent cube corner array manufacturing process for Head Up Display applications,” to be published in Photonics West 2021
19. W. He, Y. Fu, Y. Zheng, L. Zhang, J. Wang, Z. Liu, and J. Zheng, “Polarization properties of a corner-cube retroreflector with three-dimensional polarization ray-tracing calculus,” Appl. Opt. 52(19), 4527–4535 (2013). [CrossRef]
