Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Occlusion-robust sensing method by using the light-field of a 3D display system toward interaction with a 3D image

Not Accessible

Your library or personal account may give you access

Abstract

The recent developments in 3D display technology are remarkable. Owing to such developments, the importance of methods for interacting with 3D images has been increasing. In particular, the ability to perform input operations by directly touching displayed images is important. Therefore, a sensing method is required for recognizing the 3D positions of fingers and determining whether the fingers are touching the 3D image displayed. Conventionally, such a sensing method generally involves position sensing of fingers through image-based active sensing. However, this does not solve the following two problems: (1) the problem of positional registration, that is, the accurate matching between the displayed image and input location, and (2) the problem of occlusion robustness, that is, the achievement of successful sensing even with the presence of an occluding object between the sensor and hands to allow free movement of the hands. Our proposed method solves these problems through the following two ideas. First, we used a method called aerial imaging by retroreflection, which focuses light rays from a wider range than other 3D display systems. Second, for capturing the reflected light as well as projecting to the sensing target in active sensing, we used the light-field formed by the 3D display system. We also propose a method for obtaining rotation information, considering that the light-field formed by the reflected light changes based on the angle of the sensing target. Simulations and experiments were performed to evaluate the proposed system, and the system setup was optimized through simulations. In the first two experiments conducted on position and rotation sensing, we evaluated the sensing errors due to occlusion and found them to be less than 1.74 mm and less than 15.4 deg, respectively. In the third experiment, we constructed an interaction system with 3D images by using the proposed method and evaluated this system.

© 2019 Optical Society of America

Full Article  |  PDF Article
More Like This
3D touchable holographic light-field display

Masahiro Yamaguchi and Ryo Higashida
Appl. Opt. 55(3) A178-A183 (2016)

Three-dimensional profilometric reconstruction using flexible sensing integral imaging and occlusion removal

Xin Shen, Adam Markman, and Bahram Javidi
Appl. Opt. 56(9) D151-D157 (2017)

3D reconstruction of objects with occlusion and surface reflection using a dual monocular structured light system

Kejing He, Congying Sui, Congyi Lyu, Zerui Wang, and Yunhui Liu
Appl. Opt. 59(29) 9259-9271 (2020)

Supplementary Material (4)

NameDescription
Visualization 1       3D image of seven-segment LED and bar LED.
Visualization 2       Touching the 3D image of the light spot with a fingertip.
Visualization 3       The area at the top of the measuring device was disorganized. The operator changed the position and the inclination of his or her palm.
Visualization 4       3D image of the light spot was pressed by thumb. The other fingers and the palm blocked some of the light paths from/to the device. The thumb was moved up and down.

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (33)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (2)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (2)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All Rights Reserved