Abstract

Active illumination 3D imaging systems based on Time-of-flight (TOF) and Structured Light (SL) projection are in rapid development, and are constantly finding new areas of application. In this paper, we present a theoretical design tool that allows prediction of 3D imaging precision. Theoretical expressions are developed for both TOF and SL imaging systems. The expressions contain only physically measurable parameters and no fitting parameters. We perform 3D measurements with both TOF and SL imaging systems, showing excellent agreement between theoretical and measured distance precision. The theoretical framework can be a powerful 3D imaging design tool, as it allows for prediction of 3D measurement precision already in the design phase.

© 2017 Optical Society of America

Full Article  |  PDF Article

Corrections

27 October 2017: A typographical correction was made to the author listing.


OSA Recommended Articles
Enhanced single-photon time-of-flight 3D ranging

Rudi Lussana, Federica Villa, Alberto Dalla Mora, Davide Contini, Alberto Tosi, and Franco Zappa
Opt. Express 23(19) 24962-24973 (2015)

Real-time super-resolved 3D in turbid water using a fast range-gated CMOS camera

Petter Risholm, Jostein Thorstensen, Jens T. Thielemann, Kristin Kaspersen, Jon Tschudi, Chris Yates, Chris Softley, Igor Abrosimov, Jonathan Alexander, and Karl Henrik Haugholt
Appl. Opt. 57(14) 3927-3937 (2018)

Single-photon pulsed-light indirect time-of-flight 3D ranging

S. Bellisai, D. Bronzi, F. A. Villa, S. Tisa, A. Tosi, and F. Zappa
Opt. Express 21(4) 5086-5098 (2013)

References

  • View by:
  • |
  • |
  • |

  1. L. Pérez, Í. Rodríguez, N. Rodríguez, R. Usamentiaga, and D. F. García, “Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review,” Sensors (Basel) 16(3), E335 (2016).
    [PubMed]
  2. Ø. Skotheim, H. Schumann-Olsen, J. Thorstensen, A. N. Kim, M. Lacolle, K. Haugholt, and T. Bakke, “A Real-Time 3D Range Image Sensor based on a novel Tip-Tilt-Piston Micro-mirror and Dual Frequency Phase Shifting,” in Proc. SPIE 9393, Three-Dimensional Image Processing, Measurement (3DIPM), and Applications (2015), p. 93930A.
  3. S. J. Kim, S. W. Han, B. Kang, K. Lee, J. D. K. Kim, and C. Y. Kim, “A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure,” IEEE Electron Device Lett. 31, 1272–1274 (2010).
  4. D. Monnin, A. L. Schneider, F. Christnacher, and Y. Lutz, “A 3D Outdoor Scene Scanner Based on a Night-Vision Range-Gated Active Imaging System,” in 3D Data Processing, Visualization, and Transmission, Third International Symposium on (2006), pp. 938–945.
  5. M. Beer, B. J. Hosticka, and R. Kokozinski, “SPAD-based 3D sensors for high ambient illumination,” in 2016 12th Conference on Ph.D. Research in Microelectronics and Electronics, PRIME 2016 (2016).
  6. J. Illade-Quinteiro, V. M. Brea, P. López, D. Cabello, and G. Doménech-Asensi, “Distance Measurement Error in Time-of-Flight Sensors Due to Shot Noise,” Sensors (Basel) 15(3), 4624–4642 (2015).
    [PubMed]
  7. F. Mufti and R. Mahony, “Statistical analysis of measurement processes for time-of-flight cameras,” in Proceedings of SPIE-The International Society for Optical Engineering (2009), p. 74470I.
  8. B. Büttgen, M. H. El Mechat, F. Lustenberger, and P. Seitz, “Pseudonoise Optical Modulation for Real-Time 3-D Imaging With Minimum Interference,” IEEE Trans. Circuits Syst. I Regul. Pap. 54, 2109–2119 (2007).
  9. S. Savarese and P. Perona, “3D depth recovery with grayscale structured lighting,” Tech. Rep., Comput. Vision, Calif. Insititute Technol. (1998).
  10. Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook, “Period coded phase shifting strategy for real-time 3-D structured light illumination,” IEEE Trans. Image Process. 20(11), 3001–3013 (2011).
    [PubMed]
  11. S. J. Koppal and V. V. Appia, “Time-of-flight (TOF) assisted structured light imaging,” U.S. patent US 2015/0062558 A1 (2015).
  12. D. M. Bloom and M. Leone, “Structured light and time of flight depth capture with a MEMS ribbon linear array spatial light modulator,” U.S. patent US 8,970,827 B2 (2015).
  13. J. Geng, “Structured-light 3D surface imaging : a tutorial,” Adv. Opt. Photonics 3, 128–160 (2011).
  14. G. Sansoni, M. Carocci, and R. Rodella, “Three-dimensional vision based on a combination of gray-code and phase-shift light projection: analysis and compensation of the systematic errors,” Appl. Opt. 38(31), 6565–6573 (1999).
    [PubMed]
  15. Ø. Skotheim and F. Couweleers, “Structured light projection for accurate 3D shape determination,” in Proc. 12th Int. Conf. Exp. Mech (2004).
  16. F. Christnacher, S. Schertzer, N. Metzger, E. Bacher, M. Laurenzis, and R. Habermacher, “Influence of gating and of the gate shape on the penetration capacity of range-gated active imaging in scattering environments,” Opt. Express 23(26), 32897–32908 (2015).
    [PubMed]
  17. W. Xinwei, L. Youfu, and Z. Yan, “Multi-pulse time delay integration method for flexible 3D super-resolution range-gated imaging,” Opt. Express 23(6), 7820–7831 (2015).
    [PubMed]
  18. M. Laurenzis and E. Bacher, “Image coding for three-dimensional range-gated imaging,” Appl. Opt. 50(21), 3824–3828 (2011).
    [PubMed]
  19. C. Pfitzner, W. Antal, P. Hess, S. May, C. Merkl, P. Koch, R. Koch, and M. Wagner, “3D Multi-Sensor Data Fusion for Object Localization in Industrial Applications,” in ISR/Robotik 2014; 41st International Symposium on Robotics; Proceedings of (2014), pp. 108–113.

2016 (1)

L. Pérez, Í. Rodríguez, N. Rodríguez, R. Usamentiaga, and D. F. García, “Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review,” Sensors (Basel) 16(3), E335 (2016).
[PubMed]

2015 (3)

2011 (3)

Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook, “Period coded phase shifting strategy for real-time 3-D structured light illumination,” IEEE Trans. Image Process. 20(11), 3001–3013 (2011).
[PubMed]

J. Geng, “Structured-light 3D surface imaging : a tutorial,” Adv. Opt. Photonics 3, 128–160 (2011).

M. Laurenzis and E. Bacher, “Image coding for three-dimensional range-gated imaging,” Appl. Opt. 50(21), 3824–3828 (2011).
[PubMed]

2010 (1)

S. J. Kim, S. W. Han, B. Kang, K. Lee, J. D. K. Kim, and C. Y. Kim, “A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure,” IEEE Electron Device Lett. 31, 1272–1274 (2010).

2007 (1)

B. Büttgen, M. H. El Mechat, F. Lustenberger, and P. Seitz, “Pseudonoise Optical Modulation for Real-Time 3-D Imaging With Minimum Interference,” IEEE Trans. Circuits Syst. I Regul. Pap. 54, 2109–2119 (2007).

1999 (1)

Bacher, E.

Brea, V. M.

J. Illade-Quinteiro, V. M. Brea, P. López, D. Cabello, and G. Doménech-Asensi, “Distance Measurement Error in Time-of-Flight Sensors Due to Shot Noise,” Sensors (Basel) 15(3), 4624–4642 (2015).
[PubMed]

Büttgen, B.

B. Büttgen, M. H. El Mechat, F. Lustenberger, and P. Seitz, “Pseudonoise Optical Modulation for Real-Time 3-D Imaging With Minimum Interference,” IEEE Trans. Circuits Syst. I Regul. Pap. 54, 2109–2119 (2007).

Cabello, D.

J. Illade-Quinteiro, V. M. Brea, P. López, D. Cabello, and G. Doménech-Asensi, “Distance Measurement Error in Time-of-Flight Sensors Due to Shot Noise,” Sensors (Basel) 15(3), 4624–4642 (2015).
[PubMed]

Carocci, M.

Christnacher, F.

Couweleers, F.

Ø. Skotheim and F. Couweleers, “Structured light projection for accurate 3D shape determination,” in Proc. 12th Int. Conf. Exp. Mech (2004).

Doménech-Asensi, G.

J. Illade-Quinteiro, V. M. Brea, P. López, D. Cabello, and G. Doménech-Asensi, “Distance Measurement Error in Time-of-Flight Sensors Due to Shot Noise,” Sensors (Basel) 15(3), 4624–4642 (2015).
[PubMed]

El Mechat, M. H.

B. Büttgen, M. H. El Mechat, F. Lustenberger, and P. Seitz, “Pseudonoise Optical Modulation for Real-Time 3-D Imaging With Minimum Interference,” IEEE Trans. Circuits Syst. I Regul. Pap. 54, 2109–2119 (2007).

García, D. F.

L. Pérez, Í. Rodríguez, N. Rodríguez, R. Usamentiaga, and D. F. García, “Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review,” Sensors (Basel) 16(3), E335 (2016).
[PubMed]

Geng, J.

J. Geng, “Structured-light 3D surface imaging : a tutorial,” Adv. Opt. Photonics 3, 128–160 (2011).

Habermacher, R.

Han, S. W.

S. J. Kim, S. W. Han, B. Kang, K. Lee, J. D. K. Kim, and C. Y. Kim, “A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure,” IEEE Electron Device Lett. 31, 1272–1274 (2010).

Hao, Q.

Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook, “Period coded phase shifting strategy for real-time 3-D structured light illumination,” IEEE Trans. Image Process. 20(11), 3001–3013 (2011).
[PubMed]

Hassebrook, L. G.

Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook, “Period coded phase shifting strategy for real-time 3-D structured light illumination,” IEEE Trans. Image Process. 20(11), 3001–3013 (2011).
[PubMed]

Illade-Quinteiro, J.

J. Illade-Quinteiro, V. M. Brea, P. López, D. Cabello, and G. Doménech-Asensi, “Distance Measurement Error in Time-of-Flight Sensors Due to Shot Noise,” Sensors (Basel) 15(3), 4624–4642 (2015).
[PubMed]

Kang, B.

S. J. Kim, S. W. Han, B. Kang, K. Lee, J. D. K. Kim, and C. Y. Kim, “A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure,” IEEE Electron Device Lett. 31, 1272–1274 (2010).

Kim, C. Y.

S. J. Kim, S. W. Han, B. Kang, K. Lee, J. D. K. Kim, and C. Y. Kim, “A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure,” IEEE Electron Device Lett. 31, 1272–1274 (2010).

Kim, J. D. K.

S. J. Kim, S. W. Han, B. Kang, K. Lee, J. D. K. Kim, and C. Y. Kim, “A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure,” IEEE Electron Device Lett. 31, 1272–1274 (2010).

Kim, S. J.

S. J. Kim, S. W. Han, B. Kang, K. Lee, J. D. K. Kim, and C. Y. Kim, “A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure,” IEEE Electron Device Lett. 31, 1272–1274 (2010).

Lau, D. L.

Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook, “Period coded phase shifting strategy for real-time 3-D structured light illumination,” IEEE Trans. Image Process. 20(11), 3001–3013 (2011).
[PubMed]

Laurenzis, M.

Lee, K.

S. J. Kim, S. W. Han, B. Kang, K. Lee, J. D. K. Kim, and C. Y. Kim, “A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure,” IEEE Electron Device Lett. 31, 1272–1274 (2010).

Liu, K.

Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook, “Period coded phase shifting strategy for real-time 3-D structured light illumination,” IEEE Trans. Image Process. 20(11), 3001–3013 (2011).
[PubMed]

López, P.

J. Illade-Quinteiro, V. M. Brea, P. López, D. Cabello, and G. Doménech-Asensi, “Distance Measurement Error in Time-of-Flight Sensors Due to Shot Noise,” Sensors (Basel) 15(3), 4624–4642 (2015).
[PubMed]

Lustenberger, F.

B. Büttgen, M. H. El Mechat, F. Lustenberger, and P. Seitz, “Pseudonoise Optical Modulation for Real-Time 3-D Imaging With Minimum Interference,” IEEE Trans. Circuits Syst. I Regul. Pap. 54, 2109–2119 (2007).

Mahony, R.

F. Mufti and R. Mahony, “Statistical analysis of measurement processes for time-of-flight cameras,” in Proceedings of SPIE-The International Society for Optical Engineering (2009), p. 74470I.

Metzger, N.

Mufti, F.

F. Mufti and R. Mahony, “Statistical analysis of measurement processes for time-of-flight cameras,” in Proceedings of SPIE-The International Society for Optical Engineering (2009), p. 74470I.

Pérez, L.

L. Pérez, Í. Rodríguez, N. Rodríguez, R. Usamentiaga, and D. F. García, “Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review,” Sensors (Basel) 16(3), E335 (2016).
[PubMed]

Rodella, R.

Rodríguez, Í.

L. Pérez, Í. Rodríguez, N. Rodríguez, R. Usamentiaga, and D. F. García, “Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review,” Sensors (Basel) 16(3), E335 (2016).
[PubMed]

Rodríguez, N.

L. Pérez, Í. Rodríguez, N. Rodríguez, R. Usamentiaga, and D. F. García, “Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review,” Sensors (Basel) 16(3), E335 (2016).
[PubMed]

Sansoni, G.

Schertzer, S.

Seitz, P.

B. Büttgen, M. H. El Mechat, F. Lustenberger, and P. Seitz, “Pseudonoise Optical Modulation for Real-Time 3-D Imaging With Minimum Interference,” IEEE Trans. Circuits Syst. I Regul. Pap. 54, 2109–2119 (2007).

Skotheim, Ø.

Ø. Skotheim and F. Couweleers, “Structured light projection for accurate 3D shape determination,” in Proc. 12th Int. Conf. Exp. Mech (2004).

Usamentiaga, R.

L. Pérez, Í. Rodríguez, N. Rodríguez, R. Usamentiaga, and D. F. García, “Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review,” Sensors (Basel) 16(3), E335 (2016).
[PubMed]

Wang, Y.

Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook, “Period coded phase shifting strategy for real-time 3-D structured light illumination,” IEEE Trans. Image Process. 20(11), 3001–3013 (2011).
[PubMed]

Xinwei, W.

Yan, Z.

Youfu, L.

Adv. Opt. Photonics (1)

J. Geng, “Structured-light 3D surface imaging : a tutorial,” Adv. Opt. Photonics 3, 128–160 (2011).

Appl. Opt. (2)

IEEE Electron Device Lett. (1)

S. J. Kim, S. W. Han, B. Kang, K. Lee, J. D. K. Kim, and C. Y. Kim, “A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure,” IEEE Electron Device Lett. 31, 1272–1274 (2010).

IEEE Trans. Circuits Syst. I Regul. Pap. (1)

B. Büttgen, M. H. El Mechat, F. Lustenberger, and P. Seitz, “Pseudonoise Optical Modulation for Real-Time 3-D Imaging With Minimum Interference,” IEEE Trans. Circuits Syst. I Regul. Pap. 54, 2109–2119 (2007).

IEEE Trans. Image Process. (1)

Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook, “Period coded phase shifting strategy for real-time 3-D structured light illumination,” IEEE Trans. Image Process. 20(11), 3001–3013 (2011).
[PubMed]

Opt. Express (2)

Sensors (Basel) (2)

J. Illade-Quinteiro, V. M. Brea, P. López, D. Cabello, and G. Doménech-Asensi, “Distance Measurement Error in Time-of-Flight Sensors Due to Shot Noise,” Sensors (Basel) 15(3), 4624–4642 (2015).
[PubMed]

L. Pérez, Í. Rodríguez, N. Rodríguez, R. Usamentiaga, and D. F. García, “Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review,” Sensors (Basel) 16(3), E335 (2016).
[PubMed]

Other (9)

Ø. Skotheim, H. Schumann-Olsen, J. Thorstensen, A. N. Kim, M. Lacolle, K. Haugholt, and T. Bakke, “A Real-Time 3D Range Image Sensor based on a novel Tip-Tilt-Piston Micro-mirror and Dual Frequency Phase Shifting,” in Proc. SPIE 9393, Three-Dimensional Image Processing, Measurement (3DIPM), and Applications (2015), p. 93930A.

D. Monnin, A. L. Schneider, F. Christnacher, and Y. Lutz, “A 3D Outdoor Scene Scanner Based on a Night-Vision Range-Gated Active Imaging System,” in 3D Data Processing, Visualization, and Transmission, Third International Symposium on (2006), pp. 938–945.

M. Beer, B. J. Hosticka, and R. Kokozinski, “SPAD-based 3D sensors for high ambient illumination,” in 2016 12th Conference on Ph.D. Research in Microelectronics and Electronics, PRIME 2016 (2016).

F. Mufti and R. Mahony, “Statistical analysis of measurement processes for time-of-flight cameras,” in Proceedings of SPIE-The International Society for Optical Engineering (2009), p. 74470I.

S. Savarese and P. Perona, “3D depth recovery with grayscale structured lighting,” Tech. Rep., Comput. Vision, Calif. Insititute Technol. (1998).

C. Pfitzner, W. Antal, P. Hess, S. May, C. Merkl, P. Koch, R. Koch, and M. Wagner, “3D Multi-Sensor Data Fusion for Object Localization in Industrial Applications,” in ISR/Robotik 2014; 41st International Symposium on Robotics; Proceedings of (2014), pp. 108–113.

S. J. Koppal and V. V. Appia, “Time-of-flight (TOF) assisted structured light imaging,” U.S. patent US 2015/0062558 A1 (2015).

D. M. Bloom and M. Leone, “Structured light and time of flight depth capture with a MEMS ribbon linear array spatial light modulator,” U.S. patent US 8,970,827 B2 (2015).

Ø. Skotheim and F. Couweleers, “Structured light projection for accurate 3D shape determination,” in Proc. 12th Int. Conf. Exp. Mech (2004).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1 Illustration of TOF measurement timing.
Fig. 2
Fig. 2 Geometry for Structured Light depth calculation.
Fig. 3
Fig. 3 Theoretical distance uncertainty for TOF and SL systems.
Fig. 4
Fig. 4 Combined set up for TOF and SL distance imaging. The camera is seen in the centre of the image, the SL projector to the left, the laser to the right bottom and the beam expansion optic to the right of the camera.
Fig. 5
Fig. 5 Intensity image (left) and distance image [metres] (right) from TOF measurements. Multiple flat panels at various distance from the camera are shown. The colour bar indicates distance in metres.
Fig. 6
Fig. 6 Experimental uncertainty in distance estimate as function of signal intensity (crosses), as well as the theoretical expression for TOF measurements based on shot noise only (solid line), and total SNR, including readout noise (dashed line).
Fig. 7
Fig. 7 Intensity image (left) and distance image [metres] (right) from SL measurements. Multiple targets shown. Colorbar is distance in metres.
Fig. 8
Fig. 8 Experimental uncertainty in distance estimate vs theoretical distance noise (crosses). The line theory = experiment is shown in black.
Fig. 9
Fig. 9 Top left: Distance map from TOF. Bottom left: Distance map from SL phase and Gray-code. Top right: Distance map from TOF with added synthetic distance noise (used for Gray-code extraction in bottom right image). Bottom right: Combination image. Distance from SL phase and Gray-code from noisy TOF distance image.

Tables (2)

Tables Icon

Table 1 Parameters for theoretical TOF and SL comparison.

Tables Icon

Table 2 Experimental parameters.

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

D = s 1 s 1 + s 2 L p u l s e 2
σ T O F = ( D s 1 ) 2 ( Δ s 1 ) 2 + ( D s 2 ) 2 ( Δ s 2 ) 2 = 1 4 L p u l s e N p h
σ T O F = 1 2 2 m c τ r e s p o n s e N p h
σ T O F D
D = B sin ( θ p ) sin ( θ p + θ c )
θ p = F O V 2 + F O V φ φ max + π 2
φ w ( x , y ) = arc tan ( I 1 ( x , y ) I 3 ( x , y ) I 2 ( x , y ) I 4 ( x , y ) )
φ ( x , y ) = φ w ( x , y ) + 2 π N g c ( x , y )
Δ φ = n = 1 4 ( φ I n ) 2 ( Δ I n ) 2 1 A 2 ( I 2 I 4 ) 2 ( I 1 + I 3 ) + ( I 1 I 3 ) 2 ( I 2 + I 4 ) = 1 A A + 2 C
σ S L = D 2 B sin ( θ c ) sin ( θ p ) F O V φ max A + 2 C A
σ S L D 3
d = d 0 + c Δ t 2 ( n + ( s n 1 s n + 1 ) 2 ( s n 1 2 s n + s n + 1 ) )

Metrics