Abstract

Multi-beam confocal microscopy without any physical pinhole was demonstrated. As a key device, a custom CMOS image sensor realizing a focal-plane pinhole array effect by special pixel addressing and discarding of the unwanted photocarriers was developed. The axial resolution in the confocal mode measured by FWHM for a planar mirror was 8.9 μm, which showed that the confocality has been achieved with the proposed CMOS image sensor.

© 2013 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H. J. Tiziani and H. M. Uhde, “Three-dimensional analysis by a microlens-array confocal arrangement,” Appl. Opt.33(4), 567–572 (1994).
    [CrossRef] [PubMed]
  2. T. Wilson, Confocal Microscopy (Academic Press, 1990).
  3. J. Pawley and B. Masters, Handbook of Biological Confocal Microscopy, 2nd ed. (Springer, 1995) Chap.11.
  4. T. Tanaami, S. Otsuki, N. Tomosada, Y. Kosugi, M. Shimizu, and H. Ishida, “High-speed 1-frame/ms scanning confocal microscope with a microlens and Nipkow disks,” Appl. Opt.41(22), 4704–4708 (2002).
    [CrossRef] [PubMed]
  5. E. Fossum, “CMOS image sensors – electronic camera on a chip,” IEEE Trans. Electron. Dev.44(10), 1689–1698 (1997).
    [CrossRef]
  6. D. Scheffer, B. Dierickx, and G. Meynants, “Random addressable 2048×2048 active pixel sensor,” IEEE Trans. Electron. Dev.44(10), 1716–1720 (1997).
    [CrossRef]
  7. J. Ohta, Smart CMOS Image Sensors and Applications (CRC Press, 2007).
  8. P. Seitz and A. Theuwissen, Single-photon imaging (Springer, 2011).
  9. H. Yoon, S. Itoh, and S. Kawahito, “A CMOS image sensor with in-pixel two-stage transfer for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.56(2), 214–221 (2009).
    [CrossRef]
  10. D. U. Li, J. Arlt, J. Richardson, R. Walker, A. Buts, D. Stoppa, E. Charbon, and R. Henderson, “Real-time fluorescence lifetime imaging system with a 32 x 32 0.13μm CMOS low dark-count single-photon avalanche diode array,” Opt. Express18(10), 10257–10269 (2010).
    [CrossRef] [PubMed]
  11. Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
    [CrossRef]
  12. P. Lee, R. Gee, R. Guidash, T.-H. Lee, and E. Fossum, “An active pixel sensor fabricated using CMOS/CCD process technology,” Proc. 1995 IEEE Workshop on CCDs and AISs (1995).
  13. C. Sheppard and D. Shotton, Confocal Laser Scanning Microscopy (Springer, 1997) Chap. 3.
  14. S. Kawata, O. Nakamura, T. Noda, H. Ooki, K. Ogino, Y. Kuroiwa, and S. Minami, “Laser computed-tomography microscope,” Appl. Opt.29(26), 3805–3809 (1990).
    [CrossRef] [PubMed]
  15. K. Yasutomi, S. Itoh, and S. Kawahito, “A two-stage charge transfer active pixel CMOS image sensor with low-noise global shuttering and a dual-shuttering mode,” IEEE Trans. Electron. Dev.58(3), 740–747 (2011).
    [CrossRef]
  16. M. Furuta, Y. Nishikawa, T. Inoue, and S. Kawahito, “A high-speed, high-sensitivity digital CMOS image sensor with a global shutter and 12-bit column-parallel cyclic A/D converter,” IEEE J. Solid-state Circuits42(4), 766–774 (2007).
    [CrossRef]
  17. K. Kagawa, Y. Ogura, J. Tanida, and Y. Ichioka, “Discrete correlation processor as a building core of a digital optical computing system: architecture and optoelectronic embodiment,” Appl. Opt.38(35), 7276–7281 (1999).
    [CrossRef] [PubMed]

2012

Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
[CrossRef]

2011

K. Yasutomi, S. Itoh, and S. Kawahito, “A two-stage charge transfer active pixel CMOS image sensor with low-noise global shuttering and a dual-shuttering mode,” IEEE Trans. Electron. Dev.58(3), 740–747 (2011).
[CrossRef]

2010

2009

H. Yoon, S. Itoh, and S. Kawahito, “A CMOS image sensor with in-pixel two-stage transfer for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.56(2), 214–221 (2009).
[CrossRef]

2007

M. Furuta, Y. Nishikawa, T. Inoue, and S. Kawahito, “A high-speed, high-sensitivity digital CMOS image sensor with a global shutter and 12-bit column-parallel cyclic A/D converter,” IEEE J. Solid-state Circuits42(4), 766–774 (2007).
[CrossRef]

2002

1999

1997

E. Fossum, “CMOS image sensors – electronic camera on a chip,” IEEE Trans. Electron. Dev.44(10), 1689–1698 (1997).
[CrossRef]

D. Scheffer, B. Dierickx, and G. Meynants, “Random addressable 2048×2048 active pixel sensor,” IEEE Trans. Electron. Dev.44(10), 1716–1720 (1997).
[CrossRef]

1994

1990

Arlt, J.

Buts, A.

Charbon, E.

Dierickx, B.

D. Scheffer, B. Dierickx, and G. Meynants, “Random addressable 2048×2048 active pixel sensor,” IEEE Trans. Electron. Dev.44(10), 1716–1720 (1997).
[CrossRef]

Fossum, E.

E. Fossum, “CMOS image sensors – electronic camera on a chip,” IEEE Trans. Electron. Dev.44(10), 1689–1698 (1997).
[CrossRef]

Furuta, M.

M. Furuta, Y. Nishikawa, T. Inoue, and S. Kawahito, “A high-speed, high-sensitivity digital CMOS image sensor with a global shutter and 12-bit column-parallel cyclic A/D converter,” IEEE J. Solid-state Circuits42(4), 766–774 (2007).
[CrossRef]

Hashimoto, M.

Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
[CrossRef]

Henderson, R.

Ichioka, Y.

Inoue, T.

M. Furuta, Y. Nishikawa, T. Inoue, and S. Kawahito, “A high-speed, high-sensitivity digital CMOS image sensor with a global shutter and 12-bit column-parallel cyclic A/D converter,” IEEE J. Solid-state Circuits42(4), 766–774 (2007).
[CrossRef]

Ishida, H.

Itoh, S.

K. Yasutomi, S. Itoh, and S. Kawahito, “A two-stage charge transfer active pixel CMOS image sensor with low-noise global shuttering and a dual-shuttering mode,” IEEE Trans. Electron. Dev.58(3), 740–747 (2011).
[CrossRef]

H. Yoon, S. Itoh, and S. Kawahito, “A CMOS image sensor with in-pixel two-stage transfer for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.56(2), 214–221 (2009).
[CrossRef]

Kagawa, K.

Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
[CrossRef]

K. Kagawa, Y. Ogura, J. Tanida, and Y. Ichioka, “Discrete correlation processor as a building core of a digital optical computing system: architecture and optoelectronic embodiment,” Appl. Opt.38(35), 7276–7281 (1999).
[CrossRef] [PubMed]

Kawahito, S.

Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
[CrossRef]

K. Yasutomi, S. Itoh, and S. Kawahito, “A two-stage charge transfer active pixel CMOS image sensor with low-noise global shuttering and a dual-shuttering mode,” IEEE Trans. Electron. Dev.58(3), 740–747 (2011).
[CrossRef]

H. Yoon, S. Itoh, and S. Kawahito, “A CMOS image sensor with in-pixel two-stage transfer for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.56(2), 214–221 (2009).
[CrossRef]

M. Furuta, Y. Nishikawa, T. Inoue, and S. Kawahito, “A high-speed, high-sensitivity digital CMOS image sensor with a global shutter and 12-bit column-parallel cyclic A/D converter,” IEEE J. Solid-state Circuits42(4), 766–774 (2007).
[CrossRef]

Kawata, S.

Kosugi, Y.

Kuroiwa, Y.

Li, D. U.

Li, Z.

Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
[CrossRef]

Meynants, G.

D. Scheffer, B. Dierickx, and G. Meynants, “Random addressable 2048×2048 active pixel sensor,” IEEE Trans. Electron. Dev.44(10), 1716–1720 (1997).
[CrossRef]

Minami, S.

Nakamura, O.

Niioka, H.

Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
[CrossRef]

Nishikawa, Y.

M. Furuta, Y. Nishikawa, T. Inoue, and S. Kawahito, “A high-speed, high-sensitivity digital CMOS image sensor with a global shutter and 12-bit column-parallel cyclic A/D converter,” IEEE J. Solid-state Circuits42(4), 766–774 (2007).
[CrossRef]

Noda, T.

Ogino, K.

Ogura, Y.

Ooki, H.

Otsuki, S.

Richardson, J.

Scheffer, D.

D. Scheffer, B. Dierickx, and G. Meynants, “Random addressable 2048×2048 active pixel sensor,” IEEE Trans. Electron. Dev.44(10), 1716–1720 (1997).
[CrossRef]

Shimizu, M.

Stoppa, D.

Tanaami, T.

Tanida, J.

Tiziani, H. J.

Tomosada, N.

Uhde, H. M.

Ukon, J.

Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
[CrossRef]

Walker, R.

Yasutomi, K.

Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
[CrossRef]

K. Yasutomi, S. Itoh, and S. Kawahito, “A two-stage charge transfer active pixel CMOS image sensor with low-noise global shuttering and a dual-shuttering mode,” IEEE Trans. Electron. Dev.58(3), 740–747 (2011).
[CrossRef]

Yoon, H.

H. Yoon, S. Itoh, and S. Kawahito, “A CMOS image sensor with in-pixel two-stage transfer for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.56(2), 214–221 (2009).
[CrossRef]

Appl. Opt.

IEEE J. Solid-state Circuits

M. Furuta, Y. Nishikawa, T. Inoue, and S. Kawahito, “A high-speed, high-sensitivity digital CMOS image sensor with a global shutter and 12-bit column-parallel cyclic A/D converter,” IEEE J. Solid-state Circuits42(4), 766–774 (2007).
[CrossRef]

IEEE Trans. Electron. Dev.

K. Yasutomi, S. Itoh, and S. Kawahito, “A two-stage charge transfer active pixel CMOS image sensor with low-noise global shuttering and a dual-shuttering mode,” IEEE Trans. Electron. Dev.58(3), 740–747 (2011).
[CrossRef]

Z. Li, S. Kawahito, K. Yasutomi, K. Kagawa, J. Ukon, M. Hashimoto, and H. Niioka, “A time-resolved CMOS image sensor with draining-only modulation pixels for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.59(10), 2715–2722 (2012).
[CrossRef]

H. Yoon, S. Itoh, and S. Kawahito, “A CMOS image sensor with in-pixel two-stage transfer for fluorescence lifetime imaging,” IEEE Trans. Electron. Dev.56(2), 214–221 (2009).
[CrossRef]

E. Fossum, “CMOS image sensors – electronic camera on a chip,” IEEE Trans. Electron. Dev.44(10), 1689–1698 (1997).
[CrossRef]

D. Scheffer, B. Dierickx, and G. Meynants, “Random addressable 2048×2048 active pixel sensor,” IEEE Trans. Electron. Dev.44(10), 1716–1720 (1997).
[CrossRef]

Opt. Express

Other

T. Wilson, Confocal Microscopy (Academic Press, 1990).

J. Pawley and B. Masters, Handbook of Biological Confocal Microscopy, 2nd ed. (Springer, 1995) Chap.11.

J. Ohta, Smart CMOS Image Sensors and Applications (CRC Press, 2007).

P. Seitz and A. Theuwissen, Single-photon imaging (Springer, 2011).

P. Lee, R. Gee, R. Guidash, T.-H. Lee, and E. Fossum, “An active pixel sensor fabricated using CMOS/CCD process technology,” Proc. 1995 IEEE Workshop on CCDs and AISs (1995).

C. Sheppard and D. Shotton, Confocal Laser Scanning Microscopy (Springer, 1997) Chap. 3.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1
Fig. 1

Optical setup of the proposed multi-beam confocal microscope.

Fig. 2
Fig. 2

Principle to perform a focal-plane pinhole array effect with multi-beam scanning. (a) scanning path of the light beams, (b)(e) reset, (c)(f) accumulation, and (d)(g) read.

Fig. 3
Fig. 3

Image sensor architecture.

Fig. 4
Fig. 4

(a) Timing chart. (b) Sensor output format.

Fig. 5
Fig. 5

Differences of the beam positions due to the rolling shutter: (a) general case and (b) for experiments in this paper.

Fig. 6
Fig. 6

Photomicrograph of the prototype sensor.

Fig. 7
Fig. 7

Monitored waveforms of the position sensors of the piezo actuators. Upper: fast axis, bottom: slow axis.

Fig. 8
Fig. 8

A snapshot in the normal mode.

Fig. 9
Fig. 9

Rearranged images with (a) perfect alignment, (b) slight rotation of the microlens array, (c) smaller magnification, and (d) larger magnification.

Fig. 10
Fig. 10

Comparison of captured images along the axis. Axial displacements: (a)(d) 0 μm, (b)(e) 5 μm, (c)(f) 10 μm. (a)-(c) normal mode. (d)-(f) confocal mode.

Fig. 11
Fig. 11

Enlarged results of Fig. 10. (a)(d) 0μm, (b)(e) 5 μm, (c)(f) 10μm. (a)-(c) normal mode. (d)-(f) confocal mode.

Fig. 12
Fig. 12

Cross sections of Fig. 11: (a) A-A’ and (b) B-B’.

Fig. 13
Fig. 13

Comparison of the relationships between the axial displacement and the intensity.

Tables (2)

Tables Icon

Table 1 Specifications of the prototype chip

Tables Icon

Table 2 Measured characteristics

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

T P = T D + τ A + T R .
T F = N 2 T P .
τ R = τ CDS +M τ PIX
f r = 1 N 2 ( T D + τ A + T R ) = 1 N 2 { M( τ D + τ CDS +M τ PIX )+ τ A } .

Metrics