Abstract

We present an alternative method of color imaging. It utilizes a single standard Si-based image sensor and a clear aperture, tunable, color filter. Within this scheme, three monochromatic frames are taken; each is acquired with a single color and with a fraction of the total acquisition duration. This scheme offers several advantages compared to standard Bayer-demosaicing schemes. Among them – significantly higher resolution, flexibility in exposure durations, improved noise performance and in sum – better image quality. This paper presents both color filter design and the adjusted image pipeline. Lastly, the proposed imaging scheme and the Bayer scheme are compared in terms of MSE.

© 2014 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Nature-inspired color-filter array for enhancing the quality of images

Julien Couillaud, Alain Horé, and Djemel Ziou
J. Opt. Soc. Am. A 29(8) 1580-1587 (2012)

High-speed digital color imaging pyrometry

John M. Densmore, Matthew M. Biss, Kevin L. McNesby, and Barrie E. Homan
Appl. Opt. 50(17) 2659-2665 (2011)

Pentagraph image fusion scheme for motion blur prevention using multiple monochromatic images

Alon A. Volfman, David Mendlovic, and Ariel Raz
Appl. Opt. 55(11) 3096-3103 (2016)

References

  • View by:
  • |
  • |
  • |

  1. B. E. Bayer, "Color imaging array." U.S. Patent No. 3,971,065. 20 Jul. 1976.
  2. X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221 (2008).
  3. J. Adams, K. Parulski, and K. Spaulding, “Color processing in digital cameras,” IEEE Micro 18(6), 20–30 (1998).
    [Crossref]
  4. R. B. Merrill, "Color separation in an active pixel cell imaging array using a triple-well structure," U.S. Patent No. 5,965,875. 12 Oct. 1999.
  5. E.G. Nassimbene , "Color video record and playback system," U.S. Patent No. 3,529,080 A. 15 Sep. 1970.
  6. J. B. Sampsell, "White light enhanced color field sequential projection," U.S. Patent No. 5,233,385 A. 3 Aug. 1993.
  7. B. Lyout, “Filter monochromatique polarisane et ses applications en physique solarie,” AnnalesdAstrophysique 7, 31 (1944).
  8. I. Solc, “Birefringent chain filters,” J. Opt. Soc. Am. 55(6), 621–625 (1965).
    [Crossref]
  9. J. Evans, “Solc birefringent filter,” J. Opt. Soc. Am. 48(3), 142–145 (1958).
    [Crossref]
  10. C. K. Liu, K. T. Cheng, and A. Y. G. Fuh, “Designs of high color purity RGB color filter for liquid crystal displays applications using Fabry-Perot etalons,” J. Display Technol. 8(3), 174–178 (2012).
    [Crossref]
  11. J. Jiang, D. Liu, J. Gu, and S. Süsstrunk, Camera spectral sensitivity database,” http://www.cis.rit.edu/jwgu/research/camspec/camspec_database.txt
  12. J. H. Correia, M. Bartek, and R. F. Wolffenbuttel, “Bulk-micromachined tunable Fabry-Perot for the visible spectral range,” Sens. Actuators A. 76, 191–196 (1999).
    [Crossref]
  13. Imatest, “Nikon D3100 Review”, http://www.imaging-resource.com/PRODS/D3100/D3100IMATEST.HTM
  14. Imatest, “Canon 1DS Mark III Review”, http://www.imaging-resource.com/PRODS/E1DSMK3/E1DSMK3IMATEST.HTM
  15. H. S. Malvar, L. He, and R. Cutler, "High-quality linear interpolation for demosaicing of Bayer-patterned color images,” in Proceedings of IEEE conference on Acoustics, Speech, and Signal Processing (IEEE, 2004), 485-488.
    [Crossref]
  16. B. Zitova and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21(11), 977–1000 (2003).
    [Crossref]
  17. S. Dai, A. Honda, and R. Stephens "Multi-modal image registration with hyperspectral data," (2009).
  18. G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graphic 23(3), 664–672 (2004).
    [Crossref]
  19. D. H. Foster, S. M. C. Nascimento, and K. Amano, “Information limits on neural identification of colored surfaces in natural scenes,” Vis. Neurosci. 21(3), 331–336 (2004).
    [Crossref] [PubMed]
  20. J. F. Hamilton and J. E. Adams, "Adaptive color plan interpolation in single sensor color electronic camera," U.S. Patent No. 5,629,734. 13 May 1997.
  21. B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Color plane interpolation using alternating projections,” IEEE Trans. Image Process. 11(9), 997–1013 (2002).
    [Crossref] [PubMed]

2012 (1)

2008 (1)

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221 (2008).

2004 (2)

G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graphic 23(3), 664–672 (2004).
[Crossref]

D. H. Foster, S. M. C. Nascimento, and K. Amano, “Information limits on neural identification of colored surfaces in natural scenes,” Vis. Neurosci. 21(3), 331–336 (2004).
[Crossref] [PubMed]

2003 (1)

B. Zitova and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21(11), 977–1000 (2003).
[Crossref]

2002 (1)

B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Color plane interpolation using alternating projections,” IEEE Trans. Image Process. 11(9), 997–1013 (2002).
[Crossref] [PubMed]

1999 (1)

J. H. Correia, M. Bartek, and R. F. Wolffenbuttel, “Bulk-micromachined tunable Fabry-Perot for the visible spectral range,” Sens. Actuators A. 76, 191–196 (1999).
[Crossref]

1998 (1)

J. Adams, K. Parulski, and K. Spaulding, “Color processing in digital cameras,” IEEE Micro 18(6), 20–30 (1998).
[Crossref]

1965 (1)

1958 (1)

1944 (1)

B. Lyout, “Filter monochromatique polarisane et ses applications en physique solarie,” AnnalesdAstrophysique 7, 31 (1944).

Adams, J.

J. Adams, K. Parulski, and K. Spaulding, “Color processing in digital cameras,” IEEE Micro 18(6), 20–30 (1998).
[Crossref]

Agrawala, M.

G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graphic 23(3), 664–672 (2004).
[Crossref]

Altunbasak, Y.

B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Color plane interpolation using alternating projections,” IEEE Trans. Image Process. 11(9), 997–1013 (2002).
[Crossref] [PubMed]

Amano, K.

D. H. Foster, S. M. C. Nascimento, and K. Amano, “Information limits on neural identification of colored surfaces in natural scenes,” Vis. Neurosci. 21(3), 331–336 (2004).
[Crossref] [PubMed]

Bartek, M.

J. H. Correia, M. Bartek, and R. F. Wolffenbuttel, “Bulk-micromachined tunable Fabry-Perot for the visible spectral range,” Sens. Actuators A. 76, 191–196 (1999).
[Crossref]

Cheng, K. T.

Cohen, M.

G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graphic 23(3), 664–672 (2004).
[Crossref]

Correia, J. H.

J. H. Correia, M. Bartek, and R. F. Wolffenbuttel, “Bulk-micromachined tunable Fabry-Perot for the visible spectral range,” Sens. Actuators A. 76, 191–196 (1999).
[Crossref]

Dai, S.

S. Dai, A. Honda, and R. Stephens "Multi-modal image registration with hyperspectral data," (2009).

Evans, J.

Flusser, J.

B. Zitova and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21(11), 977–1000 (2003).
[Crossref]

Foster, D. H.

D. H. Foster, S. M. C. Nascimento, and K. Amano, “Information limits on neural identification of colored surfaces in natural scenes,” Vis. Neurosci. 21(3), 331–336 (2004).
[Crossref] [PubMed]

Fuh, A. Y. G.

Gunturk, B.

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221 (2008).

Gunturk, B. K.

B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Color plane interpolation using alternating projections,” IEEE Trans. Image Process. 11(9), 997–1013 (2002).
[Crossref] [PubMed]

Honda, A.

S. Dai, A. Honda, and R. Stephens "Multi-modal image registration with hyperspectral data," (2009).

Hoppe, H.

G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graphic 23(3), 664–672 (2004).
[Crossref]

Li, X.

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221 (2008).

Liu, C. K.

Lyout, B.

B. Lyout, “Filter monochromatique polarisane et ses applications en physique solarie,” AnnalesdAstrophysique 7, 31 (1944).

Mersereau, R. M.

B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Color plane interpolation using alternating projections,” IEEE Trans. Image Process. 11(9), 997–1013 (2002).
[Crossref] [PubMed]

Nascimento, S. M. C.

D. H. Foster, S. M. C. Nascimento, and K. Amano, “Information limits on neural identification of colored surfaces in natural scenes,” Vis. Neurosci. 21(3), 331–336 (2004).
[Crossref] [PubMed]

Parulski, K.

J. Adams, K. Parulski, and K. Spaulding, “Color processing in digital cameras,” IEEE Micro 18(6), 20–30 (1998).
[Crossref]

Petschnigg, G.

G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graphic 23(3), 664–672 (2004).
[Crossref]

Solc, I.

Spaulding, K.

J. Adams, K. Parulski, and K. Spaulding, “Color processing in digital cameras,” IEEE Micro 18(6), 20–30 (1998).
[Crossref]

Stephens, R.

S. Dai, A. Honda, and R. Stephens "Multi-modal image registration with hyperspectral data," (2009).

Szeliski, R.

G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graphic 23(3), 664–672 (2004).
[Crossref]

Toyama, K.

G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graphic 23(3), 664–672 (2004).
[Crossref]

Wolffenbuttel, R. F.

J. H. Correia, M. Bartek, and R. F. Wolffenbuttel, “Bulk-micromachined tunable Fabry-Perot for the visible spectral range,” Sens. Actuators A. 76, 191–196 (1999).
[Crossref]

Zhang, L.

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221 (2008).

Zitova, B.

B. Zitova and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21(11), 977–1000 (2003).
[Crossref]

ACM Trans. Graphic (1)

G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graphic 23(3), 664–672 (2004).
[Crossref]

AnnalesdAstrophysique (1)

B. Lyout, “Filter monochromatique polarisane et ses applications en physique solarie,” AnnalesdAstrophysique 7, 31 (1944).

IEEE Micro (1)

J. Adams, K. Parulski, and K. Spaulding, “Color processing in digital cameras,” IEEE Micro 18(6), 20–30 (1998).
[Crossref]

IEEE Trans. Image Process. (1)

B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Color plane interpolation using alternating projections,” IEEE Trans. Image Process. 11(9), 997–1013 (2002).
[Crossref] [PubMed]

Image Vis. Comput. (1)

B. Zitova and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21(11), 977–1000 (2003).
[Crossref]

J. Display Technol. (1)

J. Opt. Soc. Am. (2)

Proc. SPIE (1)

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221 (2008).

Sens. Actuators A. (1)

J. H. Correia, M. Bartek, and R. F. Wolffenbuttel, “Bulk-micromachined tunable Fabry-Perot for the visible spectral range,” Sens. Actuators A. 76, 191–196 (1999).
[Crossref]

Vis. Neurosci. (1)

D. H. Foster, S. M. C. Nascimento, and K. Amano, “Information limits on neural identification of colored surfaces in natural scenes,” Vis. Neurosci. 21(3), 331–336 (2004).
[Crossref] [PubMed]

Other (10)

J. F. Hamilton and J. E. Adams, "Adaptive color plan interpolation in single sensor color electronic camera," U.S. Patent No. 5,629,734. 13 May 1997.

S. Dai, A. Honda, and R. Stephens "Multi-modal image registration with hyperspectral data," (2009).

Imatest, “Nikon D3100 Review”, http://www.imaging-resource.com/PRODS/D3100/D3100IMATEST.HTM

Imatest, “Canon 1DS Mark III Review”, http://www.imaging-resource.com/PRODS/E1DSMK3/E1DSMK3IMATEST.HTM

H. S. Malvar, L. He, and R. Cutler, "High-quality linear interpolation for demosaicing of Bayer-patterned color images,” in Proceedings of IEEE conference on Acoustics, Speech, and Signal Processing (IEEE, 2004), 485-488.
[Crossref]

B. E. Bayer, "Color imaging array." U.S. Patent No. 3,971,065. 20 Jul. 1976.

J. Jiang, D. Liu, J. Gu, and S. Süsstrunk, Camera spectral sensitivity database,” http://www.cis.rit.edu/jwgu/research/camspec/camspec_database.txt

R. B. Merrill, "Color separation in an active pixel cell imaging array using a triple-well structure," U.S. Patent No. 5,965,875. 12 Oct. 1999.

E.G. Nassimbene , "Color video record and playback system," U.S. Patent No. 3,529,080 A. 15 Sep. 1970.

J. B. Sampsell, "White light enhanced color field sequential projection," U.S. Patent No. 5,233,385 A. 3 Aug. 1993.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (3)

Fig. 1
Fig. 1 (a) schematic layout of the FPI. (b) Transmission spectra common CFAs (Nikon D3X – circles, Pentax Q – asterisks, and Canon 1 Mark III – squares, The FPI transmittances (bold) were computed for CRA of 0° (solid) and of −10 o (dashed); angular difference between two marginal rays is 20°. Red, green, blue and clear we obtained by adjusting the air gap to 340 nm, 290 nm, 230 nm and 20 nm.
Fig. 2
Fig. 2 Standard image pipeline that starts with a hyperspectral cube and results in a sRGB image. Auto white balance (AWB), color correction matrix (CCM) and tone mapping (LUT-based) are usually executed within the image signal processor (ISP).
Fig. 3
Fig. 3 Representative cropped images. (a) original images, synthesized form hyperspectral cubes. (b) SF images. (c)-(e) demosaiced Bayer images.

Tables (1)

Tables Icon

Table 1 Comparison of SF Scheme with Several Demosaicing Algorithms

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

I I 0 = 1 1+F sin 2 ( 2πndcosθ λ )
S N = N photons Q e T N photons Q e T+DT+ N r 2
MSE= 1 MN 3MN ( I( x,y,color )I'( x,y,color ) ) 2

Metrics