Abstract

A new filter array and a demosaicking method for snapshot multispectral polarization imaging are proposed in this paper. The proposed filter array is a thin-film wavy multilayer structure regarded as a photonic crystal that can be fabricated using the autocloning method. The multispectral polarization filter array is developed by altering the wave structure of the photonic crystal at each pixel. In addition, we propose a demosaicking method for multispectral polarization images by considering snapshot imaging as a linear model. In the experiments, we evaluated the recovered spectrum error in some color charts and showed various demosaicked images such as multispectral polarization images, specific-band degree of linear polarization images, polarized RGB images, and non-polarized RGB images.

© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Hybrid-resolution multispectral imaging using color filter array

Yuri Murakami, Masahiro Yamaguchi, and Nagaaki Ohyama
Opt. Express 20(7) 7173-7183 (2012)

Spatio-spectral binary patterns based on multispectral filter arrays for texture classification

Sofiane Mihoubi, Olivier Losson, Benjamin Mathon, and Ludovic Macaire
J. Opt. Soc. Am. A 35(9) 1532-1542 (2018)

Snapshot, reconfigurable multispectral and multi-polarization telecentric imaging system

Shaun Pacheco and Rongguang Liang
Opt. Express 22(13) 16377-16385 (2014)

References

  • View by:
  • |
  • |
  • |

  1. J. Brauers and T. Aach, “A color filter array based multispectral camera,” Proc. of Workshop Farbbildverarbeitung (2006).
  2. F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
    [Crossref]
  3. Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
    [Crossref] [PubMed]
  4. J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
    [Crossref]
  5. V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
    [Crossref] [PubMed]
  6. X. Zhao, A. Bermak, F. Boussaid, and V. G. Chigrinov, “Liquid-crystal micropolarimeter array for full Stokes polarization imaging in visible spectrum,” Opt. Express 18(17), 17776–17787 (2000).
    [Crossref]
  7. D. S. Sabatke, M. R. Descour, E. L. Dereniak, W. C. Sweatt, S. A. Kemme, and G. S. Phipps, “Optimization of retardance for a complete Stokes polarimeter,” Opt. Lett. 25(11), 802–804 (2000).
    [Crossref]
  8. 4D Technology, “Polarcam polarization camera,” http://www.4dtechnology.com .
  9. T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
    [Crossref] [PubMed]
  10. Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
    [Crossref]
  11. V. Gruev, J. V. d. Spiegel, and N. Engheta, “Dual-tier thin film polymer polarization imaging sensor,” Opt. Express 18(18), 19292–19303 (2010).
    [Crossref]
  12. G. Myhre, W. L. Hsu, A. Peinado, C. LaCasse, N. Brock, R. A. Chipman, and S. Pau, “Liquid crystal polymer full-stokes division of focal plane polarimeter,” Opt. Express 20(25), 27393–27409 (2012).
    [Crossref] [PubMed]
  13. T. Ohfuchi, M. Sakakura, Y. Yamada, N. Fukuda, T. Takiya, Y. Shimotsuma, and K. Miura, “Polarization imaging camera with a waveplate array fabricated with a femtosecond laser inside silica glass,” Opt. Express 25(20), 23738–23754 (2017).
    [Crossref] [PubMed]
  14. X. Tu, L. Jiang, M. I. Elhaj, and S. Pau, “Design, fabrication and testing of achromatic elliptical polarizer,” Opt. Express 25(9), 10355–10367 (2017).
    [Crossref] [PubMed]
  15. Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
    [Crossref]
  16. Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
    [Crossref]
  17. C. Fu, H. Arguello, B. M. Sadler, and G. R. Arce, “Compressive spectral polarization imaging by a pixelized polarizer and colored patterned detector,” J. Opt. Soc. Am. A 32(11), 2178–2188 (2015).
    [Crossref]
  18. S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).
  19. M. Kulkarni and V. Gruev, “Integrated spectral-polarization imaging sensor with aluminum nanowire polarization filters,” Opt. Express 20(21), 22997–23012 (2012).
    [Crossref] [PubMed]
  20. X. Tu and S. Pau, “Optimized design of N optical filters for color and polarization imaging,” Opt. Express 24(3), 3011–3024 (2016).
    [Crossref] [PubMed]
  21. T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
    [Crossref]
  22. M. Garcia, C. Edmiston, R. Marinov, A. Vail, and V. Gruev, “Bio-inspired color-polarization imager for real-time in situ imaging,” Optica 4(10), 1263–1271 (2017).
    [Crossref]
  23. J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
    [Crossref]
  24. Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
    [Crossref]
  25. Y. Ohtera, D. Kurniatan, and H. Yamada, “Design and fabrication of multichannel Si/SiO2 autocloned photonic crystal edge filters,” Appl. Opt. 50(9), 50–54 (2011).
    [Crossref]
  26. S. Kawakami, “Fabrication of submicrometre 3D periodic structures composed of Si/SiO2,” Electron. Lett. 33(14), 1260–1261 (1997).
    [Crossref]
  27. T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).
  28. S. Gao and V. Gruev, “Gradient-based interpolation method for division-of-focal-plane polarimeters,” Opt. Express 21(1), 1137–1151 (2013).
    [Crossref] [PubMed]
  29. J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24(18), 20799–20807 (2016).
    [Crossref] [PubMed]
  30. A. Ahmed, X. Zhao, V. Gruev, J. Zhang, and A. Bermak, “Residual interpolation for division of focal plane polarization image sensors,” Opt. Express 25(9), 10651–10662 (2017).
    [Crossref] [PubMed]
  31. R. A. Chipman, “Polarimetry,” in Handbook of Optics, 2nd ed. (McGraw-Hill, 1995), Chap 22 pp. 1–37.
  32. S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013).
    [Crossref] [PubMed]
  33. W. K. Pratt, Digital Image Processing (Wiley, 1978), pp. 356–367.
  34. W. K. Pratt and D. E. Mancill, “Spectral estimation techniques for the spectral calibration of a color image scanner,” Appl. Opt. 15(1), 73–75 (1976).
    [Crossref] [PubMed]
  35. K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
    [Crossref]

2017 (5)

2016 (3)

2015 (2)

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

C. Fu, H. Arguello, B. M. Sadler, and G. R. Arce, “Compressive spectral polarization imaging by a pixelized polarizer and colored patterned detector,” J. Opt. Soc. Am. A 32(11), 2178–2188 (2015).
[Crossref]

2014 (1)

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

2013 (2)

2012 (2)

2011 (1)

2010 (3)

2009 (4)

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
[Crossref]

2007 (2)

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

2004 (1)

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

2000 (2)

1997 (1)

S. Kawakami, “Fabrication of submicrometre 3D periodic structures composed of Si/SiO2,” Electron. Lett. 33(14), 1260–1261 (1997).
[Crossref]

1976 (1)

Aach, T.

J. Brauers and T. Aach, “A color filter array based multispectral camera,” Proc. of Workshop Farbbildverarbeitung (2006).

Achilefu, S.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Ahmed, A.

Aoyama, T.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Arce, G. R.

Arguello, H.

Baba, A.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Barnard, K. J.

J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
[Crossref]

Bermak, A.

Boussaid, F.

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

X. Zhao, A. Bermak, F. Boussaid, and V. G. Chigrinov, “Liquid-crystal micropolarimeter array for full Stokes polarization imaging in visible spectrum,” Opt. Express 18(17), 17776–17787 (2000).
[Crossref]

Brauers, J.

J. Brauers and T. Aach, “A color filter array based multispectral camera,” Proc. of Workshop Farbbildverarbeitung (2006).

Brock, N.

Chang, Z.

Charanya, T.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Chigrinov, V. G.

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

X. Zhao, A. Bermak, F. Boussaid, and V. G. Chigrinov, “Liquid-crystal micropolarimeter array for full Stokes polarization imaging in visible spectrum,” Opt. Express 18(17), 17776–17787 (2000).
[Crossref]

Chipman, R. A.

Cronin, T. W.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Deng, X.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Dereniak, E. L.

Descour, M. R.

Edmiston, C.

Elhaj, M. I.

Engheta, N.

Fu, C.

Fukuda, N.

Gao, S.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

S. Gao and V. Gruev, “Gradient-based interpolation method for division-of-focal-plane polarimeters,” Opt. Express 21(1), 1137–1151 (2013).
[Crossref] [PubMed]

Garcia, M.

Gruev, V.

M. Garcia, C. Edmiston, R. Marinov, A. Vail, and V. Gruev, “Bio-inspired color-polarization imager for real-time in situ imaging,” Optica 4(10), 1263–1271 (2017).
[Crossref]

A. Ahmed, X. Zhao, V. Gruev, J. Zhang, and A. Bermak, “Residual interpolation for division of focal plane polarization image sensors,” Opt. Express 25(9), 10651–10662 (2017).
[Crossref] [PubMed]

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

S. Gao and V. Gruev, “Gradient-based interpolation method for division-of-focal-plane polarimeters,” Opt. Express 21(1), 1137–1151 (2013).
[Crossref] [PubMed]

S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013).
[Crossref] [PubMed]

M. Kulkarni and V. Gruev, “Integrated spectral-polarization imaging sensor with aluminum nanowire polarization filters,” Opt. Express 20(21), 22997–23012 (2012).
[Crossref] [PubMed]

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

V. Gruev, J. V. d. Spiegel, and N. Engheta, “Dual-tier thin film polymer polarization imaging sensor,” Opt. Express 18(18), 19292–19303 (2010).
[Crossref]

Hasegawa, M.

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Hashimoto, N.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Hayasaki, Y.

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Hirakawa, K.

J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
[Crossref]

Hsu, W. L.

Hui, B.

Inoue, Y.

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

Ishikawa, W.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Iso, D.

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

Jia, J.

J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
[Crossref]

Jiang, L.

Junger, S.

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

Kahan, L.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Kawakami, S.

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

S. Kawakami, “Fabrication of submicrometre 3D periodic structures composed of Si/SiO2,” Electron. Lett. 33(14), 1260–1261 (1997).
[Crossref]

Kawashima, T.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Kemme, S. A.

Kikuchi, S.

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

Kulkarni, M.

Kurniatan, D.

LaCasse, C.

Lake, S. P.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Liu, X.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Luo, H.

Mancill, D. E.

Marinov, R.

Marshall, J.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Mitsunaga, T.

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

Miura, K.

T. Ohfuchi, M. Sakakura, Y. Yamada, N. Fukuda, T. Takiya, Y. Shimotsuma, and K. Miura, “Polarization imaging camera with a waveplate array fabricated with a femtosecond laser inside silica glass,” Opt. Express 25(20), 23738–23754 (2017).
[Crossref] [PubMed]

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Monno, Y.

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

Myhre, G.

Nayar, S. K.

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

Ohfuchi, T.

Ohkubo, H.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Ohta, J.

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Ohtera, Y.

Y. Ohtera, D. Kurniatan, and H. Yamada, “Design and fabrication of multichannel Si/SiO2 autocloned photonic crystal edge filters,” Appl. Opt. 50(9), 50–54 (2011).
[Crossref]

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Okutomi, M.

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

Onuki, T.

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

Pan, Q.

Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
[Crossref]

Pana, Q.

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Pau, S.

Peinado, A.

Perkins, R.

Phipps, G. S.

Powell, S. B.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013).
[Crossref] [PubMed]

Pratt, W. K.

Raman, B.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Roberts, N. W.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Sabatke, D. S.

Sadler, B. M.

Saha, D.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Sakakura, M.

Sasagawa, K.

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Sasaki, Y.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Sato, T.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Sciortino, P.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Shimotsuma, Y.

Shinoda, K.

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Spiegel, J. V. d.

Sweatt, W. C.

Takiya, T.

Tanaka, M.

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

Tokuda, T.

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Tschekalinskij, W.

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

Tu, X.

Vail, A.

Verwaal, N.

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

Walters, F.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Wang, J. J.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Weber, N.

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

Xiaojin, Z.

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

Yamada, H.

Y. Ohtera, D. Kurniatan, and H. Yamada, “Design and fabrication of multichannel Si/SiO2 autocloned photonic crystal edge filters,” Appl. Opt. 50(9), 50–54 (2011).
[Crossref]

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Yamada, Y.

Yanagi, Y.

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Yasuma, F.

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

York, T.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

Zhang, D.

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Zhang, J.

Zhang, L.

Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
[Crossref]

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Zhao, X.

Zhao, Y.

Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
[Crossref]

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Appl. Opt. (3)

Appl. Phys. Lett. (1)

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Computer Vision and Image Understanding (1)

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Electron. Lett. (1)

S. Kawakami, “Fabrication of submicrometre 3D periodic structures composed of Si/SiO2,” Electron. Lett. 33(14), 1260–1261 (1997).
[Crossref]

IEEE J. Lightwave Technol. (1)

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

IEEE Photon. Technol. Lett. (1)

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

IEEE Trans. Biomed. Circuits Syst. (1)

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

IEEE Trans. Image Process. (1)

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

IEEE Trans. on Image Process. (2)

J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
[Crossref]

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

IEICE Trans. Electron. (1)

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

J. Opt. Soc. Am. A (1)

Opt. Express (12)

M. Kulkarni and V. Gruev, “Integrated spectral-polarization imaging sensor with aluminum nanowire polarization filters,” Opt. Express 20(21), 22997–23012 (2012).
[Crossref] [PubMed]

X. Tu and S. Pau, “Optimized design of N optical filters for color and polarization imaging,” Opt. Express 24(3), 3011–3024 (2016).
[Crossref] [PubMed]

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

X. Zhao, A. Bermak, F. Boussaid, and V. G. Chigrinov, “Liquid-crystal micropolarimeter array for full Stokes polarization imaging in visible spectrum,” Opt. Express 18(17), 17776–17787 (2000).
[Crossref]

V. Gruev, J. V. d. Spiegel, and N. Engheta, “Dual-tier thin film polymer polarization imaging sensor,” Opt. Express 18(18), 19292–19303 (2010).
[Crossref]

G. Myhre, W. L. Hsu, A. Peinado, C. LaCasse, N. Brock, R. A. Chipman, and S. Pau, “Liquid crystal polymer full-stokes division of focal plane polarimeter,” Opt. Express 20(25), 27393–27409 (2012).
[Crossref] [PubMed]

T. Ohfuchi, M. Sakakura, Y. Yamada, N. Fukuda, T. Takiya, Y. Shimotsuma, and K. Miura, “Polarization imaging camera with a waveplate array fabricated with a femtosecond laser inside silica glass,” Opt. Express 25(20), 23738–23754 (2017).
[Crossref] [PubMed]

X. Tu, L. Jiang, M. I. Elhaj, and S. Pau, “Design, fabrication and testing of achromatic elliptical polarizer,” Opt. Express 25(9), 10355–10367 (2017).
[Crossref] [PubMed]

S. Gao and V. Gruev, “Gradient-based interpolation method for division-of-focal-plane polarimeters,” Opt. Express 21(1), 1137–1151 (2013).
[Crossref] [PubMed]

J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24(18), 20799–20807 (2016).
[Crossref] [PubMed]

A. Ahmed, X. Zhao, V. Gruev, J. Zhang, and A. Bermak, “Residual interpolation for division of focal plane polarization image sensors,” Opt. Express 25(9), 10651–10662 (2017).
[Crossref] [PubMed]

S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013).
[Crossref] [PubMed]

Opt. Lett. (1)

Optica (1)

Optical Review (1)

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Proceedings of the IEEE (1)

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Other (5)

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

J. Brauers and T. Aach, “A color filter array based multispectral camera,” Proc. of Workshop Farbbildverarbeitung (2006).

4D Technology, “Polarcam polarization camera,” http://www.4dtechnology.com .

W. K. Pratt, Digital Image Processing (Wiley, 1978), pp. 356–367.

R. A. Chipman, “Polarimetry,” in Handbook of Optics, 2nd ed. (McGraw-Hill, 1995), Chap 22 pp. 1–37.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1
Fig. 1 Conceptual chart of photonic crystal.
Fig. 2
Fig. 2 Outline of the proposed multispectral polarization filter array imaging.
Fig. 3
Fig. 3 Designed multispectral polarization filter array.
Fig. 4
Fig. 4 Transmittance of filter array.
Fig. 5
Fig. 5 RGB reproduction of filter array (4 × 4pixel).
Fig. 6
Fig. 6 Implementation of the proposed snapshot camera with filter array.
Fig. 7
Fig. 7 Spectral transmittance in 16 pixels.
Fig. 8
Fig. 8 Spectral reflectance comparison of color chart.
Fig. 9
Fig. 9 RGB reproduction of color chart from original spectrum.
Fig. 10
Fig. 10 Demosaicked RGB image of color chart.
Fig. 11
Fig. 11 RMSE of DOLP.
Fig. 12
Fig. 12 Captured scene. This image is captured by an RGB camera, and the color is not calibrated to sRGB correctly.
Fig. 13
Fig. 13 Snapshot and demosaicked images.

Tables (1)

Tables Icon

Table 1 RMSE of spectral reflectance and CIE76 delta E.

Equations (20)

Equations on this page are rendered with MathJax. Learn more.

s x , y , λ = [ 1 0 1 1 0 1 1 2 1 ] I x , y , λ = PI x , y , λ .
g x , y = λ T λ aR θ x , y M x , y , λ R θ x , y s x , y , λ ,
g x , y = λ T λ [ [ m ] x , y , λ 0 , 0 [ m ] x , y , λ 0 , 1 cos 2 θ x , y [ m ] x , y , λ 0 , 2 sin 2 θ x , y [ m ] x , y , λ 0 , 1 sin 2 θ x , y + [ m ] x , y , λ 0 , 2 cos 2 θ x , y ] T s x , y , λ ,
[ m ] x , y , λ 0 , 0 = [ U ] x , y , λ T E + [ U ] x , y , λ T M 2 ,
[ m ] x , y , λ 0 , 1 = [ U ] x , y , λ T E [ U ] x , y , λ T M 2 ,
[ m ] x , y , λ 0 , 2 = 0 ,
g x , y = 1 2 λ T λ [ [ U ] x , y , λ T E + [ U ] x , y , λ T M ( [ U ] x , y , λ T E [ U ] x , y , λ T M ) cos 2 θ x , y ( [ U ] x , y , λ T E [ U ] x , y , λ T M ) sin 2 θ x , y ] T s x , y , λ = 1 2 λ T λ M x , y , λ s x , y , λ .
g = TM s = TM P I = HI ,
T = 1 2 E X Y [ T λ 0 T λ 1 T λ L 1 ] ,
M = diag ( M 0 , 0 , λ 0 , M 0 , 0 , λ 1 , M 0 , 0 , λ L 1 , M 0 , 1 , λ 0 , , M X 1 , Y 1 , λ L 1 ) ,
P = E X Y L P ,
I ^ = Wg .
arg min W I I ^ 2 s . t . I ^ = Wg .
I I ^ 2 = Tr [ R I 2 R I H T W T + WH R I H T W T ] ,
W = R I H T ( H R I H T ) 1 .
I ^ msi = W msi g ,
W msi = ( E X Y L [ 1 0 1 ] ) W .
I ^ rgb = W rgb g ,
W rgb = ( E X Y ( CTL ) ) W msi ,
[ I dolp ] x , y , λ = [ s 1 2 + s 2 2 s 0 ] x , y , λ .

Metrics