Abstract

Vital signs such as pulse rate and breathing rate are currently measured using contact probes. But, non-contact methods for measuring vital signs are desirable both in hospital settings (e.g. in NICU) and for ubiquitous in-situ health tracking (e.g. on mobile phone and computers with webcams). Recently, camera-based non-contact vital sign monitoring have been shown to be feasible. However, camera-based vital sign monitoring is challenging for people with darker skin tone, under low lighting conditions, and/or during movement of an individual in front of the camera. In this paper, we propose distancePPG, a new camera-based vital sign estimation algorithm which addresses these challenges. DistancePPG proposes a new method of combining skin-color change signals from different tracked regions of the face using a weighted average, where the weights depend on the blood perfusion and incident light intensity in the region, to improve the signal-to-noise ratio (SNR) of camera-based estimate. One of our key contributions is a new automatic method for determining the weights based only on the video recording of the subject. The gains in SNR of camera-based PPG estimated using distancePPG translate into reduction of the error in vital sign estimation, and thus expand the scope of camera-based vital sign monitoring to potentially challenging scenarios. Further, a dataset will be released, comprising of synchronized video recordings of face and pulse oximeter based ground truth recordings from the earlobe for people with different skin tones, under different lighting conditions and for various motion scenarios.

© 2015 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express 16, 21434–21445 (2008).
    [Crossref] [PubMed]
  2. M.-Z. Poh, D. McDuff, and R. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58, 7–11 (2011).
    [Crossref]
  3. Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16, 077010 (2011).
    [Crossref] [PubMed]
  4. L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
    [Crossref] [PubMed]
  5. J. M. Saragih, S. Lucey, and J. F. Cohn, “Deformable model fitting by regularized landmark mean-shift,” Int. J. Comp. Vis. 91, 200–215 (2011).
    [Crossref]
  6. B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” Proceedings DARPA Image Understanding Workshop (1981), pp. 674–679.
  7. C. Tomasi and T. Kanade, “Detection and tracking of point features,” Technical Report MU-CS-91-132, Carnegie Mellon University (1991).
  8. F. P. Wieringa, F. Mastik, and A. F. W. van der Steen, “Contactless multiple wavelength photoplethysmographic imaging: a first step toward ”SpO2 camera” technology,” Annals Biomed. Eng. 33, 1034–1041 (2005).
    [Crossref]
  9. K. Humphreys, T. Ward, and C. Markham, “Noncontact simultaneous dual wavelength photoplethysmography: a further step toward noncontact pulse oximetry,” Rev. Sci. Instrum. 78, 044304 (2007).
    [Crossref] [PubMed]
  10. M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation,” Opt. Express 18, 10762–10774 (2010).
    [Crossref] [PubMed]
  11. B. D. Holton, K. Mannapperuma, P. J. Lesniewski, and J. C. Thomas, “Signal recovery in imaging photoplethysmography,” Physiological Measurement 34, 1499–1511 (2013).
    [Crossref] [PubMed]
  12. D. McDuff, S. Gontarek, and R. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61, 2593–2601 (2014).
    [Crossref] [PubMed]
  13. J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiological Measurement 28, R1 (2007).
    [Crossref] [PubMed]
  14. S. Hu, V. Azorin-Peris, and J. Zheng, “Opto-physiological modeling applied to photoplethysmographic cardiovascular assessment,” J. Healthcare Engineering 4, 505–528 (2013).
    [Crossref]
  15. D. Brennan, “Linear diversity combining techniques,” Proceedings of the IEEE 91, 331–356 (2003).
    [Crossref]
  16. M. Nitzan, B. Khanokh, and Y. Slovik, “The difference in pulse transit time to the toe and finger measured by photoplethysmography,” Physiological Measurement 23, 85–93 (2002).
    [Crossref] [PubMed]
  17. J. Shi and C. Tomasi, “Good features to track,” in “, 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1994. Proceedings CVPR ’94,” (1994), pp. 593–600.
  18. Z. Kalal, K. Mikolajczyk, and J. Matas, “Forward-backward error: Automatic detection of tracking failures,” in “2010 20th International Conference on Pattern Recognition (ICPR),” (2010), pp. 2756–2759.
  19. M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
    [Crossref]
  20. M. Fernandez, K. Burns, B. Calhoun, S. George, B. Martin, and C. Weaver, “Evaluation of a new pulse oximeter sensor,” American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses 16, 146–152 (2007).
    [PubMed]
  21. J. A. J. Heathers, “Smartphone-enabled pulse rate variability: an alternative methodology for the collection of heart rate variability in psychophysiological research,” International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology 89, 297–304 (2013).
    [Crossref] [PubMed]
  22. Philips, “Philips vital signs camera,” http://www.vitalsignscamera.com/ .
  23. Cardiio, “Cardiio,” http://www.cardiio.com/ .

2014 (1)

D. McDuff, S. Gontarek, and R. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61, 2593–2601 (2014).
[Crossref] [PubMed]

2013 (4)

J. A. J. Heathers, “Smartphone-enabled pulse rate variability: an alternative methodology for the collection of heart rate variability in psychophysiological research,” International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology 89, 297–304 (2013).
[Crossref] [PubMed]

S. Hu, V. Azorin-Peris, and J. Zheng, “Opto-physiological modeling applied to photoplethysmographic cardiovascular assessment,” J. Healthcare Engineering 4, 505–528 (2013).
[Crossref]

L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
[Crossref] [PubMed]

B. D. Holton, K. Mannapperuma, P. J. Lesniewski, and J. C. Thomas, “Signal recovery in imaging photoplethysmography,” Physiological Measurement 34, 1499–1511 (2013).
[Crossref] [PubMed]

2011 (3)

J. M. Saragih, S. Lucey, and J. F. Cohn, “Deformable model fitting by regularized landmark mean-shift,” Int. J. Comp. Vis. 91, 200–215 (2011).
[Crossref]

M.-Z. Poh, D. McDuff, and R. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58, 7–11 (2011).
[Crossref]

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16, 077010 (2011).
[Crossref] [PubMed]

2010 (1)

2008 (1)

2007 (3)

K. Humphreys, T. Ward, and C. Markham, “Noncontact simultaneous dual wavelength photoplethysmography: a further step toward noncontact pulse oximetry,” Rev. Sci. Instrum. 78, 044304 (2007).
[Crossref] [PubMed]

M. Fernandez, K. Burns, B. Calhoun, S. George, B. Martin, and C. Weaver, “Evaluation of a new pulse oximeter sensor,” American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses 16, 146–152 (2007).
[PubMed]

J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiological Measurement 28, R1 (2007).
[Crossref] [PubMed]

2005 (1)

F. P. Wieringa, F. Mastik, and A. F. W. van der Steen, “Contactless multiple wavelength photoplethysmographic imaging: a first step toward ”SpO2 camera” technology,” Annals Biomed. Eng. 33, 1034–1041 (2005).
[Crossref]

2003 (1)

D. Brennan, “Linear diversity combining techniques,” Proceedings of the IEEE 91, 331–356 (2003).
[Crossref]

2002 (1)

M. Nitzan, B. Khanokh, and Y. Slovik, “The difference in pulse transit time to the toe and finger measured by photoplethysmography,” Physiological Measurement 23, 85–93 (2002).
[Crossref] [PubMed]

1981 (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Aarts, L. A. M.

L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
[Crossref] [PubMed]

Allen, J.

J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiological Measurement 28, R1 (2007).
[Crossref] [PubMed]

Azorin-Peris, V.

S. Hu, V. Azorin-Peris, and J. Zheng, “Opto-physiological modeling applied to photoplethysmographic cardiovascular assessment,” J. Healthcare Engineering 4, 505–528 (2013).
[Crossref]

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16, 077010 (2011).
[Crossref] [PubMed]

Bambang Oetomo, S.

L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
[Crossref] [PubMed]

Bolles, R. C.

M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Brennan, D.

D. Brennan, “Linear diversity combining techniques,” Proceedings of the IEEE 91, 331–356 (2003).
[Crossref]

Burns, K.

M. Fernandez, K. Burns, B. Calhoun, S. George, B. Martin, and C. Weaver, “Evaluation of a new pulse oximeter sensor,” American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses 16, 146–152 (2007).
[PubMed]

Calhoun, B.

M. Fernandez, K. Burns, B. Calhoun, S. George, B. Martin, and C. Weaver, “Evaluation of a new pulse oximeter sensor,” American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses 16, 146–152 (2007).
[PubMed]

Chambers, J.

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16, 077010 (2011).
[Crossref] [PubMed]

Cleary, J. P.

L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
[Crossref] [PubMed]

Cohn, J. F.

J. M. Saragih, S. Lucey, and J. F. Cohn, “Deformable model fitting by regularized landmark mean-shift,” Int. J. Comp. Vis. 91, 200–215 (2011).
[Crossref]

Fernandez, M.

M. Fernandez, K. Burns, B. Calhoun, S. George, B. Martin, and C. Weaver, “Evaluation of a new pulse oximeter sensor,” American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses 16, 146–152 (2007).
[PubMed]

Fischler, M. A.

M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

George, S.

M. Fernandez, K. Burns, B. Calhoun, S. George, B. Martin, and C. Weaver, “Evaluation of a new pulse oximeter sensor,” American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses 16, 146–152 (2007).
[PubMed]

Gontarek, S.

D. McDuff, S. Gontarek, and R. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61, 2593–2601 (2014).
[Crossref] [PubMed]

Greenwald, S.

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16, 077010 (2011).
[Crossref] [PubMed]

Heathers, J. A. J.

J. A. J. Heathers, “Smartphone-enabled pulse rate variability: an alternative methodology for the collection of heart rate variability in psychophysiological research,” International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology 89, 297–304 (2013).
[Crossref] [PubMed]

Holton, B. D.

B. D. Holton, K. Mannapperuma, P. J. Lesniewski, and J. C. Thomas, “Signal recovery in imaging photoplethysmography,” Physiological Measurement 34, 1499–1511 (2013).
[Crossref] [PubMed]

Hu, S.

S. Hu, V. Azorin-Peris, and J. Zheng, “Opto-physiological modeling applied to photoplethysmographic cardiovascular assessment,” J. Healthcare Engineering 4, 505–528 (2013).
[Crossref]

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16, 077010 (2011).
[Crossref] [PubMed]

Humphreys, K.

K. Humphreys, T. Ward, and C. Markham, “Noncontact simultaneous dual wavelength photoplethysmography: a further step toward noncontact pulse oximetry,” Rev. Sci. Instrum. 78, 044304 (2007).
[Crossref] [PubMed]

Jeanne, V.

L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
[Crossref] [PubMed]

Kalal, Z.

Z. Kalal, K. Mikolajczyk, and J. Matas, “Forward-backward error: Automatic detection of tracking failures,” in “2010 20th International Conference on Pattern Recognition (ICPR),” (2010), pp. 2756–2759.

Kanade, T.

B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” Proceedings DARPA Image Understanding Workshop (1981), pp. 674–679.

C. Tomasi and T. Kanade, “Detection and tracking of point features,” Technical Report MU-CS-91-132, Carnegie Mellon University (1991).

Khanokh, B.

M. Nitzan, B. Khanokh, and Y. Slovik, “The difference in pulse transit time to the toe and finger measured by photoplethysmography,” Physiological Measurement 23, 85–93 (2002).
[Crossref] [PubMed]

Lesniewski, P. J.

B. D. Holton, K. Mannapperuma, P. J. Lesniewski, and J. C. Thomas, “Signal recovery in imaging photoplethysmography,” Physiological Measurement 34, 1499–1511 (2013).
[Crossref] [PubMed]

Lieber, C.

L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
[Crossref] [PubMed]

Lucas, B. D.

B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” Proceedings DARPA Image Understanding Workshop (1981), pp. 674–679.

Lucey, S.

J. M. Saragih, S. Lucey, and J. F. Cohn, “Deformable model fitting by regularized landmark mean-shift,” Int. J. Comp. Vis. 91, 200–215 (2011).
[Crossref]

Mannapperuma, K.

B. D. Holton, K. Mannapperuma, P. J. Lesniewski, and J. C. Thomas, “Signal recovery in imaging photoplethysmography,” Physiological Measurement 34, 1499–1511 (2013).
[Crossref] [PubMed]

Markham, C.

K. Humphreys, T. Ward, and C. Markham, “Noncontact simultaneous dual wavelength photoplethysmography: a further step toward noncontact pulse oximetry,” Rev. Sci. Instrum. 78, 044304 (2007).
[Crossref] [PubMed]

Martin, B.

M. Fernandez, K. Burns, B. Calhoun, S. George, B. Martin, and C. Weaver, “Evaluation of a new pulse oximeter sensor,” American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses 16, 146–152 (2007).
[PubMed]

Mastik, F.

F. P. Wieringa, F. Mastik, and A. F. W. van der Steen, “Contactless multiple wavelength photoplethysmographic imaging: a first step toward ”SpO2 camera” technology,” Annals Biomed. Eng. 33, 1034–1041 (2005).
[Crossref]

Matas, J.

Z. Kalal, K. Mikolajczyk, and J. Matas, “Forward-backward error: Automatic detection of tracking failures,” in “2010 20th International Conference on Pattern Recognition (ICPR),” (2010), pp. 2756–2759.

McDuff, D.

D. McDuff, S. Gontarek, and R. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61, 2593–2601 (2014).
[Crossref] [PubMed]

M.-Z. Poh, D. McDuff, and R. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58, 7–11 (2011).
[Crossref]

McDuff, D. J.

Mikolajczyk, K.

Z. Kalal, K. Mikolajczyk, and J. Matas, “Forward-backward error: Automatic detection of tracking failures,” in “2010 20th International Conference on Pattern Recognition (ICPR),” (2010), pp. 2756–2759.

Nelson, J. S.

L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
[Crossref] [PubMed]

W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express 16, 21434–21445 (2008).
[Crossref] [PubMed]

Nitzan, M.

M. Nitzan, B. Khanokh, and Y. Slovik, “The difference in pulse transit time to the toe and finger measured by photoplethysmography,” Physiological Measurement 23, 85–93 (2002).
[Crossref] [PubMed]

Picard, R.

D. McDuff, S. Gontarek, and R. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61, 2593–2601 (2014).
[Crossref] [PubMed]

M.-Z. Poh, D. McDuff, and R. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58, 7–11 (2011).
[Crossref]

Picard, R. W.

Poh, M.-Z.

M.-Z. Poh, D. McDuff, and R. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58, 7–11 (2011).
[Crossref]

M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation,” Opt. Express 18, 10762–10774 (2010).
[Crossref] [PubMed]

Saragih, J. M.

J. M. Saragih, S. Lucey, and J. F. Cohn, “Deformable model fitting by regularized landmark mean-shift,” Int. J. Comp. Vis. 91, 200–215 (2011).
[Crossref]

Shi, J.

J. Shi and C. Tomasi, “Good features to track,” in “, 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1994. Proceedings CVPR ’94,” (1994), pp. 593–600.

Slovik, Y.

M. Nitzan, B. Khanokh, and Y. Slovik, “The difference in pulse transit time to the toe and finger measured by photoplethysmography,” Physiological Measurement 23, 85–93 (2002).
[Crossref] [PubMed]

Sun, Y.

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16, 077010 (2011).
[Crossref] [PubMed]

Svaasand, L. O.

Thomas, J. C.

B. D. Holton, K. Mannapperuma, P. J. Lesniewski, and J. C. Thomas, “Signal recovery in imaging photoplethysmography,” Physiological Measurement 34, 1499–1511 (2013).
[Crossref] [PubMed]

Tomasi, C.

C. Tomasi and T. Kanade, “Detection and tracking of point features,” Technical Report MU-CS-91-132, Carnegie Mellon University (1991).

J. Shi and C. Tomasi, “Good features to track,” in “, 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1994. Proceedings CVPR ’94,” (1994), pp. 593–600.

van der Steen, A. F. W.

F. P. Wieringa, F. Mastik, and A. F. W. van der Steen, “Contactless multiple wavelength photoplethysmographic imaging: a first step toward ”SpO2 camera” technology,” Annals Biomed. Eng. 33, 1034–1041 (2005).
[Crossref]

Verkruysse, W.

L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
[Crossref] [PubMed]

W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express 16, 21434–21445 (2008).
[Crossref] [PubMed]

Ward, T.

K. Humphreys, T. Ward, and C. Markham, “Noncontact simultaneous dual wavelength photoplethysmography: a further step toward noncontact pulse oximetry,” Rev. Sci. Instrum. 78, 044304 (2007).
[Crossref] [PubMed]

Weaver, C.

M. Fernandez, K. Burns, B. Calhoun, S. George, B. Martin, and C. Weaver, “Evaluation of a new pulse oximeter sensor,” American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses 16, 146–152 (2007).
[PubMed]

Wieringa, F. P.

F. P. Wieringa, F. Mastik, and A. F. W. van der Steen, “Contactless multiple wavelength photoplethysmographic imaging: a first step toward ”SpO2 camera” technology,” Annals Biomed. Eng. 33, 1034–1041 (2005).
[Crossref]

Zheng, J.

S. Hu, V. Azorin-Peris, and J. Zheng, “Opto-physiological modeling applied to photoplethysmographic cardiovascular assessment,” J. Healthcare Engineering 4, 505–528 (2013).
[Crossref]

Zhu, Y.

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16, 077010 (2011).
[Crossref] [PubMed]

American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses (1)

M. Fernandez, K. Burns, B. Calhoun, S. George, B. Martin, and C. Weaver, “Evaluation of a new pulse oximeter sensor,” American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses 16, 146–152 (2007).
[PubMed]

Annals Biomed. Eng. (1)

F. P. Wieringa, F. Mastik, and A. F. W. van der Steen, “Contactless multiple wavelength photoplethysmographic imaging: a first step toward ”SpO2 camera” technology,” Annals Biomed. Eng. 33, 1034–1041 (2005).
[Crossref]

Commun. ACM (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Early Hum. Dev. (1)

L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study,” Early Hum. Dev. 89, 943–948 (2013).
[Crossref] [PubMed]

IEEE Trans. Biomed. Eng. (2)

M.-Z. Poh, D. McDuff, and R. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58, 7–11 (2011).
[Crossref]

D. McDuff, S. Gontarek, and R. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61, 2593–2601 (2014).
[Crossref] [PubMed]

Int. J. Comp. Vis. (1)

J. M. Saragih, S. Lucey, and J. F. Cohn, “Deformable model fitting by regularized landmark mean-shift,” Int. J. Comp. Vis. 91, 200–215 (2011).
[Crossref]

International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology (1)

J. A. J. Heathers, “Smartphone-enabled pulse rate variability: an alternative methodology for the collection of heart rate variability in psychophysiological research,” International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology 89, 297–304 (2013).
[Crossref] [PubMed]

J. Biomed. Opt. (1)

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16, 077010 (2011).
[Crossref] [PubMed]

J. Healthcare Engineering (1)

S. Hu, V. Azorin-Peris, and J. Zheng, “Opto-physiological modeling applied to photoplethysmographic cardiovascular assessment,” J. Healthcare Engineering 4, 505–528 (2013).
[Crossref]

Opt. Express (2)

Physiological Measurement (3)

B. D. Holton, K. Mannapperuma, P. J. Lesniewski, and J. C. Thomas, “Signal recovery in imaging photoplethysmography,” Physiological Measurement 34, 1499–1511 (2013).
[Crossref] [PubMed]

J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiological Measurement 28, R1 (2007).
[Crossref] [PubMed]

M. Nitzan, B. Khanokh, and Y. Slovik, “The difference in pulse transit time to the toe and finger measured by photoplethysmography,” Physiological Measurement 23, 85–93 (2002).
[Crossref] [PubMed]

Proceedings of the IEEE (1)

D. Brennan, “Linear diversity combining techniques,” Proceedings of the IEEE 91, 331–356 (2003).
[Crossref]

Rev. Sci. Instrum. (1)

K. Humphreys, T. Ward, and C. Markham, “Noncontact simultaneous dual wavelength photoplethysmography: a further step toward noncontact pulse oximetry,” Rev. Sci. Instrum. 78, 044304 (2007).
[Crossref] [PubMed]

Other (6)

B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” Proceedings DARPA Image Understanding Workshop (1981), pp. 674–679.

C. Tomasi and T. Kanade, “Detection and tracking of point features,” Technical Report MU-CS-91-132, Carnegie Mellon University (1991).

J. Shi and C. Tomasi, “Good features to track,” in “, 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1994. Proceedings CVPR ’94,” (1994), pp. 593–600.

Z. Kalal, K. Mikolajczyk, and J. Matas, “Forward-backward error: Automatic detection of tracking failures,” in “2010 20th International Conference on Pattern Recognition (ICPR),” (2010), pp. 2756–2759.

Philips, “Philips vital signs camera,” http://www.vitalsignscamera.com/ .

Cardiio, “Cardiio,” http://www.cardiio.com/ .

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1

Camera-based PPG estimation: strength of PPG signal obtained from different skin patch in the face is different due to spatial variations in blood perfusion. DistancePPG combines the average pixel intensity signal yi(t) from different regions of the face using a weighted average to maximize the SNR of the overall estimate of the PPG signal, yraw(t) shows the PPG signal obtained from a single pixel in the forehead (marked in red) for comparison.

Fig. 2
Fig. 2

Overall steps involved in distancePPG algorithm for estimating camera-based PPG.

Fig. 3
Fig. 3

Working of MRC: (a) illustrates the goodness metric definition based on the area under the power spectrum density (PSD) of PPG, (b) shows a face with the goodness metric overlay, red regions have higher goodness metric, blue regions have lower goodness metric, (c) shows the PPG signal extracted from four different regions marked on the face, also shows the weighted average camera PPG estimate which is very similar in shape to pulse-ox PPG signal (ground truth)

Fig. 4
Fig. 4

distancePPG performance comparison for different skin tones: Plot (a) shows SNR improvement due to distancePPG for subjects clubbed into three categories: fair/light (4), olive (4), brown (4). The SNR gain due to distancePPG is more for darker skin tones. Plot (b)–(d) compares typical camera PPG waveform estimated using face averaging method (top) and distancePPG (middle) with the ground truth pulse oximeter signal (bottom).

Fig. 5
Fig. 5

Bland-Altman plot: comparison of PR derived from camera PPG and from ground truth pulse oximeter for different skin tones, 12 subjects having fair (4), olive (4) and brown (4) skin tones categories. Note that different y-axis scales are used for distancePPG and face averaging method to accommodate all the points.

Fig. 6
Fig. 6

Pulse rate variability (PRV) estimation performance for different skin tones: 12 subjects having fair (4), olive (4) and brown (4) skin tones categories. (Left) Bar-plot shows the RMSE in the timings of the peaks in the camera-based PPG waveform when compared with pulse oximeter based ground truth. (Right) bar-plot shows the percentage of missing peaks in camera-based PPG waveform. DistancePPGsignificantly reduces the percentage of missing peaks.

Fig. 7
Fig. 7

Performance of distancePPGunder various motion scenario: Plot (a) shows the improvement in SNR due to distancePPG, (b–d) shows typical snippets of PPG waveform estimated using face averaging method (top) and distancePPG (middle) during different types of motion. Comparing the camera PPG waveform with ground truth pulse oximeter (bottom) clearly shows the improvement due to distancePPG during small-medium motion (≤ 4 px per frame). Under large motion (≥ 5 px per frame), distancePPG suffers due to uncompensated motion artifacts. Average motion magnitude captures only one aspect of motion and thus cannot always predict exact performance deterioration under various types of motion like turning, tilting etc.

Fig. 8
Fig. 8

Bland-Altman plot: comparison of PR derived from camera PPG and from ground truth pulse oximeter for 5 subjects having different skin tones under three motion scenarios — (i)Reading text, (ii) watching video, (iii) talking.

Fig. 9
Fig. 9

Pulse rate variability (PRV) estimation performance under three motion scenario: (i) running, (ii) watching, (iii) talking for 5 subjects of varying skin tones. (Left) Bar-plot shows the RMSE in the timings of the peaks in the camera-based PPG waveform when compared with pulse oximeter based ground truth. (Right) bar-plot shows the percentage of missing peaks in camera-based PPG waveform.

Fig. 10
Fig. 10

SNR of camera-based PPG using distancePPG and face averaging method under various lighting conditions. DistancePPG provides on an average 6.5 dB SNR improvement for brown skin tone (squares), and provides on an average 1.9 dB SNR improvement for fair skin tone person under various lighting conditions

Fig. 11
Fig. 11

Scatter plot between goodness metric (dB) and SNR (dB): Goodness metric defined in this paper is a good substitute for signal quality (SNR) in regions having goodness metric greater than −3 dB (linear region of the scatter plot).

Tables (1)

Tables Icon

Algorithm 1 distancePPG Region Tracker algorithm

Equations (16)

Equations on this page are rendered with MathJax. Learn more.

V ( x , y , t ) = I ( x , y , t ) R ( x , y , t ) .
y i ( t ) = I i Illumination ( α i p ( t ) subsurface reflectance + b i Surface reflectance ) Reflectance + q i ( t ) Quantization noise ,
y ^ 1 ( t ) = A 1 p ( t ) + w 1 ( t ) , y ^ 2 ( t ) = A 2 p ( t ) + w 2 ( t ) , y ^ n ( t ) = A n p ( t ) + w n ( t ) ,
p ^ ( t ) = i = 1 n G i y ^ i ( t ) .
G i = A i w i ( t ) 2 .
p ^ ( t ) = i = 1 n G i y ^ i ( t ) I ( y ^ max , i y ^ min , i < A th ) ,
G i ( PR ) = PR b PR + b Y ^ i ( f ) d f B 1 B 2 Y ^ i ( f ) d f PR b PR + b Y ^ i ( f ) d f ,
k ( t ) = A 1 p ( t ) + n k ( t )
z ( t ) = A 2 p ( t ) + n z ( t )
n k ( t ) , n z ( t ) 0
n k ( t ) or n z ( t ) , p ( t ) 0 .
k ( t ) , z ( t ) z ( t ) , z ( t ) A 1 A 2 p ( t ) 2 A 2 2 p ( t ) 2 + n z ( t ) 2
A 1 A 2 k ( t ) , z ( t ) z ( t ) , z ( t ) .
n k ( t ) k ( t ) k ( t ) , z ( t ) z ( t ) , z ( t ) z ( t )
s k ( t ) k ( t ) , z ( t ) z ( t ) , z ( t ) z ( t ) ,
SNR s k ( t ) 2 n k ( t ) 2 .

Metrics