Open Access
Add to BibSonomyAbstract
Remote photoplethysmography (rPPG) imaging is an optical technique to remotely assess the local cutaneous microcirculation. Despite its potential for enabling health-related applications, the current understanding of the outcome images/maps remains incomplete. In this paper, we present a model and supporting experiments confirming the contribution of skin inhomogeneity to the morphology of PPG waveforms. Since rPPG imagers rely on the complex inner-product operator and may combine multiple wavelengths, the derived phase measurements reflect morphological heterogeneity of PPG signals to a larger extent than propagation-related phase differences. The influence of light penetration depth on PPG was observed and modeled on the green and red wavelengths at the hand region. We further show how our work contributes to understanding reproducibility issues in recent papers on pulse wave velocity (PWV) estimation.
© 2016 Optical Society of America
1. Introduction
Photoplethysmography (PPG) is a non-invasive optical method to detect blood volume variations in the microvascular bed of tissue [1]. An extension of the PPG concept is imaging, remotely, the local perfusion at the level of the cutaneous microcirculation. Remote-PPG (rPPG) imaging was first proposed, in visible light, by Verkruysse et al. [2] to describe the effect of blood coagulation on the amplitude and phase delay of PPG signals in skin, while Kamshilin et al. [3] has subsequently shown enhanced resolution by computing rPPG-images as a similarity metric—the complex inner-product using the Hilbert operator—between a reference PPG-function and an array of PPG-virtual sensors spanning a given skin region of interest (RoI). Camera-based rPPG imagers have been proposed to improve remote pulse rate extraction and to detect medical conditions in the skin tissue [2–4]. However, exploring the potential of this developing technique requires a better understanding of the spatially varying phase and pulsating strength that is inherent to mapping PPG signals using the complex inner-product. As a matter of fact, amplitude and phase maps are obtained, but the current published evidence remains insufficient to support the clinical value that these maps may have [5, 6].
Previously, we have identified ballistocardiographic (BCG) skin motion as an artifact in rPPG-images and attempted to eliminate this interfering source by combining camera channels; i.e., sensor arrays with different wavelength sensitivity [5, 7]. Although the improved validity was unequivocal in amplitude maps, the artifact-corrected imagers still contain phase variations which have no obvious interpretation. Figure 1 illustrates the problem, where the Pulse Blood Volume (PBV [5, 8]) imager and the Blood Pulsation imager (BPI [3]; using the green channel) are provided as examples of a multiple- and single-channel imager, respectively. It is visible that channel-combining by PBV addresses motion artifacts successfully, e.g., at the wrist it is visible that the amplitude map becomes rather homogenous (as expected) and also that the phase map is much more consistent. Still, the phase difference between wrist and palm changes rapidly, while the smaller phase variations also have no obvious explanation.
Fig. 1 Amplitude and phase maps, acquired at the hand and wrist of one test subject. The PBV algorithm combines RGB channels, whereas BPI uses the green channel only. Arrows highlight artifact mitigation at the wrist and palm. Scale is identical for PBV and BPI maps (units: phase, degrees; amplitude, normalized AC/DC).
In this investigation, we aim at improving our understanding of phase variations in PPG images. To this end, we build on the anatomical description of the skin tissue, which is simplified as stacked horizontal layers (e.g., epidermis, dermis and hypodermis; see Fig. 2). From the deep vascular plexus to the skin surface, there are changes in the caliber and elasticity of the arterial vessels. Previous studies, conducted using the contact-PPG modality, have featured parallel multispectral measurements of the skin to support that vessels pulsate differently as a function of depth [9–11]. Hence, it is logical to expect that the morphology and amplitude of the measured pressure waveform and/or blood-volume variations reflects these changes. As the pressure wave propagates through the vasculature, different harmonics of the PPG-signal may be attenuated and delayed differently; i.e., phase “dispersion” in PPG phase maps occurs. This supports the suggestion that different wavelengths (i.e., light penetration depths) result in different morphologies for the remotely-acquired PPG-signals—an hypothesis which is compatible with the earlier observations of Spigulis et al. [9] using contact-PPG and also N. Vahdani-Manaf and T. Kayıkçıoğlu, who reported gradual phase delays, from blue to infrared (IR) [12].
Fig. 2 The multi-layered structure of the skin, with ramifications and decreasing caliber of arterial vessels from hypodermis to epidermis and contrast in penetration depths for red and green illumination. From the interaction of light with the skin, PPG signals are formed. Red and green wavelengths reflect differently weighted light energy sums over depth [14, 15].
Later in this paper, clear differences will be shown between the rPPG signals from the red and green camera channels. Wavelength-dependent differences in the scattered light from the skin, even between collocated sites, is consistent with the spatial heterogeneity of the skin architecture (see schematics in Fig. 2), including the cutaneous vasculature [13]. Unfortunately, skin-site variability is ignored in imaging (mapping) the phase of PPG signals, possibly leading to erroneous interpretations.
rPPG-images are the outcome of a two-step approach, where signals are first extracted from each collocated skin site, recorded by a video camera and then correlated with a Hilbert-transformed reference function for the remote-PPG signal [3]. Such reference is the same for all sensor locations. Typically, it is the signal that results from processing a user-defined region of interest (RoI) at the skin followed by Hilbert-transformation. Each pixel location in the rPPG-image accounts for the correlation between the sensor signal and a reference PPG-signal. There is a validity issue in this approach because the complex inner-product provides a phase map which has an unclear meaning if the shape of the signal changes over the skin (e.g., see Fig. 1). Thus, PPG phase-maps may reflect the shape variability of PPG to a comparable or an even greater extent than the contribution from the propagation of the cardiac pulse pressure wave.
In the remainder of this paper, we shall confirm the expectations from our simplified skin model. In the first experiment (Section 2.2), we demonstrate waveform deformation from small arteries to the bed tissue by contrasting the waveforms of skin (i.e., radial artery) motion at the wrist (wMOT), transmission-mode PPG using red illumination (TxPPG) and reflection-mode PPG (RxPPG). In a second experiment (Section 2.3), it is verified that the green-RxPPG waveforms are a distorted and delayed version of their red-RxPPG counterparts and that compressing the microvasculature reduces waveform dissimilarity between red and green. Finally, we shall discuss important consequences from the model that have been neglected in recent publications and led to erroneous conclusions. Also we shall show how the model leads to a better understanding of phase and amplitude maps with the emerging PPG-imaging technique.
2. Analysis and methods
2.1. Modeling light interaction within the skin
It can be helpful to briefly revise the influence of wavelength selection on light penetration depth within the skin, which unequivocally confirms that the skin layers are wavelength-selective. Anderson and Parrish [16] highlighted that the chromophores of the epidermis and stratum corneum, in general, determine the attenuation of radiation in these layers to a larger extent than the optical scattering mechanism. Also, although the blood chromophores (Hb, HbO2, and bilirubin) determine dermal absorption of wavelengths >320 nm, scattering by collagen fibers largely determines the depths to which these wavelengths penetrate the dermis [17]. When scattering is marked, most photons experience multiple scattering before being absorbed or back-scattered from the sample [16].
The epidermis (depth range, 0–80 µm; however, the thickness of skin epidermis largely depends on body site and can vary between 50 and 200 µm) may be sampled by all wavelengths but absorbs mostly blue (<490 nm). The superficial dermis (depth range, 50–200 µm) has Hb and HbO2 as main absorbers and is reached by wavelengths above >500 nm. For example, the estimated penetration depth of the 544 nm (green) wavelength is about 0.3 mm, which concerns the reticular dermis; i.e., a highly diffusing fibrous structure (situated at the mid-dermis) which severely limits the contributions to the PPG-signal from deeper vascular structures for green-to-yellow wavelengths. This means that the upper (subepidermal) vascular plexus is responsible for almost the entire absorption of the green light. Thus, only the diffusely reflected light using red and infrared wavelengths (> 590 nm) reach the deep dermis and hypodermis, which contain the deep vascular plexus and large vessels [14].
The effect of light penetration depth in the remote-PPG signal morphogenesis was specifically investigated by Huelsbusch [15]. By simulating the visitation probabilities of photons at different skin penetration depths, he concluded that the diffusely reflected green illumination samples mostly the superficial (average depth, 50 µm) from the superficial vascular plexus, while light in red reaches the deep vascular plexus. It is noteworthy that the exact numbers reported in the literature are inconsistent, which is possibly due to contrasts between data acquired in vitro or in vivo, modeling simplifications and tissue variability among skin sites or populations [18,19].
Since the light penetration depth is wavelength-dependent, the pulsating component of the scattered light (i.e., the PPG signal) is a unique mixture of contributions from each of the skin layers, where deeper wavelengths scan deeper layers and vice versa. Accordingly, the remote PPG signal may be modeled as a weighted average of the contributions from different layers:
where the coefficients, C1, C2 and C3, depend on the wavelength λ (due to blood absorption) and location , where x and y are the surface coordinates within the skin, and z refers to the penetration depth dimension. The dependence on reflects, therefore, the inhomogeneity of the skin-layers in the remote signal. Clearly, we acknowledge that the above is a discrete simplification and in reality the signal results as an integral over weighted contributions. The proposed model is consistent with the following predictions:- Although shorter wavelengths (particularly yellow-green) only reach the less pulsatile superficial skin layers, its sensitivity for blood-volume variations is stronger than for red and infrared (IR). Hence, for blue-to-yellow illumination, low pulsatility at a shallow skin layer suffices to generate a stronger signal than by using red and IR, yet the latter reaches more pulsating vessels. The non-linear relation between the strength of PPG signals and the blood absorption curve was shown by Corral et al. [20]. His measurements showed that PPG-strength is fairly correlated to blood-absorption for the deeply wavelengths between 600 nm and 900 nm. The PPG strength rises less than proportional to the blood absorption towards the hemoglobin absorption peak at 550 nm and falls towards 400 nm.
- Signals extracted using less penetrating wavelengths are more delayed because they “see” mostly smaller vessels. For the same reason, shallower penetration wavelengths have a decreasing high-frequency (HF) content of the pulse signal. We expect that PPG-signals obtained with red light have a relatively small delay and little loss of HF-content compared to signals reflecting arterial wall motion (which can be measured as a BCG-motion at the skin surface above larger arteries). However, PPG from green light is expected to show much more HF-loss and a further increased delay compared to PPG in red;
- Compressing the skin is expected to decrease the morphological difference between red and green PPG-signals, as blood is removed from the superficial plexus and light-modulation is more determined by deeper vessels for all wavelengths. We expect a resulting rise in PPG amplitude for all wavelengths and a decrease in relative phase shifts.
This investigation involved two experiments at the right upper limb which aim to test the above predictions. The study was approved by the Internal Committee Biomedical Experiments of Philips Research and an informed consent was obtained from each subject.
2.2. Experiment 1: waveform changes across the terminal arterial path
The measurements for experiment 1 were carried out in four healthy subjects (ages, 22 to 30 yrs; 2 male). Subjects with painted skin/nails or skin diseases were excluded. We further required skin motion at the wrist to be visible to the human eye under lateral lighting conditions. For each participant, we jointly acquired and compared surrogates of the pressure signal at the radial artery and at contrasting levels of the microvascular bed of tissue. This allowed us to demonstrate progressive deformation of the pulse pressure waveform across the terminal arterial path, which is currently not accounted for in PPG-imaging algorithms.
Sampling PPG at the surface of the dermis, comprising arteriole endings and capillaries [15], was achieved by using the shallow penetrating green-channel from the camera (hereafter denoted as RxPPG-green). Similarly, PPG was sampled deeper (also including the dermal artery) by inspecting the transmission-mode PPG (TxPPG) signal from the finger pad. Extracting TxPPG by using a camera was first demonstrated by Rubins et al. [21] and was recently validated for heart-rate monitoring by Amelard et al. [22].
Complementarity, remotely probing skin motion in the vicinity of pulsating arteries is a means to assess the morphology of the cardiac pressure waveform prior to deformation across the skin tissue. The rationale of this approach is that vessel wall movements and local arterial pressure are almost linearly related for small arteries such as the radial artery (diameter, 2–3 mm), although we acknowledge that differences may exist between signals of skin displacement, blood flow and arterial pressure within the vicinity of an artery. We shall denote the remotely-acquired motion signal at the wrist as wMOT. Remotely extracting skin motion in the vicinity of the radial artery displacement was demonstrated by Moço et al. [5].
2.2.1. Data acquisition
The setup for experiment 1 is illustrated in Fig. 3. An RGB camera detected, simultaneously, surrogate signals of the cardiac pressure wave: PPG (at the palm and finger) and skin motion (in the vicinity of the radial artery). Illumination was provided by a red LED and two fluorescent (FL) lamps. The red LED was powered in DC, while the fluorescent lamps are of a dimmable type, equipped with an internal power supply and are operated in AC-mode with a very high frequency around 22 kHz. This frequency is high enough to prevent interference with the camera sampling rate. Recordings were performed under mixed lighting conditions: side illumination for the wrist, bilateral for the palm and mostly punctiform red retro-illumination for the finger pad. wMOT signals were acquired under lateral lighting conditions (i.e., by covering the outer side of the wrist with black cardboard; see Fig. 3). Reflection-mode PPG signals were formed over the whole skin surface, although lighting conditions were most even (and thus less contaminated by motion artifacts) at the palm. At the finger pad, the LED provided back-illumination at the pointing finger so as to result in the TxPPG signal. We note that, at the finger, the transmissive signal is a much larger contribution than reflection-mode red-PPG, causing the latter to be negligible. The hand was supported on a rigid horizontal surface, while the LED was attached to the back of the finger nail and did not move during video recordings. For minimized inter-channel interference, the camera was equipped with a dual-band optical filter (passbands: green, 500–560 nm; red, 620–680 nm). The green and red channels of the color camera selected the green and red peaks of the FL illumination (see normalized spectrum in Fig. 3). The LED provided illumination within the 620–680 nm band (evenly for this range). Recordings were performed for three minutes at a sampling rate of 50 Hz.
Fig. 3 Setup for inspecting pressure waveform changes across the terminal arterial path at the upper limb. Signals are collected at the wrist (wMOT, lateral illumination), finger (TxPPG) and palm (RxPPG-green, bilateral illumination).
2.2.2. Signal processing
Figure 4 shows the processing pipelines of the procedures applied in experiment 1: a) PPG-imaging and b) ensemble-averaging from user-defined or refined skin RoIs. Both were implemented in MATLAB. Each procedure is described in the remaining of this section.
Fig. 4 Processing pipelines for (a) PPG-imaging using the denoised TxPPG signal from the finger as reference for image formation and (b) ensemble-averaging.
PPG-imaging
rPPG-maps (here interchangeably referred to as rPPG-images) were computed for the red and green channels. To this end, frames were first registered with respect to (w.r.t.) the central frame of the video sequence [23] to stabilize small involuntary movements. The frames were subsequently blurred with a Gaussian kernel (size, 45) and resized, horizontally and vertically, by a factor of 1/5. Each pixel in the obtained PPG-images shall be subsequently referred to as a sensor-element, while m and n are the vertical and horizontal coordinates within the rPPG-image, respectively.
The processing in each sensor-element (m, n) began with the corresponding raw RGB streams, , being normalized as “AC over DC”, where AC and DC stand for the cardiac-related variations and low-frequency components of the input signal, respectively. To this end, sensors were lowpass filtered (LPF) to extract its “DC” component (i.e., low-frequency non-cardiac related) with a Butterworth filter (9th order; cutoff frequency, 30 bpm), resulting in . The normalized signal was obtained as . Despite its simplicity, AC/DC conversion confers invariance to local brightness levels and robustness to non-cardiac low-frequency oscillations in the signal (e.g., drifting or respiration). For additional improvements in signal quality, rPPG streams were subsequently adaptively bandpass filtered (ABPF) so that only the fundamental of the pulse-rate was selected. ABPF processing was performed in the frequency domain in a stride processing approach. The signals were processed in strides of 512 samples (i.e., about 10 cardiac cycles) with an overlapping factor of 50%. Each stride was detrended, multiplied with a Hanning window and filtered in the frequency domain by selecting the fundamental of the pulse-rate frequency. The camera-based TxPPG was used as a reference for pulse-rate estimation in the ABPF processing.
Single-channel PPG-maps were computed as the complex inner-product between individual skin sensors (normalized as AC/DC), in the red and green camera channels, within the recording frames and a reference function. Accordingly, the value of the rPPG-image at location (m, n) is the normalized inner-product between xRef and ; i.e, , with l being the current frame in the range 1 … L and L is the frame length of the video sequence. xRef denotes the reference signal. The reference remote-PPG signal was set to be the Hilbert-transformed TxPPG signal (normalized as AC/DC), as extracted from the finger pad. Similar to regular sensor streams, the reference signal was denoised by ABPF. The signal was finally Hilbert-transformed and normalized to unit norm as .
Ensemble averaging
Firstly, RoIs were defined for the extraction of RxPPG-green, TxPPG and wMOT signals. For RxPPG-green, the skin RoI was user-defined at the palm, at a site where the corresponding location in PPG amplitude-maps was fairly uniform. We ensured that the RxPPG amplitude at the selected RoI was relatively strong in green (e.g., >0.005 in normalized AC/DC), while the amplitude in the red channel was at least an order of magnitude below the sites of strong pulsation in the wrist. In this manner, the RxPPG-green signals have a minimal interfering contribution from skin motion. TxPPG signals were extracted from a RoI defined at the finger pad. wMOT signals were extracted (within the vicinity of the radial artery) at the skin locations that correspond to local hotspots in the PPG-amplitude map of the red channel. For selection of motion spots, thresholds in the range of 0.005-0.01 proved effective. For some subjects, skin motion sensors from neighboring clusters had opposite signs (an effect of skin depression/dilation in each cardiac cycle); when negative, the polarity of wrist motion signals was inverted. Secondly, all signals were preprocessed by amplitude conversion to AC/DC and ABPF, using the conservative range of the first 8 harmonics of the pulse-rate. Finally, ensemble averaging was performed in two steps: 1) Individual cycles were re-scaled to the median number of samples per cardiac cycle (at a sampling rate of 50 Hz); 2) the cycles were combined by using the median. The first and last 3 cycles were excluded from the ensemble-averaging procedure. Contact-PPG signals from pulse oximetry were used as a reference for peak detection and instantaneous pulse-rate estimation.
2.3. Experiment 2: finger compression
We conducted an additional experiment (hereafter referred to as experiment 2) where the finger pad was inspected in normal conditions and under externally-applied compression against a glass plate. The rationale of the experiment is that mild compression applied to the skin “optically clears” blood from the dermis/subdermis and partly or fully occludes dermal arterial vessels. In response to increased vascular resistance and an immediate reduction of capillary blood from the skin, vasodilation (by mediation of nitric oxide) is induced locally to protect the skin tissue [24, 25]. This intervention complements experiment 1 by showing causality between conducting properties of the skin layers and the morphology of PPG signals. The measurements for experiment 2 included 5 subjects (ages, 20–35 yrs; 4 male). Subjects with painted skin or nails were excluded.
2.3.1. Data acquisition
Figure 5 illustrates the experimental setup for data acquisition, which reuses the RGB camera (with bandpass filter) and two FLs (bilaterally placed around the finger) from experiment 1. Both the FLs and camera were covered by polarizing film (cross-polarizing scheme).
Fig. 5 Setup and rationale for the finger compression experiment. Acronym: APD, average penetration depth.
We selected the finger pad as a convenient region of interest. Finger recordings were made, first, without contact with a surface and then under mild applied compression. PPG waveforms were simultaneously acquired in the green and red wavelengths. While green mostly samples arteriole endings and capillaries, the diffusely reflected red light reaches deeper and larger vessels [15]; having these contrasting light penetration depths allowed us to investigate, non-obtrusively, waveform dissimilarity for the baseline and compressed states. Recordings were performed consecutively for two minutes per each stage at a sampling rate of 30 Hz.
2.3.2. Signal processing
Figure 6 illustrates the signal processing pipelines required for ensemble averaging. The out-coming cycles were used to study the effect of compression and to derive the transfer function between non-compression and compression conditions.
Fig. 6 Processing pipelines for ensemble-averaging finger RxPPG signals, at the red and green camera channels.
RxPPG signals were acquired from the finger pad. The same user-defined skin RoI was used to extract RxPPG-red and RxPPG-green signals at the red and green camera channels, respectively. Apart from the fact that the reference for pulse-rate estimation and for peak detection was extracted from a contact-PPG signal (acquired by using a finger oximeter attached to the right middle finger), the processing and ensemble-averaging procedure was similar to experiment 1.
3. Results
3.1. Experiment 1
For compactness, results are shown for subjects whose PPG-waveforms are representative of two identified types reported in the literature [26]. Subject A has no clear dicrotic notch (reflected wave) in the PPG-waveforms while subject B does (see Fig. 8). Figure 7 depicts the spectral density of fragments of RxPPG, TxPPG and the wMOT signals (normalized AC/DC; duration, 20 seconds). Alongside the input AC/DC signals are their denoised versions by ABPF around the first 8 harmonics of the fundamental of the pulse-rate. This is a conservative harmonic range since the cardiac-related peaks are mostly confined to the first 3–4 harmonics of the pulse-rate frequency.
Fig. 8 Sample wrist motion (wMOT) and rPPG ensemble-averaged waveforms acquired at RoIs of the wrist, palm and finger (abbreviations: TxPPG-r, transmission-mode PPG signal in the red channel; RxPPG-g, reflection-mode PPG signal in the green channel). The amplitude maps at the red channel were used to extract the wMOT signal.
For each subject in Fig. 8, ROIs were demarcated at the finger, palm and wrist for the extraction of TxPPG, RxPPG-green and wMOT, respectively. When the ensemble-averaged cycles from these signals are plotted jointly, morphological heterogeneity is clearly seen. With respect to wMOT, TxPPG seems to be less distorted than RxPPG-green (see Fig. 8(a) and Fig. 8(b)). Similar observations hold for the remaining two test subjects recruited for experiment 1.
The above insights were confirmed by the frequency responses of the skin tissue (modeled as transfer functions; TF), from wMOT to RxPPG-green at the hand palm (TF1) and wMOT to TxPPG (TF2). These are plotted in Fig. 9(a) and Fig. 9(b) for subjects A and B, respectively. We considered up to the first 4–6 harmonics of the pulse-rate for both subjects as higher order harmonics were increasingly more compromised by noise and provided marginal added value for the analysis. It was observed that TF1 and TF2 have a low-pass characteristics harmonics of the pulse-signal, which is more pronounce for TF2. Stated another way, the frequency content drops progressively as the pressure waveform propagates from the radial artery (2–3 mm diameter) to the upper dermis. The phase response further indicates that the signals are also delayed from wMOT to RxPPG-green and from wMOT to TxPPG.
Fig. 9 Transfer functions (TF) from wrist motion (wMOT) to RxPPG (green) and TxPPG (black) at two different sites evidence lowpass characteristics and delay, which are more pronounced for RxPPG. Abbreviations: exp., experimental; mod., modeled response.
3.2. Experiment 2: compressing the skin against a glass plate
We evaluated four subjects (ages, 25 to 35 years old; one female, two males) for whom skin motion at the wrist was visible under lateral lighting conditions. Data is shown for two subjects whose PPG-waveforms are representative of the main types reported in the literature: subject A has no clear dicrotic notch (reflected wave) in the PPG-waveforms while subject B does (see Fig. 8(a) and Fig. 8(b)).
Subjects underwent two consecutive recordings at the finger pad. Initially the skin had no contact to the glass plate. The second recording was performed under contact against a glass plate. The applied pressure at the finger was estimated to be within 100–200 g per 0.5 – 1 cm × 1 cm. This mild pressure was sufficient to cause blanching to the skin (partial obstruction to blood flow). During the compression stage, three subjects reported feeling pulsations in the finger and, in 2 subjects, the PPG signal was clearly visible by the naked eye. Figure 10 shows the ensemble-averaged waveforms for subjects A and B. To ease comparison, data is presented both as AC/DC and scaled to 0–1. For each plot, the signals from the red and green camera channels are plotted jointly.
Fig. 10 Ensemble averaged remote-PPG waveforms before and during finger compression, for 2 test subjects in the red and green camera channels.
The most obvious change induced by skin compression in an increase in the strength of signals. For both subjects, the amplitude gain exceeded a twofold increase for the red channel and a fourfold increase in the green camera channel. Compression changes are also visible as morphological changes in the remote PPG signals. Both in red and green, we notice that the morphology of PPG signals change into a dilatory, parabolic-like profile, with an increased crest time (i.e., the duration between the foot and edge of the PPG waveform increases). Concomitantly, the removal of venous blood from the upper dermis (observed by increased average brightness of the probed skin w.r.t. baseline values) and partial of full occasion of capillary units allows the green light to penetrate deeper into the skin. Upon application of pressure to the skin, it appears that the shape of the green-rPPG resembles more the shape of the red-rPPG waveform. To confirm this observation, we may inspect of the TFs between the PPG signals in the red and green camera channels (input and output, respectively), which are provided in Fig. 11 for 2 subjects, in the range of 0 to 5 Hz. It is evidenced that the skin has a clearly more pronounced low pass filter characteristics in the baseline condition than in the compression condition. Under compression, the attenuation is within about 90% of the fundamental of the pulse rate in the 0–5 Hz range and the introduced phase delay is halved for subjects A and B. These results, therefore, indicate that the microvasculature of the skin becomes transparent when venous blood and small caliber arterioles and capillaries are “cleared” from the optical path of the scattered red and green light.
Fig. 11 Illustration of the frequency characteristics of the skin in the finger pad, before and during finger compression [test subjects A and B]. The input and output signals are the red and green camera channels, respectively.
4. Discussion
Experiment 1
Experiment 1 illustrated waveform deformation across the terminal arterial path (from the radial artery to the upper dermis). We observed gradual delay and lowpass filter characteristics in the TFs between wMOT to TxPPG-red and from wMOT to RxPPG-green. Such observations are consistent with earlier expectations that the microvascular bed of tissue introduces delay and distortion.
Waveform deformation is a confounding effect in propagation-related phase measurements, particularly those obtained by inspecting selected PPG sensors from phase-maps. However, point-to-point algorithms meant for estimating peripheral PWV are also affected by this error source. To give a better impression of the problem, we scaled the waveforms to unity and attempted to retrieve transit time as the temporal lag for which proximal and distal waveforms are at half of its relative height. Based on Fig. 8, it clear that PWV measurements are unreliable in reflection-mode at the green channel, possibly because of the lower SNR of signals w.r.t. wMOT and TxPPG and because shape dispersal is higher between spaced locations. Estimating PWV based on the TxPPG at finger and green-rPPG at the palm would return invalid results if either a point-to-point approach or if the inner-product was used.
When PWV measurements are performed between less deformed waveforms, the validity of results improve. At the wrist (wMOT) and finger (TxPPG), the temporal delay in the order of 1/50 Hz. For a wrist-to-finger spacing of about 17 – 20 cm and heart rate within 50 – 70 bmp, this transit time measurement is consistent with aortic PWV measurements (range 4 to 12 m/s [27]), although minor changes in parameters (e.g., subtle changes in the RoI selection for wMOT and TxPPG signal extraction) affects measurements to different values. For subjects A, B, C and D, peripheral PWV values were grossly estimated as 10–11 m/s, 8–9 m/s, 9–10 m/s and 4–5 m/s, respectively. Typical values for healthy young individuals would be within 6–8m/s [27]; the found discrepancy is likely due to a) added propagation delay of TxRed w.r.t. wMOT; and b) the fact that PWV increases from the aortic vessels to periphery [28].
Our results of waveform deformation and consistent with earlier observations of Bernjak et al. [29], which showed deformation (shape and delay) between tonometric signals from the radial artery and laser Doppler flowmetry signals from the fingertip within the same limb. Future work is warranted to assess whether a combination of remote-PPG and skin-motion measurements (as an alternative to laser Doppler flowmetry and pressure) enables simultaneous characterization of the states of the macro and microvasculature.
Experiment 2
In experiment 2, external pressure was applied to the skin to influence the elastic properties of the vessels. Locally increasing the dilation of the finger pad vessels resulted in increased propagation-related delay, crest time, waveform similarity and signal strength in red and green wavelengths. As in the previous experiment, these insights were derived from qualitative observation on the ensemble-averaged waveforms in red and green and also by inspecting the system response of the skin (given RxPPG-red and RxPPG-green as inputs and outputs, respectively). Overall, the finger compression intervention support the hypothesis that regional variations in arterial stiffness introduce spatial variability in PPG-waveforms and dispersal in phase images, particularly when similarity metrics are used.
There are a number of studies of the effect of compression in PPG signals. For example, Bergstrand et al. [24] evaluated a multi-modality system combining laser Doppler flowmetry, infrared-PPG, and green-rPPG in a single probe for the simultaneous measurement of blood flow at different depths in the tissue. When pressure was applied to the sacral skin region, the signal strength of green-rPPG had a modest but significant increase of about 40% for 50.0 mmHg, whereas the IR sensor registered a twofold increase, also for 50.0 mmHg. Within the setting of remote PPG-imaging, Kamshilin et al. also obtained results of increased PPG signal strength in green lighting when gentle compression (about 0.3 N per cm 2) was applied to the palm of the hand by contact with a glass plate [30].
Unfortunately, the above studies do not target, specifically, morphological changes upon applied pressure to the skin. Still, Teng et al. [31] partly overcame this gap in the literature by reporting on how pressure affects the transit time between a proximal reference and the PPG signal at a distal site. Accordingly, PPG signals were collected by using a finger oximeter in infrared lighting conditions [31–33]. The experimental results and theoretical model further support increases in signal strength when pressure is applied to the finger. Also, it is verified that compression slightly increases PTT between a proximal reference and the foot of the PPG signals. Although our current study does not aim at transit time, we could confirm increased delay in the front wave of the PPG waveforms, both in red and in the green wavelengths, when pressure was applied to the finger pad. To the best of our knowledge, compressing the skin has not been reported in the literature as a means of showing that the penetration depth of green approaches red when venous blood is removed from the bed tissue and the microvasculature is partly to fully occluded.
Implications
Light interaction with the skin
Our model preconizes that the vessel elasticity (compliance) at the microvasculature is depth-dependent and is compatible with the consensus that PPG signals result from blood volume variations [1]. We support that volume variations have origin in the dilation/collapse dynamics (i.e., filling and emptying) of the microvasculature and that the modulation of the rPPG-signal (amplitude and morphology) is depth-dependent. Thus, we diverge from the recently proposed model of Kamshilin et al. [30], where the emphasis is on the elasticity of the connective tissue; i.e., pulse oscillations of arterial transmural pressure would mechanically deform the connective-tissue components of the dermis, resulting in periodical changes in the optical properties of the skin.
Pulse rate and PPG-amplitude
In light of our findings on waveform heterogeneity for different depths, a pitfall is identified in existing remote-PPG algorithms, which is the achievement of motion robustness by linear combination of camera channels with the grounding assumption of signal redundancy over color/camera channels. Fortunately, in visible light applications (e.g., using an RGB camera with overlapping optical filters [8]) the PPG signal is dominated by contributions from wavelengths around 550 nm. Thus, the approximation error is small / acceptable for pulse rate detection and PPG-amplitude estimation. This strategy has been proven effective in large motion scenarios [34] and also against motion artifacts in PPG-imaging [5]. The approximation is also valid for applications using red-to-infrared wavelengths as the penetration depths are most similar [16, 35].
Phase measurements
However, caution is advised when it comes to applications with stricter requirements for phase accuracy, e.g. when measuring the arterial travelling time of the cardiac pressure wave propagating from heart to peripheral sites [28, 36]. Such PPG point-to-point measurements with a camera in the visible spectrum, just as the emerging PPG-imaging methods, often rely on the complex inner-product for measuring phase differences between remote PPG-sensors [37, 38]. Consequently, waveform heterogeneity is an important source of error. Within a single body site, significant PPG-phase shifts between RxPPG sensors may occur due to skin inhomogeneity as we have shown, but they should not be confused with arterial pulse wave travelling times, which are negligible between neighboring skin-sites. We further note that the relative time lag between red-IR PPG signals is not necessarily an independent indicator of health, as hypothesized by Vahdani-Manaf and Kayıkçıoğlu [12]. Their data just shows that blood absorption changes with breath-holding: for lower SpO2 values, the blood absorption of red and IR becomes similar. Hence, light penetration becomes comparable and the relative delay reduces (as supported from our model). We can expect that a change in skin temperature may affect superficial skin perfusion and the relative delays as a consequence.
Towards clinical applications of PPG-imaging
Mapping the skin microvasculature at different depths has been pursued by a decade [13, 39, 40]. This publication contributes to this goal by providing observations of waveform heterogeneity in the PPG signals at two contrasting wavelengths, as well as a supporting physiology-based skin model. Future work could be directed towards exploring the potential of PPG maps as features for characterizing the depth-varying perfusion state of the microvasculature.
Further exploration of remote SpO2 is also worthy of future investigation. Particularly, we expect that calibration using visible light, as attempted by Guazzi et al. [41], is likely challenged by the mentioned skin non-homogeneity affecting green and blue PPG-strength relative to red. Thus, we are concerned that calibration of SpO2 measurements is likely possible, but only for deeply penetrating light (λ > 620 nm) [42]. Lastly, we highlight the potential of PPG-imaging as an intermediate processing step in algorithms aiming at pulse wave analysis or even PVW estimation. Yet, PPG-imaging remains to be carefully interpreted due to the still limited understanding of the effects of skin inhomogeneity in rPPG-images, stress/pressure, temperature and other confounding factors [43].
Limitations
Experiment 1 provided valuable information about waveform heterogeneity across the terminal branches of the arterial path and also on the root cause of phase variations in PPG phase-maps. Yet, we acknowledge a number of methodological limitations: a) despite stabilization of the hand, the red and green channels may contain remnants of skin motion artifacts; b) the signal-to-noise ratio in the red camera channel at the overall skin and the green at the fingers was low due to reduced brightness. We were able to assume that RxPPG at the palm is much stronger that the local BCG-artifacts and the sensor noise level, and still use the green channel as the basis for RxPPG-green signal extraction and processing. However, the same considerations for RxPPG-red are not possible. Consequently, we could only contrast the morphology of ensemble waveforms of RxPPG in green and red in experiment 2. The results from both experiments are specific for the hand and position, including elevation w.r.t. the heart level; other settings may result in differences in the shapes of signals or as response to compression [44].
5. Conclusion
This investigation showed that light penetration depth affects the morphology of remote-PPG signals. Shallower penetrating green illumination results in higher distortion and phase delay w.r.t. to the pressure wave in large arteries, while sampling deeper (red illumination) into the skin minimizes this effect, though at the cost of lower signal quality. Regional variations in PPG maps were explained by recognizing the waveform shape variability is ignored if rPPG imagers rely on the complex inner product as a similarity metrics. PPG phase maps may be more suited for tissue characterization than for retrieving propagation-related measurements. Publications on imaging a single body site (which rely on the complex inner product) for retrieving PWV should be interpreted with caution.
References and links
1. J. Allen, “Photoplethysmography and its application in clinical physiological measurements,” Physiol. Meas. 28(3), 1–40 (2007). [CrossRef]
2. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express 16(26), 21434–21445 (2008). [CrossRef] [PubMed]
3. A. A. Kamshilin, S. Miridonov, V. Teplov, R. Saarenheimo, and E. Nippolainen, “Photoplethysmographic imaging of high spatial resolution,” Biomed. Opt. Express 2(4), 996–1006 (2011). [CrossRef] [PubMed]
4. J. Spigulis, “Biophotonic technologies for non-invasive assessment of skin condition and blood microcirculation,” Latvian Journal of Physics and Technical Sciences 49(5), 63 (2012).
5. A. Moço, S. Stuijk, and G. de Haan, “Motion robust PPG-imaging through color channel mapping,” Biomed. Opt. Express 7(5), 1737–1754 (2016). [CrossRef] [PubMed]
6. N. Zaproudina, V. Teplov, E. Nippolainen, J. A. Lipponen, A. A. Kamshilin, M. Narhi, P. A. Karjalainen, and R. Giniatullin, “Asynchronicity of facial blood perfusion in migraine,” PLoS One 8(12), 80189 (2013). [CrossRef]
7. A. Moço, S. Stuijk, and G. de Haan, “Ballistocardiographic artifacts in PPG imaging,” IEEE Trans. Biom. Eng. 63(9), 1804–1811 (2016). [CrossRef]
8. G. de Haan and A. van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature,” Physiol. Meas. 3(9), 1913–1926 (2014). [CrossRef]
9. J. Spigulis, L. Gailite, A. Lihachev, and R. Erts, “Simultaneous recording of skin blood pulsations at different vascular depths by multiwavelength photoplethysmography,” Appl. Optics 46, 1754–1759 (2007). [CrossRef]
10. J. Spigulis, L. Gailite, R. Erts, and A. Lihachev, “Contact probe pressure effects in skin multi-spectral photoplethysmography,” Proc. SPIE 6628, 66281F (2007). [CrossRef]
11. H. Ugnell and P. A. Oberg, “Time variable photoplethysmographic signal: its dependence on light wavelength and sample volume,” Proc. SPIE 2331, 23310 (1995).
12. N. Vahdani-Manaf and T. Kayıkçıoğlu, “Comparison of time difference variations of photoplethysmographic signals with different wavelengths,” in 23th Signal Processing and Communications Applications Conference (IEEE, 2015), pp. 1134–1137.
13. Z. Marcinkevics, U. Rubins, J. Zaharans, A. Miscuks, E. Urtane, and L. Ozolina-Moll, “Imaging photoplethysmography for clinical assessment of cutaneous microcirculation at two different depths,” J. Biomed. Optics 21(3), 035005 (2016). [CrossRef]
14. P. Agache, “Skin Structural Components: Physiology and Metrology,” in Measuring the skin, P. Agache and P. Humbert, eds. (Springer-Verlag, Berlin, 2014), pp. 17–399.
15. M. Huelsbusch, “An image-based functional method for opto-electronic detection of skin-perfusion,” Phd thesis, RWTH Aachen dept. of EE. (in German) (2008).
16. R. R. Anderson and J. A. Parrish, “The optics of human skin,” J. Invest. Dermatol. 77(1), 13–19 (1981). [CrossRef] [PubMed]
17. R. R. Anderson, J. Hu, and J. A. Parrish, “Optical radiation transfer in the human skin and applications in in vivo remittance spectroscopy” in Bioengineering and the Skin: Based on the Proceedings of the European Society for Dermatological Research Symposium, held at the Welsh National School of Medicine, Cardiff, July 1979, R. Marks and P. A. Payne, eds. (SpringerNetherlands, Dordrecht, 1981), pp. 253–265.
18. T. Lister, P. A. Wright, and P. H. Chappell, “Optical properties of human skin,” J. Biomed. Opt. 17(9), 090901 (2012). [CrossRef]
19. J. L. Reuss, “Multilayer modeling of reflectance pulse oximetry,” IEEE Trans. Biom. Eng. 52(2), 153–159 (2005). [CrossRef]
20. L. F. Corral, G. Paez, and M. Strojnik, “Optimal wavelength selection for non-contact reflection photoplethysmography,” Proc. SPIE 8011, 801191 (2011). [CrossRef]
21. U. Rubins, R. Erts, and V. Nikiforovs, “The blood perfusion mapping in the human skin by photoplethysmography imaging,” Proc. IFMBE 29, pp. 304–306 (2010). [CrossRef]
22. R. Amelard, C. Scharfenberger, F. Kazemzadeh, K. J. Pfisterer, B. S. Lin, D. A. Clausi, and A. Wong, “Feasibility of long-distance heart rate monitoring using transmittance photoplethysmographic imaging (PPGI),” Sci. Rep. 5, 14637 (2015). [CrossRef] [PubMed]
23. S. Roujol, M. Ries, B. Quesson, C. Moonen, and B. Denis de Senneville, “Real-time MR-thermometry and dosimetry for interventional guidance on abdominal organs,” Magn. Reson. Med. 63(4), 1080–1087 (2010). [CrossRef] [PubMed]
24. S. Bergstrand, L. G. Lindberg, A. C. Ek, M. Linden, and M. Lindgren, “Blood flow measurements at different depths using photoplethysmography and laser Doppler techniques,” Skin. Res. Technol. 15(2), 139–147 (2009). [CrossRef] [PubMed]
25. H. Tsukahara, H. Ende, H. I. Magazine, W. F. Bahou, and M. S. Goligorsky, “Molecular and functional characterization of the non-isopeptide-selective ETB receptor in endothelial cells. Receptor coupling to nitric oxide synthase,” J. Biol. Chem. 269(34), 21778–21785 (1994). [PubMed]
26. S. C. Millasseau, J. M. Ritter, K. Takazawa, and P. J. Chowienczyk, “Contour analysis of the photoplethysmographic pulse measured at the finger,” J. Hypertens. 24(8), 1449–1456 (2006). [CrossRef] [PubMed]
27. K. Groenewegen, H. den Ruijter, G. Pasterkamp, J. Polak, M. Bots, and S. A. Peters, “Vascular age to determine cardiovascular disease risk: A systematic review of its concepts, definitions, and clinical applications,” Eur. J. Prev. Cardiol. 23(3), 264–274 (2016). [CrossRef]
28. R. Mukkamala, J. Hahn, O. T. Inan, L. K. Mestha, C. Kim, H. Töreyin, and S. Kyal, “Towards Ubiquitous Blood Pressure Monitoring via Pulse Transit Time: Theory and Practice,” IEEE Trans. Biom. Eng. 62(8), 1879–1901 (2015). [CrossRef]
29. A. Bernjak and A. Stefanovska, “Pulse transit times to the capillary bed evaluated by laser Doppler flowmetry,” Physiol. Meas. 30(3), 245–260 (2009). [CrossRef] [PubMed]
30. A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015). [CrossRef] [PubMed]
31. X. F. Teng and Y. T. Zhang, “The effect of contacting force on photoplethysmographic signals,” Physiol. Meas. 25(5), 1323 (2004). [CrossRef] [PubMed]
32. X. F. Teng and Y. T. Zhang, “The effect of applied sensor contact force on pulse transit time,” Physiol. Meas. 27(8), 675–684 (2006). [CrossRef] [PubMed]
33. X. F. Teng and Y. T. Zhang, “Theoretical study on the effect of sensor contact force on pulse transit time,” IEEE Trans. Biomed. Eng. 54(8), 1490–1498 (2007). [CrossRef] [PubMed]
34. W. Wang, S. Stuijk, and G. de Haan, “Exploiting Spatial Redundancy of Image Sensor for Motion Robust rPPG,” IEEE Trans. Biomed. Eng. 62(2), 415–425 (2015). [CrossRef]
35. M. van Gastel, S. Stuijk, and G. de Haan, “Motion robust remote-PPG in infrared,” IEEE Trans. Biom. Eng. 62(5), 1425–1433 (2015). [CrossRef]
36. The Reference Values for Arterial Stiffness Collaboration, “Determinants of pulse wave velocity in healthy people and in the presence of cardiovascular risk factors: establishing normal and reference values,” Eur. Heart. J. 31(19), 2338–2350 (2010). [CrossRef] [PubMed]
37. K. Nakano, Y. Aoki, R. Satoh, A. Hoshi, H. Suzuki, and I. Nishidate, “Non-contact measurement of pulse wave velocity using RGB cameras,” Proc. SPIE 715, 971515 (2015).
38. H. Lee, E. Seo, J. Yang, S. Park, and K. Hong, “Video-based Bio-Signal Measurements for a Mobile Healthcare System,” pp. 1–5 (ACM, 2016).
39. F. P. Wieringa, F. Mastik, and A. F. W. van der Steen, “Contactless Multiple Wavelength Photoplethysmographic Imaging: A First Step Toward “SpO2 Camera” Technology,” Annals Biom. Eng. 33(8), 1034–1041 (2005). [CrossRef]
40. D. Iakovlev, V. Dwyer, S. Hu, and V. Silberschmidt, “Noncontact blood perfusion mapping in clinical applications,” Proc. SPIE 887, 988712 (2016).
41. A. R. Guazzi, M. Villarroel, J. Jorge, J. Daly, M. C. Frise, P. A. Robbins, and L. Tarassenko, “Non-contact measurement of oxygen saturation with an RGB camera,” Biomed. Opt. Express 6(9), 3320–3338 (2015). [CrossRef] [PubMed]
42. W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of Contactless Pulse Oximetry,” Anesth. Analg. (posted 31 May 2016, in press). [CrossRef] [PubMed]
43. J. Allen and K. Howell, “Microvascular imaging: techniques and opportunities for clinical physiological measurements,” Physiol. Meas. 35(7), 91–141 (2014). [CrossRef]
44. M. Hickey, J. P. Phillips, and P. A. Kyriacou, “The effect of vascular changes on the photoplethysmographic signal at different hand elevations,” Physiol. Meas. 36(3), 425–440 (2015). [CrossRef] [PubMed]
