Open Access
Add to BibSonomy
Abstract
The remote acquisition of photoplethysmographic (PPG) signals via a video camera, also known as photoplethysmography imaging (PPGI), is not yet standardized. In general, PPGI is investigated with test persons in a laboratory setting. While these in-vivo tests have the advantage of generating real-life data, they suffer from the lack of repeatability and are comparatively effort-intensive because human subjects are required. Consequently, studying changes in signal morphology, for example, due to aging or pathological effects, is practically impossible. As a tool to study these effects, a hardware PPG simulator has been developed: this is a phantom which simulates and generates both 1D and locally resolved 2D optical PPG signals. Here, we demonstrate that it is possible to generate PPG-like signals with various signal morphologies by means of a purely optoelectronic setup, namely an LED array, and to analyze them by means of PPGI. Signals extracted via a camera show good agreement with simulated generated signals. In fact, the first phantom design is suitable to demonstrate this qualitatively.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Photoplethysmography (PPG) is a non-invasive method for optical measurement of changes in tissue blood volume. The basic setup consists of a light source irradiating the tissue under examination, and a detector registering changes in light intensity due to light-tissue interaction. Changes in intensity are modulated by the blood pumped from the heart to the periphery, but are also dependent on other physiological characteristics, as well as the measurement equipment used. Because PPG allows non-invasive monitoring of hemodynamics via the skin, and that at relative low cost, it is widely used, not only in clinical applications, but also in fitness trackers. Details of the PPG method and its clinical applications are published in e.g. [1].
In the 1990s, researchers began to investigate whether the method could be used with a different kind of sensor, namely video cameras. Measurements taken with cameras do not require contact with the skin and are, therefore, remote. Moreover, by using a camera, multiple spots can be measured simultaneously because groups of pixels can be viewed as virtual sensors. Thus, it is also possible to conduct simultaneous measurements on multiple parts of the body with a single detector. In addition, a camera can capture spatially resolved representations of hemodynamics. The photoplethysmography imaging (PPGI) technique was inspired by transillumination imaging and was first applied by Such [2] in 1996 to study venous hemodynamics. Further development led to acquisition systems for arterial signals, as described, for example, by Wu et al. [3]. The development from conventional PPG to a contactless method was reviewed in more detail by Sun et al. [4] and Zaunseder [5], who also emphasized the challenges related to the method.
In accordance with Sun et al., we agree that the employed systems lack a standardized design mature enough to be effectively used in a clinical setting. While PPG is a widely used method and a clinical standard, PPGI is still in an early developmental stage, where specialized hardware (e.g. a customized light source) has to be designed, or models have to be invented which explain the processes of signal formation. Therefore, this work aims at studying the imaging performance of video cameras with respect to dynamic PPG signals. For this purpose, we present the modeling and the generation of signals by means of an optoelectronic tissue phantom.
A model is generally used to describe the function of a system in such a simplified way that the real system is sufficiently approximated for the application. The successful simplification of complex relationships improves the understanding of systems and is beneficial for the design of a device. Especially for medical questions, the effort to generate data is time-consuming and requires many resources; for example, test subjects must be acquired and examined in appropriate studies. Here, models that can be used to generate reproducible data are a good alternative to the use of test persons and patients.
In fact, several ways exist to synthesize physiological signals such as the PPG signal: on the one hand, there are models which mimic the physiology, e.g. parts of the cardiovascular system (CVS), thus allowing to generate signal changes according to alterations in, e.g. cardiac output. On the other hand, physiological signals can be approximated by mixing different signals. Examples of the latter are the synthesis of the signal by using three different sinusoidal source signals [6], the combinations of two or more Gaussian pulses [7–9], the use of generative models involving learning methods [10], or dynamic systems [11].
Besides simulations, there are various approaches which also use phantoms. These artificial objects are created in a way that they allow to study factors like PPG configurations or light-tissue interactions in a controlled environment, not only in silico but with actual equipment [12–16].
In 2006, Wieringa [12] described an invention of a phantom which allows to mimic anatomical structures and optical characteristics of biological tissues. The system is described to not only use static components but also dynamic ones: one core component are liquid crystal devices (LCDs) which are electronically switchable. In particular, the phantom is meant to assess imaging performance as well as to calibrate devices for oxygen saturation measurements. Amongst others, liquid dyes were suggested to be mixed with the liquid crystals. Further, it was thought about realizing 3D structures by stacking multiple layers. In addition, another dynamic phantom was presented consisting of three main components: artificial skin (polyoxymethylene), artificial PPG generator and blood reservoir. Then, the artificial signals were generated as simple block waves toggling an LCD shutter between on and off states.
In 2011, Nishidate et al. [13] conducted experiments using tissue-like agar gel phantoms to simulate the transition from oxygenated to deoxygenated blood. For this reason, the phantoms two layers represent epidermis and dermis. Scattering was realized by an intralipid stock solution. Further, melanin was simulated with a coffee solution. Moreover, horse blood was added to the layers. In the end, the transition process (120 min) was observed with a camera.
In that same year, Wijshoff et al. [15] investigated the effect of sensor deformation and resulting change of emitter-detector distance in classical PPG. For this, they built a flowcell that models skin perfusion. Specifically, pulsatile blood flow was modeled using milk pumped by a roller pump and optical characteristics of the skin were mimicked by a plastic (polyoxymethylene) window. In addition, the influence of sensor displacement was also investigated, albeit using only the polyoxymethylene skin as a phantom. Important to realize, the artificial skin has been moved by a loudspeaker using sinusoidal excitation[16].
In 2013, Myllylä et al. [14] fabricated a multilayered optical phantom designed to mimic the optical properties of tissue layers of the forehead. In this case, the phantom was used to study pulsations coming from a grey matter-like layer using near-infrared spectroscopy. With this in mind, a large vessel was modeled by a silicone tube (4 mm) filled with a liquid and air bubbles. Then, movement was induced by pulsating the liquid.
Similar to the approach which was proposed by Wieringa, we aimed to implement a phantom which can mimic the dynamic characteristics of blood volume in the skin. We deliberately did not use mechanical components or fluids for the phantom to simplify handling. Instead, we aimed to synthesize a PPG-like waveform and to control the shape of the signal by means of an optoelectronic device. Thus, the synthesized signal can be measured directly via PPGI and, equally important, no conversion from fluid volume shifts to light intensity changes is necessary. Moreover, the signal simulation is based on a Windkessel (German for air chamber) model (WKM), which simplifies parts of the CVS. The signal is then passed to an emitter, a light-emitting diode (LED) array, where the light signal is sent to the camera. For one thing, using an array allows to not only generate a single signal, but also spatially distributed signals, for example, to simulate the effects of different tissue properties or wave propagation along the vessel network.
The remainder of this paper is structured as follows: first the measurement principle of PPG/PPGI is reviewed to develop the engineering model of the phantom. Then, based on the engineering model, realization of the hardware is discussed. Furthermore, simulating suitable signals according to the hardware realization is addressed before the results of PPGI measurements with the phantom are shown and discussed in the results. Finally, we conclude by summarizing the most important points and discussing future prospects.
2. Measurement principle
In this section, we discuss the measurement principle of in-vivo measurements and which aspects are mimicked in measurement using our phantom.
2.1 In-vivo measurement
In a PPGI measurement setup, light is directed into the tissue of interest and the reflected or transmitted light is measured by a video camera, as depicted in Fig. 1(a). The intensity signal received at the camera is modulated by (amongst others) the amount of blood in the vascularity, which is influenced by cardiac activity. Consequently, more light is absorbed during systole than in diastole. A quasi constant amount of light is absorbed by the remaining tissue. In addition, the signal shows differences depending on the anatomical location where it is measured: due to different properties of the prevalent vessels (along the circulatory system, as well as the local structure of the skin) signal morphology is changed.
Fig. 1. (a) PPGI measurement principle: a camera measures the amount of light which is reflected and backscattered by the tissue; (b) The engineering model used to mimic the signal generation: modulated light is projected on a screen and then detected by a camera. The switchable light sources model the effect modulating the photon flux in the process of light-tissue interaction in an in-vivo measurement.
2.2 Phantom measurement
In a typical PPG measurement, light which reaches the detector has entered the tissue, interacted with it mainly by scattering and left the skin. In the process, the photon flux is modulated by the dynamic tissue components, creating a PPG signal at the detector. In our phantom, there is no external light source. As opposed to the in-vivo measurement, the light source is situated within the tissue. Thus, the modulation of light can be modeled by electronically switchable light sources. Furthermore, the top layer of the artificial tissue is represented by a passive component which can be probed by a detector. The engineering principle of said phantom is depicted in Fig. 1(b): the phantom mimics the behavior of perfused skin on the signal level by modulating light sources below a projection screen (we use a diffuser). Although this is a simplified model, it is sufficient to test the capabilities of the measurement equipment. The top layer of skin acts as the source of modulated light. Here, the amount of light is modulated over time to simulate the effects of systole and diastole. The artificial signals are created with light to be received directly by typical detectors. Furthermore, no external light source, or mechanical components or fluid compositions are needed. As a result, the overall system is kept compact and simple. Changes in signal morphology which would correspond to different anatomical locations are taken into account by changing the waveform of the synthesized signal. The phantom allows to generate spatially distributed signals using multiple light sources which can be controlled individually; this is in order to consider the properties of cameras as detectors for spatio-temporal signals.
2.3 PPG waveform
A typical PPG waveform or, more precisely, one pulse is provided in Fig. 2. The signal consists of two components: the pulsatile AC component shown here (which reflects the arterial blood volume variations), and the slow DC component (which mainly represents tissue and the influence of venous blood). Besides the systolic peak, signals originating from healthy, compliant vessels, show more characteristic features, such as a diastolic peak and a dicrotic notch. It is important to realize that the wave is often inverted (as shown here) before it is presented to the physician to better correspond with other vital signs, in this case, pressure waveforms.
Fig. 2. Typical PPG waveform of a healthy, compliant vessel network: morphologic features including systolic and diastolic peak, as well as dicrotic notch, are present. Additional features can be derived using these parameters, e.g. area under the curve, ratio between amplitudes, pulse width, etc.
Waveform analysis can be used to extract additional features from the waveform. Here, the focus is on the qualitative reproduction of the waveform with systolic peak, diastolic peak and dicrotic notch present. Further, the time between one pulses at different tissue locations is simulated; this transit time is related to the pulse wave velocity (PWV). This parameter is attributed to blood pressure and vessel elasticity and can be measured, especially, by applying two PPG sensors at known distance [17]. In like manner, with a camera, the signals can be extracted at arbitrary locations, making PPGI interesting for such approaches. This further motivates the use of an array of independent light sources for the phantom and the option to generate different signals at the same time, for instance, the same signal with a phase delay.
3. Hardware realization
Schematically, the system presented in this paper consists of three main blocks: wave generator, controller and emitter unit. These components are depicted in Fig. 3(a). For simulating PPGI measurements with the phantom, the setup sketched in Fig. 3(b) was used; this consists of the simulator, the camera, and a computer for video acquisition and evaluation (not shown). The following sections describe the individual components and the experimental setup.
Fig. 3. Components of the hardware simulator: (a) BeagleBone Black; FPGA and LED matrix; (b) Sketch of the measurement setup showing the LED matrix with attached screen (here a diffusor) at distance $d_{2}= {30} \,\textrm{mm}$ and a camera on top as sensor at distance $d_1= {280} \,\textrm{mm}$.
3.1 Wave generator
The wave generator creates a PPG-like waveform which is either produced in real-time or is a recording. The chosen hardware is the BeagleBone Black (BBB) (BeagleBoard.org Foundation, USA), a low-cost development platform which was programmed as a real-time target via Matlab/Simulink (The MathWorks, Inc., USA). Then, the wave generator passes the signals to the controller from where they are forwarded to the LED matrix which emits the signals to be picked up by a detector. Because the BBB is not powerful enough to simultaneously generate signals and address the phantoms emitter components individually, an additional controller was employed for this task.
3.2 Controller
For the controller, we chose a Field Programmable Gate Array (FPGA): the Spartan-3E (XC3S16000E-4FG320) (Xilinx Inc., USA) which has 104 digital I/O pins and a clock oscillating at 125 MHz. In order to use the ports, we combined the FPGA with the prototyping carrier board TE0303-01 (Trenz Electronic GmbH, Germany) and programmed it via Xilinx ISE (V. 14.7) in Very High Speed Integrated Circuit Hardware Description Language (VHDL).
3.3 Emitter unit
The light emitting unit consists of three boards stacked on top of each other: the interface board, which connects to the controller, the LED driver board and the LED matrix board. The matrix itself is divided into two logical parts with three columns each.
The final LED configuration was chosen to maximize packaging density given the physical constraints of our printed circuit board and LED package. However, we also considered to use the equations given by Moreno et al. [18]: a method was presented to compute the LED-to-LED spacing for several LED configurations so that the irradiance pattern would be maximally flat. Thus, a configuration based on the presented conceptual framework would have resulted in an LED configuration which emits a uniform irradiance pattern on a target screen. Unfortunately, for our design parameters and available hardware, the formulations would not have allowed a physical realization (LED-to-LED distance too big). Hence, a design which uses the conceptual framework of [18] is a topic for future work.
Finally, the self-built LED matrix board measures 40 mm by 70 mm and consist of 42 near-infrared (NIR)-LEDs which are arranged in a $7 \times 6$ configuration. The LEDs are spaced with the following distances between their respective centers: ${\Delta }x = {11} \,\textrm{mm}$ and ${\Delta }y = {6} \,\textrm{mm}$. The panel is equipped with VSMY3850 (Vishay Intertechnology, Inc., USA) surface-mounted device (SMD) LEDs. The package of the LED is the PLCC-2. The peak wavelength is rated at $\lambda _{\textrm {P}}= {850} \,\textrm{nm}$ and the angle of half intensity is $\phi =\pm {60}^{\circ}$.
The rationale for using NIR-LEDs for the simulator is that NIR is not visible to the human eye and, thus, this light source can be used 24/7 in a real measurement. Moreover, in our lab there is less noise from other light sources compared to the visible band.
3.4 Signal generation
The LEDs are controlled in such a way that a PPG waveform can be received by a detector. For this, the light needs to be modulated and the AC component of the PPG waveform needs to be sufficiently resolved.
Dimming strategy
The light modulation was achieved by pulse-width modulation (PWM). Thus, dimming is achieved by simply toggling the LED between the ‘on’ and ‘off’ state. In other words, the received light intensity is then determined by the duty-cycle $D$, the ratio between pulse duration $\tau$ and signal period $T$:
Here, the signal period $T$ is of fixed length, while $\tau$ is variable and, thus, the light intensity can be controlled by modulating $\tau$. Hence, the LED is operated at only two operating points where voltage and current are fixed.Signal discretization
Two different signal discretization strategies have been implemented:
- • the first strategy uses 25 of 100 levels to represent the AC component,
- • the second strategy two uses 8 of 500 levels to represent AC component.
The two discretization strategies for the AC component are illustrated in Fig. 4(a) and Fig. 4(b), respectively: when using only 8 levels, the information on the dicrotic notch is lost. This is due to the amplitude difference between the dicrotic notch and peak of the dicrotic wave: compared with the systolic peak, this difference is very small as it only equals about 6 % of the peak. In case of the depicted signal, 6 % of 25 levels equals 1.5 levels, while for 8 levels it equals 0.48 levels and, thus, cannot be resolved. The signal values are given in arbitrary units (AUs).
Fig. 4. (a) Discretized PPG waveform with 25 of 100 levels; and with (b) only 8 of 500 levels. Using less discretization levels for the AC component results in the loss of the dicrotic notch. Even though this loss in waveform morphology is not desired, the compromise made to allocate the discretization steps in this manner, allows us to simulate realistic AC/DC ratios like in in vivo measurements without modifying hardware. In this case, the problem is how to realize as many discretization steps as possible.
Experimental setup
For the measurements, a thin screen (Lee filter 216, a diffuser, by Panavision Inc., USA) was applied to the emitter at distance $d_2= {30} \,\textrm{mm}$ from the LEDs. To reduce the effect of unwanted light sources, the measurements were conducted in a room without ambient light. The only other light source was a computer monitor which was directed away from the measurement site. The receiver was a Grasshopper3 (GS3-U3-23S6M-C) 2.3 MP Mono USB 3 Vision camera (FLIR Systems, Inc., USA) equipped with a Fujinon CF12.5HA-1 (Fujifilm Holdings Corporation, Japan) lens. The lens aperture was fixed to f/22. On the receiver, stray light is reduced by using the BN850 narrow band-pass machine vision filter (Midwest Optical Systems, Inc., USA). With the signal morphology in mind, we set the camera to record measurements at 250 Hz and ${512} \times {512} \,\textrm{pixel}$, while for AC/DC measurements we reduced the framerate to 50 Hz.
4. PPG signal simulation
We used Matlab Simscape and Simulink to run PPG signal models on the BBB. For this, we implemented a framework which allows to create up to 14 independent signals which can be distributed over the LED matrix. Here, one can choose between constant wave, sinusoidal wave or PPG wave, extracted either from patient data or simulated signals coming from the model (described below).
Windkessel model
Cardio vascular systems can be modeled with electrical circuit analogs, where blood pressure can be represented by voltage $V$, blood flow by electrical current $I$, vascular resistance by electrical resistance $R$, vascular compliance by capacitance $C$, blood inertance by inductance $L$, and so on. The Windkessel model is a lumped-parameter model where the total peripheral resistance and all arterial compliances are combined in a parallel circuit of a resistance and capacitance. Thus, it is possible to generate approximations of arterial pressure pulses when using the model with an appropriate, pulsatile, input current.
Multi-compartment models
For certain questions, the WKM is too simple. In contrast to that, models using multiple compartments can be anatomically more accurate and allow virtual measurements at different anatomical locations: each compartment lumps parts of the CVS. At the same time, a higher model complexity requires more processing power from the signal generator.
Hence, we investigated and tested four models of different complexity and anatomical accurateness which are listed here in order of descending complexity to run on the BBB:
- • the model of Hassani et.al. [19], a 42-segment model which describes the CVS and also considers the pulmonary system;
- • a simplified version of the model of Chen et al. [20] which describes the vascular tree from head to extremities and which we simplified by considering only the left arm;
- • the model by Zahedi et al. [21] which consists of four sections and represents the vessels from the aorta to the capillaries and allows to simulate effects of vascular aging;
- • a modified Dual Windkessel Model (DWKM) which describes the PPG wave and its features with reasonable accuracy.
Dual Windkessel model
The model that is further investigated and which we used for signal simulation is given in Fig. 5. The components in the model (adapted from Parlikar [22]) represent the following parts of the CVS: the compliance $C_{\textrm {p}}$ of the larger and proximal vessels, the inertance $L$ of blood, the compliance $C_{\textrm {d}}$ and resistance $R_{\textrm {d}}$ of the distal vessels. For convenience, the hemodynamical parameters are represented by their electrical analogs. As given below, the model can be described by Eqs. (2) – (3) in the Laplace domain:
where $s$ is a frequency parameter. Combining these equations, the resulting transfer function is then given by Eq. (5):Fig. 5. Diagram of the DWKM used to simulate waveforms. A proximal and a distal compartment describe the whole arterial vessel network. Modified from Parlikar [22].
Table 1. Value ranges for parameters used in the DWKM.
Current source
First, we tested the input current with the cardiac output (CO) wave generated by a Simulink implementation of the model described by Heinke et al. [23]. Again, because of the real-time requirement, the model for the input current had to be replaced by a simpler one. Therefore, the CO was simulated for each heart cycle by a parabola-shaped wave as given by:
Results of the simulations with more detailed parameter combinations and corresponding PPGI measurements are presented below.
5. Results
The following sections present a selection of measurements performed with the phantom. We show the qualitative results of the measurements when varying model parameters in the DWKM, look at waves propagating along the matrix, and observe the perceivable AC/DC ratio for a typical physiological value.
If not stated otherwise, rectangular $9 \times {9} \,\textrm{pixel}$ regions of interest (ROIs) have been used to retrieve a signal. As a matter of fact, a waveform results from spatially averaging the pixels of a ROI when applied on a video sequence. The positions of the LED centers (and with them the positions of the ROIs) were automatically detected by thresholding and blob analysis prior to measurement.
For the measurements, different spatial pattern schemes were used: as examples, we chose the following schemes presented in Fig. 6. The schemes were configured as follows:
- • Scheme ${\textbf {I}}$ two sections A and B are left and right of each other;
- • Scheme ${\textbf {II}}$ four rectangular sections A to D form a rectangle;
- • Scheme ${\textbf {III}}$ A to G are row-wise sections;
- • Scheme ${\textbf {IV}}$ one section A with a single LED emitting the signals.
Fig. 6. Four different spatial test patterns (Schemes $\textbf {I -- IV}$) were used to test the imaging performance using the hardware simulator. The patterns are distributed over the $7 \times 6$ LED matrix.
The values used for the model parameters in the simulations are given in Table 2.
Table 2. Investigated parameters and used values for the signal generation.
5.1 Effects of changing model parameters
As mentioned above, 25 of 100 levels of intensity were enough to modulate the AC component with its wave features. Here, different parameter of the DWKM were changed to verify whether the simulated changes would also be visible in the generated waveforms. This was tested by generating different waveforms by varying the parameters $L$, $C_{\textrm {p}}$ and $C_{\textrm {d}}$.
Influence of the inductor L
The inductor $L$ was simulated for 21 H and 41 H simultaneously using Scheme ${\textbf {I}}$. The synthetized waveforms are depicted in Fig. 7(a) and the resulting measurements in Fig. 7(b). As an example, the effect of the parameter is observed along line 5: from column 1 to column 6, the transformation of the waveform is evident, but more prominent when comparing the extremities (two bottom graphs). While the dicrotic notch is evident in the wave measured in column 1, it is less pronounced in column 6. This is in accordance with the simulation where lower values of $L$ result in more oscillatory behavior after the systolic peak in favor of an accentuated dicrotic notch.
Fig. 7. (a) Simulation of the PPG time dynamics with different values of $L$ in distinct sections of Scheme ${\textbf {I}}$. Each section consists of $2 \times 4$ LEDs; (b) Light intensities measured through line 5, for the simulation with different values of $L$ in distinct sections affecting oscillatory behavior of the waves.
Influence of the capacitor $C_{\textrm {p}}$
The synthesized signals for $C_{\textrm {p}}$ using 870 µF and 670 µF are displayed in Fig. 8(a), the corresponding measurements can be found in Fig. 8(b). The effects of the changes in $C_{\textrm {p}}$ are shown for line 4: smaller values result in a steeper diastolic decay as can be seen, for example, with the graphs of column 2 and column 5 (in the middle). Then again, the difference in the systolic amplitude is not correctly reproduced for the column pairs. The difference between the troughs can be observed correctly, the minima of column 4 to column 6 lie beneath their corresponding columns (3 to 1).
Fig. 8. (a) Simulation of the PPG wave with different values of $C_{\textrm {p}}$ in distinct sections. Each section consists of $2 \times 4$ LEDs (Scheme ${\textbf {I}}$); (b) Light intensities measured through line 4, for the simulation with different values of $C_{\textrm {p}}$ in distinct sections . The decay in the diastolic phase (in the dicrotic wave) is affected.
Influence of the capacitor $C_{\textrm {d}}$
The simulated waves for different values of $C_{\textrm {d}}$ in the range 50 µF to 110 µF are shown in Fig. 9(a) and the corresponding measurements are shown in Fig. 9(b), respectively. The signals were emitted according to Scheme ${\textbf {II}}$.
Fig. 9. (a) Simulation of the PPG wave with different values of $C_{\textrm {d}}$ in distinct sections of Scheme ${\textbf {II}}$. Each section consists of four LEDs; (b) Light intensities measured through column 4, for the simulation with different values of $C_{\textrm {d}}$ in distinct sections affecting the dicrotic notch.
For the different values of $C_{\textrm {d}}$, in Fig. 9(b) it is possible to observe the differences of the waveforms along one line or one column of the array. The waves of column 4 are represented as examples where, along that column, the value of the capacitor increases. This translates to a more evident dicrotic notch in accordance with the simulation. Whereas for lines 1, 2 and 3 the dicrotic notch is soft, its evidence gradually increases along the lines, being most pronounced in lines 6 and 7, where there is some influence of the simulated signal corresponding to $C_{\textrm {d}}= {90}$ µF and more of the signal where $C_{\textrm {d}}= {110}$ µF. The difference between the signals of line 2 and line 7 (third and second signal from the bottom) is easy to discern, because the dicrotic notch is evident in line 7.
5.2 Signal propagation
To simulate the signal propagation in space, the whole matrix was used to output the same signal with time delays. Therefore, Scheme $\textbf {III}$ was applied, where LEDs of the same line have the same phase, while along the column, a delay of 15 ms was sent to the LEDs. The scheme mimics the propagation of the blood volume pulse along a body segment, for example, from heart to finger.
The simulated signals are shown in Fig. 10(a). As an example, the measured waves for column 3 are shown in Fig. 10(b). We can identify the presence of a delay by visually checking the graphs for line 1 and 7 (i.e. lowest graphs). Nonetheless, this detected delay is only approximately half the simulated delay.
Fig. 10. (a) Simulation of the PPG wave with different delays for each line. A $\Delta t$ of 15 ms per line was added; (b) Measurement results are given for the third column, showing the evidence of a delay which is smaller than the simulated one.
5.3 AC/DC ratio
Testing AC/DC ratios is performed to evaluate the limits of the measurement camera at typical settings for current cameras. Hence, the sampling rate of the camera is reduced to 50 Hz and videos were recorded with 8 bit pixel depth. For this test, the projection screen was not applied because spatial distribution was not to be studied. A single LED (Scheme ${\textbf {IV}}$) was used with a simulated signal and AC/DC ratio of 1.63 % using 8 of 500 values for the AC component. The simulated signal is presented in Fig. 11(a). The measurement result for a $19 \times 19\,\textrm {pixel}$ ROI is shown in Fig. 11(b). As discussed above, the systolic peak is the only prominent feature. The measured AC/DC ratio is ${\leq {0.1} \,{\%}}$ and, thus, deviates by a factor of 10 from the simulation.
Fig. 11. (a) Evaluation of the AC/DC ratio of the simulation with a ratio of 1.63 %; (b) The measured ratio is smaller than 0.1 % when using a $19 \times {19} \,\textrm{pixel}$ region of interest.
6. Discussion
Here, we discuss the decisions made during the development of the hardware phantom and their implications for the measurement results. Further, we discuss some characteristics of the presented system with the phantoms described in the literature.
We opted for a very simplistic model of signal origin, which reduces to light signals originating from a surface. The ideal model of a phantom implementation with regard to the presented engineering model would allow to model and generate signals for each point of a surface/screen: thus, instead of a passive screen, a video screen would be desired. We consider micro LED displays with thousands of freely controllable pixels to be a good candidate to build a more sophisticated phantom. In contrast, only a small number of LEDs were used in our phantom, and the homogeneity was low (32.17 %). Due to the imaging characteristics, it is not easy to compare signals detected spatially next to each other.
The camera and phantom are not synchronized. This can be a problem as the light signals are generated via PWM, while for each frame the camera integrates light during the shutter time. In the remainder of the time, the sensor readout takes place. Consequently, if PWM and shutter are not synchronized, amplitude readings are corrupted.
These circumstances were the basis for our rationale not to (largely) evaluate the results quantitatively. Notwithstanding, we could generate qualitative results for important questions in camera-based sensing applications: to be more specific, for the signals created by varying the model parameters, we could show that a camera system can register changes in morphology. Even if amplitudes could not be reproduced exactly, the generated signals show the differences. At the same time, the determination of delays is more challenging; this requires a very fast sampling camera and, moreover, was affected (e.g.) by the PWM method used, due to amplitude alterations. Although the AC/DC ratio from the simulation was not reproduced, it was possible to show that even smaller ratios can be resolved with a camera.
These results can be used as clues for the limits of algorithms used in measurements with humans. Although these results may not be generalized, they can be used as a benchmark.
In the introduction, we presented phantoms for PPG- and camera-based applications. In general, these phantoms only imitate partial aspects of a measurement. The one presented here is no exception: we focused on the aspects of fast dynamic signal changes and only considered the optical signal. In essence, this design choice simplifies handling compared to designs which use fluids or mechanical parts [14,15]. Unlike [12,13], we did not cover oxygen saturation, nor did we study the effect of motion [15,16]. Instead, the generation of complex waveforms and the option to allocate these signals on a 2D plane are distinguishing features. The phantom using electronically switchable LCDs which was envisioned by Wieringa et al. [12] would share these characteristics.
7. Conclusion and outlook
In this work, we designed and evaluated the capabilities of a hardware simulator (phantom) for photoplethysmographic signals and demonstrated the imaging performance of a current video camera. The development of a simulator is motivated by the necessity to investigate PPG measurement devices under controlled conditions, which cannot be achieved with human subjects. Furthermore, besides the generation of 1D signals, we examined the possibility to study spatial distribution and propagation of signals. Therefore, the signal emitting unit of the phantom favors optoelectronic components as compared to other approaches which rely, for example, on a fluid-mechanics design.
We have shown that the simulator is suitable to investigate the quickly varying AC component of the PPG waveform, as well as shape-related features which were simulated by a Dual Windkessel model in real time. Medically relevant signal changes can, therefore, be successfully synthesized and emitted by the developed system and detected by a camera. Specifically, we demonstrated that changes to the dicrotic notch can be examined. At the same time, we faced problems resolving time delays between spatially distributed signals which we attribute to phantom and camera settings. As has been noted, the presented results are qualitative, and more quantitative evaluations are needed.
For future studies, (e.g. spatial distribution and oxygen saturation) the hardware design needs refinement, i.e. besides the problems caused by light inhomogeneity, at least a second wavelength needs to be added for more complex assessments. Furthermore, alternative hardware for the signal generator should be tested, which allows to simulate more complex models.
To sum up, using signal models and the optoelectronic phantom, studies without humans are feasible. It was demonstrated that it is possible to resolve clinically-relevant changes in morphology in the PPG waveform using cameras; this has implications for the range of applications of PPGI. Especially, vascular diagnostics could benefit from spatially resolved imaging.
Funding
Philips.
Acknowledgments
The authors thank Philips Research for providing measurement equipment and funding parts of this research.
Disclosures
MP: Philips (F); AFM: none; CHA: none; VB: (P) ; SL: Philips (F)
References
1. J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiol. Meas. 28(3), R1–R39 (2007). [CrossRef]
2. O. Such, “Near infrared CCD imaging of hemodynamics,” in Proceedings of the 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 1997), pp. 2105–2106.
3. T. Wu, V. Blazek, and H. J. Schmitt, “Photoplethysmography imaging: a new noninvasive and noncontact method for mapping of the dermal perfusion changes,” in Proceedings of Optical Techniques and Instrumentation for the Measurement of Blood Composition, Structure, and Dynamics, vol. 4163A. V. Priezzhev and P. A. Oberg, eds. pp. 62–70 (2000).
4. Y. Sun and N. Thakor, “Photoplethysmography revisited: From contact to noncontact, from point to imaging,” IEEE Trans. Biomed. Eng. 63(3), 463–477 (2016). [CrossRef]
5. S. Zaunseder, A. Trumpp, D. Wedekind, and H. Malberg, “Cardiovascular assessment by imaging photoplethysmography - a review,” Biomed. Tech. 63(5), 617–634 (2018). [CrossRef]
6. H. Ozaki and Y. Omura, “An advanced nonlinear signal model to analyze pulsation-derived photoplethysmogram signals,” Int. J. Smart Sens. Intell. Syst. 8(2), 921–943 (2015). [CrossRef]
7. U. Rubins, “Finger and ear photoplethysmogram waveform analysis by fitting with gaussians,” Med. Biol. Eng. Comput. 46(12), 1271–1276 (2008). [CrossRef]
8. L. Xu, S. Feng, Y. Zhong, C. Feng, M. Meng, and H. Yan, “Multi-gaussian fitting for digital volume pulse using weighted least squares method,” 2011 IEEE International Conference on Information and Automation, pp. 544–549 (2011)
9. R. Couceiro, P. Carvalho, R. Paiva, J. Henriques, I. Quintal, M. Antunes, J. Muehlsteff, C. Eickholt, C. Brinkmeyer, M. Kelm, and C. Meyer, “Assessment of cardiovascular function from multi-gaussian fitting of a finger photoplethysmogram,” Physiol. Meas. 36(9), 1801–1825 (2015). [CrossRef]
10. S. Nabar, A. Banerjee, S. Gupta, and R. Poovendran, “Resource-efficient and reliable long term wireless monitoring of the photoplethysmographic signal,” in Wireless Health (Springer,2011).
11. C. Hoog Antink, S. Leonhardt, and M. Walter, “A synthesizer framework for multimodal cardiorespiratory signals,” Biomed. Phys. & Eng. Express 3(3), 035028 (2017). [CrossRef]
12. F. Wieringa, “Pulse oxigraphy and other new in-depth perspectives through the near infrared window,” Ph.D. thesis, Erasmus Universiteit Rotterdam (2007).
13. I. Nishidate, N. Tanaka, T. Kawase, T. Maeda, T. Yuasa, Y. Aizu, and K. Niizeki, “Noninvasive imaging of human skin hemodynamics using a digital red-green-blue camera,” J. Biomed. Opt. 16(8), 086012 (2011). [CrossRef]
14. T. Myllylä, A. Popov, V. Korhonen, A. Bykov, and M. Kinnunen, “Optical sensing of a pulsating liquid in a brain-mimicking phantom,” in SPIE Proceedings Volume 8799, Diffuse Optical Imaging IV, vol. 8799 of SPIE ProceedingsP. Taroni and H. Dehghani, eds. (SPIE, 2013)
15. R. W. C. G. R. Wijshoff, J. Veen, A. M. van der Lee, L. Mulder, M. Stijnen, S. van Tuijl, and R. M. Aarts, “PPG motion artifact handling using a self-mixing interferometric sensor,” in SPIE Proceedings Volume 7894, Optical Fibers, Sensors, and Devices for Biomedical Diagnostics and Treatment XI, I. Gannot, ed. (SPIE, 2011), p. 78940F.
16. R. Wijshoff and R. Aarts, “Self-mixing interferometric sensor displacement reference for PPG motion artifact handling,” Proc. 21st ProRISC Work. STW.ICT Conf., pp. 1–4 (2012).
17. F. Giancarlo and G. Valerio, “PPG-based methods for non invasive and continuous blood pressure measurement: an overview and development issues in body sensor networks,” in 2010 IEEE International Workshop on Medical Measurements and Applications, pp. 10–13 (2010).
18. I. Moreno, M. Avendaño-Alejo, and R. Tzonchev, “Designing light-emitting diode arrays for uniform near-field irradiance,” Appl. Opt. 45(10), 2265–2272 (2006). [CrossRef]
19. K. Hassani, M. Navidbakhsh, and M. Rostami, “Modeling of the aorta artery aneurysms and renal artery stenosis using cardiovascular electronic system,” Biomed. Eng. Online 6(1), 22 (2007). [CrossRef]
20. S. Chen, S. Zhang, Y. Gong, K. Dai, M. Sui, Y. Yu, and G. Ning, “The role of the autonomic nervous system in hypertension: a bond graph model study,” Physiol. Meas. 29(4), 473–495 (2008). [CrossRef]
21. E. Zahedi, K. Chellappan, M. A. M. Ali, and H. Singh, “Analysis of the effect of ageing on rising edge characteristics of the photoplethysmogram using a modified Windkessel model,” Cardiovasc. Eng. 7(4), 172–181 (2007). [CrossRef]
22. T. Parlikar, “Non-invasive vascular assessment using photoplethysmography - modeling and monitoring of cardiovascular dynamics for patients in critical care,” Ph.D. thesis, Massachusetts Institute of Technology (2007).
23. S. Heinke, C. Pereira, S. Leonhardt, and M. Walter, “Modeling a healthy and a person with heart failure conditions using the object-oriented modeling environment dymola,” Med. Biol. Eng. Comput. 53(10), 1049–1068 (2015). [CrossRef]
