Open Access
Add to BibSonomy
Abstract
Sensitive and accurate assessment of dermatologic inflammatory hyperemia in otherwise grossly normal-appearing skin conditions is beneficial to laypeople for monitoring their own skin health on a regular basis, to patients for looking for timely clinical examination, and to primary care physicians or dermatologists for delivering effective treatments. We propose that mathematical hyperspectral reconstruction from RGB images in a simple imaging setup can provide reliable visualization of hemoglobin content in a large skin area. Without relying on a complicated, expensive, and slow hyperspectral imaging system, we demonstrate the feasibility of determining heterogeneous or multifocal areas of inflammatory hyperemia associated with experimental photocarcinogenesis in mice. We envision that RGB-based reconstructed hyperspectral imaging of subclinical inflammatory hyperemic foci could potentially be integrated with the built-in camera (RGB sensor) of a smartphone to develop a simple imaging device that could offer affordable monitoring of dermatologic health.
© 2017 Optical Society of America
1. Introduction
Skin cancer is the most common and frequently diagnosed type of cancer in the United States, with 5 million new cases annually in 2017 [1,2]. One in five Americans is at risk for skin cancer over the course of their lives [3]. In addition, actinic keratosis (also known as a solar keratosis), which is a premalignant lesion that may develop into non-melanoma skin cancer, has a reported prevalence of ~60% [4]. As a result, the average annual total cost for skin cancer has substantially increased by 126.2% from $3.6 billion (2002 – 2006) to $8.1 billion (2007 – 2011), compared to the increase of 25.1% for all other cancers [1]. In this respect, cost-effective risk assessment and prevention of skin cancer are of paramount importance in the clinical management. One possible strategy to achieve this objective is to make a simple and cost-effective imaging device available for personal monitoring of dermatologic health to detect early changes for delivering effective treatments. Such reliable yet affordable detection of skin abnormalities, which otherwise would be a grossly normal-appearing skin condition, is crucial to laypeople for monitoring their own dermatologic health status on a regular basis and to primary care clinicians/dermatologists for delivering effective prevention treatments in a timely manner.
Inflammatory hyperemia, which can be detected by hemoglobin (Hgb) content in tissue, can be useful as an endogenous optical biomarker of increased risk for tumors within an area exposed to carcinogenesis [5–9]. Indeed, inflammation with increased hyperemia and/or angiogenesis has been observed in the tumor environment, often early before tumors are formed [7–10]. Specifically, inflammatory angiogenesis during early skin carcinogenesis is commonly seen in humans and animals [8,9,11]. Skin cancer is often derived from environmental exposures [12], which promote inflammatory hyperemia or angiogenesis. In our previous animal study, we demonstrated that a spatial and temporal analysis of focal areas of inflammatory hyperemia can predict not only whether tumors will form but also where the tumors will form during experimental photocarcinogenesis of non-melanoma skin cancer [6]. A recent clinical study with spatial frequency domain imaging has successfully shown that the optical properties including Hgb content of the skin can be used to characterize skin abnormalities from mild photodamage to actinic keratosis in a clinically relevant area [5].
There are several different label-free tissue imaging methods available, such as reflectance confocal microscopy, optical coherent tomography, diffuse optical tomography, spatial frequency domain imaging and laminar optical tomography [13–15]. In terms of blood monitoring, laser speckle imaging is also useful for monitoring blood flow in an affordable imaging setup [16,17]. Using simultaneous 3-wavelength laser illumination, smartphone snapshot imaging is used for mapping skin pigments, including melanin and Hgb content [18]. In addition, spectroscopic analyses of light absorption of Hgb are commonly used to quantify Hgb content in biological tissue [6,19,20]. Specifically, Hgb has a unique characteristic absorption spectrum in the visible and NIR range [21]. For accurate and precise determination of Hgb content, common methods rely on the use of a spectrometer, an imaging spectrograph, or a tunable filter. On the other hand, the acquisition and processing of the spectral data using such hyperspectral imaging systems fundamentally limit the development of an uncomplicated, compact, rapid, and cost-effective imaging system. In this work, we propose to investigate whether it is feasible to determine spatial and temporal changes in Hgb content computed from RGB images without using a bulky hyperspectral imaging system.
Fortunately, mathematical reconstruction of hyperspectral (with relatively high spectral resolution) or multispectral (with several spectral measurements) imaging from RGB image data acquired using a conventional 3-color (RGB) camera is currently an active area of research. Since the reconstruction of multispectral images from RGB images was originally developed in basic color sciences [22], this approach has recently received considerable attention in biomedical applications [23–28]. There are several methods available for reconstructing full reflectance spectra from RGB data, including pseudo-inverse, regression, Wiener estimation, and demultiplexing [23–25,29,30]. In particular, Wiener estimation depends largely on noise variances and spectral autocorrelation matrix calculations [31,32], requiring a priori knowledge of the spectral sensitivity of the sensor and the spectral power distribution of the illumination. On the other hand, multivariate regression does not require any extra information but only uses reflectance spectra and corresponding RGB camera responses for training models, making it more applicable in general. Thus, using multivariate regression, we apply a data-driven approach to reconstruct hyperspectral reflectance image data from RGB reflectance images that are acquired with a conventional camera or a smartphone.
In this study, we report that the combination of an RGB-based hyperspectral reconstruction algorithm and a simple imaging setup can accurately and precisely determine heterogeneous or multifocal areas of inflammatory hyperemia on the animal skin in a preclinical setting. First, we conduct an experimental non-melanoma skin cancer study during early skin photocarcinogenesis. As a testing model, SKH-1 hairless mice are irradiated with a carcinogenic dose of UVB, which dominates the carcinogenic effect of sunlight as an environmental risk of skin cancer. Second, we construct a working prototype of dual imaging system that acquires hyperspectral image data and RGB image data under the identical illumination and detection configuration. Our intention is that eventually only RGB image data will be required to map out detailed spatial distributions of inflammatory hyperemia on the skin, simplifying the instrumentation, without requiring additional bulky and expensive hardware attachment. Third, we apply RGB-based hyperspectral reconstruction to visualize spatiotemporal changes in inflammatory hyperemia on a mesoscopic (between microscopic and macroscopic) imaging scale. Finally, we analyze spatial and temporal extent of Hgb content to characterize inflammatory hyperemia associated with photocarcinogenesis in mice.
2. Materials and methods
2.1 Mouse model of inflammatory hyperemia during early photocarcinogenesis
We used 26 SKH-1 hairless albino mice (Charles River Laboratories) as a model system of both UVB irradiation and optical imaging. As SKH-1 hairless albino mice are immunocompetent and have numerous similarities with human skin [33,34], they are commonly used for experimental skin studies. In particular, they have a similar response of dermal inflammation often seen in humans, in part because of permanently arrested hair cycling in the catagen phase [33]. For an experimental group, we irradiated 20 mice with one erythema dose of UVB (2,240 J/m2) three times per week and discontinued UVB irradiation after 10 weeks as previously described [6]. The cumulative UVB dose of 67.2 kJ/m2 was equivalent to a carcinogenic UVB dose [35]. For a control group, six mice did not receive any UVB irradiation. We imaged the surface of the dorsal skin of the mice over time until 17 weeks after the cessation of UVB irradiation. To obtain sequential images from the identical areas over time, the skin of each mouse was tattooed with a small reference marker (~0.5 mm in size). Immediately before imaging of the dorsal surface of the mouse, the mice were lightly sedated using intraperitoneal injection of ketamine (50 mg/kg) and xylazine (5 mg/kg) or isoflurane inhalation (~4.0% isoflurane in oxygen) for immobilization.
2.2 Telecentric imaging of biological tissue
Given the highly diffusive nature of light propagation in biological tissue, it is always beneficial to remove the diffusive light component in tissue imaging in a simple and cost-effective manner. In this respect, we have recently shown that a telecentric lens provides partial removal of the diffusive light to an extent in a simple mesoscopic imaging platform for tissue imaging setup [36]. A telecentric lens is a compound lens system modified with an additional aperture for achieving constant magnification and focus over a long distance [37]. Thus, telecentric lenses have often been used in the machine vision industry, because they remove the parallax error that makes closer objects appear to be larger than objects farther from the lens. Regarding tissue imaging, a small acceptance angle of ~0.1° of a telecentric lens was used to remove stray light in optical tomography (transmission mode) [38] and to overcome the curvature or distortion of the objective of interest in optical coherence tomography [39,40] and digital holographic microscopy [41]. However, the use of telecentric lenses for imaging biological tissue in a reflectance mode has not commonly been explored yet.
Specifically, telecentric imaging in a reflection mode can offer directional angular gating [20,42,43], which suppresses a relative amount of unwanted scattered or diffusive light in biological tissue given that the scattering in biological tissue is highly anisotropic [20,43]. The main idea of telecentric imaging is that a small imaging acceptance angle in a telecentric lens can remove off-axis unwanted diffuse light, because the exit angle of the light diffused from biological tissue at the tissue surface is large. i) The imaging acceptance angle in the telecentric lens is characterized as a telecentric angle. A typical telecentric angle is as narrow as ~0.5°, compared with conventional lenses with an average angle of ~5°. ii) Light traveling in biological tissue has a high directional tendency, as described by a high value of the scattering anisotropy factor, which is defined by g = <cos θ>, in a range of g = 0.8 – 0.95. Our previous system achieved this feature by using a 4-f lens system with a limited image area of 15 mm × 15 mm [6,20]. In our recent study [36], to quantify how much unwanted diffused is removed, we computed an image resolution of a black-and-white target embedded in tissue phantoms and mouse skin. Telecentric imaging showed ~25% enhancement in the image resolution, compared with conventional nontelecentric lens systems. In addition, the incorporation of a telecentric lens can enhance the field of view, the resolution, and the imaging depth for mesoscopic tissue imaging [36]. In particular, this simple optical design could easily be incorporated into an instrument of modest price for practical adoption.
2.3 Imaging setup for hyperspectral and RGB imaging
To obtain a reliable hyperspectral reconstruction matrix that converts RGB image data to hyperspectral imaging data, it is critical to have a dual data set (i.e., hyperspectral and RGB image data) from the same subject under the same imaging configuration, because reflectance (absorption and scattering) spectra are highly sensitive to illumination and detection geometries [44,45]. We note that this critical issue has often been ignored, resulting in inconsistent spectral reconstructions [25]. We carefully designed an imaging setup such that both hyperspectral and RGB image data can be acquired under the identical system configuration, minimizing potential spectral discrepancies resulting from different illumination and detection configurations (Fig. 1).
Fig. 1 Schematics of the dual imaging setup to acquire hyperspectral and RGB image data under the identical illumination and detection configurations, consisting of a telecentric lens. The illumination beam is coupled with a liquid crystal tunable filter (LCTF) to generate hyperspectral imaging data of the light reflected from the dorsal surface of a mouse. For RGB imaging, LCTF is temporally removed and a 3-color CCD camera is used. Inset: Spectral response of the 3-color CCD (Sony ICX625) used in this imaging setup. Although the telecentric lens system is not required, this offers back-directional angular gating for anisotropic scattering media (i.e., biological tissue), in part discarding unwanted scattered or diffusive light in biological tissue [20,42,43].
i) For hyperspectral imaging, an illumination beam from a white-light source (Xenon lamp) was coupled with a liquid crystal tunable filter (LCTF) (CRI VariSpec) in the delivery arm to scan a wavelength λ and was illuminated onto the sample via a ring illuminator. Specifically, diffuse light with a divergence angle of ~0.9° was illuminated on the mouse skin, avoiding specular surface reflection [36]. The light reflected from the sample at a distance (non-contact mode) was collected by a telecentric lens (magnification of 0.3 × , Schott Moritex Co.) and was recorded using a mono CCD camera. The telecentric lens imaged the mouse skin with a small acceptance (or telecentric) angle of ~0.5° [36]. The wavelength of LCTF was varied from 400 nm to 720 nm with a spectral resolution of 10 nm, generating a data set (x, y, and wavelength λ) for spectral analyses. ii) As for RGB image acquisition, LCTF was temporarily removed from the configuration to only allow the white light to illuminate the sample. For an RGB image sensor, we used a 3-color CCD camera embedded with Sony ICX 625. Overall, this imaging setup offered a relative large imaging area of 50 mm × 50 mm with a typical pixel size of 100 µm and an imaging depth up to ~1 mm from the tissue surface, assuming the scattering path length of light in the mouse skin is 70 – 100 µm [36]. To control the imaging setup, we implemented automated image acquisition with Virtual Instrument using LabVIEW (National Instruments). The total acquisition time for hyperspectral and RGB imaging was approximately 4 minutes per mouse. After the acquisition of a raw image data set on the mouse skin, we also measured a background data set and a reference image data set using a reflectance standard (Labsphere) to compensate for the system response such that
2.4 Reconstruction of hyperspectral image data from RGB
We previously developed a hyperspectral image reconstruction method that can reliably predict detailed hyperspectral information from RGB data obtained from a conventional 3-color (RGB) CCD camera [27,28]. In particular, we successfully tested this method using plants, in which the main pigment is chlorophyll [28], and redness in the eyelid of animals for anemia detection [27]. In this study, we extended this method to an animal model of skin carcinogenesis in which Hgb is the major pigment in the tissue and is an intermediate optical biomarker of inflammatory hyperemia. The RGB-based reconstruction algorithm for generating a full spectrum information of mouse skin can be summarized as follows: First, an RGB signal from a 3-color camera can be modeled such that
where is a vector corresponding to RGB signals, is a matrix of the RGB spectral response of the 3-color sensor (Sony ICX625), is an vector that has a spectral reflectance intensity as a function of λ, and is a vector of the system noise with zero mean. indicates the number of wavelengths; because LCTF in our imaging setup was varied from 400 nm to 720 nm with a spectral step of 10 nm, = 33. The spectral response of the 3-color CCD sensor (inset of Fig. 1) was used to obtain RGB signals from the measured reflectance spectra using Eq. (2). Second, once RGB signals are obtained for each spectrum , and can be constructed by stacking the reflectance spectra and RGB data such thatwhere and are the collections of the hyperspectral reflectance and RGB data, respectively. In our study, m is the number of different cases that is a product of the number of pixels of the camera and the number of animal data sets (i.e., number of mice × number time-points of sequential images). With this data set, we estimated the transformation matrix that can convert RGB data into hyperspectral reflectance data. To solve Eq. (3) for the unknown and to improve the prediction accuracy for the hyperspectral reconstruction, we utilized multivariate 2nd order polynomial regression. Finally, we predicted a reflectance spectrum from a new RGB signal by multiplying by . We note that a new testing set was employed to strengthen the validation process of the reconstruction model, in which the testing data set did not include any training data sets. Specifically, given the mouse data set, we built the reconstruction model using 100 mouse skin image data taken over time from 26 mice (i.e., the training data set). In addition, we separately employed unused 94 mouse skin image data from 12 mice (i.e., the test data set) to evaluate the performance of the model. Thus, we were able to avoid potential overfitting of the model by not reusing the same data set for both training and testing.2.5 Spectroscopic quantification of hemoglobin (Hgb) content
Once hyperspectral reflectance image data were reliably reconstructed from RGB image data, we quantified Hgb content on the mouse skin using a spectral absorption analysis that estimates Hgb levels in the individual pixel of the imaged area. There are several spectral analysis methods available and some of them assume that light-scattering spectra would be an exponential decay or a power-law decay function over the wavelength. In this current study, we made use of our previously developed method [20], in which the spectral shape from the scattering contribution is modeled as a linear decay function of the wavelength. The sum of the difference between the modeled spectrum (i.e., no absorption) and the measured spectrum within a spectral range of 510 – 620 nm was used to estimate Hgb content.
We ensured that this spectral analysis offered reliable quantification of Hgb content in tissue phantoms in a similar manner of our previous studies [6,20]. Specifically, to reliably extract a value of Hgb content from a spectral absorption area in our current imaging setup, we refined the previous developed calibration curve converting the spectral absorption area to the corresponding total Hgb content using a nonlinear least-square solver. For this refinement, we conducted additional tissue phantom studies consisting of aqueous suspensions of microspheres (i.e., scatters) and lyophilized Hgb (i.e., absorbers); the scattering properties (i.e., transport mean free path length and anisotropy factor) of the scattering media were estimated using the Mie calculations [46]. Polystyrene microspheres with a nominal diameter of 0.36 and 0.50 (Thermo Fisher Scientific Inc.) were used, resulting in the scattering path lengths of 673 and 305 and the anisotropy factor of 0.80 and 0.85, respectively. For each scattering medium, Hgb content was gradually increased by adding lyophilized Hgb (Sigma-Aldrich) into the scattering suspension from 0 to ~3.3 mg/mL. For each step, hyperspectral reflectance signals from the tissue phantoms were recorded using the imaging setup.
2.6. Statistical analysis of longitudinal data
We used two-stage analyses (also known as response feature analyses and NIH method), in which multiple responses on each animal are reduced to a single response over time and then an analysis of variance (ANOVA) or nonparametric methods equivalent of ANOVA are applied [47]. For the single response over time, linear regression analyses were used to evaluate slope estimates between Hgb variables and time. The assumptions of normality and uniform variance for the parametric analyses were also checked. We performed the statistical analyses using Stata 13.
3. Results
3.1 Hyperspectral image data reconstructed from RGB image data
Using the training data set from 100 mouse skin image data, we initially evaluated that our hyperspectral reconstruction method with polynomial regression can reliably reconstruct detailed spectral information. For this initial evaluation, we conducted numerical experiments for predicting hyperspectral reflectance data on the mouse skin using RGB data. After obtaining the hyperspectral image data using LCTF, we applied the spectral response of the 3-color CCD camera (Sony ICX 625 and the inset of Fig. 1) to numerically generate RGB image data that theoretically correspond to RGB image data directly acquired from the 3-color CCD camera. Figure 2(a) and 2(b) illustrate three-dimensional data structures of hyperspectral reflectance images made up with two-dimensional spatial information and one-dimensional spectral information for representative UVB-irradiated and non-irradiated mice, respectively. Figure 2(c) and 2(d) show the corresponding average reflectance spectra that were directly measured (red solid line) and reconstructed from the RGB data (blue circles) for the UVB-irradiated and non-irradiated mouse skin. The unique optical signature of Hgb around 420, 530, and 570 nm in the visible range is clearly seen in the average reflectance spectra of the imaged tissue (Fig. 2(d)). Compared to the control mouse in Fig. 2(d), the UVB-irradiated one in Fig. 2(c) also shows that the average reflectance intensity of the UVB-irradiated mouse is lower than that of the control mouse, due to the light absorption of Hgb. This representative case supports the idea that the reconstructed spectral reflectance from the RGB data can significantly resemble the originally measured reflectance spectra.
Fig. 2 (a) and (b) Representative 3-dimensional data structures of hyperspectral reflectance images for a UVB-irradiated mouse skin at 17 weeks after the cessation of UVB irradiation and a non-irradiated (control) mouse skin, respectively. (c) and (d) Corresponding reflectance spectra originally measured (red solid line) and numerically reconstructed from RGB signals (blue circle) averaged from the UVB-irradiated and control mouse skin, respectively.
To evaluate the resemblance between the measured reflectance spectra and the reconstructed ones for the entire data set, we conducted a statistical analysis using goodness-of-fit metrics: coefficient of determination R2 and root mean square error (RMSE). The average RMSE from 94 mouse image data was 0.008 (min = 0.003 and max = 0.021) and the average R2 was 0.995 (min = 0.975 and max = 0.999). Furthermore, Fig. 3(a) describes errors of spectral reconstruction from RGB data over and 95% confidence intervals of mean differences (dark blue line) between the originally measured reflectance spectra and the reconstructed reflectance spectra at each . In Fig. 3(b), the goodness-of-fit metrics from 94 mouse image data taken from 12 mice over time, which were used as a new testing set, clearly support reliable prediction of the spectral reconstruction with R2 > 0.98 and RMSE < 0.02.
Fig. 3 Performance characteristics of hyperspectral reconstruction from RGB data. (a) 95% confidence intervals of mean differences (dark blue line) between the original reflectance spectra and the reconstructed spectra at each wavelength. (b) Goodness-of-fit metrics of R2 and RMSE from 94 mouse skin image data taken over time from 12 mice used as a testing data set.
3.2 Experimental comparisons of Hgb maps generated from the dual imaging setup
Using the dual imaging setup, we compared Hgb maps directly computed from the hyperspectral image data (i.e., hyperspectral imaging mode) and Hgb maps computed from the reconstructed hyperspectral image data from the measured RGB image data (i.e., RGB imaging mode). These data sets were acquired from the same animal under the same illumination and detection configuration as depicted in Fig. 1. Due to the extended imaging time, only six mice were utilized for the dual imaging setup as a pilot test. As shown in the representative case of Fig. 4, the Hgb map from the RGB image is significantly similar to that from the hyperspectral image data. In Fig. 4(b) and 4(c), the tumor formation can clearly be located by the area with the high Hgb level in both Hgb maps. For quantitative evaluation of the similarity, a calculated 2D correlation coefficient C2D between Fig. 4(b) and 4(c) resulted in C2D = 0.99. This representative case supports the reliability of Hgb content imaging from the reconstructed hyperspectral image data.
Fig. 4 (a) White-light image of a representative mouse skin. Four small dots (forming a square shape) are tattoo marks for sequential imaging. (b) and (c) Hgb content maps are computed from the hyperspectral image data and the RGB image data using the hyperspectral reconstruction method, respectively. (d) Histogram of 2D correlation coefficients between Hgb content maps using the original hyperspectral image data and the reconstructed hyperspectral image data from RGB data for 94 mouse skin images from 12 mice that are used as a separate testing data set.
We further evaluated similarities of Hgb maps obtained from the hyperspectral image data and the numerically generated RGB image data. We used a separate testing set (94 mouse skin images) from 12 mice to avoid potential overfitting of the hyperspectral reconstruction model, in which the testing data did not include any training data obtained from 26 mice. In Fig. 4(d), the histogram of two-dimensional correlation coefficients calculated from 94 mouse skin images from 12 mouse samples show that subclinical focal areas of inflammatory hyperemia in the tissue microenvironment can be sensitively and accurately extracted using the RGB data, taking advantage of the hyperspectral reconstruction method built on a prior information.
3.3 Visual tumor count and histology of UVB-irradiated mice
Figure 5(a) shows a significant increase in durable tumors (tumor size > 0.5 – 1 mm) for UVB-irradiated mice after the cessation of UBV irradiation. In contrast, non-UVB irradiated mice failed to develop tumors. Indeed, the UVB irradiation for 10 weeks consistently resulted in initial tumor formation after the final UVB irradiation. In the histology analysis of later photocarcinogenesis in Fig. 5(b), the Hgb hotspots were significantly associated with alterations in the tissue microenvironments, in particular stromal hyperemia. Specifically, the areas of high Hgb content (i.e., hotspots) exhibited i) epidermal hyperplasia, ii) increased inflammation (red blood cells), iii) dermal expansion and follicular hyperplasia in the dermis, and iv) loss of deep adipose tissue.
Fig. 5 (a) Visible tumor multiplicity in UVB-irradiated mice counted as average numbers of microscopically observed tumors per mouse as a function of weeks after the cessation of UVB irradiation. (b) Microscopy images (200 × ) of H&E stain of hyperemic foci at approximately 20 weeks after the cessation of UVB irradiation. Areas of high Hgb content have epidermal hyperplasia and exhibit inflammatory infiltrate and dermal expansion.
3.4 Spatiotemporal changes in inflammatory hyperemic foci in tissue microenvironment
We investigated spatiotemporal changes in multifocal areas of inflammatory hyperemia from 12 mice used as the testing set during cutaneous photocarcinogenesis. Hgb maps were computed over time after the cessation of UVB irradiation, using both the original hyperspectral image and the reconstructed one from the RGB data. The visualization of Hgb content during the early photocarcinogenesis shows that inflammatory hyperemia preceded the visible tumor occurrence, specifically as shown in 4 or 6 weeks in Fig. 6. The multifocal areas of increased inflammatory hyperemia were identified, which persisted or formed over the ensuing weeks from a representative UVB-irradiated mouse. Compared to the control mouse in Fig. 7, the UVB irradiated mouse shows a significant increase in the area of high Hgb over time after stopping the UVB irradiation as depicted in Fig. 6(b) and 6(c). These persistent focal areas of inflammatory hyperemia seem to expand over time. We note that the hyperemic areas during early photocarcinogenesis were not easily observed in the conventional photographs, due to the lack of the contrast as shown in Fig. 6(a). Also, subsequent tumor formation appeared within the focal areas of the increased inflammatory hyperemia. Overall, tumor-associated hyperemia can sensitively and accurately be visualized using our RBG-based hyperspectral imaging of Hgb as markedly elevated Hgb content.
Fig. 6 Spatiotemporal extent in Hgb content maps of UVB-induced hyperemic foci following different weeks after the cessation of carcinogenic UVB irradiation from the same mouse in a sequential manner. (a) Photographs obtained using a digital camera. During early photocarcinogenesis (i.e., 4 – 9 weeks after the cessation of UVB irradiation), the conventional photographs are not sensitive to the changes of hyperemia. (b) Hgb maps computed using the hyperspectral image data from the UVB-irradiated mouse skin. (c) Hgb maps computed using the reconstructed hyperspectral image data from the RGB data. (b) and (c) show the significant similarity due to the reliable hyperspectral reconstruction. In both cases, focal areas of intense hyperemia precede tumor formation, indicating sites at high risk for tumor occurrence; the hyperemic areas not only persist, but also suggest a link between dermal hyperemia and ongoing epidermal tumor development.
Fig. 7 Hgb content maps from a representative non-irradiated mouse skin from the same mice in a sequential manner. (a) Photographs obtained using a digital camera from the control mouse. (b) Hgb maps computed using the hyperspectral image data. (c) Hgb maps computed using the reconstructed hyperspectral data from RGB data for the same mouse. The significant similarity between (b) and (c) in the case of low Hgb content also supports the reliability of hyperspectral reconstruction.
For more quantitative assessment of Hgb maps from 94 mouse skin image data from 12 mice, we set up a threshold of Hgb content to 2.0 mg/mL. This range of threshold value was determined in our previous study for effectively delineating Hgb hotspots in the photocarcinogenesis of SHK-1 mice [6]. Then, we computed Hgb hotspot areas above this threshold and average Hgb content within the corresponding Hgb hotspot areas over time. The quantitative analyses shown in Fig. 8(a) and 8(b) support the same observation that the areas of inflammatory hyperemia not only persisted before tumor formation but also expanded in size up to tumor formation over time after the cessation of UVB irradiation. For Hgb content computed from the hyperspectral image data, the slope estimates of the linear regression for hotspot areas and Hgb content over time in UVB-irradiated mice were statistically significant with p-values of 0.025 and 0.017, respectively. The differences in the hotspot area and Hgb content slopes between the groups were statistically significant with p-values of 0.007 and 0.012, respectively. In other words, the difference of multifocal areas of inflammatory hyperemia between the UVB-irradiated and control groups was statistically significant. Similarly, for Hgb content computed from the RGB image data using the hyperspectral reconstruction method, the slope estimates of hotspot areas and Hgb content in UVB-irradiated mice returned p-values of 0.105 and 0.015, respectively. The higher p-value for the slope estimate of hotspot areas may be attributable to the larger variation of hotspot areas from the RGB image data as shown in Fig. 8(b). Similar to Fig. 8(a), the differences in the hotspot and Hgb content slopes between the groups, computed from the reconstructed hyperspectral image data, were statistically significant with p-values of 0.028 and 0.005, respectively.
Fig. 8 (a) Analyses of Hgb content maps computed from the hyperspectral reflectance image data. After thresholding hotspot areas with Hgb content more than 2.0 mg/mL, mean areas of hyperemic foci are calculated. The areas of high Hgb content expand after the cessation of UVB irradiation in UVB-irradiated mice but not in control mice (i.e., non UVB-irradiated). The difference of multifocal hyperemic areas between the UVB-irradiated and control groups is statistically significant with p-value = 0.007. Average Hgb content within the hotspots is determined for each mouse at each time-point. (b) Corresponding analyses of Hgb content maps computed from the RGB image data, using the hyperspectral reconstruction method. The Hgb content analyses using the RGB-based hyperspectral reconstruction significantly resemble the Hgb analyses using the original hyperspectral data. All of the error bars are standard deviations.
4. Discussions
Several smartphone applications or handheld dermoscopy devices (e.g., FotoFinder, DermLite, SIAscope, and MelaFind) are already available for analyzing moles and pigmented spots for skin cancer, in particular melanoma, detection. There are also several smartphone-based Hgb measurement technologies (e.g., ToucHb, HemoGlobe, Eyenaemia, and HemaApp) for anemia. However, smartphone-based Hgb imaging of skin inflammation has not yet been commercially available, to the extent of our knowledge. In this current study, we demonstrated mathematically reconstructed hyperspectral reflectance imaging fed with RGB image data for reliably mapping out subclinical inflammatory hyperemic foci associated with photocarcinogenesis
Our current study has several limitations. Simultaneous acquisition of hyperspectral and RGB image data was not possible. The current dual imaging setup avoided potential spectral variations resulting from the light scattering contribution, because both hyperspectral image and RGB image data were acquired under the identical illumination and detection configuration, but still acquired the data in a sequential manner with a time delay; RGB image data were taken after the acquisition of hyperspectral image data, causing a time delay in the current setup. Considering potential clinical implication in humans, the relatively long acquisition time of ~4 minutes may cause involuntary movement to negatively affect the quality of raw image data. In this respect, we are currently developing a next generation of dual imaging system that allows for concurrent and simultaneous acquisition of hyperspectral and RGB image data.
The hyperspectral reconstruction from originally measured RGB image data may involve technical adjustment issues, such as different pixel sizes between mono and RGB cameras and RGB color interpolation of adjacent pixels. Indeed, most common 3-color sensors (e.g., Bayer filter sensor) interpolate missing color information in selected pixel positions using a demosaicing algorithm [48]. Each pixel records only of the RGB colors in a format of a grid or a 3-color checkerboard. On the other hand, a Foveon X3 sensor captures all of the three colors within the same pixel. The manufacturer claims that Foveon X3 sensors can avoid this limitation, improving the RGB image quality. Thus, a Foveon X3 sensor would be more ideal for our RGB-based hyperspectral reconstruction.
The training-based hyperspectral reconstruction method requires a prior hyperspectral image data of interest and heavily depends on the quality of training data and the similarity of testing data to training data sets. Even though SKH-1 mouse skin has similarities with human skin, our current conversion matrix from RGB to hyperspectral image data would be limited only to SKH-1 mouse skin samples. Although the murine model is highly valuable to provide detailed information on human inflammation, there are potential differences between murine and human skin. The human dermis is relatively thicker and has fewer hair follicles than the mouse dermis. The human epidermis is generally 6 - 10 cell layers thick (> 100 μm) while the murine epidermis consists of 2 - 3 cell layers [49]. For visualization of spatiotemporal changes in inflammatory hyperemia in humans, training hyperspectral data from humans with different degrees of hyperemia under several clinical scenarios would be necessary. A human study that we are currently considering is to image skin areas of persistent focally increased Hgb content in individuals with photodamaged skin and sun-protected skin. In this case, the degree of photodamage can be assessed using a clinically utilized 9-point photonumeric McKenzie photodamage scale [50]. We note that this limitation applies to any training-based reconstruction methods, involving an appropriate selection of samples.
5. Conclusion
We have shown that the unique combination of hyperspectral reconstruction and telecentric reflectance imaging can serve as a simple and affordable imaging system to visualize quantitative spatiotemporal changes of subclinical inflammatory hyperemia in a relatively large tissue area. We have conducted the comparative investigation of Hgb content maps between the hyperspectral and RGB image data at different time-points of photocarcinogenesis to validate our imaging method in the experimental setting. The current hyperspectral reconstruction method could potentially be extended to examine the tissue microenvironment in humans, when a training data set is readily available. Because inflammation is a common component of many dermatologic abnormalities, accurate and reliable detection of inflammatory hyperemia will be beneficial to the clinical management. The main advantages of RGB-based hyperspectral reconstruction imaging of Hgb are i) the sensitivity of Hgb content, ii) the simplicity for instrumentation, and iii) the high temporal resolution of data acquisition, all of which suggest that this technology could easily be incorporated into an instrument of modest price such as a smartphone. Indeed, as the smartphone ownership is common, the ‘right’ technology should take advantage of the built-in camera (i.e., RGB sensor) in a smartphone without requiring additional bulky and expensive hardware attachment. We expect that such noninvasive, cost-effective, and user-friendly imaging devices will play a critical role in initial screening and risk-stratification for personalized monitoring of inflammatory skin diseases, leading to timely clinical examination.
Funding
This work was supported by Samsung Global Research Outreach Program, Trask Innovation Fund from Purdue Research Foundation, Indiana Clinical and Translational Sciences Institute (NIH UL1TR001108), and NIH R21TW010620.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References and links
1. G. P. Guy Jr, S. R. Machlin, D. U. Ekwueme, and K. R. Yabroff, “Prevalence and costs of skin cancer treatment in the U.S., 2002-2006 and 2007-2011,” Am. J. Prev. Med. 48(2), 183–187 (2015). [PubMed]
2. American Cancer Society, Cancer Facts & Figs.2017.
3. A. Jemal, “Recent trends in cutaneous melanoma incidence and death rates in the United States, 1992-2006,” J. Am. Acad. Dermatol. 65(5), e1–e11 (2011).
4. L. A. Drake, R. I. Ceilley, R. L. Cornelison, W. L. Dobes, W. Dorner, R. W. Goltz, G. F. Graham, C. W. Lewis, S. J. Salasche, and M. L. Turner, “Guidelines of care for actinic keratoses,” J. Am. Acad. Dermatol. 32(1), 95–98 (1995). [PubMed]
5. J. B. Travers, C. Poon, D. J. Rohrbach, N. M. Weir, E. Cates, F. Hager, and U. Sunar, “Noninvasive mesoscopic imaging of actinic skin damage using spatial frequency domain imaging,” Biomed. Opt. Express 8(6), 3045–3052 (2017). [PubMed]
6. R. L. Konger, Z. Xu, R. P. Sahu, B. M. Rashid, S. R. Mehta, D. R. Mohamed, S. C. DaSilva-Arnold, J. R. Bradish, S. J. Warren, and Y. L. Kim, “Spatiotemporal assessments of dermal hyperemia enable accurate prediction of experimental cutaneous carcinogenesis as well as chemopreventive activity,” Cancer Res. 73(1), 150–159 (2013). [PubMed]
7. B. Hu, E. Castillo, L. Harewood, P. Ostano, A. Reymond, R. Dummer, W. Raffoul, W. Hoetzenecker, G. F. Hofbauer, and G. P. Dotto, “Multifocal epithelial tumors and field cancerization from loss of mesenchymal CSL signaling,” Cell 149(6), 1207–1220 (2012). [PubMed]
8. A. Albini, F. Tosetti, R. Benelli, and D. M. Noonan, “Tumor inflammatory angiogenesis and its chemoprevention,” Cancer Res. 65(23), 10637–10641 (2005). [PubMed]
9. D. Hanahan and J. Folkman, “Patterns and emerging mechanisms of the angiogenic switch during tumorigenesis,” Cell 86(3), 353–364 (1996). [PubMed]
10. A. K. Tiwari, S. E. Crawford, A. Radosevich, R. K. Wali, Y. Stypula, D. P. Kunte, N. Mutyal, S. Ruderman, A. Gomes, M. L. Cornwell, M. De La Cruz, J. Brasky, T. P. Gibson, V. Backman, and H. K. Roy, “Neo-angiogenesis and the premalignant micro-circulatory augmentation of early colon carcinogenesis,” Cancer Lett. 306(2), 205–213 (2011). [PubMed]
11. D. Hanahan and R. A. Weinberg, “Hallmarks of cancer: The next generation,” Cell 144(5), 646–674 (2011). [PubMed]
12. P. Anand, A. B. Kunnumakkara, C. Sundaram, K. B. Harikumar, S. T. Tharakan, O. S. Lai, B. Sung, and B. B. Aggarwal, “Cancer is a preventable disease that requires major lifestyle changes,” Pharm. Res. 25(9), 2097–2116 (2008). [PubMed]
13. D. A. Boas, C. Pitris, and N. Ramanujam, Handbook of Biomedical Optics (CRC press, 2016).
14. V. V. Tuchin, Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis (SPIE press, 2015).
15. L. V. Wang and H.-i. Wu, Biomedical optics: principles and imaging (John Wiley & Sons, 2012).
16. B. Choi, N. M. Kang, and J. S. Nelson, “Laser speckle imaging for monitoring blood flow dynamics in the in vivo rodent dorsal skin fold model,” Microvasc. Res. 68(2), 143–146 (2004). [PubMed]
17. L. M. Richards, S. M. Kazmi, J. L. Davis, K. E. Olin, and A. K. Dunn, “Low-cost laser speckle contrast imaging of blood flow using a webcam,” Biomed. Opt. Express 4(10), 2269–2283 (2013). [PubMed]
18. J. Spigulis, I. Oshina, A. Berzina, and A. Bykov, “Smartphone snapshot mapping of skin chromophores under triple-wavelength laser illumination,” J. Biomed. Opt. 22(9), 91508 (2017). [PubMed]
19. R. K. Wali, H. K. Roy, Y. L. Kim, Y. Liu, J. L. Koetsier, D. P. Kunte, M. J. Goldberg, V. Turzhitsky, and V. Backman, “Increased microvascular blood content is an early event in colon carcinogenesis,” Gut 54(5), 654–660 (2005). [PubMed]
20. Z. Xu, “Back-directional gated spectroscopic imaging for diffuse light suppression in high anisotropic media and its preclinical applications for microvascular imaging,” IEEE J. Sel. Top. Quant. 16(4), 815–823 (2010).
21. S.-H. Tseng, P. Bargo, A. Durkin, and N. Kollias, “Chromophore concentrations, absorption and scattering properties of human skin in-vivo,” Opt. Express 17(17), 14599–14617 (2009). [PubMed]
22. H. Haneishi, T. Hasegawa, A. Hosoi, Y. Yokoyama, N. Tsumura, and Y. Miyake, “System design for accurately estimating the spectral reflectance of art paintings,” Appl. Opt. 39(35), 6621–6632 (2000). [PubMed]
23. K. Xiao, Y. Zhu, C. Li, D. Connah, J. M. Yates, and S. Wuerger, “Improved method for skin reflectance reconstruction from camera images,” Opt. Express 24(13), 14934–14950 (2016). [PubMed]
24. K. Yoshida, I. Nishidate, T. Ishizuka, S. Kawauchi, S. Sato, and M. Sato, “Multispectral imaging of absorption and scattering properties of in vivo exposed rat brain using a digital red-green-blue camera,” J. Biomed. Opt. 20(5), 051026 (2015). [PubMed]
25. I. Nishidate, T. Maeda, K. Niizeki, and Y. Aizu, “Estimation of melanin and hemoglobin using spectral reflectance images reconstructed from a digital RGB image by the Wiener estimation method,” Sensors (Basel) 13(6), 7902–7915 (2013). [PubMed]
26. I. Nishidate, N. Tanaka, T. Kawase, T. Maeda, T. Yuasa, Y. Aizu, T. Yuasa, and K. Niizeki, “Noninvasive imaging of human skin hemodynamics using a digital red-green-blue camera,” J. Biomed. Opt. 16(8), 086012 (2011). [PubMed]
27. T. Kim, S. H. Choi, N. Lambert-Cheatham, Z. Xu, J. E. Kritchevsky, F. R. Bertin, and Y. L. Kim, “Toward laboratory blood test-comparable photometric assessments for anemia in veterinary hematology,” J. Biomed. Opt. 21(10), 107001 (2016). [PubMed]
28. T. Kim, J.-I. Kim, M. A. Visbal-Onufrak, C. Chapple, and Y. L. Kim, “Nonspectroscopic imaging for quantitative chlorophyll sensing,” J. Biomed. Opt. 21(1), 16008 (2016). [PubMed]
29. J. Deglint, F. Kazemzadeh, D. Cho, D. A. Clausi, and A. Wong, “Numerical demultiplexing of color image sensor measurements via non-linear random forest modeling,” Sci. Rep. 6, 28665 (2016). [PubMed]
30. S. Chen, Y. H. Ong, X. Lin, and Q. Liu, “Optimization of advanced Wiener estimation methods for Raman reconstruction from narrow-band measurements in the presence of fluorescence background,” Biomed. Opt. Express 6(7), 2633–2648 (2015). [PubMed]
31. N. Shimano, K. Terai, and M. Hironaga, “Recovery of spectral reflectances of objects being imaged by multispectral cameras,” J. Opt. Soc. Am. A 24(10), 3211–3219 (2007). [PubMed]
32. H. L. Shen and J. H. Xin, “Estimation of spectral reflectance of object surfaces with the consideration of perceptual color space,” Opt. Lett. 32(1), 96–98 (2007). [PubMed]
33. F. Benavides, T. M. Oberyszyn, A. M. VanBuskirk, V. E. Reeve, and D. F. Kusewitt, “The hairless mouse in skin research,” J. Dermatol. Sci. 53(1), 10–18 (2009). [PubMed]
34. C. A. Cole, R. E. Davies, P. D. Forbes, and L. C. D’Aloisio, “Comparison of action spectra for acute cutaneous responses to ultraviolet radiation: man and albino hairless mouse,” Photochem. Photobiol. 37(6), 623–631 (1983). [PubMed]
35. A. P. Pentland, J. W. Schoggins, G. A. Scott, K. N. M. Khan, and R. Han, “Reduction of UV-induced skin tumors in hairless mice by selective COX-2 inhibition,” Carcinogenesis 20(10), 1939–1944 (1999). [PubMed]
36. M. A. Visbal Onufrak, R. L. Konger, and Y. L. Kim, “Telecentric suppression of diffuse light in imaging of highly anisotropic scattering media,” Opt. Lett. 41(1), 143–146 (2016). [PubMed]
37. M. Watanabe and S. K. Nayar, “Telecentric optics for focus analysis,” IEEE Trans. Pattern. Anal. 19(12), 1360–1365 (1997).
38. A. Thomas, J. Newton, and M. Oldham, “A method to correct for stray light in telecentric optical-CT imaging of radiochromic dosimeters,” Phys. Med. Biol. 56(14), 4433–4451 (2011). [PubMed]
39. A. Tao, Y. Shao, J. Zhong, H. Jiang, M. Shen, and J. Wang, “Versatile optical coherence tomography for imaging the human eye,” Biomed. Opt. Express 4(7), 1031–1044 (2013). [PubMed]
40. R. P. McNabb, P. Challa, A. N. Kuo, and J. A. Izatt, “Complete 360° circumferential gonioscopic optical coherence tomography imaging of the iridocorneal angle,” Biomed. Opt. Express 6(4), 1376–1391 (2015). [PubMed]
41. A. Doblas, E. Sánchez-Ortiga, M. Martínez-Corral, G. Saavedra, and J. Garcia-Sucerquia, “Accurate single-shot quantitative phase imaging of biological specimens with telecentric digital holographic microscopy,” J. Biomed. Opt. 19(4), 046022 (2014). [PubMed]
42. Z. Xu, A. K. Somani, and Y. L. Kim, “Scattering anisotropy-weighted mesoscopic imaging,” J. Biomed. Opt. 17(9), 090501 (2012). [PubMed]
43. Z. Xu, J. Liu, and Y. L. Kim, “Diffuse light suppression of back-directional gating imaging in high anisotropic media,” J. Biomed. Opt. 14(3), 030510 (2009). [PubMed]
44. S. C. Gebhart, S. K. Majumder, and A. Mahadevan-Jansen, “Comparison of spectral variation from spectroscopy to spectral imaging,” Appl. Opt. 46(8), 1343–1360 (2007). [PubMed]
45. H.-L. Shen, J. H. Xin, and S.-J. Shao, “Improved reflectance reconstruction for multispectral imaging by combining different techniques,” Opt. Express 15(9), 5531–5536 (2007). [PubMed]
46. Y. L. Kim, “Simultaneous measurement of angular and spectral properties of light scattering for characterization of tissue microarchitecture and its alteration in early precancer,” IEEE J. Sel. Top. Quant. 9(2), 243–256 (2003).
47. W. D. Dupont, Statistical Modeling for Biomedical Researchers: A Simple Introduction to the Analysis of Complex Data (Cambridge University Press, 2009).
48. P. A. Cheremkhin, V. V. Lesnichii, and N. V. Petrov, “Use of spectral characteristics of DSLR cameras with Bayer filter sensors,” J. Phys. Conf. Ser. 536(1), 012021 (2014).
49. P. A. Khavari, “Modelling cancer in human skin tissue,” Nat. Rev. Cancer 6(4), 270–280 (2006). [PubMed]
50. N. E. McKenzie, K. Saboda, L. D. Duckett, R. Goldman, C. Hu, and C. N. Curiel-Lewandrowski, “Development of a photographic scale for consistency and guidance in dermatologic assessment of forearm sun damage,” Arch. Dermatol. 147(1), 31–36 (2011). [PubMed]
