An optimized system for fast, high-resolution spectral imaging of in vivo human skin is developed and evaluated. The spectrograph is composed of a dispersive prism in combination with an electron multiplying CCD camera. Spectra of autofluorescence and second harmonic generation (SHG) are acquired at a rate of 8 kHz and spectral images within seconds. Image quality is significantly enhanced by the simultaneous recording of background spectra. In vivo spectral images of 224 × 224 pixels were acquired, background corrected and previewed in real RGB color in 6.5 seconds. A clear increase in melanin content in deeper epidermal layers in in vivo human skin was observed.
©2011 Optical Society of America
Nonlinear optical techniques such as two-photon excitation (TPE) microscopy and second harmonic generation (SHG) are powerful tools for the investigation of tissues [1–11]. Excitation only occurs in the focus of the microscope; out-of focus absorption and photobleaching are strongly reduced . Importantly, near infrared (NIR) light used for TPE penetrates much deeper into tissue than visible light used in one photon excitation.
Multi-photon excitation can be employed to excite the autofluorescence of naturally occurring fluorophores in tissues [2,4,6,7]. This affords non-invasive imaging of skin with subcellular resolution without adding dyes [2,4,6–8,12]. Autofluorescent components that can be observed include NADH , keratin , elastin , FAD , melanin [17,18] and in addition the SHG of certain types of collagen can be detected [10,19]. Images of the layered structure of skin can be easily recorded with a quality comparable to histological sections [8,17].
The combination of tissue imaging and spectroscopic detection such as fluorescence lifetime imaging , spectral imaging  or hyperspectral imaging , enhances image contrast and allows fluorescent compounds to be discriminated based on differences in fluorescence decay kinetics or emission spectra. Lifetime imaging methods typically utilize time correlated single photon counting  or time gating , spectral imaging can be performed using a set of emission filters [21,23] or dispersive optics and a PMT array  or CCD camera . The advantage of the latter approach is that it affords detailed (high resolution) spectroscopic analysis; this was for instance demonstrated in in vivo mouse skin imaging experiments [7,8]. Rapid visualization of both spectral and morphological information can be achieved in real time by “real color” RGB representation of the spectral images [7,8]. Endogenous fluorophores can be identified based on their spectral signature  and spectral unmixing can be employed in an offline analyses to determine the relative contributions of individual fluorophores to the signal .
The disadvantage of conventional CCD camera based spectral imaging systems is their slow spectral acquisition rate. For instance, we previously reported a nonlinear spectral imaging microscope with a spectral readout of 500 Hz, resulting in a 2-minute image acquisition time for images of 224 × 224 pixels . The background signal in these experiments was typically not constant over the image. To correct for this background, background spectra were extracted from low intensity regions-of-interest; an elaborate procedure that does not account for variations in the background within an image. The slow acquisition time makes the system sensitive for motion artifacts and the errors in the background correction complicate obtaining accurate spectral information. These limitations make the system less well suited for in vivo imaging.
Here, we optimized the image acquisition hardware and procedure of nonlinear spectral imaging for in vivo applications. The most important improvements are: (1) use of an electron multiplying CCD (EMCCD) based spectrograph that can acquire 8000 spectra/s, and (2) simultaneous acquisition and correction for the background signal. This approach proved to be fast and robust; it accurately corrects for stray light and temporal drift of the background. The emission is detected with high quantum efficiency by the EMCCD camera; at normal excitation power levels (typically 5-20mW) there is sufficient signal for RGB visualization and detailed spectral analyses of ROI’s.
The complete acquisition procedure, including background correction, and RGB visualization can now be performed within 6.5 seconds for a 224x224 pixel image. Motion artifacts are strongly reduced because of reduced acquisition time. The system is validated in in vivo imaging experiments on human skin of Caucasian and Asian phototype. The experiments revealed clear differences in the melanin content between the different epidermal skin layers.
2. Nonlinear spectral imaging setup and spectra processing
The nonlinear spectral imaging microscope is based on a homemade inverted microscope system described in previous work  and diagram is shown in Fig. 1 . Briefly, the excitation light source (tuned to 760 nm) consists of a mode-locked titanium:sapphire (Ti:Sa) laser (Tsunami, Spectra-Physics, Sunnyvale, CA). The laser light is scanned in the XY direction using a galvanometer mirror scanner (040EF, LSK, Stallikon, Switzerland). In addition to the beam-scanner, the microscope is equipped with a Z-piezo objective translation stage (Mad City Labs, Madison, WI, USA). The laser light is focused on the sample by a microscope objective and the TPE and SHG signals are collected by the same objective lens. The results reported here are acquired using an infinity-corrected water-immersion objective (40 × , NA = 0.8, Nikon, Japan) with a long working distance (2 mm). The emission passes through a dichroic mirror (680 nm shortpass) and is filtered by a multiphoton emission filter (FF01-680/SP-25, Semrock, Rochester, NY, USA). Note that the emission has to be descanned to keep the position of the spectrum on the CCD chip fixed. The typical reduction in detection sensitivity and penetration depth in a descanned detection layout  is circumvented here by positioning the scan mirror directly underneath the objective .
The basic design of the spectrograph is simple: it consists of a fused silica equilateral dispersive prism (Linos, Göttingen, Germany), a focusing lens (f = 90 mm) and an EMCCD camera (Cascade 128 + EMCCD, Photometrics, Tucson, AZ, USA). The back illuminated CCD is equipped with on-chip multiplication gain, where the signal is amplified before the readout electronics. Therefore, this camera is very suitable for detecting low light levels. An EM gain of 30%-of-maximum has been used in all experiments. Spectral acquisition is controlled by software written in V++ (Digital Optics, Auckland, New Zealand). A low level camera readout script was written and a customized driver is used for optimized performance.
The high spectral acquisition speed is the result of the optimized read-out scheme of the EMCCD. The rate limiting step is the speed of A/D conversion. Therefore, the number of read out pixels is reduced and the collection of photons occurs simultaneously with the read out of the data of previous pixels. The spectrum is projected on to the EMCCD chip parallel to the read out register (see Fig. 1, right). Three sections of 10 rows of 128 pixels are defined. The emission spectrum is projected on the central section. The other two sections are used as background reference spectra. The readout script is synchronized with the pixel clock of the microscope. For each voxel of the acquired image, the sensor area of the camera is exposed until the next pixel clock trigger. After the trigger, the content of the sensor area is shifted to the frame transfer area of the CCD chip. While the emission and background spectra of the next voxel are collected in the sensor area, the three sections of interest are shifted from the frame transfer area to the serial read-out register. The 10 vertical rows of each section are binned and shifted to the first multiplication register. The spectra are transferred through four multiplication registers before being read out. Also the transfer of the spectra through the registers is optimized. During normal operation, the data of three voxels are simultaneously recorded on the EMCCD chip: one in the sensor area, one in the frame transfer area and one in the register. The resulting raw data file is 128 × 3 × N2 pixels for an N × N voxel image.
Experimentally the minimum dwell time was found to be 85 μs per voxel. In all the experiments reported here, a pixel dwell time of 128 μs was used and all images are 224 × 224 pixels. This results in an image acquisition time of 6.5 seconds.
Wavelength calibration was carried out with great care. In the nonlinear spectral microscope, the emission from the focal volume of the objective is dispersed and imaged onto the CCD. Small shifts in the focal position affect the calibration of the spectrograph. This makes it very difficult to use an external light source for calibration purposes; multiphoton excited emission has to be used instead. Here, wavelength calibration is accomplished by inserting a multiband bandpass filter (Semrock FF01-390/482/563/640-25) in the emission path of the microscope. The filter has steep edges (~1 nm) and by dividing spectral recordings with and without the filter, positions of the edges are obtained. Calibration is finalized by fitting the pixel positions with a 2nd order polynomial. The correction for the wavelength dependent sensitivity of the system (flat-field) is composed of the spectral response of the camera multiplied by the transmission curves of the optical components in the emission path (e.g., dichroic and filter) and a correction for the wavelength dependent dispersion of the prism. The flat-field correction is constructed from data supplied by manufacturers and the material properties of the prism.
The raw data file is processed in V++ by a script written in the built-in programming language Vpascal. This script performs the following operations: (1) the background spectra are averaged and subtracted from the fluorescence spectrum, (2) a threshold is applied, (3) 8-bit auto-scaled intensity images are created by summation over the whole spectrum, (4) 8-bit RGB coloring is applied. Conversion of the spectral images to real color RGB images [7,8] is performed by multiplying the emission spectrum by the spectra for the red, green and blue sensitivities of the human eye (flat-field correction is included). The sums of these multiplied spectra give the R, G, and B values, respectively. The R/G/B values are scaled from 0 to 1 (the latter being the largest of R, G, and B) and multiplied by the intensity (8-bit) of the pixel. The emission below 400 nm (i.e. SHG) is made visible by adding the “false color” purple.
3. Results and discussion
The nonlinear spectral imaging system is applied to in vivo imaging of the lower inner arm skin of Caucasian and Asian volunteers. Accurate background subtraction is critical in the analysis of the (nonlinear) spectral images, and background correction procedures are frequently time-consuming and prone to artifacts. In previous work on spectral imaging of skin [7,8], the analysis time was dominated by accurate background correction. Here, we improve the speed and accuracy of this procedure by simultaneously recording the emission and background spectra. The raw data file of the image contains three spectra per pixel: one emission spectrum and two background spectra. In Fig. 2 , a raw spectral image of human skin is analyzed. The emission spectrum of the specimen is projected on row 2; here the autofluorescence of epidermal cells is observed. The image quality for row 2 is, however, compromised by the background. The background gradually increases from the top to the bottom of the image, as can be readily observed in the background images of row 1 and 3. Moreover, the background shows a (reproducible) periodic signal, which results in diagonal lines in the images. Using background subtraction (row2-0.5* [row1 + row3], see Fig. 2), these effects are corrected for. Also, the baseline level of the spectrum is now close to zero (was ~180, see spectra in Fig. 2) and the effect of background on RGB conversion is minimized (see RGB images in Fig. 2).
The above background correction procedure significantly improves the quality of the images and was applied to all the images shown here. A Z-stack of 10 XY images of in vivo Caucasian human skin is shown in Fig. 3 . In addition, an XZ image through the center of the stack is shown in the same figure (top left). The pixel dwell time was 128 µs per pixel, 6.5 seconds for a 224 × 224 pixel image and ~1 minute for the total stack. The excitation power was 15 mW, sufficiently low to avoid significant photodamage [25,26]. Various structures can be distinguished. A part of the Stratum Corneum, corresponding to a furrow, extends into the recorded stack and exhibits bright green autofluorescence that probably originates from clusters of keratin. Epidermal cells have predominantly blue/green emission because of NADH, keratin and flavins. Some cells are melanized, which results in additional yellow/green emission. In the dermis, violet second harmonic generation (SHG) of collagen fibers is observed, as well as blue/green autofluorescence of elastic fibers. Vessel-like structures with orange emission can also be distinguished in the papillary dermis (upper region of the dermis) and correspond to blood capillaries.
A major advantage of the high-resolution EMCCD based spectrograph is that full autofluorescence spectra of skin components can be recorded. This is demonstrated in Fig. 4 for Caucasian and Asian skin. The images were recorded at the junction between epidermis and dermis. The fluorescence spectra are averaged on four regions of interest (ROIs) that contain: (1) epidermal cells, (2) melanized epidermal cells, (3) collagen fibers and (4) elastin fibers. In the dermis, two major components can be distinguished: a second harmonic generation (SHG) band at 380 nm from collagen and autofluorescence that mainly originates from the closely located elastin fibers. Because of the overlap of the networks formed by these two major components of the dermis, there is some bleed-through in their spectra (Fig. 4). The spectra of the collagen fibers SHG and elastin fibers autofluorescence are the same for both volunteers. Our data indicate that the epidermal cells, however, contain a skin type dependent contribution of melanin. Already from the RGB images, it is clear that melanin contribution (yellow/green autofluorescence) is stronger in the Asian skin type. Since melanin is composed of heterogeneous oligomers of fluorophores, its emission spectrum strongly depends on experimental conditions (as shown for synthetic melanin by Teuchner et al [18,27]). Here, the spectra of melanin enriched cells exhibit an additional autofluorescence component at wavelengths longer than 550 nm. For the least melanized epidermal cells, the autofluorescence peaks at 480 nm, and the emission is a combination of NADH (450 nm), keratin (470 nm) and FAD (520 nm) autofluorescence. The spectra of nonmelanized epidermal cells of the two skin types are comparable. The melanized cells in Asian skin exhibit much stronger melanin autofluorescence than in Caucasian skin. For comparison, the spectrum of nonmelanized epidermal cells is added (gray line) to the spectrum in Fig. 4.
To confirm that the additional yellow emission is due to melanin, a series of images at the papillary epidermis/dermis interface were recorded (Fig. 5 ). Here, autofluorescence of melanin can be detected . For three volunteers with Caucasian skin, the same observation is made: clusters of cells exhibiting yellow or green emission are located around dermal papilla. The difference in apparent color is likely due to different levels of tanning; low melanin levels appear green in the RGB images due to the presence of the red melanin fluorescence and blue autofluorescence (NADH, keratin). An increasing level of melanization results in a red shift of the observed color from green to yellow.
Improvements in acquisition speed reduce the amount of detected signal. Due to the use of an EMCCD in our setup this is partly compensated. EMCCDs have a high sensitivity and low readout noise even at fast readout speeds. The sensitivity is much higher than systems equipped with multiplier tubes (PMTs) and it extends well into the red. The effect of signal level is exemplified in Fig. 5, where the signal-to-noise (S/N) ratio is varied by averaging the spectrum in each pixel by binning (N = 1, 9 (3 × 3) and 25 (5 × 5)). Note that the spectra are averaged, and that the resulting R, G, and B values are scaled and multiplied by the original (not-averaged) intensity value in the pixel. Spectral averaging does not visually improve the color contrast in the images. The spectrum recorded in 1 pixel (128 μs, ~15mW, image acquired at the epidermal-dermal junction) is sufficient for RGB conversion. Since RGB conversion only involves multiplication of spectra, it is fast. On a standard computer, an image of 224 × 224 pixels is converted in less than a second.
The real color representation of the spectral images can be used to discriminate between different structures in tissue. To investigate the sensitivity of the real color RGB representation to noise and its uniqueness for the structural features in skin, ternary plots of the RGB values are made (see Fig. 6 ). These ternary plots display the relative contribution of the R, G, and B values for selected pixels. Five ROIs are selected: epidermal cells (EC), partly melanized EC, melanized EC, collagen and elastic fibers. The values of the pixels in these ROIs are overlaid in one plot. The ternary representation of unbinned pixels is not sufficient to uniquely identify components. By averaging over N = 9 and N = 25, however, clearly non-overlapping clouds of pixels are observed for most of the ROIs. The RGB values of the pixels containing elastic fibers are, however, very similar to the ones that represent partly melanized epidermal cells. This is not surprising since the spectra are very similar (see Fig. 4). Since they are located in a different layer of the skin, it is nevertheless easy to separate them.
In conclusion, the fast nonlinear spectral imaging method presented here provides a powerful means to record images of in vivo human skin. Due to the use of an EMCCD based spectrograph, image acquisition time is significantly reduced and sensitivity is increased compared to systems equipped with regular CCD cameras. Importantly, this system affords real time correction for background by the simultaneous recording of background spectra. In combination with real color visualization it provides a rapid and photon efficient way to obtain color contrasted images that can be employed to readily identify regions of interest in the specimen. Further analyses of such regions can be carried out by utilizing the full wavelength resolution of the spectrograph. Here, clear differences it is shown that different emission spectra are obtained for the epidermal cells with different levels of melanin. Moreover, the ternary representation of R, G, and B values of their spectra allows for the discrimination between the different epidermal cells: melanized, partly melanized and nonmelanized cells. Future work will be devoted to spectral unmixing of the individual spectral components in the skin. This is of special interest for studies on e.g. the distribution of melanin between the different skin phototypes.
References and links
1. C. Xu, W. Zipfel, J. B. Shear, R. M. Williams, and W. W. Webb, “Multiphoton fluorescence excitation: new spectral windows for biological nonlinear microscopy,” Proc. Natl. Acad. Sci. U.S.A. 93(20), 10763–10768 (1996). [CrossRef] [PubMed]
4. K. Koenig and I. Riemann, “High-resolution multiphoton tomography of human skin with subcellular spatial resolution and picosecond time resolution,” J. Biomed. Opt. 8(3), 432–439 (2003). [CrossRef] [PubMed]
5. W. R. Zipfel, R. M. Williams, R. Christie, A. Y. Nikitin, B. T. Hyman, and W. W. Webb, “Live tissue intrinsic emission microscopy using multiphoton-excited native fluorescence and second harmonic generation,” Proc. Natl. Acad. Sci. U.S.A. 100(12), 7075–7080 (2003). [CrossRef] [PubMed]
7. J. A. Palero, H. S. de Bruijn, A. van der Ploeg-van den Heuvel, H. J. C. M. Sterenborg, and H. C. Gerritsen, “In vivo nonlinear spectral imaging in mouse skin,” Opt. Express 14(10), 4395–4402 (2006). [CrossRef] [PubMed]
8. J. A. Palero, H. S. de Bruijn, A. van der Ploeg van den Heuvel, H. J. C. M. Sterenborg, and H. C. Gerritsen, “Spectrally resolved multiphoton imaging of in vivo and excised mouse skin tissues,” Biophys. J. 93(3), 992–1007 (2007). [CrossRef] [PubMed]
9. A. M. Pena, A. Fabre, D. Débarre, J. Marchal-Somme, B. Crestani, J. L. Martin, E. Beaurepaire, and M. C. Schanne-Klein, “Three-dimensional investigation and scoring of extracellular matrix remodeling during lung fibrosis using multiphoton microscopy,” Microsc. Res. Tech. 70(2), 162–170 (2007). [CrossRef] [PubMed]
10. M. Strupler, A. M. Pena, M. Hernest, P. L. Tharaux, J. L. Martin, E. Beaurepaire, and M. C. Schanne-Klein, “Second harmonic imaging and scoring of collagen in fibrotic tissues,” Opt. Express 15(7), 4054–4065 (2007). [CrossRef] [PubMed]
11. M. Strupler, M. Hernest, C. Fligny, J. L. Martin, P. L. Tharaux, and M. C. Schanne-Klein, “Second harmonic microscopy to quantify renal interstitial fibrosis and arterial remodeling,” J. Biomed. Opt. 13(5), 054041 (2008). [CrossRef] [PubMed]
13. D. W. Piston, B. R. Masters, and W. W. Webb, “Three-dimensionally resolved NAD(P)H cellular metabolic redox imaging of the in situ cornea with two-photon excitation laser scanning microscopy,” J. Microsc. 178(Pt 1), 20–27 (1995). [PubMed]
14. A. M. Pena, M. Strupler, T. Boulesteix, and M. C. Schanne-Klein, “Spectroscopic analysis of keratin endogenous signal for skin multiphoton microscopy,” Opt. Express 13(16), 6268–6274 (2005). [CrossRef] [PubMed]
15. N. Ramanujam, “Fluorescence spectroscopy in vivo,” in Encyclopedia of Analytical Chemistry (Wiley, 2000), pp. 20–56.
17. E. Dimitrow, I. Riemann, A. Ehlers, M. J. Koehler, J. Norgauer, P. Elsner, K. König, and M. Kaatz, “Spectral fluorescence lifetime detection and selective melanin imaging by multiphoton laser tomography for melanoma diagnosis,” Exp. Dermatol. 18(6), 509–515 (2009). [CrossRef] [PubMed]
18. K. Teuchner, W. Freyer, D. Leupold, A. Volkmer, D. J. S. Birch, P. Altmeyer, M. Stücker, and K. Hoffmann, “Femtosecond two-photon excited fluorescence of melanin,” Photochem. Photobiol. 70(2), 146–151 (1999). [PubMed]
19. G. Cox, E. Kable, A. Jones, I. Fraser, F. Manconi, and M. D. Gorrell, “3-dimensional imaging of collagen using second harmonic generation,” J. Struct. Biol. 141(1), 53–62 (2003). [CrossRef] [PubMed]
20. J. A. Palero, G. Latouche, H. S. de Bruijn, A. van der Ploeg van den Heuvel, H. J. Sterenborg, and H. C. Gerritsen, “Design and implementation of a sensitive high-resolution nonlinear spectral imaging microscope,” J. Biomed. Opt. 13(4), 044019 (2008). [CrossRef] [PubMed]
21. A. J. Radosevich, M. B. Bouchard, S. A. Burgess, B. R. Chen, and E. M. C. Hillman, “Hyperspectral in vivo two-photon microscopy of intrinsic contrast,” Opt. Lett. 33(18), 2164–2166 (2008). [CrossRef] [PubMed]
22. J. Sytsma, J. M. Vroom, C. J. De Grauw, and H. C. Gerritsen, “Time-gated fluorescence lifetime imaging and microvolume spectroscopy using two-photon excitation,” J. Microsc. 191(1), 39–51 (1998). [CrossRef]
23. L. H. Laiho, S. Pelet, T. M. Hancewicz, P. D. Kaplan, and P. T. So, “Two-photon 3-D mapping of ex vivo human skin endogenous fluorescence species based on fluorescence emission spectra,” J. Biomed. Opt. 10(2), 024016 (2005). [CrossRef] [PubMed]
24. V. E. Centonze and J. G. White, “Multiphoton excitation provides optical sections from deeper within scattering specimens than confocal imaging,” Biophys. J. 75(4), 2015–2024 (1998). [CrossRef] [PubMed]
25. F. Fischer, B. Volkmer, S. Puschmann, R. Greinert, W. Breitbart, J. Kiefer, and R. Wepf, “Risk estimation of skin damage due to ultrashort pulsed, focused near-infrared laser irradiation at 800 nm,” J. Biomed. Opt. 13(4), 041320 (2008). [CrossRef] [PubMed]
27. K. Teuchner, J. Ehlert, W. Freyer, D. Leupold, P. Altmeyer, M. Stücker, and K. Hoffmann, “Fluorescence studies of melanin by stepwise two-photon femtosecond laser excitation,” J. Fluoresc. 10(3), 275–281 (2000). [CrossRef]
28. K. Sugata, S. Sakai, N. Noriaki, O. Osanai, T. Kitahara, and Y. Takema, “Imaging of melanin distribution using multiphoton autofluorescence decay curves,” Skin Res. Technol. 16(1), 55–59 (2010). [CrossRef] [PubMed]