We have developed a low-cost, near-infrared (NIR) reflectance confocal microscope (RCM) to overcome challenges in the imaging depth and speed found in our previously-reported smartphone confocal microscope. In the new NIR RCM device, we have used 840 nm superluminescent LED (sLED) to increase the tissue imaging depth and speed. A new confocal detection optics has been developed to maintain high lateral resolution even when a relatively large slit width was used. The material cost of the NIR RCM device was still low, ~$5,200. The lateral resolution was 1.1 µm and 1.3 µm along the vertical and horizontal directions, respectively. Axial resolution was measured as 11.2 µm. In vivo confocal images of human forearm skin obtained at the imaging speed of 203 frames/sec clearly visualized characteristic epidermal and dermal cellular features of the human skin.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Skin biopsy histopathologic evaluation is the standard method for making a diagnostic assessment of most dermatological conditions. However, diagnosis often relies on clinical examination alone in remote and/or resource-scarce sites, which can lead to incorrect or delayed diagnosis [1,2] and inadequate treatment. Recently, smartphone-based microscopy devices have been developed with a goal of providing microscopy images at the point of care and subsequently improving the disease diagnosis in low-resource or distant settings. However, most of the smartphone-based microscopy devices are tailored for imaging excised and thinly-sectioned samples [3–7]. The sample acquisition and slide preparation remains challenging in these settings due to the lack of required equipment and trained personnel.
Reflectance confocal microscopy (RCM) is an in vivo microscopy technology that can examine cellular features of the skin without having to invasively sample the suspicious lesions . RCM has been evaluated for the diagnosis of various skin diseases and shown to provide high diagnostic accuracy for major skin cancers in developed countries [9,10]. Recently, RCM has been also tested for imaging skin diseases prevalent in low-resource settings such as Kaposi’s Sarcoma and Xeroderma Pigmentosum [11,12]. Clinical adaptation of RCM in low-resource or remote settings, however, has not been realized yet due mainly to the relatively high cost associated with the device.
We had previously developed a low-cost, smartphone-based confocal microscope and demonstrated human skin imaging in vivo . The smartphone confocal microscope utilized slit confocal apertures, a broadband LED, and diffraction gratings to image multiple lines of the tissue simultaneously with each line associated with a distinctive wavelength . As a result, two-dimensional confocal images were obtained without using any beam scanning devices and the confocal microscope was built at low cost (material cost = $4,200). While the smartphone confocal microscope successfully visualized cellular features of the skin, there were remaining technological challenges. Use of a relatively short wavelength (central wavelength = 590 nm) made it challenging to clearly image cellular features in the dermis. The color filter on the smartphone CMOS sensor inherently reduced the light collection efficiency, which in part limited the imaging speed to 4.3 frames/sec. The suboptimal imaging speed could lead to motion artifacts during in vivo skin imaging.
In this paper, we report the development of a low-cost, near-infrared (NIR) confocal microscope to address the aforementioned challenges. A new confocal detection optics is described and theoretical resolution simulation reported. Preliminary in vivo confocal images of the human forearm skin are presented.
2. Materials and methods
2.1 Low-cost near-infrared reflectance confocal microscope
The schematic of the low-cost confocal microscope is shown in Fig. 1. Light from a super-luminescent light emitting diode (sLED; EXS210040-01, Exalos; central wavelength = 840 nm; bandwidth = 50 nm; power = 10.5 mW; working current = 150 mA) was collimated by an aspheric singlet (f = 8 mm). The collimated light was then diffracted by a diffraction grating (grating 1; groove density = 1764.7 lpmm), focused by a cylindrical lens (focal length = 30 mm) and an objective lens (CFI60 Apochromat 40x, Nikon; NA = 0.8; water immersion) on the tissue. Use of the cylindrical lens generated a focused line on the tissue for each wavelength. A square aperture with the width of 2.8 mm and offset of 2.6 mm from the objective lens optical axis was used between the grating 1 and cylindrical lens, which resulted in the effective illumination NA of 0.28 and chief ray angle on the tissue of 21.4°.
Reflected light from the tissue was collected by the objective lens and collimated. A circular aperture with a diameter of 3.5 mm and offset of 2.25 mm from the objective lens optical axis was used near the proximal end of the objective lens. The circular aperture made the effective detection aperture 0.35 and chief ray angle 18.7°. Light after the circular aperture was diffracted by another grating (grating 2; 1764.7 lpmm) and focused by a focusing lens (f = 30 mm). A detection slit with the width of 50 µm and length of 3 mm was positioned at the focal plane of the focusing lens. After the detection slit, the light was collimated by the collimation lens (f = 30 mm), diffracted by the grating 3 (1800 lpmm), and focused on the CMOS sensor (1280 × 1024 pixels; pixel size = 4.8 µm). A USB 3.0 cable was used to transfer the data from the CMOS sensor to a laptop (Surface Book Pro, Microsoft).
2.2 Confocal detection optics
We have developed a new confocal detection optics that provides better lateral resolution than the detection optics used in our previous smartphone confocal microscope. In RCM, a relatively large detection aperture is often used to reduce the speckle noise . The sLED has a wide spectral bandwidth and therefore has a reduced speckle noise when the entire bandwidth is focused on the same point. In our confocal microscope, however, the effective spectral bandwidth that each pixel is detecting is small, which increases the coherence length and subsequent speckle noise. Therefore, a wide detection slit width is still needed even when the sLED with wide bandwidth is used as the source.
With the detection optics of the previous smartphone confocal microscope , an increase of the detection slit width directly degraded the lateral resolution. Comparison between the previous and present detection optics is shown in Fig. 2. The light beam color in Fig. 2 does not represent the actual light color of the light source but is indicative of the relative wavelength for the given beam path, red being a longer wavelength and blue being a shorter wavelength. The separation between the focused beams on the detection slit in Fig. 2 is exaggerated to better demonstrate the difference between the previous and present confocal detection optics. For an example detection slit width of 50 µm, which is also the separation between the blue and red beams, the corresponding spectral bandwidth between the red and blue beams was 0.6 nm.
Illumination optics is identical between the previous and present configurations. In the previous smartphone confocal microscope (Fig. 2(a)) , the grating in front of the camera lens (grating 3) was parallel to the grating in the detection path of the confocal optics (grating 2). When the detection light incident on a particular pixel on the CMOS sensor is backtraced to the tissue, the longer wavelength (red rays) passes through the top edge of the detection slit. The longer wavelength is then incident on the grating 2 with a smaller angle than the central wavelength (green rays), which results in a larger field angle between the grating 2 and objective. This larger angle makes the detection point-spread function (PSF) left-shifted (red detection PSF). Likewise, the shorter wavelength (blue) can be back-traced to a right-shifted detection PSF (blue detection PSF). In summary, the detection pixel is not conjugate to a single point on the tissue but to multiple, laterally-shifted points. Confocal PSF for each pixel can be calculated as the sum of confocal PSF’s for all the wavelengths detected by that pixel. The confocal PSF for each wavelength is calculated by multiplying the illumination PSF with the detection PSF for the given wavelength. Therefore, in the previous detection optics configuration (Fig. 2(a)), the confocal PSF (dotted ellipse) becomes wider along the lateral direction as the detection slit width increases. Since confocal images are usually taken en face, resolution degradation along the lateral dimension deteriorates the image quality more severely than along the axial dimension.
In the present detection optics configuration (Fig. 2(b)), the grating 3 and grating 2 are mirror-symmetric relative to the detection slit. Due to this mirror symmetry, all the wavelengths detected by a particular pixel is back-traced to have the same field angle on the objective lens. Therefore, each pixel is conjugate to a single point on the tissue and produces a detection PSF without any lateral shift (white ellipse). The slit width determines the effective bandwidth each pixel is detecting. The confocal PSF for each pixel can then be calculated by summing the products between the detection PSF (white ellipse) and the illumination PSFs that correspond to the effective bandwidth (e.g., red, green, and blue illumination PSFs in Fig. 2(b)). As shown in Fig. 2(b), the lateral extent of the confocal PSF (dotted ellipse) is mainly determined by the lateral extent of the detection PSF and remains small even when the slit width is increased. The slit width increase, however, stretches the axial extent of the confocal PSF and degrades the axial resolution.
2.3 Resolution simulation
We have simulated the confocal PSFs for various slit widths for both previous and present detection optics (Fig. 3) using a custom Matlab code (Mathworks, Natick, MA). The simulation parameters were set according to the specifications of the components as described in section 2.1. The FWHM of the focused spot on the detection slit was 7.3 µm with the given detection beam diameter of 3.5 mm and focusing lens focal length of 30 mm. For the present detection optics, the resolution was calculated both along the lateral and axial directions of the tissue space (FWHMx and FWHMz) and along the minor and major axes of the confocal PSF (FHWMu and FWHMv). With the previous detection optics, the lateral resolution degraded significantly from 2.0 µm to 8.52 µm as the slit width increased from 10 µm to 50 µm, while the axial resolution did not change much. With the present detection optics, the lateral resolution (FWHMu) was maintained high, around 1 µm, while the axial resolution was increased significantly from 3.40 µm to 11.26 µm. While smaller axial FHWM is desirable for confocal microscopy, axial resolution of 10-15 µm was shown useable to visualize cellular features of human tissues [16,17] as long as the lateral FHWM is maintained small.
2.4 Imaging performance test
Lateral resolution of the low-cost NIR confocal microscope was measured by imaging a USAF resolution target. FWHM of the line spread function (LSF) was calculated along the spectrally-encoded and slit-length directions. Axial resolution was measured by translating a mirror along the objective lens optical axis with a motorized stage and calculating the FWHM of the axial response curve. The source power was attenuated during the resolution measurement to ensure that the pixel values were not saturated.
Tissue imaging performance was evaluated by imaging human forearm in vivo at different imaging depth levels. The forearm skin surface was placed parallel to the focal plane of the objective lens. Ultrasound gel with a similar refractive index to that of water was applied between the forearm and objective lens. The exposure time was set at 4.8 msec and the resulting frame rate was 203 fps. The microscope was translated relative to the forearm using a motorized stage. The motor speed was set to 1 mm/sec and the scan range 500 µm. The maximum acceleration of the motor was 4 mm/sec2, which produced the acceleration time of 0.25 sec and deceleration time 0.25 sec. At the center of the axial scanning, the uniform speed of 1 mm/sec was maintained over 250 µm range. Within the uniform speed region, the axial step size between frames was 5 µm. The skin surface was located at the beginning of the uniform velocity region. A bi-directional axial scan was conducted. The resulting 3D volume acquisition rate was 1.33 volumes/sec.
Images were saved as an AVI file using a custom LabVIEW code (National Instruments, Austin, TX). At the end of each axial scanning, the confocal FOV was manually moved to a new imaging location and the axial scanning was conducted at the new imaging location. After image acquisition, the AVI file was segmented into multiple image stacks with each stack representing one axial scan. The image stacks were analyzed in ImageJ . The background intensity level was measured and subtracted. 3D rending of the image stacks was conducted using 3D Slicer . The speckle noise contrast was calculated by analyzing dermis images and dividing the standard deviation of the intensity values by the mean value at four 100 × 100-pixel regions that exhibited grossly uniform reflectivity without observable cellular features. The speckle noise contrast was measured at three different imaging depth levels.
A photograph of the low-cost confocal microscope is shown in Fig. 4. The confocal microscope had a dimension of 15 cm (W) × 16 cm (H) × 4.5 cm (D), and the weight was 0.57 kg. The material cost for the confocal microscope was $5,188. The optical power on the specimen was 2.2 mW.
A confocal image of the USAF resolution target is shown in Fig. 5(a). The smallest line pattern in group 9, element 3 (linewidth = 0.78 µm) was clearly distinguished along the slit length direction. The line pattern in group 9, element 2 (linewidth = 0.87 µm) was well resolved along the spectrally encoded direction. Lateral resolutions were measured as 1.05 ± 0.05 µm and 1.31 ± 0.06 µm along the slit length and spectrally encoded directions, respectively. The axial response curve is shown in Fig. 3(b). The axial FWHM was measured as 11.24 ± 0.13 µm. Both measured lateral and axial resolutions were in good agreement with the theoretically expected values shown in Fig. 3.
Figure 6 shows representative confocal images of human forearm skin in vivo. Each image had an image size of 678 µm × 543 µm. The image taken at the depth of 25 µm (Fig. 6(a)) shows high reflectivity of the stratum corneum. The image at the 60 µm depth (Fig. 6(b)) visualizes keratinocytes with dark cytoplasm and bright cell borders (arrows). At a deeper imaging depth of 80 µm (Fig. 6(c)), the keratinocytes have a smaller cell size. Melanocytes or melanin-containing basal cells are visualized as bright dots (arrowheads) and dermal papillae (yellow asterisk) as dark openings at this depth. As the imaging depth is increased to 125 µm (Fig. 6(d)), the bright cells are distributed more towards the center of the FOV (dotted region), which is also observed in the video of the skin confocal images (Visualization 1). More dermal papillae are visualized at this depth. At a larger imaging depth of 195 µm (Fig. 6(e)), the fiber network in dermis generates bright signals while blood vessels are visualized as dark areas (white asterisks). Finally, at the imaging depth of 265 µm (Fig. 6(f)), the fiber network and blood vessels are visualized with reduced contrast. The speckle noise contrast was measured as 0.28, 0.26, and 0.24 for the imaging depth of 125 µm, 195 µm, and 265 µm, respectively.
Visualization 1 shows a video of the sequential 2D en face confocal images from the first axial scan and a video of 3D rendering of 10 consecutive axial scans. The movie of the 2D confocal images is played at a 25 times slower speed than the image acquisition speed. The 3D rendering movie shows the possibility of rapidly examining multiple tissue lesions in 3D without having to identify the depth of the dermal-epidermal junction (DEJ).
In this paper, we have demonstrated that high-speed imaging of human skin in vivo with the low-cost NIR confocal microscope is feasible. The confocal images visualized characteristic cellular features of the skin. Our new confocal microscope achieved a fast image acquisition rate of 203 frames/sec, approximately 50 times faster than our previous smartphone confocal microscope and 20 times faster than commercial confocal microscope devices. The high imaging speed can be used to conduct real-time 3D confocal imaging of a region of interest or large-area imaging of the entire skin lesion within a short procedural time. We also expect that the low cost of the device will facilitate a wide adaption of the device in various clinical settings.
There were several remaining technological challenges found during the preliminary testing. Even though a relatively wide slit was used, the speckle noise was still prominent in confocal images, which hindered the image interpretation. Use of the wide slit degraded the axial resolution. In the future development, we will address these two issues by using a high-power LED, which has a significantly reduced spatial coherence and therefore allows for use of a narrow slit width. The volumetric imaging rate was limited to 1.33 volumes/sec mainly due to the acceleration and deceleration of the axial scanning stage. A piezoelectric transducer (PZT)-based scanner can be used to achieve higher volumetric imaging rate. In the new confocal detection optics, the CMOS sensor is located on the same side as the tissue, which will make it challenging to image certain anatomical locations such as back or face. A fold mirror can be used between the grating and camera lens to move the CMOS sensor away from the tissue and allow for imaging of a wider range of skin locations. In the future study of imaging suspicious skin lesions, we will evaluate the image quality of our microscope in comparison with the commercial confocal microscope and evaluate feasibilities of large-area scanning and real-time 3D imaging.
National Institutes of Health/Fogarty International Center (R21TW010221).
This research was supported in part by the National Institutes of Health (R21TW010221).
DK and CG are inventors on a provisional patent application filed by the University of Arizona on the technology presented.
1. M. O. Laker-Oketta, M. Wenger, A. Semeere, B. Castelnuovo, A. Kambugu, R. Lukande, F. C. Asirwa, N. Busakhala, N. Buziba, L. Diero, K. Wools-Kaloustian, R. M. Strother, M. Bwana, W. Muyindike, E. Amerson, E. Mbidde, T. Maurer, and J. Martin, “Task Shifting and Skin Punch for the Histologic Diagnosis of Kaposi’s Sarcoma in Sub-Saharan Africa: A Public Health Solution to a Public Health Problem,” Oncol. (2015).
2. E. Amerson, N. Buziba, H. Wabinga, M. Wenger, M. Bwana, W. Muyindike, C. Kyakwera, M. Laker, E. Mbidde, C. Yiannoutsos, K. Wools-Kaloustian, B. Musick, P. LeBoit, T. McCalmont, B. Ruben, P. Volberding, T. Maurer, and J. Martin, “Diagnosing Kaposi’s Sarcoma (KS) in East Africa: How accurate are clinicians and pathologists?” 13th Int. Conf. Malig. AIDS Other Acquir. Immunodefic. ICMAOI 2011 Bethesda, MD United States (2012). [CrossRef]
3. H. Im, C. M. Castro, H. Shao, M. Liong, J. Song, D. Pathania, L. Fexon, C. Min, M. Avila-Wallace, O. Zurkiya, J. Rho, B. Magaoay, R. H. Tambouret, M. Pivovarov, R. Weissleder, and H. Lee, “Digital diffraction analysis enables low-cost molecular diagnostics on a smartphone,” Proc. Natl. Acad. Sci. U.S.A. 112(18), 5613–5618 (2015). [CrossRef] [PubMed]
5. G. L. Damhorst, C. Duarte-Guevara, W. Chen, T. Ghonge, B. T. Cunningham, and R. Bashir, “Smartphone-Imaged HIV-1 Reverse-Transcription Loop-Mediated Isothermal Amplification (RT-LAMP) on a Chip from Whole Blood,” Engineering (Beijing) 1(3), 324–335 (2015). [CrossRef] [PubMed]
7. M. Kühnemund, Q. Wei, E. Darai, Y. Wang, I. Hernández-Neuta, Z. Yang, D. Tseng, A. Ahlford, L. Mathot, T. Sjöblom, A. Ozcan, and M. Nilsson, “Targeted DNA sequencing and in situ mutation analysis using mobile phone microscopy,” Nat. Commun. 8(1), 13913 (2017). [CrossRef] [PubMed]
8. M. Rajadhyaksha, A. Marghoob, A. Rossi, A. C. Halpern, and K. S. Nehal, “Reflectance confocal microscopy of skin in vivo: From bench to bedside,” Lasers Surg. Med. 49(1), 7–19 (2017). [CrossRef] [PubMed]
10. G. Pellacani, P. Guitera, C. Longo, M. Avramidis, S. Seidenari, and S. Menzies, “The impact of in vivo reflectance confocal microscopy for the diagnostic accuracy of melanoma and equivocal melanocytic lesions,” J. Invest. Dermatol. 127(12), 2759–2765 (2007). [CrossRef] [PubMed]
11. T. C. Grazziotin, C. Cota, R. B. Buffon, L. Araújo Pinto, A. Latini, and M. Ardigò, “Preliminary evaluation of in vivo reflectance confocal microscopy features of Kaposi’s sarcoma,” Dermatology (Basel) 220(4), 346–354 (2010). [CrossRef] [PubMed]
12. S. Segura, S. Puig, C. Carrera, M. Lecha, V. Borges, and J. Malvehy, “Non-invasive management of non-melanoma skin cancer in patients with cancer predisposition genodermatosis: A role for confocal microscopy and photodynamic therapy,” J. Eur. Acad. Dermatol. Venereol. 25(7), 819–827 (2011). [CrossRef] [PubMed]
13. E. E. Freeman, A. Semeere, H. Osman, G. Peterson, M. Rajadhyaksha, S. González, J. N. Martin, R. R. Anderson, G. J. Tearney, and D. Kang, “Smartphone confocal microscopy for imaging cellular structures in human skin in vivo,” Biomed. Opt. Express 9(4), 1906–1915 (2018). [CrossRef] [PubMed]
15. C. Glazowski and M. Rajadhyaksha, “Optimal detection pinhole for lowering speckle noise while maintaining adequate optical sectioning in confocal reflectance microscopes,” J. Biomed. Opt. 17(8), 085001 (2012). [CrossRef] [PubMed]
16. D. Kang, M. J. Suter, C. Boudoux, H. Yoo, P. S. Yachimski, W. P. Puricelli, N. S. Nishioka, M. Mino-Kenudson, G. Y. Lauwers, B. E. Bouma, and G. J. Tearney, “Comprehensive imaging of gastroesophageal biopsy samples by spectrally encoded confocal microscopy,” Gastrointest. Endosc. 71(1), 35–43 (2010). [CrossRef] [PubMed]
17. D. Kang, S. C. Schlachter, R. W. Carruth, M. Kim, T. Wu, N. Tabatabaei, A. R. Soomro, C. N. Grant, M. Rosenberg, N. S. Nishioka, and G. J. Tearney, “Large-area spectrally encoded confocal endomicroscopy of the human esophagus in vivo,” Lasers Surg. Med. 49(3), 233–239 (2017). [CrossRef] [PubMed]
18. C. T. Rueden, J. Schindelin, M. C. Hiner, B. E. DeZonia, A. E. Walter, E. T. Arena, and K. W. Eliceiri, “ImageJ2: ImageJ for the next generation of scientific image data,” BMC Bioinformatics 18(1), 529 (2017). [CrossRef] [PubMed]
19. “3D Slicer,” https://www.slicer.org.