1. Introduction

Endoscopic surgery remains the most effective treatment option for gastrointestinal (GI) tract cancer. Conventional endoscopy procedures alone cannot provide histopathologic information; unnecessary biopsy or the removal of suspected lesions using surgical tools is sometimes required for diagnosis, leading to multiple patient visits for additional biopsies or examinations [1].

Neoplastic lesions often overexpress specific enzymes or receptors in transformed mucosa. These differences between healthy and abnormal tissue allow for an effective screening method using fluorescence imaging techniques. Proteases are potential candidates for molecular targets because proteolytic enzymes play an important role in cell proliferation, invasion, angiogenesis, and metastasis. These enzymes provide important means for the detection and diagnosis of cancer in the GI tract, including colon cancer [2,3]. Choosing protein targets with orthogonal tumorigenesis pathways may increase the sensitivity of tumor detection. The molecular targets for GI tract cancer are epidermal growth factor receptor, human epidermal growth factor receptor 2 (HER2), and phosphoinositide 3-kinase (PI3 K) [2,4,5]. These molecular target-based fluorescent probes have the disadvantage of dimming over time. However, fluorescently labeled, bacteria-based molecular probes show increasing fluorescence intensity over time.

The presence of bacteria and viruses in human tumors has been recognized for more than 50 years. Significant amounts of bacteria as well as viral particles have been found in tumors excised from patients. To demonstrate the survival of bacteria in tumors, spores of the obligate anaerobe Clostridium pasteurianum were injected intravenously into tumor-bearing mice and found to replicate in the hypoxic center of the tumor. Some researchers have shown that bacteria injected intravenously into live animals entered and replicated in solid tumors and metastases [6]. Bacteria was first actively used in the treatment of cancer, which ultimately led to the study of immunomodulation. In an animal tumor model, bacteria target and multiply selectively within tumors, thus amplifying intratumoral gene delivery and therapeutic effects [7]. They can effectively overcome the barrier of extravascular transport in solid tumors [8]. The tumor-specific amplification process was visualized in real time using luciferase-catalyzed luminescence and green fluorescent protein fluorescence, which revealed the locations of tumors and metastases.

To provide accurate assessments of tumor resection, advanced surgical navigation techniques based on fluorescence imaging have guided crucial decisions [9]. Many fluorescence imaging systems that use FDA-approved fluorescent agents have been used in clinical settings during first-in-human trials [10–12]. Fluorescence endoscopic imaging with a wide field of view is an emerging technology that can screen precancerous lesions in real time based on molecular signatures, when used in combination with clinical endoscopy [13,14]. Because fluorescence imaging can provide highly sensitive and real-time information of the surgical tissue area, it is an upcoming technique for the removal of lesion tissues. The fluorescence signal detected with these techniques arises either from endogenous or exogenous fluorophores. Several FDA-approved fluorescence imaging devices used in combination with fluorescein isothiocyanate (FITC), methylene blue, or indocyanine green (ICG) for visualization have been developed, including fluorescence-assisted resection and exploration (FLARE) and frequency domain photon migration imaging. Most fluorescence imaging devices use either a single image sensor to capture visible-light and fluorescence images sequentially or multiple cameras to image spectra sequentially or simultaneously.

Multicolor fluorescence imaging has been developed for improving cancer detection rates instead of using a targeted single fluorescence signal. Although multicolor fluorescence imaging techniques provide more detailed spectral information, the systems are usually complex. A multispectral imaging technique using a complementary metal–oxide–semiconductor (CMOS) sensor with a four-channel Bayer pattern was recently developed [15]. Although this multispectral imaging achieves only a low spectral resolution, it has high spatial sampling. This method can be used in applications that require only a few spectral bands in the visible and NIR wavelengths. Although multispectral fluorescence imaging techniques also provide more detailed spectral information, the systems are correspondingly more complex. The typical configuration includes one monochromatic camera with a filter wheel and two or more cameras with dichroic beam splitters. Some researchers have developed a multispectral imaging technique using an RGB-IR CMOS sensor with a four-channel Bayer pattern. A photodetector with both a charge-coupled device (CCD) sensor and CMOS has a wide spectral sensitivity range, from UV to NIR. NIR light often degrades the image quality of a sensor with the conventional red, green, and blue (RGB) color filter array (CFA). Therefore, a window with low-pass coating is often used to block NIR light. However, even when the window blocks the NIR signals, it is impossible to extract the NIR information from a conventional sensor with an RGB CFA. To obtain both visible and NIR light information from a single sensor, a new type of sensor with four-color filter arrays has been developed [16]. Crosstalk between each channel occurs in the sensor. Consequently, color separation algorithms will be needed to extract NIR and RGB information from a single image. Recently, an NIR fluorescence imaging device was developed for contrast-enhanced NIR fluorescence imaging using a mobile phone [17].

In the present study, we describe a low-cost and portable multiple fluorescence imaging system based on a single RGB-IR sensor that uses near-infrared (NIR) fluorescence imaging of 3D multicellular tumor spheroids with fluorescently labeled probiotics for cancer targeting. The home-built fluorescence imaging system was tested and evaluated for guided vessel-mapping on an ex vivo chicken breast tissue with acutely formed fluorescent vessel-mimicking in a pilot study.

2. Materials and methods

2.1 Single RGB-IR imaging system for multicolor fluorescence imaging

The imaging detection module comprises a 4-megapixel OV4682-based RGB-IR camera sensor (OmniVision, USA), coupled with an f/2.0 ML12 lens system (6020PL001, WITHROBOT, South Korea) and a notch optical filter (#67-111 and #67-108, Edmund Optics, USA) (Fig. 1(a)). Figure 1(a) shows the system configuration of the imaging device. The center wavelengths of the notch filter are 488 nm and 632 nm. The camera module has four color channels (B, G, R, and NIR).

 

Fig. 1. Configuration of RGB-IR fluorescence imaging system. (a) System scheme of imaging device. (b) Four-channel format of RGB-IR sensor. (c) Optical power for excitation light at a distance. (d) Representative RGB-WL image and fluorescence image. (e-g) The optical power for each excitation light at a distance.

Download Full Size | PDF

The filter for blocking was placed between the lens system and the camera, with the angle of incidence and cut-off wavelength carefully chosen to ensure appropriate fluorescence detection in the desired field of view. The RGB-IR CMOS sensor with a four-color filter array was used to capture the white-light color image and NIR image. The intensiometric measurements from the camera sensor were obtained at 10-bit depth. Also, the frame rate was 60 fps. The B, G, R, and NIR signals were extracted from the corresponding channels in the raw image.

The outer diameter of the device is 44 mm, including an excitation light source, detection module, and PCB sensor. The device provides a field of view of approximately 450 mm² in front of the specimen at a distance of 30 mm. The light source module consists of two white-light LEDs for color imaging, two 470 nm LEDs, two 630 nm LEDs, and two 730 nm LEDs for selective excitation. The illumination field is 80 mm ${\times} $ 80 mm, and the working distance is 50 mm. The averaged intensities of the excitation light sources are 2.80 mW/cm², 2.63 mW/cm², and 2.03 mW/cm² at a distance of 50 mm from the specimen (Fig. 1(c)). The optical intensity of each excitation light was measured using an optical power meter (PM100A, Thorlabs, USA) connected to a photodiode power sensor (S120C, Thorlabs, USA). The CMOS sensor PCB (e-con Systems, USA) was used for primary imaging acquisition. When the NIR excitation light is incident to ICG, emission light over 780 nm is detected by the IR sensor. However, IR light cannot be detected by an RGB sensor. We used Otsu threshold segmentation for enhancing the contrast in the fluorescence image. Figure 1(d) shows an overlaid ICG fluorescence image on a white-light color image.

2.2 RGB-IR imaging processing

Producing full-resolution RGB-IR imaging from the output of such a sensor requires inferring the three missing channels at every pixel. IR light signal above 650 nm acts as a high noise in the RGB sensor, and usually removes it using IR-cut filter. However, RGB-IR sensor is equipped with an RGB-IR filter array. Its quality is affected by both the RGB-IR sensor characteristics, such as a filter array pattern (FAP) and spectral sensitivity and the imaging pipeline converting the mosaic data to the output RGB and NIR images. In Fig. 2(d) and (e), the signal processing and imaging principle of the imager is explained. A unique pixel configuration of the imager in the camera system is the existence of an R + IR pixel, a G + IR pixel, a B + IR pixel, and an IR pixel [18]. The RGB-IR Bayer CFA is similar to the RGB Bayer CFA, but one of the two G datapoints is replaced by IR data in the former. (Figure 2(b))

 

Fig. 2. RGB-IR imaging processing. (a) RGB-IR sensor Bayer array. (b) RGB-IR sensor Bayer array. (c) RGB-IR sensor sensitivity. (d) RGB window imaging processing and (e) IR window imaging processing

Download Full Size | PDF

Image processing was performed separately for the RGB and IR screens.

RGB image processing was performed as follows:

The data in the RGB-IR layer CFA were color coded through the color correction matrix, excluding IR data. Thereafter, through the demosaicing process, IR data were placed next to the color-corrected G data. Additionally, the IR data were extracted via a Bayer demosaicing process. The size of the IR screen is the same as the RGB screen to facilitate further image processing.

2.3 System characterization

The spatial resolution of the system was determined using a USAF 1951 target (#38-710, Edmund Optics, USA) and measured using the slanted edge method. The test target was imaged at a distance of 50 mm, and the resolution was determined using a white-light image. The fluorescence imaging performance of our imaging system was characterized by FITC, Cy5, and ICG concentrations. First, the sensitivity of this system for the over 800 nm fluorescence channel was determined by light signals from samples of ICG diluted in deionized (DI) water at concentrations ranging from 65 nM to 64.5 µM. A significant proportion of ICG was bounded to 5 mg/mL albumin in deionized water to improve quantum yield. Albumin stabilizes and maintains the distance between the ICG molecules, and different albumin concentrations can affect the NIR signal intensity of ICG [19,20].

Second, the fluorescence signal of Cy5 diluted in dimethyl sulfoxide (DMSO) was measured at concentrations ranging from 130 nM to 1312 µM. Third, the signal of FITC diluted in DI water was measured at concentrations ranging from 100 nM to 1000 µM. The measurements were conducted at a distance of 50 mm from specimens with exposure times of 16 ms. We repeated the experiment thrice for the measurement of fluorescence sensitivity. All fluorophores were contained in black 96-well plates at a volume of 300 µL. For multiple fluorescence imaging, FITC and ICG were mixed in the same sample. When the 480-nm and 730-nm LEDs irradiated the fluorophores simultaneously, the fluorescence signals were detected in each channel (green and NIR) of a single RGB-IR CMOS camera module.

2.4 Fluorescence vessel-mimicking imaging on ex vivo chicken breast tissue

The technical feasibility of a single-camera fluorescence imaging system to visualize vessel morphology was studied ex vivo on chicken breast tissues (60 mm${\; } \times 17\; $mm ${\times} \; $3 mm). To simulate the blood vessel, a silicone tube ($\emptyset {\; }$2 mm) was inserted from the top of the tissue until it reached an insertion distance of 50 mm and an inserting depth of 15 mm. Solutions of each fluorophore (0.2 mL) (ICG 12.9 µM, Cy5 13.1 µM, FITC 20.0 µM, phosphate-buffered saline (PBS) control) were injected into the tubes to simulate blood flow. The fluorescence emissions of fluorophores in the tubes were acquired and projected back onto the ex vivo tissue. The integration time of the camera was 16 ms, and each fluorophore was injected into the ex vivo tissue. We also evaluated the imaging depth of the system in chicken breast tissues of varying thickness.

2.5 Tumor spheroids with fluorescent-labeled probiotics for cancer detection

2.5.1 Mammalian cell culture

Human colon cancer cells (HCT-116, ATCC CCL-247) were used for the experiments. The cells were seeded in a T-25 flask containing McCoy's 5a medium supplemented with 10% (v/v) fetal bovine serum and incubated at 37 °C with 5% CO₂. As cell confluence reached 80%, cells were lifted with 1 mL of 0.25% Trypsin-EDTA solution (ATCC, Manassas, USA) and diluted after cell density was estimated using a hemocytometer.

2.5.2 Tumor spheroid formation

A total of 15,000 cancer cells were suspended in 200 µL of complete culture medium and seeded in a 96-well, ultralow-attachment treated plate (Corning Inc., Corning, NY). The plates were centrifuged at 1000 × g for 10 min and incubated at 37 °C and 5% CO₂. When the diameter of tumor spheroids reached 1 mm, bacterial incubation experiments were conducted. The detailed procedure for tumor spheroid formation has been described previously [8].

2.5.3 Bacterial culture and incubation experiment

The probiotic Escherichia coli Nissle 1917 (Ardeypharm GmbH, Herdecke, Germany) with an integrated gene cassette coding for a red fluorescent protein (smURFP and HO-1) was grown at 37 °C and 100 rpm in lysogeny broth (LB; 1% w/v of tryptone, 1% w/v of NaCl, 0.5% w/v of yeast extract, pH 7.0) until the optical density at 600 nm reached 1.0. A 1-mL aliquot of the culture, corresponding to a 1.2 × 10⁹ colony of forming units (CFU), was then centrifuged at 1700 × g for 5 min at room temperature and diluted to 1.8 × 10⁸ CFU in 200 µL of complete culture media with the corresponding cancer cell (HT-29). The prepared tumor spheroids were incubated with the bacterial solution at 37 °C with 5% CO₂ on a vortex mixer at 500 rpm. After 12 h of incubation, the tumor spheroids were carefully rinsed with Dulbecco’s phosphate buffered saline and prepared for imaging.

3. Results

3.1 Optical characteristics

The achievable spatial resolution for the imaging device was characterized by a 1951 USAF resolution test target on the RGB channel (Fig. 3(a)). The limits of resolving power were determined by the largest group of element pairs where the spacing pattern was no longer discernible. The 1951 USAF test target showed that the imaging device can achieve spatial resolutions of 200 µm at 50 mm from the tip of the imaging device. Besides, we measured the fluorescence signal in a 0.5 mm silicon tube phantom with each fluorescent dye for full width at half-maximum (FWHM). The cross-section lines for diameters show FWHM. Thus, targets with widths less than the diameter of the cross-section were un-resolvable with the imaging system. FWHMs of approximately 238 µm in FITC and 262 µm in Cy5 were obtained (Fig. 3(e) and (f)). The NIR resolution of the ICG shows 340 µm, which is poorer than the resolution in the visible range.

 

Fig. 3. Characterization of the fluorescence imaging system with the RGB-IR sensor. (a) White-light image of USAF 1951 resolution test target. (b-d) The intensity changes of FITC, Cy5, and ICG at varying concentrations. (e-g) Evaluation of the spatial resolution of the imaging system (results from linear-scan) Normalized intensity profiles along lines as depicted in (a-c). The spatial resolution of the imaging system was determined by the FWHM of point spread function

Download Full Size | PDF

To provide insights into how the different fluorescence imaging options of the single-camera fluorescence imaging device relate to the detection sensitivity for the individual dyes (FITC, Cy5, ICG), a dilution series of the dyes was generated and measured. The results are depicted in Fig. 3(b)–(d). In the black 96-well plate setup, the minimum sensitivity for FITC was found to be 0.200 µM. For Cy5 and ICG, the sensitivities were 0.130 µM and 0.065 µM, respectively. NIR fluorescence imaging was very sensitive to small amounts of ICG in the NIR channel, and the signal was not linearly correlated, unlike those for the other fluorophores. The ICG dye has a quenching effect, which is a phenomenon where the signal decreases with increasing concentrations of ICG because closely crowded ICG molecules absorb the NIR signal from nearby ICG molecules. Therefore, the NIR signal of ICG is not linearly correlated to the amount of ICG [19,21]. Consequently, the NIR signal of ICG is the strongest at concentrations between 0.001– 0.05 mg/mL and decreases as the concentration changes. Conventional concentration quenching is a manifestation of the Forster nonradiative energy transfer phenomenon, and it is characterized by a decrease in the fluorescence quantum yield as the fluorophore concentration is increased [22]. However, the fluorescence intensity of FITC and Cy5 demonstrated a linear increment in this concentration [23].

3.2 Multiple fluorescence imaging with FITC and ICG

For multiple fluorescence imaging, we mixed FITC and ICG without Cy5 because of the limited ability to split the spectrum of the two fluorophores. The combined volume of the fluorophores was 300 µL on a black plate. Figure 4 shows the RGB images and each fluorescence image for FITC and ICG, simultaneously, with all LEDs turned on.

 

Fig. 4. Multiple fluorescence imaging with FITC, ICG, and PBS for control. Cyan: FITC, Yellow: ICG, Green: Mixture of FITC and ICG.

Download Full Size | PDF

The single RGB-IR sensor of the imaging device can acquire multiple fluorescence signals using the split channel. Through the overlay of FITC with a green pseudo color and the ICG with a red pseudo color, the yellow color showed a mixed distribution of the two fluorophores. In the PBS solution, the device acquired no signal from the G and IR channels. The RGB-IR sensor can detect multiple fluorescence signals with a split spectrum channel using two FDA-approved fluorophores and Cy5 for potential candidates. Cy5 has the potential to be a useful fluorophore for fluorescence-guided surgical interventions due to it’s adaptability to a variety of therapeutic ligands.

3.3 Fluorescence vessel-mimicking imaging on ex vivo chicken breast tissue

Figure 5 shows the RGB color, fluorescence, and overlay images for FITC, Cy5, and ICG on ex vivo chicken breast tissue, respectively. When acquiring images only the LED suitable for each fluorescence was used and all of the LEDs were not turned on simultaneously. In the below figure, we have shown each color combination (470 nm white light LED, 630 nm white light LED, and 730 nm white light LED) in the RGB image channel. When the 470 nm white light LED was used, the RGB image was violet in color. The ICG solution in PBS accumulated at the end of the tube and diffused into the surrounding tissue area to simulate blood flow. Fluorescence emission of ICG in the silicone tube was acquired and projected back onto the ex vivo chicken breast tissue. The images were acquired at an exposure time of 16 ms with real-time display acquisition at a frame rate of approximately 62.5 fps.

 

Fig. 5. Fluorescence imaging of a vessel-mimicking tube in ex vivo chicken breast tissue. FL images show the pseudo color with green. (a) Brief experimental configuration. (b-d) Fluorescence vessel-imaging with FITC, Cy5, and ICG.

Download Full Size | PDF

We evaluated the imaging depth of the system by embedding a 13.1 µM Cy5 solution in ex vivo chicken breast tissue. The average fluorescence signals of Cy5 and FITC in the region of interest (ROI) of blood vessel mimicking model were maximized on the surface. The value on the surface was set to 1. The relative intensity was calculated by comparing the average intensity in ROI and background intensity in an adjacent area at the same skin depth. The range of relative intensity was 0 to 1. This Fig. 5 shows how the fluorescence signal decreases with skin depth for confirming effective imaging depth. According to Fig. 6(a)-(b), the fluorescence signal remained marginally above the background at 4 to 6 mm. Figure 6(a) and Fig. 6(b) show the extracted RGB and FL images when the excitation light and white light were turned on. The fluorescence images are displayed in pseudo color, as shown in Fig. 6(a) and Fig. 6(b). The half-signal and highest signal at the surface using Cy5 and ICG were 4.0 mm and 4.9 mm, respectively (Fig. 6(d)).

 

Fig. 6. Fluorescence vessel-mimicking imaging on ex vivo chicken breast tissue with depth for Cy5 and ICG. (a-b) Each vessel-mimicking imaging; WL: white-light image (RGB image with all LEDs turned on), FL: fluorescence image. (c) Scheme of experimental configuration in ex vivo chicken breast tissue. (d) The graph of fluorescence intensity changes with tissue depth for Cy5 and ICG.

Download Full Size | PDF

3.4 Fluorescent-labeled probiotic imaging on 3D multicellular tumor spheroids

Figure 7 shows the targeted fluorescence imaging in 3D tumor spheroid. We observed the fluorescence signal of smURFP-labeled probiotics using an Axio Observer Z1 Inverted Microscope (Carl Zeiss, Germany) with 20× magnification, which is fluorescent and brightfield-equipped with differential interference contrast. Additionally, we evaluated the impact of fluorophore conjugation on cancer cell invasion in model 3D multicellular in vitro tumor spheroids (MCTS) derived from the HT-29 human colon cancer cell line. We chose in vitro 3D MCTS as the tumor model for an intratumoral transport assay because they have been shown to effectively recover essential structural function and mass transport properties of avascular tumor tissue [24,25]. To compare the fluorescence signals, HT-29 colon cancer cells (control), smURFP-labeled probiotics, Cy5-labeled probiotics with HT-29 colon cancer cells, and smURFP-labeled probiotics in 3D MCTS were used. The fluorescence images were acquired with an exposure time of 250 ms and $5 \times 5$ binning using ZEN software (Zeiss Microscopy, Germany). The smURFP conjugation was the observed invasion of cancer cells and intratumoral transport in 3D MCTS. The size of each 3D MCTS was approximately 900 µM.

The smURFP-labeled probiotics adhered well to the outside of the tumor spheroid but not to the inside of the 3D MCTS. Fluorescent light was strongly emitted by smURFP excited at a wavelength of 630 nm. Although we used conventional fluorescence microscopy, which cannot produce 3D images, it did provide 2D images. The spheroid fluorescence intensity is a difference of background signal in the blank wall plate and the average intensity in the specimens. Additionally, the signal was also detected in the HT-29 cells used as a control group, but we believe this was a reflection signal and not a fluorescence signal. The signal of smURFP-labeled probiotics in 3D MCTS was higher than that of smURFP-labeled probiotics in the HT-29 cell line.

4. Discussion

Early detection of structural or molecular changes in tumor tissues with fluorescence imaging may be crucial for improving survival rates. In this study, we introduce a customized fluorescence imaging system that consists of an RGB-IR sensor, three light sources, and add-on notch filters. The RGB-IR sensor can provide visible and NIR images simultaneously. However, a conventional RGB-IR sensor causes aliasing and has lower fluorescence sensitivity than multiple scientific CCD cameras. However, the RGB-IR sensors also have advantages of ease of use for multiple fluorescence imaging and application to chip-on-the-tip fluorescence endoscopy.

To enhance the system performance, Color correction is important when using single-sensor RGB-IR imaging. In conventional, current RGB sensors, the NIR-cut filter is placed in front of the sensor to avoid undesirable effects of spectral crosstalk [18]. However, the IR cut filter needs to be removed for single-sensor RGB-NIR imaging, resulting in color shifts of the acquired RGB image. The RGB image of the sensor was redder and less color-saturated than that of the conventional RGB sensor. The aim of color correction was to correct such color shifts and reproduce an image with the desired color representation.

A color correction process was developed to obtain accurate color information. The standard printed ColorChecker chart was used as the calibration target. The printed ColorChecker chart was designed to deliver true-to-life image reproduction so that photographers can predict and control how the color will appear under various illuminations. The color was considerably different from that in the standard RGB camera with the optimized color correction process. Therefore, color correction was required to obtain accurate color information from this sensor. White balance is one of the most critical functions necessary for cameras to achieve high-quality images. From the standard printed ColorChecker chart, we used six gray patches for white-balance correction and the simple global white-balance algorithm, also known as the gray world algorithm [26].

Using the home-built imaging device, we demonstrated the feasibility and potential of multiple fluorescence imaging using a single RGB-IR sensor imaging system with FITC, Cy5, and ICG dyes. The minimum sensitivities for FITC, Cy5, and ICG were 0.200 µM, 0.130 µM, and 0.065 µM, respectively, in black 96-well plates. When an identical photon flux was assumed” is controlling for constant illumination intensities to excite the fluorescence the penetration depth of fluorophores emitting in the visible spectrum was less when compared to the penetration depth of emission in the NIR window. ICG in-depth detection seemed to be better than that of Cy5 in-depth detection. The fact that Cy5 performs this well may be related to its superior brightness relative to ICG caused by a different quantum yield. Optical imaging in the NIR range holds considerable promise due to reduced light scattering by the tissue and increased penetration depth when compared to the visible range [27].

Thresholding is an effective and simple method for separating objects from the background for identification of only the target image; it also allows observation of changes in vessel morphology (Fig. 8). Otsu thresholding improves the image segmentation effect, which is a prevalent global automatic thresholding technique [28]. The imaging processing method is based on the principle that the gray level for which the between-class variance is maximum or within-class variance (${\sigma _B}$) is minimum is selected as the threshold [29,30].

 

Fig. 7. Targeted fluorescence imaging in 3D tumor spheroid. (a) Fluorescence microscope image of the cell, smURFP-labeled probiotics, and 3D tumor spheroid with smURFP-labeled probiotics. (b) Scheme of the experimental configuration. (c) Fluorescence endoscopy image of smURFP-labeled probiotics. (d) Comparison of fluorescence intensity of each group.

Download Full Size | PDF

 

Fig. 8. Pseudo fluorescence vessel diameter in skin depth. (a) Otsu thresholding of fluorescence image. (b) Data graph for each fluorophore (FITC, Cy5, and ICG) at skin depth (0–6 mm).

Download Full Size | PDF

The vessel morphologies in tumor tissue have different patterns compared to normal GI tract tissue. Fluorescence imaging can improve the visualization of the vascular pattern. A normal colonic presents a regular vessel pattern. However, colon inflammation causes thicker vessels and variable vascular density in the polyps [31]. We have tested effective fluorescence imaging depth to show ∼ 4 mm deeper vessel morphologies in a subdermal blood vessel phantom with Cy5 and ICG.

To study tissue penetration, the experiment was performed with silicone tubes containing individual fluorophores as a substitute for distal veins and arteries in ex vivo chicken breast tissue. The quantum yield is kwon for the fluorophores (FITC ∼ 0.90, Cy5 ∼ 0.2 and ICG ∼ 0.13) [32,33]. FITC emitted the highest number of photons/s. Although the quantum yield of ICG is relative low, we used high-power NIR light source for excitation of ICG molecules to acquire similar fluorescence intensity. The difference in penetration depth, which could be visibly assessed on-screen, was exemplified. The highest tissue penetration was obtained with Cy5 to a skin depth of 4 mm. The fluorescence of ICG has been reported to be visible through approximately 0.5 to 1.0 cm of soft tissue [34,35]. In ex vivo chicken tissue, the maximum penetration of NIR fluorescence imaging with ICG is ∼ 6 mm. The tissue penetration of FITC was restricted to a mere 2 mm in the ex vivo chicken breast tissue phantom. After comparing the diameter change of the fluorescence vessel with Otsu thresholding, the emitted NIR signal penetrated deep tissue because of low scattering and tissue absorption. When light penetrates ex vivo tissue, it is attenuated depending on the light wavelength [36]. However, tissue penetration with ICG can be restricted in vivo because of tissue autofluorescence. Our preliminary ex vivo tissue study suggests that Cy5 and ICG are good candidates for deep tissue imaging for lesion detection because of the reduced scattering effect of FITC. Besides, ICG, with a density of 0.05 mg/ml, was injected into the primary tumor during the sentinel lymph node surgery [36]. The penetration depth for practical usage is a maximum of 0.5 cm. The imaging system with 0.05 mg/ml ICG can be applied to the sentinel lymph node surgery in 0.5 mm depth skin.

Cy5 will most likely become highly significant for fields where FITC and ICG are routinely used in clinics. The chemical freedom provided by the Cy5 fluorophore structure makes it a suitable fluorescent label for various types of receptor targeting [37]. However, ICG conjugation is limited to specific ligands for lesion binding because of high non-specific binding [38].

Previously, we conducted an experiment for selective cancer attachment to confirm the selective cancer accumulation of probiotics. In vitro invasion assays in normal and cancer cells were performed to determine the extent of selective cancer accumulation by comparing the number of probiotics that penetrated cells or adhered to cell walls. Consequently, the number of probiotics attached to cancer cells was several times higher than that of normal cells. In addition, we studied how probiotics penetrate a three-dimensional cancer tissue, which is a collection of cells rather than a single cell [8]. The invasion mechanism was confirmed after the invasion of probiotics into the 3D multicellular tumor spheroids, which were determined to be well-simulated cancer environments. Thus, the number of probiotic cells penetrating between the cancer cells was found to be more than 90% of the total population. We found that the performance of probiotics could change depending on the cohesion of the cancer cells. Additionally, we verified the detection performance of fluorescent probiotics using our single-camera fluoroscope. Localization of light-emitting microorganisms could be followed in real time through fluorescence imaging with bacteria. Attenuated bacterial strains such as V. cholerae, S. typhimurium, and L. monocytogenes were found to enter and replicate in the tumor tissue, indicating tumor-specific localization [6]. Intercellular self-replication and translocation are the dominant mechanisms in bacterial intratumoral penetration. Adapting this system to various different microenvironemental sensing readouts in bacteria could make this model system a useful tool in studying multicellular tumor dynamics. Based on this research, multiple fluorescence endoscopy imaging with fluorescently labeled probiotics has the potential to improve the detection of tumors within the GI tract.

In the future, we will focus on reducing the outer diameter of the current home-built RGB-IR imaging system because it is currently over 40 mm, which limits the application for endoscopic imaging. For endoscopic procedures, a size smaller than 10 mm is required. The image of RGB-IR OV 4682 sensor is 5.440 mm by${\; }$3.072 mm. However, the camera sensor PCB diameter is 30 mm. The outer diameter needs to be smaller than 13 mm because the imaging diameter of clinical colonoscopy (CF-HQ290L/I, Olympus, Japan) is 13.2 mm. To reduce the overall size to 13.2 mm or lesser, the size of the sensor PCB should be 10 mm or lesser. Furthermore, we will perform below 20 fps imaging to improve the signal to noise of the fluorescence images to apply fluorescence-based lesion screening in the GI tract.

The white-light RGB imaging, in combination with multiple fluorescence imaging, may help enhance precision during endoscopic surgical procedures.

5. Conclusion

In summary, we presented a home-built single-camera fluorescence imaging device based on an RGB-IR sensor for simultaneous white-light-reflected imaging and multiple fluorescence imaging to improve precision margin screening. Compared with other fluorescence imaging systems using a single camera, the proposed imaging device had the advantage of detecting multiple fluorescence signals and white-light signals simultaneously. Among FITC, Cy5, and ICG, the ICG has characteristics beneficial for deep tissue penetration imaging in ex vivo chicken skin tissue because of the low absorption and scattering in the NIR spectrum. Future investigations will focus on improving the color correction algorithms and reducing the outer diameter of the imaging device for GI tract screening. In addition to ex vivo tissue imaging, we will perform in vivo tumor-targeted imaging with fluorescently labeled probiotics in a preclinical animal model. Furthermore, the 10 mm single camera fluorescence imaging system with fluorescent probiotics can offer additional detection of small or flat tumors and monitoring of tumor progression in real-time with full fluorescence capabilities in clinical study colon cancer.