Intraoperative fluorescence imaging informs decisions regarding surgical margins by detecting and localizing signals from fluorescent reporters, labeling targets such as malignant tissues. This guidance reduces the likelihood of undetected malignant tissue remaining after resection, eliminating the need for additional treatment or surgery. The primary challenges in performing open-air intraoperative fluorescence imaging come from the weak intensity of the fluorescence signal in the presence of strong surgical and ambient illumination, and the auto-fluorescence of non-target components, such as tissue, especially in the visible spectral window (400–650 nm). In this work, a multispectral open-air fluorescence imaging system is presented for translational image-guided intraoperative applications, which overcomes these challenges. The system is capable of imaging weak fluorescence signals with nanomolar sensitivity in the presence of surgical illumination. This is done using synchronized fluorescence excitation and image acquisition with real-time background subtraction. Additionally, the system uses a liquid crystal tunable filter for acquisition of multispectral images that are used to spectrally unmix target fluorescence from non-target auto-fluorescence. Results are validated by preclinical studies on murine models and translational canine oncology models.
© 2017 Optical Society of America
Surgical resection is the key step in the treatment of a range of disorders including cancer in which patient survival is directly correlated with the precision and completeness of tumor mass removal. Successful complete resection of malignant tissue, including satellite nodules and micro-metastases, determines the effectiveness of the treatment and survival of the patient. Meanwhile, normal tissue surrounding malignancies should be spared from removal to the greatest possible extent. Distinguishing normal and malignant tissue visually is challenging and prone to error, particularly for small tumors in satellite locations away from primary malignancies. Pre-surgical imaging using modalities such as magnetic resonance imaging (MRI) offers only static pre-operative images for tumor localization. While techniques have been developed for combining pre-operative images from these modalities with intraoperative data , these imaging solutions do not provide real-time guidance and cannot be easily integrated in intraoperative workflows, considering their complexity, size, and high cost. Recent advances in the development of fluorescent reagents that label tumors have made intraoperative fluorescence imaging a compact and cost-effective solution for surgical guidance [2–4]. To use fluorescence imaging for real-time intraoperative guidance, two primary challenges must be addressed. First, the fluorescence signal intensity is weak compared to the ambient and surgical illumination present in intraoperative workflows. Second, auto-fluorescence from non-target sources such as tissue, especially in the visible wavelengths (400–650 nm), results in false positive fluorescence signals from non-malignant normal tissue. In this Letter, we present an intraoperative fluorescence imaging platform, Solaris™, that overcomes these challenges. Solaris offers temporal image processing and background correction for real-time fluorescence imaging in the presence of surgical lighting. It also performs open-air intraoperative spectral unmixing for separating auto-fluorescence from target fluorescence signals.
Solaris supports four fixed fluorescence channels at wavelengths of 500, 690, 770, and 800 nm in addition to a multispectral channel with tunability in the 520–620 nm range. The four channels of Solaris allow for imaging widely used fluorescent reagents, e.g., indocyanine green (ICG) and fluorescein isothiocyanate (FITC), in addition to application-specific near-infrared (NIR) fluorescent reporters, such as ProSense® 750 . Surgical illumination and fluorescence excitation is provided by multiple banks of light emitting diodes (LEDs) housed in the imaging head, as shown in Fig. 1. Each LED bank has two white-light and four excitation LEDs, each with of illumination power, at 470, 660, 750, and 790 nm corresponding to the four fixed spectral channels. The LED banks provide multi-angle illumination with above 90% illumination uniformity across a field of view of 10 cm. The white-light LEDs generate an illuminance of at the surgical field. The system is equipped with two cameras for simultaneous fluorescence and bright-field (color) imaging. The fluorescence image is captured by a scientific complementary metal oxide semiconductor (sCMOS) camera. The white-light image is captured by a red-green-blue (RGB) color camera. As depicted in Fig. 1(b), the surgical field is imaged using a relay and objective lens to the sCMOS camera through emission filters populating a motor-controlled filter wheel. The 5.5 megapixel cooled sCMOS camera provides 60% peak quantum efficiency (at 600 nm), read noise of , dark current of , and 16-bit read-out at up to 100 frames per second (fps) allowing for video-rate capture at a dynamic range of 1:33000. The surgical field is imaged through a pick-off mirror by an RGB camera, as illustrated in Fig. 1(b). The system is equipped with an ultrasonic range finder and passive software-based autofocus to ensure that the head remains within the optical working distance from the surgical field, and the fluorescence image captured by the sCMOS camera stays in focus.
In order to obtain the pure fluorescence image under open-air surgical illumination, non-fluorescence components must be removed from the frames captured by the sCMOS camera. To do this, the excitation LEDs are pulsed in synchronization with the sCMOS image acquisition, as illustrated in a timing diagram in Fig. 2. This synchronized LED pulsing results in a succession of foreground (bright-field and fluorescence) and background (bright-field only) frames in the stream captured by the sCMOS camera.
As illustrated in Fig. 2, in foreground frames, the sCMOS global clear signal is fired immediately before the excitation LED is turned on. In the background frames, global clear is fired immediately after the excitation LED is turned off. Thus, background frames contain only non-fluorescence bright-field components. The background frames are subtracted from the foreground at video rate in the software to render a fluorescence stream. This is done with careful management of the byte ordering and data access. The sCMOS camera is set to acquire at 70 fps ( per each frame) in sync with the LED pulsing. With background subtraction, Solaris renders a real-time fluorescence-only stream at 35 fps with a noise floor of . The fluorescence stream is overlaid with the color camera stream, as shown in Fig. 3(a). To characterize the performance of Solaris fluorescence imaging, signal-to-noise ratio and sensitivity studies were performed in all four channels using dilution series of commonly used reagents in well plates. As shown in the results presented in Fig. 3(b), Solaris offers detection sensitivity of below 50 nM at video rate in all channels. This has been verified in other recent independent studies .
Auto-fluorescence from sources such as biological tissue results in false positive signals in in vivo fluorescence imaging . This is especially problematic in the visible spectral window where FITC-based reagents are widely used . To overcome this, Solaris uses a multispectral imaging channel equipped with a liquid crystal tunable filter (LCTF) comprised of electronically tunable liquid crystal wave-plates located between polarizers to allow for wavelength tunability, as depicted in a schematic in Fig. 4(a). LCTF filter spectra are plotted in Fig. 4(b) with a pitch of 10 nm. Though the out-of-band rejection (optical density) of LCTF filtering is only around 2, it provides sufficient spectral distinction for unmixing target and auto-fluorescence signals. The primary sources of auto-fluorescence in in vivo imaging are tissue and food, i.e., chow, in the gastrointestinal (GI) tract. The auto-fluorescence signal from these sources needs to be unmixed from the reagent fluorescence signal, e.g., FITC, labeling targets such as tumors. By fast tuning the LCTF to the spectral bands shown in Fig. 4(b), a multispectral image stack can be acquired. The fast spectral tunability of LCTF ( settling time) allows for acquisition of multispectral snapshots with durations of at 10-nm spectral resolution, which is much finer than multispectral cameras used in prior work . As such, while the fixed-channel fluorescence imaging in Solaris is performed at video rate, the multispectral imaging is performed as single snapshots. Though the multispectral imaging is not performed at video rate, it takes less than a second and can provide direct intraoperative guidance. The multispectral fluorescence image is obtained by performing background subtraction, as illustrated in Fig. 4(c).
The next step in performing spectral unmixing is obtaining the spectral signatures of the fluorescent and auto-fluorescent components. This can be done by extracting pure spectra of auto-fluorescence and reagent fluorescence from regions of interest (ROI) of the multispectral images, as shown in Fig. 5(a). When direct spectral characterization of pure spectra is not possible, one can use automated techniques, such as vertex component analysis (VCA), to recover the spectral signatures . Once the spectral signatures are obtained, the linear unmixing equation can be established which relates the unmixed fluorescence signal vector for each pixel to the multispectral data through the spectral signature matrix as follows:1) describes the linear unmixing problem where, given the data vector, , and the spectral matrix, , the unmixed fluorescence vector, , is calculated. To perform spectral unmixing, non-negative least squares is applied to Eq. (1) to solve for the unmixed component vector that best fits the unmixing equations. For the in vivo imaging data presented in Fig. 5(a), the spectral unmixing algorithm yields three component images corresponding to food, tissue, and FITC, as shown in Fig. 5(c).
By applying the spectral unmixing framework to intraoperative multispectral snapshots, margins of FITC-labeled tumors can be identified. An example is shown in Fig. 6 where a murine model with an abdominal tumor nodule underwent multispectral intraoperative imaging. As illustrated in Figs. 6(a) and 6(b), the white-light and fluorescence images of the abdominal area do not provide clear definitions of the tumor. The fluorescence image contains false positive signals from tissue and food auto-fluorescence. The spectral unmixing result is shown in Fig. 6(c) as a colorized unmixed image, which is the composite of the three unmixed components with blue, red, and green representing tissue, food, and FITC signals, respectively. After spectral unmixing, the tumor nodule located adjacent to the GI tract and tagged by FITC-Dextran can be identified with improved specificity.
Proof-of-concept in vivo studies were performed on two nu/nu murine models with metastatic malignancies in the abdominal cavity labeled by FITC-based reagents to validate the spectral unmixing technique. The pure spectra of the primary fluorescent sources, i.e., FITC, tissue, and food, were obtained, as illustrated in Fig. 5. The results are presented in Fig. 7. In the first study, the animal was given an intra-splenic injection of HCT116-red-fluc tumor cells and was splenectomized 10 min post injection. In the second study, the animal was given an intraperitoneal injection of HT29-luc2 colorectal cancer cells. After about one week of tumor growth, the first animal was injected with FITC-Dextran, which pools into the interstitial space in malignant masses. Twenty-four hours post injection of FITC-Dextran, the first animal underwent multispectral intraoperative imaging. For the second animal, multispectral imaging was performed after 10 days of tumor growth with topical administration of ProteoGREEN (Goryo Chemical), which is activated in cancer cells and in the pancreas . Considering this, the pancreas region of the second animal was covered during intraoperative fluorescence imaging. Prior to surgery, bioluminescence imaging was performed on both animals to validate the presence and location of tumors. As shown in Figs. 7(a) and 7(d), bioluminescence signals from tumors were observed in the upper abdomen of the first animal and in the upper and lower abdominal areas of the second animal. In Figs. 7(b) and 7(e), the intraoperative fluorescence images of the animals are shown with signals from the target as well as false positive signals from the skin, abdominal tissue, and GI tract. The spectral unmixing results are shown in Figs. 7(c) and 7(f).
The spectral unmixing of the first animal data identifies the signal from a tumor nodule adjacent to the stomach as FITC fluorescence and characterizes signals from other areas of the animal as tissue and food auto-fluorescence, as illustrated in Fig. 7(c). The spectral unmixing of the second animal data identifies signals from a large mass in the lower left abdomen and a tumor nodule adjacent to the stomach as FITC fluorescence, as depicted in Fig. 7(f). These agree with the bioluminescence data shown in Figs. 7(a) and 7(d). The correlation of the spectral unmixing results with the bioluminescence data substantiates the improved specificity in tumor detection and intraoperative margin determination offered by spectral unmixing in the presence of strong auto-fluorescence.
Though the proof-of-concept studies presented here were focused on preclinical models, the methodology directly translates to veterinary and clinical applications. Preliminary studies have utilized fixed fluorescence channels of Solaris for tumor margin identification using NIR fluorescent reagents in canines undergoing surgery, as shown in Fig. 8 and Visualization 1. While spectral unmixing is applicable to veterinary and clinical applications, fluorescence imaging in NIR typically does not require spectral unmixing. In the veterinary canine study shown in Fig. 8, ProSense 750 FAST, an NIR fluorescent probe, was administered intravenously twenty-four hours prior to surgery. Imaging with the Solaris clearly demarcated the tumor margin compared to normal tissue, as presented in Visualization 1.
In summary, the Solaris imaging system provides robust intraoperative fluorescent imaging and can be used to image in multiple channels at video rate under surgical lighting with nanomolar sensitivity. Solaris also offers multispectral imaging and spectral unmixing for improved specificity in FITC-based imaging.
We thank Dr. Arno Roos, Dr. Clemens Lowik, and Dr. Jurgen Tan at the Veterinary Referral Clinic in Gouda, Netherlands, for providing canine surgery data.
1. F.-Y. Li, X.-L. Chen, and B.-N. Xu, Chronic Dis. Transl. Med. 2, 181 (2016).
2. V. Ntziachristos, Annu. Rev. Biomed. Eng. 8, 1 (2006). [CrossRef]
3. S. Keereweer, P. B. Van Driel, T. J. Snoeks, J. D. Kerrebijn, R. J. B. de Jong, A. L. Vahrmeijer, H. J. Sterenborg, and C. W. Löwik, Clin. Cancer Res. 19, 3745 (2013).
4. Q. T. Nguyen and R. Y. Tsien, Nat. Rev. Cancer 13, 653 (2013). [CrossRef]
5. J. L. Figueiredo, H. Alencar, R. Weissleder, and U. Mahmood, Int. J. Cancer 118, 2672 (2006). [CrossRef]
6. A. V. DSouza, H. Lin, E. R. Henderson, K. S. Samkoe, and B. W. Pogue, J. Biomed. Opt. 21, 080901 (2016). [CrossRef]
7. Y. Urano, M. Sakabe, N. Kosaka, M. Ogawa, M. Mitsunaga, D. Asanuma, M. Kamiya, M. R. Young, T. Nagano, and P. L. Choyke, Sci. Transl. Med. 3, 110ra119 (2011). [CrossRef]
8. N. Dimitriadis, B. Grychtol, L. Maertins, T. Behr, G. Themelis, and N. C. Deliolanis, Opt. Lett. 41, 1173 (2016). [CrossRef]
9. J. M. Nascimento and J. M. Dias, IEEE Trans. Geosci. Remote Sens. 43, 898 (2005). [CrossRef]