1. Introduction

There is a need for a faster imaging methodology to obtain images that replicate the current hematoxylin and eosin (H&E) staining technique. Solid tumor cancer outcomes are closely linked to positive surgical margins (PSM), typically confirmed by the review of H&E stained formalin-fixed paraffin embedded (FFPE) tissue or performance of a pre-operative frozen section procedure (FS). FFPE margin assessment is thorough and extensive, enabling clear histological visualization of the abundantly sampled margins, but it is time-consuming, labor intensive, and, most importantly, it is not done in real time during the surgery, preventing meaningful re-excision to ensure negative margin status during the surgical procedure. This means that 11-38 % of radical prostatectomy specimens (RP) will end up having PSM, which will increase the likelihood of biochemical recurrence, local disease recurrence and the need for salvage cancer treatments [1]. FS procedures, which represent the gold standard for per-operative histopathological margin evaluation takes approximately 20 minutes to perform, from sample acquisition to reporting in real-time [2,3]. FS accuracy of RP is only 81% due to freezing artifacts in fatty tissue and margin sampling [4,5]. Even with the quick turnaround time for FS, increasing surgical times has been shown to increase infectious complication (IC) rates at 14% per half hour [6], and hospital length of stays increasing (LOS) at 6% per extra half hour [7]. For this reason, as well as the avoidance of freezing artifacts, another faster imaging method is desired.

Many optical microscopy techniques have been proposed to solve this dilemma but each has their own drawbacks such as speed, cost, coupling, or the need for molecular labeling. Optical approaches are rapidly being proposed due to the high resolutions and potential for faster scanning capabilities. Optical coherance tomography and reflectance confocal microscopy are two such optical imaging modalities with high spatial resolutions and sensitivity, however it has low molecular specificity due to the scattering of light [8,9]. Nonlinear microscopy methods have been shown to be able to image label free with high spatial resolutions but with significantly more expensive equipment needed and slower imaging speeds [10–12]. Another modality, photoacoustic microscopy (PAM) has been used to acquire virtual hematoxylin images without labels using absorption contrast by measuring the induced pressures from a ultraviolet (UV) excitation laser beam [13]. Ultraviolet-PAM (UV-PAM) has recently demonstrated fast scanning using a fast hybridized, optomechnical scanning method. However, it still requires physical coupling with the sample which limits its use interoperatively [14].

Here we propose the use of photoacoustic remote sensing (PARS) as it is label-free, non-contact, inexpensive, and has demonstrated the potential for faster imaging speeds. PARS is similar to PAM as it uses an excitation laser pulse that is absorbed by molecules of interest, where they undergo rapid heating and create a large pressure gradient. The difference arises in the measurement of this pressure wave with PAM directly measuring the pressure waves using an ultrasonic transducer whereas PARS uses a separate interrogation laser beam to measure the refractive index modulations imposed by the large pressure gradient. By not requiring stains or labels, PARS uses absorption contrast to immediately image after resection and will not have any human variability that stains and labeling may possess [15–17]. PARS uses UV, strongly absorbed by nucleic acids, to obtain absorption contrast of cell nuclei [18]. Recent work with ultraviolet-PARS (UV-PARS) has shown excellent comparisons with traditional FFPE hematoxylin stained sections in a variety of tissue samples [19–21]. PARS has also been utilized to replicate the eosin-like contrast of cells’ cytoplasm where a variety of different methods have been explored for the eosin-like contrast. Most recently three different methodologies have been proposed: using an additional absorption wavelength of 420-nm to target cytokines absorption, frequency decomposition then principal component analysis of frequency information, and the low-pass filtering of the back-scattered interrogation beam [22–24]. This work utilizes the back-scattered interrogation beam low-pass filtering as a virtual eosin stain as it does not take any additional equipment or acquisition time to aid in the simplicity and speed of the proposed microscopy system. Multiple iterations of photoacoustic remote sensing systems have been demonstrated but all have lacked adequate imaging acquisition speed to have the potential of imaging in an interoperative setting [20,22,25]. Here we present a microscopy system that can acquire 10mm x 5mm 73 Megapixel images of sectioned tissue in under 3.5 minutes, 185 times faster than previous PARS implementations with the same pixel density. We can quickly swap between taking larger scans at a lower resolution to pinpoint areas of interest and subsequent high-resolution images using the same system. Previous scanning iterations have placed the galvanometer mirror(s) before the reflective objective [19,21,24,26], whereas our optomechanical setup consists of a single galvanometer mirror (GM) placed after the reflective objective and a mechanical stage translation to scan over the sample at significantly faster speeds. The working distance (WD) of previous reflective objectives was not long enough to accommodate any GMs so we had to compromise and utilize a lower NA objective with a longer WD to physically fit in our GM. This setup allowed us to increase the speed of the system albeit with a lower NA objective which lowered the system’s lateral resolution to 0.86µm. One of the flaws in these previous set-ups is that the reflective objective has an obscuration of 26%. When the light is scanned through the reflective objective, the obscuration will vary leading to a dynamic intensity pattern on the sample which may not have enough fluence to impose a discernible photoacoustic signal. Additionally, scanning the beam into the reflective objective requires either clipping the beam which will degrade the focusing or reducing the beams size (to avoid clipping) which will reduce the effective numerical aperture. In contrast, when placing a GM after the reflective objective, an ideally focused spot is scanned across the sample albeit with a varying height of the focal position. For these reasons we have chosen to place the GM after the reflective objective. In this manuscript we discuss the system modifications and scanning methodology to obtain fast virtual H&E images, alongside the utilization of a colormap matching algorithm to present more comparable virtual H&E images with the gold standard of FFPE H&E stained tissue. We demonstrate excellent agreement between the gold standard and our virtual H&E images in prostate tissue and continue to discuss tissue structure and architecture observed by pathologists in the virtual H&E images.

2. Methods

2.1 System set-up

Figure 1 depicts the system diagram of our microscopy system. This system was built upon previous iterations as described in the previous work of Restall et al. [24] but with the additional constant velocity scanning implementation and GM placement. To briefly summarize, a 532-nm nanosecond excitation laser (IPG Photonics, GLP-10) is frequency doubled to 266-nm then rapidly pulsed on the sample. This causes a rapid heating which translates to large pressure gradients that modulate the local refractive index. These modulations are detected by the backscatter of the 1043-nm (SLD1050S-A60) circularly polarized, continuous wave interrogation beam. Both the 266-nm and 1043-nm beams are shaped using Galilean beam expansion to 8mm (aperture size of the objective) then combined using a harmonic beam splitter (Semrock, Di01-R355-25x36) and co-aligned into a 0.3 NA reflective objective (Thor Labs, LMM15X-UVV). The light is focused onto a single galvanometer (GM) (Thor Labs, GVS412) and scanned under the sample. The working distance of the reflective objective is 23.8mm, which included a separation of 7.5mm to the GM and subsequent 16.3mm to the sample. The reflected light passes back through the system and is redirected towards a 75-MHz balanced photodiode (Thorlabs, PDB420C-AC) with a polarized beam splitter (PBS) (Thorlabs, CCM1-PBS254). The balanced photodiodes radio frequency (RF) signal is passively highpass filtered over 130kHz (Thor Labs, EF505) to form the virtual hematoxylin data and its DC scattering signal is lowpass filtered at 100kHz (KR Electronics, 2928) to form the virtual eosin data. It is important to note that the RF signal is effectively bandpassed with the combination of the 130kHz highpass filter and 75MHz lowpass limit on the balanced photodiode. As the beams scan across the sample, the DC scattering signal will change at each position due to the heterogeneity of scatterers within the sample. As a result, the acquired RF signal will have additional noise at higher frequencies. This signal must be removed or else artifacts will occur along the edges of the tissue in the virtual eosin image. The highpass filter is used to remove these artifacts with a higher cut-off frequency. The digital data was acquired from 3 channels: the mean galvanometer signal for the y- positional data, the mean backscattered DC signal for the virtual hematoxylin, and the absolute maximum of the RF signal for the virtual eosin.

 

Fig. 1. a) System Diagram: Individual components can be identified as: balanced photodiode (BPD), beam dump (BD), caesium lithium borate crystal (CLBO), collimator (C), galvanometer mirror (GM), half-wave plate (HWP), harmonic beam splitter (HB), lens (L), mirror (M), polarized beam splitter (PBS), prism (P), quarter-wave plate (QWP), and reflective objective (ROBJ). b) The scanning trajectory is shown with each horizontal strip being acquired then the mechanical stage will translate vertically in preparation for the new strip acquisition.

Download Full Size | PDF

Our optomechanical scanning method is similar the UV-PAM technique of Li et al.[14] but our approach is non-contact and 8 times faster while also providing a virtual eosin contrast above and beyond the virtual hematoxylin contrast in their work [14]. The images were acquired in horizontal strips, then amalgamated together to form a large field of view (FOV) virtual H&E image. For each strip, the sample was mechanically translated horizontally with a constant velocity while the GM scanned the laser vertically in a sinusoidal pattern as seen in Fig. 1(b). Using this methodology, a 220µm x 10mm strip was acquired every 9s with a constant stage velocity of 1.1mm/s and point spacing of 1µm. Due to the point spacing being larger than our systems lateral resolution of 0.86µm, we will consider the point spacing to be the limiting factor for the purposes of this imaging system. After each strip is acquired the mechanical stage will translate 220µm vertically to begin acquiring the next horizontal strip. After scanning the mechanical stage will return to its original position and stop. A 10mm x 5mm image was acquired in 207s with 23 of these strips acquired all together, 185-300 times faster than previous point scanning embodiments of PARS [22,25] as well as 8 times faster than the UV-PAM embodiment of this similar scanning methodology [14]. These comparisons are made in terms of area scanned per second and with similar resolutions to the 0.86µm lateral resolution acquired with this system. One critical difference in our system is the placement of the GM being after the objective. The larger strip size achieved by this system compared to previous iterations is due to the GM scanning after the objective and larger depth of focus (DOF) of the objective. The galvanometer was scanned at an angle of $\sim \pm 0.424^\circ$ leading to a focal point deviation of up to 0.5µm but still within the 1.48µm DOF of the reflective objective. However, to accommodate the added GM, a longer focal length reflective objective was utilized with a lower NA. This offered a decrease in fluence which needed to be compensated with higher overall power of the interrogation beam and UV pulse energies. Of course, with faster scanning and a reduction in point density that same image can be acquired in 50s with a point spacing of 10µm and stage velocity of 5mm/s. For each of the constant velocity images an algorithm is used to automatically adjust scanning parameters for equal point spacing and maximal scanning speeds. The mechanical stage velocity, the galvanometer mirror frequency, and laser pulse repetition rate (PRR) are adjusted for the desired point spacing and in turn the resolution of the desired image. The optimal density of 1pt/µm utilized 266-nm pulse energies of 4.8nJ corresponding to a fluence of 2.03J/$cm^2$ at the focal point and 11.2mW or 4.7MW/$cm^2$ at the focal point for the 1043-nm light. This translates to similar fluences demonstrated in previous work as this work uses a lower 0.3NA objective [20,23–25].

2.2 Tissue preparation

Quartered formalin-fixed paraffin embedded RP specimens were obtained from prostate cancer patients after pathology cases were closed and tissues were otherwise flagged for disposal as per approved ethics (HREBA (Cancer) / HREBA.CC- 20-0145). Tissue sample patient information was redacted and research staff and pathologists reading true and virtual H&E images were blinded to all patient and diagnostic information. This work focuses on thin sections which are slide mounted but unstained. Virtual H&E images however were compared with H&E stained slides for validation.

3. Results

In brief, eosin and hematoxylin colormaps were extracted from a true H&E reference image using a blind stain separation algorithm previously described in Li et al.[27,28]. Our acquired PARS and scattering data is normalized to values between 0 and 1 before undergoing a histogram match with the aforementioned reference images. A subsequent median filter is applied to both data sets, and an alpha threshold of 10% is applied to the virtual hematoxylin image before it is overlaid on top of the virtual eosin image and the virtual colormaps are applied as described in Haven et al.[25] and shown in Fig. 2(a). Next, an intensity smoothing algorithm is employed to perform an intensity gradient transformation for each individual strip acquired then the entire image. The algorithm works as follows: A generalized intensity gradient is taken from the mean of each strip in the images and then the entire dataset in both the x direction and y direction to compute a counter gradient intensity transformation. This is applied to the data set with a symmetric nearest neighbor filter to achieve a more uniform image, progressing as shown in Fig. 2(b). Figure 2(c) represents the mean intensity of each row after the colormap algorithm is applied. There are large intensity outliers along the edges of each strip, a slightly increasing intensity gradient near the top of each strip, and an overall increasing intensity gradient moving towards the bottom of the image. These are intensity artifacts are compensated with the intensity smoothing algorithm that leaves the images with greater contrast and fewer artifacts as seen in Fig. 2(d).

 

Fig. 2. a) This figure shows the steps taken for the image processing progression from Haven et al.’s colormap algorithm to (b) the intensity gradient smoothing for the final image. Scale bars are all 2mm. c) The mean row intensity, after undergoing the colormap algorithm, is plotted along the y axis. d) The mean row intensity, after undergoing the intensity smoothing algorithm, is plotted along the y axis.

Download Full Size | PDF

Figure 3 compares the large FOV images (10mm × 5mm) with the gold standard, FFPE with H&E staining. Here we can identify nuclei by nuclei characteristics with Fig. 3(b) and Fig. 3(d) showing a similar distribution of cell nuclei around a glandular benign prostatic hyperplasia (BPH), as well as comparable gross morphology between the two images. The area outlined in blue provides evidence of a benign area due to the nodular and lobular architecture of the prostate glands as well as the lack of infiltrative glands which would suggest malignancy. The two subset images, Fig. 3(b) and Fig. 3(d), support the benign diagnosis due to a variety of factors such as the undulating gland lumen and the low ratio of nuclei to cytoplasm, which favors benign tissue. Basal cell identification is difficult on both the virtual H&E image, Fig. 3(b), and the true H&E image, Fig. 3(d), where their presence would confirm the benign status. This highlights the benefits of improved magnification and resolution to be pursued in the future. Building on this excellent agreement between our new imaging technique and the gold standard, we demonstrate various other virtual H&E images from different areas of interest in the RP. Figure 4(a) and Fig. 4(b), taken from the peripheral zone (PZ) of the prostate, visualize a variety of different architectures such as luminal epithelial cells and benign glands of varying shape and size. These well-defined glands are important for diagnosis as well as Gleason score grading of the prostate tissue. Figure 4(c) has a defined area of benign glands, indicated in blue, that have led to a compression effect on the adjacent glands outside the region of interest (ROI), indicated by the red arrows. The green area is indicative of a luminal structure. However, the long and thin architecture could favor a thin-walled blood vessel such as a lymphatic or vein structure rather than a benign prostate gland. Figure 4(d) and e are two enlarged images of a ROI with a potential for malignancy in the specimen. The darker gland area with higher nuclei to cytoplasm ratio are particularly worrisome. It would be preferable to have a higher magnification to definitively evaluate whether it is malignant or atrophic glands. Despite the need for higher magnification scans the current work demonstrates the rapid scanning capabilities of the current system for gross evaluation. Our recent work could be used for a higher magnification scanning over a small FOV [25].

 

Fig. 3. a) Full FOV virtual H&E stain of RP with blue subsections showing architecture of nodular/lobular arrangement. b) Subsection of prostate glands with potential benign area from virtual H&E image. c) Full FOV of true H&E stain of RP with blue subsections showing architecture of nodular/lobular arrangement. d) Subsection of prostate glands with potential benign area from true H&E image.

Download Full Size | PDF

 

Fig. 4. a) Virtual H&E stain from RP FFPE prostate tissue with a variety of benign glands from the PZ. Scale bar is 1mm. b) Virtual H&E stain from FFPE prostate tissue with benign glands of varying shape and size. Scale bar is 1mm. c) Virtual H&E stain from FFPE prostate tissue with observed luminal structure as well as benign glands and compressed glands outside the ROI. Scale bar is 1mm. d) Virtual H&E image of the ROI with a higher density of darker, more prominent nuclei. Scale bar is 500µm. e) True H&E stain of the same region. The higher density and darker nuclei may indicate atrophic glands or malignancy. Scale bar is 500µm.

Download Full Size | PDF

This prostate specimen has previously been graded with a tertiary score of 5, meaning that less than 5% of the prostate is grade 5, which precipitates a significant chance of biochemical recurrence-free survival [29,30]. These characterization of the tumor assist the patient the care in the future to prevent re-occurrence. One significant area of improvement for this technique would be to increase the resolution up to at least a resolving power of 400X and improve specificity to resolve the nucleolus inside the nuclei. Pathologists will observe the prominence, size and color to help predict tumour prognosis [31].

4. Conclusion

We demonstrate the improved imaging capabilities of a PARS system by adding a 1D galvanometer post-objective to utilize higher laser repetition rates for significantly faster scanning speeds, a 185-fold speed increase from our previous work [25]. We have demonstrated the capability to acquire a 10mm x 5mm virtual hematoxylin and eosin (H&E) image in less than 3.5 minutes allowing surgeons to have the option to make multiple tissue resections during the surgery with feedback for each resection. Stitching artifacts between the strips still need to be addressed as this can cause loss of detail in a specific region of interest (ROI). Our optical system uses our acquisition algorithm for large field of view gross imaging or a smaller ROI with comparable results to the gold standard of formalin-fixation, paraffin embedded H&E staining. The smaller NA objective in the system allows for more consistent, faster scanning capabilities. However, the increased speed is at the cost of a lower NA and corresponding reduction in resolution when compared to our previous work. Pathologists need the equivalent of up to 400X magnification for reliable diagnosis. This requirement is beyond the capabilities of the current system and will be addressed in future work. Speed is also limited by the galvanometer scanning frequency which has an upper limit of 1kHz, but this could be solved with the implementation of a resonant galvanometer or polygon scanning apparatus. Each of these obstacles will need to be overcome for this imaging system to be viable for diagnosis and prognostication of prostate cancers. Overall, this microscopy system presents an opportunity to greatly improve patient outcomes by lowering overall surgical times reduces positive surgical margins, infectious complication rates and length of stay. In the future, images could be acquired up to 5 times faster with a faster laser repetition rate of 2MHz, and a resonant mirror for galvanometer scanning at 4kHz.