1. INTRODUCTION

Flow cytometers (FCs), which analyze fluorescence emission from labeled cells or bioparticles, have become one of the most important diagnostic tools in both research and clinical fields [1–5]. However, the use of FCs is primarily restricted to centralized laboratories because they comprise bulky, expensive, and structurally sophisticated optical components to excite, spectrally filter, amplify, and detect fluorescence signals. There has been a great deal of interest in miniaturizing FCs for point-of-care (POC) diagnostics [6–9]. So far, POC FCs have largely relied on the widely available RGB cameras on consumer electronics that provide three spectral channels in the visible band. RGB camera-based POC FCs have been used for white blood cell counting [10], protein droplet detection [11], immunophenotyping [12], and cell multispectral imaging [13]. Nonetheless, the three partially overlapped spectral channels of RGB cameras make it difficult to classify fast-moving fluorescence signals with subtle differences in the emission spectrum. Consequently, state-of-the-art POC FCs have only demonstrated classification of four distinct fluorophores [12], falling far behind the performance of conventional FCs, which are typically capable of classifying eight or more fluorophores and their combinations, as a result of the increased channel number and purposely engineered filter functions.

Metasurfaces, composed of planar arrays of metallic or dielectric nanostructures, offer the ability to control the amplitude, phase, polarization, and spectrum of transmitted or reflected light as a function of spatial position [14–24]. The recent development of compact metasurface-integrated optical systems with extended functionalities has emerged as a promising option to build near-future sensing platforms [25]. For example, metasurface-based spectral filters have recently been integrated with commercial cameras and used for ultracompact reconstructive spectrometry [26–31] which can be extended to detect biological molecules with high sensitivity [32–36]. However, most metasurface-integrated biological sensing platforms are based on near-field optical interaction between fixed analytes and metasurfaces, which limits their application in sensing fast-moving fluorescence signals. To date, metasurface-based filters have not been used for sensing fast-moving fluorescence signals from cells or bioparticles for flow cytometry applications. Here, we present an optofluidic system that integrates spatially varying metasurfaces with microfluidics to drastically increase the number of spectral channels for enhanced accuracy of POC FCs. These metasurfaces serve as filters with spectral transmittance that is tailored to a particular fluorophore panel using inverse design. Single-shot imaging of fluorescently labeled particles moving through the view field multiplexes information from each of the spectral channels into a spectral barcode for end-to-end classification using a pre-trained neural network. The increased spectral diversity allows for a 97.2% classification accuracy when identifying 13 different combinations of up to four different types of fluorophores, which is significantly better than an RGB-only system. The enhanced capability of metasurface-based POC FCs could greatly facilitate their applications in rapid diagnostics, especially in resource-scarce settings.

2. DESIGN PRINCIPLES

Figure 1 depicts the different fluorescence read-out strategies for traditional and POC filter-on-chip flow cytometers. As shown in Fig. 1(a), a conventional FC uses laser excitation and splits the fluorescence signal into several beam paths with dichroic mirrors and uses narrow-band filters to determine whether fluorescent emission in a specific spectral range exists. As such, the cost and volume of the system grows significantly with the total number of spectral channels. An RGB filter-based POC FC, as shown in Fig. 1(b), uses the camera’s spectral filters to classify fluorescence emission. The overlapping RGB spectral channels of the camera, however, limit the system to fluorophores that are spectrally far apart, posing a severe limitation to the number of unique emitters that can be classified. Note that while the spectral channels lack spectral diversity, the imaging sensor of an RGB camera possesses sufficient pixels to spatially encode the fast-moving fluorescence signal as a streak [Fig. 1(b), right panel]. Our system takes advantage of the spatially controlled optical properties of metasurface arrays to multiplex the fluorescence signal into a barcode with rich information about the signal’s spectral characteristics, as shown in Fig. 1(c).

 

Fig. 1. Different flow cytometers and their spectral channel responses. (a) Each channel in a conventional flow cytometer consists of a dichroic mirror and bandpass filter to decompose and filter the signal. Overlap between spectral responses is avoided by spectrally separating the window of each bandpass filter. (b) The spatially coded Bayer filter mosaic creates three channels. Transmittance of the RGB channels is designed for human trichroic color perception and thus there is overlap between channels. (c) Metasurface spectral filters diversify the RGB transmission spectra by creating unique spectral characteristics along the particle trajectory.

Download Full Size | PDF

There are two challenges for fluorescent signal classification using RGB filters: channel crosstalk caused by overlapping spectral response of the ${\rm R}$, ${\rm G}$, and ${\rm B}$ channels, and signal overspill due to overspreading of the emission spectrum of fluorophores. Conventional bulk FCs use narrow step function-like transmission windows to eliminate channel crosstalk, as shown in Fig. 1(a). Each spectral channel directly represents the expression strength of a targeted fluorophore in specified spectral range with classification based on intensity gating of each channel. For RGB-based on-chip FCs, there is spectral overlap between channels, as shown in Fig. 1(b), which results in channel crosstalk. Therefore, a fluorescence signal can have intensity readings in more than one channel, preventing a one-to-one mapping between channels and targeted fluorophores. While more advanced algorithms based on machine learning models such as linear regression, support vector machine, or DNNs can help to mitigate the issues of channel crosstalk and signal overspill, the performance of RGB filter-on-chip FCs remains inadequate due to the limited number of spectral channels. In contrast, spatially coded metasurface arrays can enable additional spectral channels by combining the spectral response of the metasurface and the conventional RGB filter mosaic on the camera. This approach takes advantage of the trajectory of particles, by arranging different filters along the particle motion path, taking advantage of the large sensing area to generate additional spectral channels.

In order to implement this sensing strategy, metasurface arrays were directly integrated at the bottom of microfluidic channels, as shown in Fig. 2(a). Particles carrying different combinations of fluorophores flow closely above the metasurfaces and fluorescence is excited using ultraviolet light. The fluorescence emission is first filtered by the metasurface array and then the RGB filter on the camera. From a single frame snapshot, a barcode-like RGB reading is recorded as shown in Fig. 2(b), which can be further decomposed into three grayscale ${\rm R}$, ${\rm G}$, and ${\rm B}$ barcodes. Note that the pixel intensity in these barcodes is determined by the unique transmission spectra of the metasurfaces. Each metasurface array contains 11 different metasurfaces, which, together with the intentionally designed void space between consecutive arrays, provides 12 unique spectral channels. The average pixel value of each region in the grayscale ${\rm R}$, ${\rm G}$, and ${\rm B}$ barcodes corresponds to the intensity reading of each independent spectral channel. As such, intensity readings from a total of 36 channels can be extracted from the grayscale ${\rm R}$, ${\rm G}$, and ${\rm B}$ barcodes, as shown in Fig. 2(c). The readings from all channels are then normalized to the sum of the intensity readings from the RGB channels at the void region (Supplement 1, Section 3). This 36-dimensional input is fed to a pre-trained end-to-end DNN-based classification model to identify the type of fluorescence signal, as shown in Fig. 2(d), without any intermediate compensation or gating.

 

Fig. 2. Metasurface-enabled optofluidic system for fluorescence signal classification. (a) Schematic of the metasurface-based microfluidic system for spectral filtration. (b) A single frame snapshot yields a barcode-like reading containing information from each spectral channel. (c) Intensity readings of all spectral channels extracted from the barcode image. (d) Readings of all spectral channels are fed to a pre-trained classification model based on DNN to classify the fluorescence signal from a pool of 13 distinct signals consisting of linear combinations of four unique fluorophores.

Download Full Size | PDF

3. RESULTS

The microfluidic platform is shown in Fig. 3(a) with the zoom-in views of the metasurface array and metasurface unit cell shown in Figs. 3(b) and 3(c), respectively. Electron beam lithography was used to pattern 180 nm tall periodic pillars of amorphous silicon on a silica substrate. A 400 nm thick polymethyl methacrylate (PMMA) encapsulation layer was spin-coated on the patterned pillars to form a protective layer, which provides a reusable interface between the metasurface and the polydimethylsiloxane (PDMS) microfluidic channels placed on top of it.

 

Fig. 3. (a) Photo of the integrated metasurface and microfluidic system. (b) Optical bright field transmission image of the field of view. Each microfluidic channel contains six periodic metasurface arrays aligned in the vertical direction and each metasurface array consists of 11 spectrally unique metasurfaces. The scale bar is 100 µm. (c) SEM images of the microstructures in different metasurfaces. The scale bar is 300 nm. (d) Sketch of the RGB camera-based microscopy system. (e) Transmission spectra of the 11 metasurfaces and their corresponding geometric parameters labeled (radius/period [nm]).

Download Full Size | PDF

Inverse design of the metasurfaces, based on a genetic algorithm (Fig. S1a in Supplement 1), was used to maximize the average classification accuracy. Note that the absence of metasurfaces appearing blue or green is not due to a limitation associated with the metasurface but rather these are eliminated in the selection process based on the particular spectra of the fluorophore panel. The inverse design loop accounted for various sources of experimental noise as well as fabrication errors, ensuring a metasurface platform for robust classification performance. The experimentally measured transmission spectrum of each fabricated metasurface is shown in Fig. 3(e) with the corresponding radius and period of the pillars. Note that our design scheme eliminates the need for spectrum measurements with a conventional spectrometer to calibrate the fabrication error of the metasurfaces, which is a typical limitation of metasurface-based spectrometry [27,29]. Instead, an imaging-based process is implemented to measure the RGB values of the metasurfaces which are then matched to the corresponding spectra in the library (Supplement 1, Section 1). The extracted spectrum, based on the RGB measurement, has an excellent agreement with the spectrum measured using a conventional spectrometer (Fig. S3).

The system was assembled by integrating a PDMS microfluidic circuit with the PMMA encapsulated metasurface array, as shown in Figs. 3(a) and 3(d). Fluid is withdrawn from the outlet by applying negative pressure with a syringe pump. This operation mode allows for the flow circuit formed between the PDMS and PMMA to be self-sealed under negative pressure and no permanent bonding between PMMA and PDMS is required to prevent leakage. The field-of-view (FOV) of our imaging system, shown in Fig. 3(b), is ${1000}\;\unicode{x00B5}{\rm m} \times {800}\;\unicode{x00B5}{\rm m}$. There are 13 parallel microfluidic channels (50 µm in width and 40 µm in height) each with six consecutive and identical metasurface arrays. Under the flowrate of 10 µL/min, the average speed of particles is measured to be 2.1 mm/s, which results in an average streak length of 210 µm with a 100 ms exposure time. Since the signals appear at random locations along the channel, periodic metasurface arrays are implemented to ensure that the signal spans at least one set of 11 distinctive metasurfaces, which has a longitudinal period of 100 µm.

To quantitatively benchmark the classification accuracy, 13 different classes of particles labeled with unique ratios of the fluorescent proteins BV421, BV480, BV605, and BV650 were used for performance characterization (Supplement 1, Section 2). Variation in the relative ratios between the fluorophores, uncorrelated camera noise, and correlated noise from the temporal fluctuation of the excitation intensity and flow speed were characterized and incorporated in the model (Supplement 1, Section 3). Typical RGB images of all classes of particles, recorded using the metasurface-integrated system and RGB-only system, respectively, are shown in Figs. 4(a) and 4(b). The images from the metasurface-integrated system in Fig. 4(a) demonstrate a unique barcode pattern for each class of particles, which provides much richer information regarding the fluorophores attached on the particle than the uni-color bar readings in Fig. 4(b) taken from the RGB-only system.

 

Fig. 4. (a) RGB images showing the barcodes for all classes of signals. (b) Images of the RGB-only results of all classes of signals. (c) Two-dimensional pattern clustering by tSNE based on the barcoded signal. (d) Two-dimensional pattern clustering by tSNE based on the RGB-only signal. (e) Confusion matrix using the barcoded signal. (f) Confusion matrix using the RGB-only signal.

Download Full Size | PDF

A computer vision model, based on detection of aggregate channel features, was applied to locate and read out the signal from the obtained images with (Fig. S8 in Supplement 1). This model identifies the barcode width, which corresponds to the diameter of the particle, and selects barcodes from particles with diameters ranging from 7.5 to 12 µm. This approach ensures robustness even in the presence of size variations and common occurrences of debris and particle clusters. The t-distributed stochastic neighbor embedding (t-SNE) technique, a popular dimensionality reduction method, was employed to visualize high-dimensional data in a lower-dimensional space. The t-SNE technique also preserves local similarities between data points and was used to assess the similarities between signals from both systems through unsupervised clustering. The data from both systems were then mapped and clustered into two-dimensional plots [Figs. 4(c) and 4(d)], where smaller spatial separation between data points indicates higher similarity. For the barcode system, each type of fluorescence signal is clustered into an elliptical shape at distinct locations separated from each other, while the clustered readings of the RGB-only system are more scattered with many overlapped regions among the line-shaped point clusters. Comparison between the t-SNE results indicates that our metasurface-integrated system can capture more spectral characteristics of each unique fluorophore combination, reducing the similarity between different combinations and facilitating more accurate classification.

The classification accuracy of the metasurface-integrated system was quantitatively characterized and compared with the RGB-only system. The confusion matrix, as shown in Figs. 4(e) and 4(f), demonstrates the classification accuracy for each class of particles using the two approaches. When classifying fluorescent signals at a throughput of 200 particles/second, which is comparable, or higher, than other recently reported POC flow cytometry devices [12,13], our metasurface-based system achieved an overall accuracy of 97.2%, which is 16.8% higher than the RGB-only system shown in Fig. 4(f) on average. Figure S9 in Supplement 1 shows the confusion matrix without normalization where 20,000 data points were collected for each class to calculate the accuracy. The throughput is comparable with other POC FCs and the least accurate class for both systems is class 11, for which the accuracy of the metasurface-based system is 32.5% higher than the RGB-only system. The experimentally measured accuracy demonstrates the benefit of our metasurface-enabled approach where the spatial degree of freedom along the particle trajectory is exploited to expand the spectrally diverse channels well beyond those in conventional RGB imaging cameras.

4. DISCUSSION

In this work, we integrated metasurface arrays into microfluidic platforms to construct a POC FC with spectrally diverse channels that enable classification of over 13 unique combinations of four types of fluorophores. The classification accuracy could be further improved by reducing signal overlap and channel crosstalk through the use of spectrally separated, and narrow emitters such as quantum dots [37]. In addition, in the current design, we have limited the geometry of the metasurface unit cell to cylindrical pillars which limited spectral diversity. Expanding the diversity of the unit cell geometries would yield spectral channels with more diverse features, including sharp resonances that can reduce channel crosstalk. The uncorrelated noise, which corresponds to the fluorophore expression variation of the stained microsphere, was characterized as ${\pm}\;{3.2}\%$ (Fig. S4 in Supplement 1). This level of variation is comparable with the variation in expression level of many strongly expressed surface groups of some human leukocyte and lymphocyte subsets [38]. In addition, certain classes of fluorophore combinations within the fluorophore panel can be redundant for cell classification [39]. A more robust tolerance of variation in fluorophore expression, or a higher accuracy, can be achieved by excluding redundant classes of fluorophore combinations in cell classification. These additional degrees of freedom make the system well suited to optimization for the wide range of established fluorophore panels that are currently used in FC-based diagnostics.

While we have focused on POC applications here, this technique could yield advantages in more conventional settings where more advanced cameras and light sources can be utilized. For example, an elevated signal-to-noise ratio could be achieved by using a camera with lower readout noise, such as gated intensified cameras, or using multiple excitation sources that maximize the absorption and thus emission intensity of each fluorophore in the panel. The number of microfluidic channels can also be increased by moving to optics with a larger FOV in combination with a higher resolution camera.

5. CONCLUSION

In summary, we demonstrated a new strategy of multiplexing the spectral channels of POC FCs by integrating metasurface arrays with microfluidic circuits. The resulting POC FCs provide rich spectral information of the fluorescent signals in the form of barcodes that can be deconvoluted by a pre-trained DNN module to achieve a high classification accuracy for 13 different combinations of fluorescent signals, substantially outperforming RGB-only systems. In addition, our end-to-end classification architecture removes the need for a conventional spectrometer to calibrate for fabrication errors, yielding a system that can be readily deployed in POC settings. The adaptability of the design pipeline and structural simplicity also allows for further improvement in capacity, accuracy, and throughput when targeting a specific diagnostic application. Recent advancements in complementary metal–oxide–semiconductor (CMOS)-compatible manufacturing techniques [40] and nanoimprint lithography [41,42] have significantly reduced the cost of mass production of metasurfaces, bringing our proposed system closer to practical commercial applications. These advantages could enable a broader spectrum of clinical applications that can be addressed with portable mass-produced POC devices.