1. Introduction

Ever since the demonstration of the first functional mobile phone in 1973 by Martin Cooper at Motorola [1], mobile phones have become a critical mainstay of everyday life. According to the International Telecommunication Union (ITU), there are more than 7.8 billion active cellular subscriptions around the globe. The high penetration of mobile phones is largely due to their affordability and user-oriented design. Mobile phones have great potential to connect people who are isolated from the mainstream of economic and technological development due to political and socio-economic challenges. As a result, their widespread availability and affordability in developing and underdeveloped countries is prompting new initiatives by many governmental or non-governmental organizations [2].

Early versions of mobile phones were primarily intended for voice communication and messaging applications. The rapid advancement in embedded technology, miniaturized electronics, and fast computation has accelerated the evolution of mobile phone technology, ushering in the modern-day smartphone. It is estimated that there were nearly 3.2 billion smartphone users across the globe in 2019 [3]. The modern smartphone is not merely a communication device: the enormous processing power, storage capacity and battery life of smartphones allows the integration of different consumer-oriented sensors (e.g., complementary metal-oxide semiconductor (CMOS) cameras, light emitting diode (LED) flashlights, proximity and ambient light sensors (ALS), accelerometers, global positioning system (GPS), wi-fi, graphical user interface (GUI)) with user-oriented software tools and smartphone applications (a.k.a., apps). Hence, the modern smartphone is essentially a portable personal computer and sensing platform that lowers the economic barriers to rapid development and deployment of scientific tools in traditional and need-based communities [4–7].

Since the smartphone camera has become a primary selling point of these devices, continuous efforts have been made to improve its quality over time. The modern camera phone first emerged after the development of CMOS active pixel sensors in the early 1990s [8]. In 1999, Kyocera, commercialized the first camera phone (VP-210) with a 0.11-megapixel (MP) front camera [9]. After this, many companies realized the potential of the smartphone’s market penetration, and the never-ending race to increase pixel count and pixel density began. In 2000, Samsung released their first phone with a built-in 0.35-MP camera (Samsung SCH-V200). Following the trend, Sprint released Audiovox PM8920 in the United States with a 1.3-MP camera in 2004. In 2005, Nokia introduced their 2-MP camera phone with Carl Zeiss optics, an LED flash, and autofocus capabilities. In 2007, Samsung released the first 5-MP camera phone, and this resolution remained as a high-end standard for many years [10].

Smartphone cameras with higher resolution again appeared in 2010. Since the footprint of the smartphone is small, the key strategy used to increase the pixel count is to reduce the size of pixels. Sony released a 12-MP autofocus camera with added facilities such as face detection, geotagging and smart contrast in 2010. Afterwards, in mid-2013, Nokia announced their smartphone, Lumia 1020, with a 41-MP camera sensor with a 1.2-µm pixel size, embedded with an f/2.4 Carl-Zeiss all-aspherical one-group lens [11]. Due to the high resolution of the camera, this smartphone was used to demonstrate the detection of single DNA molecules [12]. The high-pixel-count smartphone cameras further evolved as, in 2019, Samsung developed and commercialized the 64-MP and 108-MP cameras for smartphones [13]. These densities can be found in many new smartphones available on the market today: for example, in the Galaxy S21 from Samsung, the Mi 10I from Xiaomi, and the Edge+ from Motorola. The 108-MP cameras process images through pixel binning, where four or nine pixels are combined to work as a single pixel. The use of the larger pixel enables capturing more light, which results in a higher ISO rating and lower noise. A higher ISO rating enables detection in low-light settings, which is essential for applications such as fluorescence-based assays [14].

Owing to the low-cost, small foot-print, low-power requirement, and vast adaptability of the smartphone – key factors for developing affordable and point-of-care disease diagnosis platforms – different research-based and commercial biomedical devices have been demonstrated. For example, microscopic imaging is an important tool for the early detection and diagnosis of many significant diseases, such as malaria. Therefore, efforts have been made by many research groups to develop smartphone-based microscopy platforms [15–17]. Some systems simply integrate traditional optical components [18], while others use advanced digital holographic image processing techniques [19]. Using a simple lens and LED configuration, Zhu et al. demonstrated the usability of the smartphone as a cost-effective imaging tool for rapid blood analysis [20]. D’Ambrosio et al. demonstrated the first video microscopy platform in a smartphone for the detection and quantification of blood-borne filarial parasites [21].

Outside of microscopy, other implementations of smartphone-based medical devices include examples like the CellScope Oto, a commercially available smartphone-based otoscope to diagnose pediatric ear infections [22]. Another prominent use of the smartphone includes point-of-care urinalysis platforms. Using a smartphone camera and machine learning approaches, several companies and research groups have already translated laboratory-based urinalysis to home-based detection [23,24]. Some clinical applications do not rely on the smartphone hardware but rather leverage specially designed apps to exploit its computational platform to process data from external medical devices. KardiaMobile is a portable electrocardiogram (EKG) monitor that can be connected to a smartphone through wi-fi, and the EKG chart and other heart-related information can be analyzed in the corresponding phone application [25]. Butterfly iQ is a smartphone USB-powered commercial ultrasound imaging system that brings affordable and portable ultrasound imaging anywhere [26]. The commercial adaptation of these smartphone-based biomedical devices opens the door to enabling other lab-confined biomedical devices to be used as point-of-care applications.

Optical spectroscopy is a key scientific strategy to detect the presence of different biochemical analytes based on their unique interaction with light. It has been extensively used for non-invasive disease diagnosis and to detect numerous disease-specific biomarkers in complex sample matrices [27]. Of central importance to most spectroscopic methods is a spectrometer – a device that measures the constituent wavelength components of light that have been reflected from or transmitted through a sample. Many commercial spectrometers utilize multi-pixel detectors not altogether different from the camera sensor in modern smartphones. Spectrophotometry, another branch of spectroscopy that quantifies the concentration of an analyte by measuring its absorbance at a specific wavelength, utilizes a single photodetector. The embedded ALS, which is a photodiode with a spectral detection range of 350 nm-1000 nm, is ideal for such spectrophotometric applications [28]. The native flashlight in a smartphone can be used as a light source for spectroscopic applications in the visible domain since it is a bright, white LED with emission wavelengths ranging from 400 nm-700 nm [29]. Besides hardware, the existing computational power and display capabilities of a modern smartphone are ideal for developing apps for analysis, interpretation and transmission of spectral data.

Many research groups have introduced smartphone-based spectroscopic platforms for biomedical applications [30–39]. A review by McGonigle et al. discusses some instrumentational aspects of smartphone-based spectroscopic systems based on their grating configurations [40], but there is no focused review available on the recent development of systems based on spectroscopic modalities (absorption, reflectance and fluorescence spectroscopy) that have been specifically demonstrated for biomedical applications. The current review aims to provide an overview of the current state-of-the-art in smartphone spectroscopic instrumentation and the development of smartphone-based spectroscopic modalities for biomedical applications. We begin with an overview of the embedded components of a standard smartphone that enable spectroscopy. Next, we describe different modalities of smartphone spectroscopic platforms that have been reported for biomedical applications. Finally, we discuss the advantages and disadvantages of the current platforms and present potential opportunities for further exploration of this promising technology.

2. Enabling embedded components used for developing smartphone-based spectroscopic platforms

Figure 1 provides a graphical overview of the components of a typical smartphone that may be employed for various aspects of spectroscopic applications. Smartphone-based spectroscopic platforms primarily aim to leverage the embedded camera as a spectral detector [41]. In addition to the camera, the ambient light sensor (ALS), which detects the general level of light in the environment, may be used as a detector in some spectroscopic applications [28], especially those that require sensitivity to NIR light. The spectral signal detected by the camera or ALS is typically processed within the phone by using a custom-developed phone application. To demonstrate a truly self-contained platform, several research groups have deployed the embedded flashlight as a light source [42]. Alternatively, the existing USB port of the phone can be utilized to power external LEDs from the smartphone battery [43]. A detailed description of these enabling functional components is provided below.

 

Fig. 1. Typical position of the functional components of a smartphone used for developing spectroscopic platforms.

Download Full Size | PDF

2.1 Camera

The optical design of the embedded camera module may vary from phone to phone. For simplicity, it can be considered as an assembly of a focusing lens, light filters and a CMOS sensor, as shown in Fig. 2(a). The camera module is primarily designed and intended for consumer applications such as photography; therefore, its response is limited to the visible region. Although the sensor chip – typically fabricated from silicon – has sensitivity in the near infrared to nearly 900 nm, the phones usually include an infrared (IR) filter to limit the response of the camera to the wavelength range of 400 nm to 700 nm [44]. In addition, all current-generation smartphones are embedded with a Bayer image sensor: a pixel-sized array with red, green, and blue filters arranged in a Bayer pattern. The inset in Fig. 2(a) shows the schematic of the Bayer pattern and the corresponding process of digital color image formation by the CMOS sensor of the smartphone. Each pixel records red, green, or blue light; therefore, the data from a single pixel of the image sensor cannot fully specify a true color value on its own. A full-color image is obtained by using a demosaicing algorithm, which interpolates a set of complete red, green, and blue values for each super-pixel (comprising four pixels). These algorithms make use of the surrounding pixels of the corresponding colors to estimate the values for a particular super-pixel. Each pixel contributes a single 8-bit, grayscale intensity value (0 to 255 levels). Once reconstructed, images may be displayed in color on the phone screen or analyzed to extract relevant information. Note that while traditional benchtop spectroscopic systems use a 1-D photodetector array, which can detect only one spectrum at a time, the embedded camera in a smartphone is two-dimensional and can, therefore, be utilized to detect multiple spectra at the same time (e.g., for multiplexed detection of biomarkers [45]).

 

Fig. 2. General characteristics of the functional components of smartphone used for developing spectroscopic platforms. (a) Optics and image detection process. (b) Functional block diagram and wavelength response of the ALS. (c) Emission spectrum of the LED flashlight. (d) Circuit diagram to use the phone battery as a source through USB.

Download Full Size | PDF

2.2 ALS

The ALS embedded in the front panel of the smartphone is meant to optimize the consumption of battery power. The ALS controls the brightness of the display panel automatically in accordance with the surrounding environment. Almost all branded smartphones contain an Avago APDS-9930 or ams AG(TAOS) TMD2771 ambient light and proximity sensor chip [46,47]. This sensor chip has two photodiode channels: CH0 is used for light sensing and CH1 is used for proximity sensing. As shown in Fig. 2(b), the sensor chip includes on-chip integrating amplifiers, analog-to-digital converters (ADCs), accumulators, clocks, buffers, comparators, a state machine and an Inter-Integrated Circuit (I2C) interface. Upon detecting light on either photodiode channel, the amplified photodiode currents are converted to 16-bit digital values by the ADC unit. The converted digital values are then transferred to the CH0 and CH1 data registers of a microprocessor for further processing. From the microprocessor, the data are sent to the central smartphone processor through a fast, two-wire I2C serial bus. On Android phones, the ALS data can be accessed by user-designed smartphone applications using the Android Sensor Manager module. As shown in the posterior portion of Fig. 2(b), the responsivity for the CH0 photodiode ranges from 350 nm to 1000 nm, while the CH1 photodiode has a responsivity range covering 450 nm to 1000 nm. The CH0 photodiode has a dynamic range of 0 Lux to 20000 Lux with a resolution of 0.01 Lux. Due to its high dynamic range and resolution, the ALS can be an excellent alternative to a laboratory-grade photodetector, which may find usability in many spectroscopic applications.

2.3 LED flashlight

The LED flashlight used in the smartphone is a bright white LED with emission wavelengths ranging from 400 nm to 750 nm. Figure 2(c) shows the emission spectrum of the LED flash embedded in a typical smartphone. The typical power level of the flashlight is 4.9W and the LED pulse width ranges between 20 ms and 200 ms [48]. When combined with external optical filters, the embedded LED flash can be used as a light source for many sensing applications.

2.4 Micro-USB port

The micro USB port of the smartphone is typically used to charge the smartphone battery; however, the charging port can be used to interact with peripheral devices such as flash drives through the USB On-The-Go (USB-OTG) protocol, a communication specification that provides access and storage of data on the host device. The USB-OTG cable can also be used to power external LEDs. Figure 2(d) shows the circuit diagram for connecting an external LED to the micro USB port of the smartphone. The output current rating of the smartphone micro USB port at 5 V is 500 mA. A resistor of 250 Ω can be used to limit the current to illuminate an external LED.

3. Optical configurations of smartphone-based spectroscopic platforms and their biomedical applications

The advancements in smartphones have enabled the development of inexpensive, portable, and self-contained smartphone-based spectroscopic systems. These systems are largely based on absorption, reflectance or fluorescence spectroscopy. Figure 3 provides an overview of the instrumentation involved and process flow of a general smartphone-based spectroscopic system. Initially, light from a source (halogen lamp, phone flashlight, or sunlight) interacts with the sample based on the respective spectroscopic modality. The sample-modulated light (in reflection or transmission) is then dispersed either using a transmission or reflection element (typically a grating or prism) and enters the camera aperture, whereupon it is captured by the CMOS camera sensor of the phone. The spectrum, in the form of an image, can be visualized on the display unit of the smartphone. The spectrum is then digitally processed, which includes converting to the necessary color space and performing pixel-to-wavelength conversion to obtain the corresponding intensity vs wavelength curve. Generally, the analytes are detected and estimated from a calibration equation, which is generated from a calibration curve. Some biosensing applications detect a shift in wavelength, as shown in Fig. 3. Finally, the results are saved in the phone memory or transmitted to a required location. If necessary, external optical components (e.g., lens, pinhole, grating) may be enclosed in a custom-designed holder, which can be fabricated by 3D printing and attached to the smartphone. The design of the holder primarily depends on the position of the functional components in the smartphone. This requirement poses significant challenges in developing a universal smartphone sensing system, since the position of these functional components varies from phone to phone. The optical design and configurations of these spectroscopic systems are optimized to facilitate integration with the smartphone.

 

Fig. 3. Overview of possible configurations used in different smartphone-based spectroscopic systems.

Download Full Size | PDF

In 2008, Wang et al. demonstrated the first application of smartphones for visible light spectroscopy by attaching a transmission grating as a wavelength-selective element onto the lens of the smartphone camera [49]. Smartphone-integrated spectroscopy systems have since been utilized for vast biomedical applications. In what follows, we discuss the systems that have been demonstrated and their applications in biomedical science and technology, segmented by spectroscopic modality.

3.1 Smartphone spectroscopic systems developed based on absorption spectroscopy

All smartphone cameras contain an in-built lens unit to focus light from the object to its sensor; therefore, the easiest way to develop a smartphone spectrometer is to place the dispersive element directly in front of the phone camera to capture the wavelength spectrum. Smith et al. used this configuration to demonstrate the first biomedical application of a smartphone spectrometer [18]. As shown in Fig. 4(a), a light beam interacts with the sample and then propagates through a plastic holder having two slits of width 1 mm on both sides. The holder was fabricated in such a way that it makes an angle of 45 degrees with a transmission grating of 1000 lines/mm, which is essential to record the first-order diffracted wavelength spectrum by the phone camera. The system was implemented on an iPhone 2G cellphone, which contains a 1600-pixel x 1200-pixel CMOS sensor with a 2.2-µm pixel size. With this camera, they reported a spectral resolution of 5 nm over a 300-nm bandwidth. In order to demonstrate the potential biomedical applications of the reported system, the transmission spectrum of 1 cm of human tissue was recorded by inserting a finger in the path between a 60-W Tungsten bulb and the spectrometer slit. The color bands in the middle of Fig. 4(a) show the spectrum captured by the phone from the tungsten bulb and the finger, respectively, and the bottom figure represents the resulting transmission spectrum generated after data processing.

 

Fig. 4. Smartphone spectroscopic systems developed based on absorption spectroscopy with a transmission grating. (a) Schematic of smartphone-based spectroscopic system developed by Smith et al. and spectrum captured for a tungsten bulb and human finger, reproduced from Ref. [18] with permission of plos.org. (b) Schematic of the system developed by Long et al. and the spectrums captured by the system, reproduced from Ref. [50] with permission from Optical Society of America. (c) Schematic and photograph of the system developed by Dutta et al. and the plot of spectrum captured for different BSA protein concentrations, reproduced from Ref. [51] with permission from Royal Society of Chemistry.

Download Full Size | PDF

Subsequently, Long et al. used a similar configuration to perform Enzyme-Linked Immunosorbent Assays (ELISA) at biologically relevant concentrations [50]. ELISA is one of the most widely used biological assays for quantification of proteins and antibodies for diagnosis of diseases ranging from cancer to HIV. The antibody-antigen interaction in an ELISA test yields colorimetric changes to the liquid sample. The absorption of wavelengths generates a dark band in the captured spectrum, as shown in Fig. 4(b), where the bottom portion demonstrates the intensity plot vs. wavelength range for different dilution. The system was developed using an iPhone 4 embedded with a 2592-pixel × 1936-pixel CMOS image sensor, which achieved a spectral resolution of 0.334 nm/pixel with a 1200-lines/mm grating. A smartphone spectrometer in a similar transmission configuration (Fig. 4(c)) was used by Dutta et al. to demonstrate its usability for the detection of bioconjugation events using localized surface plasmon resonance (LSPR) as the sensing scheme [51].

Reflection grating-based absorption spectrometers have also been demonstrated for various biosensing applications. Wang et al. demonstrated a novel standalone smartphone sensing platform that does not require any external light source, lens or filter [42]. In this work, the flashlight of the smartphone was used as a light source, and a reflective compact disk (CD) grating placed at a distance of 50 mm from the phone served as the dispersive element. As shown in Fig. 5(a), light from the flashlight interacts with the sample solution after passing through a 1-mm pinhole; the modulated light gets dispersed and reflected by the CD grating, which is then captured by the phone camera. This system was used to detect glucose utilizing a well-known bienzymatic cascade assay. Figure 5(a) also shows the spectra captured by the phone at different times. Since this system does not need an external light source or optics, it reduces the overall complexity and showcases the potential of smartphone-based spectroscopic systems to be self-contained, which is highly useful for field testing and home diagnostics. Similarly, a reflection grating-based configuration was used by Ding et al. for the development of a spectroscopic system for quantifying creatinine concentration with high spectral accuracy [52].

 

Fig. 5. Smartphone-based spectroscopic systems based on absorption spectroscopy with a reflection grating. (a) Schematic of the system developed by Wang et al. using flashlight as light source and the corresponding spectrum recorded for different glucose concentration, reproduced from Ref. [42] with permission from Royal Society of Chemistry. (b) Spectroscopy system developed by Jian et al. and recorded monochrome spectrum for different R6G concentrations, reproduced from Ref. [53] with permission from Elsevier.

Download Full Size | PDF

One problem with using the flashlight as a light source is that its emission spectrum is not distributed evenly in the visible wavelength range, as can be seen from Fig. 2(c). Due to this reason, broadband sources such as halogen lamps are used to provide a more evenly distributed spectrum in almost all smartphone-based spectroscopic systems. These sources are difficult to integrate into a hand-held and portable smartphone spectrometer, however, due to their size and need for an optical fiber cable for light transmission, a driver circuit, and external power. To mitigate this issue, Jian et al. demonstrated the use of sunlight as the illumination source, which has a more uniform spectrum in the visible wavelength range than both the smartphone flashlight and halogen lamps [53]. The spectrometer was designed using a smartphone with a monochrome CMOS sensor (4224-pixels × 3192-pixels, 1.12-µm pixel size), which reduces the color overlapping that occurs in traditional CMOS sensors with Bayer color filters. A resolution of 0.276 nm/pixel was demonstrated with this system over a wavelength range of 380 nm to 760 nm. The stability of the developed platform was established using standard Rhodamine 6G (R6G) diluted with deionized water. Figure 5(b) shows the monochrome spectrum captured by the system and the intensity variation for different R6G concentrations. The sunlight-based smartphone spectrometer was further used for detecting avian influenza virus (AIV) H7N9 and porcine circovirus type 2 (PCV2) antibodies.

Many label-free biosensing methods such as surface plasmon resonance- (SPR), evanescent wave- (EW), or photonic crystal-based (PC) biosensors are based on the intensity variation or shift in wavelength of the transmitted light from the transducer due to the adsorption of biomolecules [54–56]. The shift in wavelength is usually measured by a spectrometer. Gallegos et al. demonstrated a label-free photonic crystal biosensor, where the smartphone camera was used as the detector [57]. In this work, the PC structure was designed to behave like a high-efficiency narrowband reflectance filter (565-nm central wavelength with 5-nm bandwidth), which allows all wavelengths to transmit through the PC except the resonantly reflected wavelength, as shown in Fig. 6(a). Upon adsorption of biomolecules on the PC surface, the effective refractive index of the resonant mode increases, which results a shift in the resonantly reflected wavelength. The magnitude of this shift is proportional to the optical density of the adsorbed molecule. A smartphone-based spectroscopic system was developed to detect this shift in wavelength, and its bio-detection capability was demonstrated by detecting immunoglobulin G (IgG) using an immobilized layer of Protein A on the PC surface. The optical design is similar to that of a smartphone-based absorption spectrometer, except the cuvette was replaced by the PC surface in the optical path. The adjacent sub-figure in Fig. 6(a) shows the schematic and the fabricated device. The dark band in the captured spectrum represents the wavelength band that is reflected resonantly, and the plot shows the corresponding transmission spectrum generated by the system. A similar configuration was used to develop an evanescent wave-coupled spectroscopic sensing system [58]. Using a right-angled glass prism, the evanescent field generated due to total internal reflection was allowed to interact with the external medium, which was attached to one face of the prism. The smartphone-spectroscopic system was used to detect the shift in wavelength and the corresponding analyte concentration.

 

Fig. 6. Biosensing systems demonstrated with smartphone-based spectroscopic platforms. (a) Schematic and process flow of the photonic crystal biosensor developed by Gallegos et al., reproduced from Ref. [57] with permission from Royal Society of Chemistry. (b) Schematic of fiber optic SPR biosensor developed by Bremer et al. and the plot of shift in spectrum due to RI, reproduced from Ref. [59] with permission from Optical Society of America.

Download Full Size | PDF

Both the photonic crystal- and evanescent wave-based sensing systems reported above are based on a free-space optical design. These systems require several external components to guide the light from the external light source to the camera via the optical transducer, which makes the overall footprint relatively bulky and costly. To reduce the overall size and cost of such wavelength- or intensity modulation-based sensing system, Bremer et al. reported the development of a fiber optic smartphone-based SPR sensing system. It has a very small footprint, the required optical coupling and alignment are simple, and there is no need for external prisms or lenses [59]. SPR sensors are based on the resonant excitation of surface plasmon waves (SPW) or the electron density oscillations in a metal-dielectric interface caused by incident light having a propagation constant equal to that of SPW. The associated transverse magnetic polarized waves of the SPW are guided parallel to the metal-dielectric interface during resonance, which produces a dip in the wavelength spectrum of the transmitted light. Since the propagation constant of the SPW depends on the refractive index of the surrounding medium, SPR can be used for highly sensitive biosensing applications. The SPR sensor was fabricated by coating 10 mm of the core of an optical fiber with a thin silver layer. As shown in Fig. 6(b), the end faces of the optical fiber were polished to 45 degrees in order to directly couple the light from the smartphone flashlight via the SPR sensing region. The wavelength shift of the SPR due to the change in refractive index of the sample was detected by dispersing the light into the camera using a grating. The adjacent sub-figure in Fig. 6(b) shows the spectrum captured by the system and the shift in wavelength due to variation of the refractive index. Since the SPR sensor is based on a fiber optic waveguide, it is possible to integrate the whole system within the protective cover of the smartphone; thus, the system can be implemented as a low-cost and disposable lab-on-a-chip system, as needed in many biomedical applications.

3.2 Smartphone spectroscopic systems developed based on reflectance spectroscopy

Most smartphone-based spectroscopic systems have been designed to measure liquid samples, but many biomedical applications require detection of analytes from solid samples such as tissue, paper-based biosensors etc. Reflectance spectroscopy is generally used for such applications. Hossain et al. demonstrated the first application of reflectance spectroscopy in a smartphone [30]. As shown in Fig. 7(a), a flexible endoscopic fiber bundle was used as a reflectance probe, which was integrated into the phone spectrometer platform. Light from the in-built phone flashlight was coupled to the endoscopic fiber with a custom-developed polymer light-guide and was delivered to the sample using six fiber bundle rings. Reflected light was collected through the collection fiber bundle as shown by the adjacent sub-figure. The collected light was then collimated using a lens and subsequently diffracted by a reflective grating. The dispersed spectrum was captured by the phone camera and later processed with a custom-designed phone application to generate the reflectance spectrum. With a Samsung Galaxy smartphone (13-MP camera), a spectral resolution as high as 2.0 nm was obtained over a bandwidth of 250 nm with a slit of width 0.7 mm.

 

Fig. 7. Smartphone spectroscopic systems developed based on reflectance spectroscopy. (a) Optical design and photograph of the smartphone reflectance spectroscopy system developed by Hossain et al., reproduced from Ref. [30] with permission from Optical Society of America. (b) Schematic of the diffused reflectance system developed by Hong et al. and the plot of recorded spectrum with human mucosa tissue, reproduced from Ref. [63] with permission from Nature publications. (c) Process flow and Schematic of G-Fresnel diffused reflectance spectroscopy system developed by Edwards et al., reproduced from Ref. [33] with permission from Nature publications.

Download Full Size | PDF

Many paper-based assays, such as urine dipsticks, are analyzed by visual comparison against a standard color chart. Since this method depends on the user’s color perception and the lighting conditions, it is difficult to differentiate color variation accurately. Moreover, the colored compound produced in the test strip usually has a complex wavelength spectrum: that is, many wavelength components are combined to produce the final visible color. Analysis of subtle color differences can provide accurate information and can be detected by analyzing the scattered light by means of reflectance spectroscopy. Woodburn et al. developed a smartphone-based reflectance spectroscopic platform for the analysis of paper-based colorimetric assays [60,61]. Similar to the work reported by Hossein et al., in this work, white light from the smartphone's flashlight was coupled to an optical fiber and made incident on test strips housed in a custom-designed cartridge. The scattered light was then collected by another optical fiber. The cartridge was designed to manually slide over the system, and the wavelength spectrum generated by a transmission grating was recorded as a video file. The video was processed with a custom-designed application to obtain the characteristic wavelength spectrum from the multi-analyte test strips. The developed system can be used for analyzing different paper-based assays to obtain accurate and precise results, more specifically for the class of assays where conventional phone-based colorimetric detection or analysis of the red-green-blue pixel values of a camera image is not sufficient to measure the complex scattered spectra. A similar strategy was used by Bayram et al. in developing a portable reflectance spectrometer for colorimetric detection of Bisphenol-A, which is a well-known endocrine disruptive agent [62].

Diffuse reflectance spectroscopy has been extensively used for many biomedical applications where non-invasive investigation is required. In this spectroscopic method, incident light penetrates deeply into the tissue, gets absorbed by chromophores and is scattered by cellular and intercellular entities. The modulated light re-emerges to the surface carrying information about chromophore concentrations and the scattering properties of the tissue. Diffuse reflectance spectroscopy systems are often bulky and costly due to the need for a traditional spectrometer and heavy computational requirements. An affordable, easy-to-use and portable diffuse reflectance spectroscopy system could significantly improve accessibility to the technology, especially in low resource settings. Hong et al. demonstrated a dual-modality smartphone-based microendoscope system that integrates quantitative diffuse reflectance spectroscopy and high-resolution fluorescence imaging for quantification of physiological and morphological properties of epithelial tissues [63]. Figure 7(b) shows the schematic diagram of the system, which consists of a Samsung Galaxy S6 smartphone, a 3D-printed attachment for holding optical components, a fiber-optic microendoscope and an app for data analysis. Light from a 20-mW white LED was delivered to the tissue through two multimode optical fibers (200-µm core diameter), and the diffusely reflected light was collected using a single detection fiber of the same core diameter. The collected light was then propagated through a 100-µm slit and collimated by a collimating lens. A transmission grating (1200 lines/mm) diffracted the collimated light and then imaged it by the phone camera. The diffuse reflectance spectrum collected by the system was wirelessly transmitted to a server through the developed app. The data processing module in the server automatically processed the data and sent back the results to the app for display. A spectral resolution of 2 nm was obtained over a spectral range of 395.5 nm to 693.3 nm. The feasibility of the system in characterizing the properties of epithelial tissue was tested in a single human subject in vivo. Spectra were recorded from oral mucosa, including labial mucosa tissue, gingival tissue and tongue dorsum tissue, where the α and β bands of oxy-hemoglobin were clearly visible, as shown in the sub-figure of Fig. 7(b). The differences in shape and intensity of the measured spectra from the oral tissues represent their underlying differences in physiological and morphological characteristics.

Hemoglobin is an important biomarker for early diagnosis of several malignancies. Hypoxia and angiogenesis are two crucial features for the growth of tumors; measurement of oxy- and deoxy-hemoglobin non-invasively could potentially be used as an indicator for early detection of different forms of cancer, such as oral cancer, cervical cancer and breast cancer. Thus, the development of an affordable and portable systems for measurement of hemoglobin in local tissue is an utmost need for point of care applications. Edwards et al. from the aforementioned group developed a similar smartphone-based system for diffusive reflectance measurement of hemoglobin in a tissue phantom [33]. The system was designed to operate over a broad wavelength range: 400 nm to 1000 nm. Since the phone camera is limited to work only in the visible range (400 nm-700 nm), an external camera, which works in the visible to near infra-red range, was used to collect the spectra. As shown in Fig. 7(c), the spectroscopic system with the external camera was connected to the phone using the micro-USB port of the smartphone for operational control. An app was developed to communicate between the phone and the USB camera to record the spectrum and compute the hemoglobin concentration. With the developed system, a mean error of 9.2% was obtained in the measurement of hemoglobin concentration in comparison to the results obtained with a commercial benchtop spectrometer. Considering the affordability and portability of the presented device, the developed system has the potential to be used as a point-of-care device for cancer screening in resource-limited settings.

3.3 Smartphone spectroscopic systems developed based on fluorescence spectroscopy

Fluorescence is an inherent property of certain molecules whereby they emit light at a higher wavelength when irradiated by light falling within a certain excitation band. The measurement of fluorescent intensity allows the determination of the presence of fluorophores and their concentration. Fluorescent tags have been extensively used for many biological applications including disease diagnosis, proteomics, drug discovery, and life science research. Although fluorescence-based detection methods are highly sensitive and specific to the target molecule, the instrumentation required is difficult to use outside of a standard laboratory. The availability of a portable and low-cost system for the detection and analysis of fluorescence signals could help to translate lab-confined methods to the point of care. Many papers have been written about smartphone-based fluorescence systems that measure a single or few wavelengths to capture fluorescence [64–66]. Very few works have been written that perform true fluorescence spectroscopy, where they are capable of capturing the full fluorescence spectrum. This section focuses on the latter.

Yu et al. demonstrated the first application of a smartphone spectroscopic system for read-out of fluorescence-based biological assays [67]. The developed system was used to perform a sensitive molecular beacon Foster resonance energy transfer (FRET) assay to detect specific nucleic acid sequences from a liquid sample. FRET is a mechanism to observe changes in the quenching efficiency between matched donor-acceptor pairs of molecules. This assay is performed by adding the analyte to be detected to a solution containing a flurophore-tagged probe molecule that specifically recognizes the target analyte, as shown in Fig. 8(a). FRET is very effective for diagnostic applications because it is a single-step assay without need of washing steps. A green laser pointer (power = 300 mW and wavelength = 532 nm) was used to excite the fluorescent emitters placed in a transparent cuvette, and the light emitted by the sample was collected through an optical fiber placed at an orthogonal angle in order to minimize the light collected from the excitation laser. The output from the optical fiber was fed to a smartphone-based spectrometer previously demonstrated by the same group [57]. The developed system performed the assay with better sensitivity and specificity than a laboratory fluorometer and detected miRNA sequences with a limit of detection of 10 pM.

 

Fig. 8. Smartphone spectroscopic systems developed based on fluorescence spectroscopy. (a) Schematic of the fluorescence spectroscopy system developed by Yu et al. and the mechanism of FRET assay, reproduced from Ref. [67] with permission from American Chemical Society. (b) Schematic of the system developed by Hossain et al. and the electronic circuit diagram used to power the external LED with smartphone battery, reproduced from Ref. [68] with permission from Optical Society of America.

Download Full Size | PDF

To make the fluorescence detection more self-contained, Hossein et al. demonstrated the use of a smartphone battery to power external light sources in a smartphone-based fluorescent spectroscopic system [68]. In this work, a custom gold-coated polymer grating was used as a dispersive element, which can be fabricated inexpensively using nano-imprinting as compared to a commercial grating. The system was designed to detect the spectrum from two analytes: a pH-sensitive amino-phthalimide fluorescent probe and a Zn2+-sensitive fluro-ionophore. As shown in Fig. 8(b), the excitation LEDs were powered using the phone battery and placed at an orthogonal angle to the sample cuvette; the emitted fluorescence spectrum from the sample was dispersed by a reflection grating and imaged by the phone camera. The captured image was then processed by a custom-developed Android application to generate the fluorescent intensity vs. wavelength curve within the app interface. Ding et al. further demonstrated a smartphone-based fiber optic fluorescent spectroscopic system for mHealth applications [52]. The developed system was used to detect creatine and urinary glucose concentration.

4. Discussion and potential roadmap for future strategies

We have described the major spectroscopic modalities that have been implemented using a smartphone. Many of them were developed with the key goal of transforming lab-confined healthcare applications into point-of-care assays to improve accessibility for people from all economic and social backgrounds. The availability of these systems will significantly impact the healthcare scenario in resource-constrained settings. According to the guidelines of the World Health Organization, systems should closely follow the ASSURED (Affordable, Sensitive, Specific, User-friendly, Rapid and robust, Equipment-free, and Deliverable to end-users) criteria to ensure proper implementation in resource-constrained settings [69]. Some of these criteria are inherently met by systems based on smartphones [70], but there is still some room to further reduce the cost. A large contributor to the cost is the use of costly optical components, such as commercial dispersive elements and lenses for collimation and focusing. For example, the cost of components involved in fabricating the smartphone spectrometer used by Gallegos et al. for biomolecular detection is $210 (excluding the smartphone), where more than half of the overall cost is due to the diffraction grating ($82.78) and the lenses ($74) [57]. Similarly, the total cost of the smartphone spectrometer system developed by Woodburn et al. for colorimetric analysis is $550 (including the smartphone) using similar optical components [60]. Keep in mind, however, that the reported costs for these devices is often higher than their cost once manufactured at scale, and demonstrated smartphone spectrometers systems are still much cheaper than portable commercial spectrometers (∼ $2000). For in-field applications where a large number of samples needs to be detected at the same time due to constraints such as low availability of skilled personnel and consumables, high-throughput and multimodal detection are equally critical as affordability and portability [71]. Unfortunately, most smartphone spectroscopic systems are designed to perform single-analyte testing at a time.

4.1 Strategies to improve affordability and compactness

To meet the need for affordability, alternative strategies of using a digital versatile disc (DVD), compact disk (CD) (∼$0.25) or custom-designed Fresnel lens as a grating element have been proposed. CDs and DVDs comprise periodic metal-coated grating structures incorporated in a polycarbonate substrate. These structures can be used as a reflection grating or a transmission grating after removing the reflective metal layer. Wang et al. demonstrated a DVD grating-based smartphone spectroscopic system to detect neurotoxins [72] using a similar configuration to that demonstrated in section 3.1 (Fig. 9(a)). The DVD used in this work has a grating period of 710 nm ± 19 nm. The usability of the system was investigated by comparing the absorbance measured for Rhodamine B with a commercial microplate reader and when using a commercial grating (1200 grooves/ mm, Thorlabs) in the same system; the authors confirmed that the DVD grating could achieve similar performance to that of the commercial grating. Similarly, Kong et al. investigated the usability of a CD as a dispersive element in a smartphone-based spectroscopic system and demonstrated its applicability for highly sensitive and cost-effective detection of ascorbic acid [73]. Zhang et al. eliminated the use of a focusing lens as a spectrometer component by using a custom-designed Fresnel lens, termed a G-Fresnel, for both focusing and dispersing light [74]. The G-Fresnel element was fabricated by sandwiching the corresponding negative PDMS molds of both a grating and a Fresnel lens. PDMS molds can be easily fabricated in an affordable way through the surface-molding method. As shown in Fig. 9(b), since the G-Fresnel element can both collimate and disperse the light, it significantly reduces the overall size of the smartphone spectroscopic system. A spectral resolution of 1.6 nm was achieved at 595 nm, which is more than sufficient for many biomedical applications. The usability of the system was demonstrated by measuring protein concentrations in the well-known Bradford assay.

 

Fig. 9. Strategies to reduce cost and size. (a) Schematic of the smartphone spectroscopic system developed using a DVD as a dispersive element and the SEM image of the DVD, reproduced from Ref. [72] with permission from American Chemical Society. (b) Schematic of the G-Fresnel element acting both as a dispersing and collimating unit and the corresponding spectroscopic system developed by Zhang et al., reproduced from Ref. [74] with permission from Royal Society of Chemistry.

Download Full Size | PDF

Besides linear gratings, other types of gratings such as the stacked, mutually rotated diffraction grating from SpectroClick are commercially available [75]. These gratings are manufactured in plastic films, which makes them very affordable ($1) for enabling the development of low-cost spectroscopic devices [76]. One way of reducing both the cost and size of spectrometers is to use pixel-level spectral filter arrays covering wavelength bands outside of those used by the traditional RGB Bayer color filters. In this configuration, the wavelength response at every pixel can be calculated using a suitable demosaicing algorithm [77]. This method is most commonly used in snapshot spectral imaging systems [78]. Since the spectral filtering is performed in the detection layer, the overall footprint of the system would be very small, making it suitable to integrate into a smartphone as a standalone spectral sensor. For example, Bao and Bawendi developed a quantum-dot spectrometer in which each pixel was covered by a filter comprising a unique, heterogeneous mixture of quantum dots with varying responsivities [79].

Linear variable filters have also been demonstrated to enable low-cost hyperspectral imaging systems [80], where the bandpass filter is directly placed above the image sensor to capture the corresponding wavelength spectrum. A miniaturized version of such filters could be used to enable the capture of spectroscopic data by directly placing it on top of the smartphone camera. In fact, such filters could even be used to enable responsivity outside of the traditional spectral sensitivity of the camera if the filters themselves are capable of converting light from one wavelength range to another. We demonstrated this general idea by using a miniature filter composed of quantum dots to capture UV data using a smartphone [81]. Another potential way of developing a standalone spectral sensing unit within a smartphone is to use interferometric techniques for wavelength filtering. Custom-designed Mach-Zehnder crystal array interferometers can be fabricated at the micro-scale using silicon photonics technology [82]. Similar to an FTIR spectrometer, the smartphone camera could be used to capture the interferogram generated by an interferometer array with known path differences, and the corresponding wavelength spectrum can be generated from the interferogram using well-known Fourier transform techniques.

4.2 Strategies to enable multiplexed operation

Owing to the 2D nature of the smartphone camera, different research groups have demonstrated high-throughput and multichannel spectroscopic detection in a smartphone. Wang et al. demonstrated the first multichannel smartphone spectroscopic systems for high-throughput point-of-care diagnostics [45]. As shown in Fig. 10(a), light from a backlight panel, which was used as a light source, initially propagates through an aperture array with an aperture diameter of 6 mm to separately illuminate eight individual micro-wells of a 96-well microplate. To reduce spectral cross-talk, another aperture array of 4-mm diameter was placed above the micro-well array. A PDMS micro-prism array integrated above the aperture array guides the transmitted light into the field of view of the phone camera, whereupon it is diffracted by a grating, and the phone camera captures the eight spectra individually, as shown in the adjacent figure. Two rails were used to translate the system in order to scan every column of the 96-well microplate. The usability of the system was validated by performing an immunoassay for human cancer biomarkers and measuring protein concentrations. The same group later used 3D printing to reduce the overall cost and clinically validated the system by detecting autoantibodies from human serum samples and comparing the results with an FDA-approved instrument [83]. A similar strategy was used by Fan et al. to develop a smartphone-based multi-spectral platform for detecting multiple biomarkers with a microfluidic chip [84]. As shown in Fig. 10(b), a micro-hole array and a micro-lens array were used to separately illuminate and record the spectrum from each channel of the microfluidic chip. This system was used to measure the concentrations of protein solutions, sucrose solutions, and serum specimens. Lo et al. demonstrated similar multichannel detection capability using a lightweight plastic aspheric concave blazed grating. Biswas et al. further exploited the multi-order characteristics of the diffraction pattern to develop a two-channel spectroscopic system [85]. All these reported works confirmed that smartphone-based spectroscopic systems have the potential to be used as a multi-testing platform when required.

 

Fig. 10. Strategies for multiplexed operation. (a) the schematic of the multiplexed system and the recorded spectrum developed by Wang et al., reproduced from Ref. [45] with permission from Elsevier. (b) Microfluidics based multiplexed detection system developed by Fan et al., reproduced from Ref. [84] with permission from MDPI.

Download Full Size | PDF

We proffer that the complexity of the proposed multiplexed systems can be further reduced by considering the use of parallel spectral acquisition. In a carefully designed experiment, this can be achieved with line illumination and can eliminate the use of multiple light sources as discussed above. Line illumination can be easily generated using a cylindrical lens. For example, the spectra from multiple microfluidic channels could be acquired simultaneously by illuminating the channels with line illumination of suitable length. Since the smartphone camera has a 2D image sensor, the spectrum from all the points of the line illumination can be captured in parallel.

4.3 Strategies to improve spectral resolution

Another key factor for any spectroscopic system is the spectral resolution, which inherently depends on the optical design and components involved to develop it. The number of pixels present in the phone camera sensor can play an important role in the overall spectral resolution of the system. Table 1 provides a comparison of some of the demonstrated spectroscopic systems in order of pixel count to showcase the impact of phone camera pixel resolution on the spectral resolution. All of the systems in this table use the same dispersive element, a transmission grating of 1200 lines/mm. It can be seen that the spectral resolution increases from 0.33 nm/pixel for a 5-MP camera smartphone to 0.19 nm/pixel for a 20.7-MP camera smartphone. Although the currently achieved spectral resolution is perfect for the demonstrated applications, other biomedical applications based on different spectroscopic imaging techniques (e.g., hyperspectral and multispectral) may require even higher spectral resolution, as the spectral resolution could significantly impacts the overall imaging capability of the system [86]. In light of Moore’s law, we anticipate that the quality of the CMOS sensor will improve over time with the integration of more pixels, which can aid in increasing the spectral resolution of smartphone-based spectroscopic systems [87]. Another way to increase the spectral resolution is to improve the integrated optical design to cover more pixels for the target wavelength range. A higher pitch grating can disperse the light more broadly, yielding higher resolution. If multiplexed operation is not necessary, one may consider designing a 2D spectrometer, such as based on an echelle grating configuration used for solar applications [88]. Besides hardware, computational algorithms such as the high-throughput computational slit (HTCS) method can be implemented as post-processing steps to enhance the spectral resolution [89], or one can implement other methods that combine compressed sensing with non-linear dispersion, which have been shown to yield better spectral resolution than could be anticipated using a traditional configuration [90].

Table 1. Number of camera pixels vs. Spectral resolution

View Table

4.4 Strategies to improve the detection range

Most of the systems discussed in the above sections were designed to work in the visible range due to the limited spectral responsivity of the embedded camera sensor. As discussed in section 2, due to the presence of the infrared cut- off filter, the camera sensor is responsive only within the visible wavelength range, 400 nm to 700 nm. Yet a vast number of biomedical applications require spectral responsivity in the ultraviolet (UV) or infrared (IR) range. One way to use a smartphone camera as a detector for such applications is to convert the light to visible range via optical transduction using nanoparticle-based methods [81]. Alternatively, one might consider using the embedded ALS as a photodetector [91] collect data in the near-infrared (NIR) spectral range,. As shown by Fig. 2(b) of section 2, the ALS is responsive from the visible to NIR wavelength range (CH0). Pereira et al. demonstrated an ultra-low-cost spectrophotometer (less than $5) using the ALS and verified its usability with a protein assay [28]. As shown by Fig. 11(a), the system simply consists of an LED-powered with a coin-cell battery and a 3D-printed cradle to hold the cuvette. Light passing through the cuvette after its interaction with the analyte sample was detected by the ALS and the developed application then quantifies the concentration. As the ALS comprises a single photodetector, it can only measure photocurrent from a single wavelength at a time, not a spectrum. Hence, one possible way to measure the spectrum for different wavelengths is to use different LED sources to capture the absorption wavelengths of interest. Hussain et al. demonstrated a compact ALS-based photometric platform that works both in the visible and NIR spectral range [35]. To make the system self-contained, the LEDs were powered by the smartphone battery using the USB-OTG protocol, as shown by Fig. 11(b). The system was used to detect iron and phosphate ions in liquid samples by measuring their absorbance at 510 nm and 880 nm, respectively. This work confirms the potential utility of the ALS for developing a smartphone-based photometric platform beyond the visible spectral range. Furthermore, ALS-based systems are very useful in developing affordable and portable biosensing systems where detection can be done at a single wavelength.

 

Fig. 11. Smartphone spectrophotometric systems in NIR spectral range. (a) different parts and photograph of ALS based spectrophotometric system developed by Pereira et al., reproduced from Ref. [28] with permission from plos.org. (b)Schematic of the photometric system developed by Hussain et al. showing the location of embedded ALS chip in a smartphone, reproduced from Ref. [35] with permission from IEEE. (c) Front panel of Apple iPhone 11 smartphone.

Download Full Size | PDF

The recent inclusion of face-recognition technology in smartphones may become a platform for the development of IR spectroscopy. Figure 11(c) shows the different components embedded in the front panel of the Apple iPhone 11. The IR camera, flood illuminator, front camera, and dot projector are together called a TrueDepth camera system and are used for face recognition [92]. Many other recent Android smartphones are equipped with similar face-recognition technology. The dot projector illuminates the face with thousands of IR dots and the IR camera captures an image of the face pattern. The IR image is then fed to a neural network to confirm its similarity with the face pattern that was used during installation of the phone and is set to unlock the phone if the pattern matches. Since iPhones use an encrypted platform, there is currently no publicly available Application Programming Interface (API) to use the IR camera for functions other than face recognition. In contrast, Android is an open-source platform, and different APIs and applications are already available to use the IR camera to capture images [93]. The availability of this technology will undoubtedly create new opportunities for developing IR imaging and spectroscopic platforms for biomedical applications.

5. Conclusions

Smartphone-based systems based on different spectroscopic modalities have been successfully introduced and applied to a vast number of biomedical applications ranging from detection of biomolecules (protein, nucleic acids etc.) to non-invasive detection of hemoglobin from human tissue. This review summarized the development of different smartphone-based spectroscopic systems by highlighting the current challenges and potential solutions in achieving affordability, portability, higher accuracy and adaptability for point-of-care applications, which are important considerations for resource-constrained settings. The rapid growth of the smartphone market paves the way for the integration of more sophisticated hardware and computational power to a smartphone over the course of time.

In addition to the research systems described above, several portable spectrometers are available commercially, such as the VS20-VIS from Horiba, the USB2000+ from Ocean Insight, and the LI-180 from Licor. Their optics are distinct from components on the phone itself, but they are able to communicate with smartphones (e.g., for data processing) using various communication protocols (i.e., wi-fi, Bluetooth, or USB). Unfortunately, the overall cost of these spectrometers is still high (∼ $2000) for applications in low-resource settings. Many companies, however, have already seized the opportunity to develop smartphone-integrated commercial spectrometers due to their affordability. Changhong H2, a smartphone developed by Consumer Physics, Israel, has an integrated spectrometer that can be directly used to assess the quality of medicine and food [94]. GoSpectro is another commercially available optical attachment for the smartphone camera designed to capture a spectrum using a phone application [95].

Besides developing the smartphone as a consumer-based product, companies like Samsung are developing smartphones for military applications [96]. Use of smartphones on the battlefield will open new avenues for rapid diagnostics testing such as monitoring wound infection, spectroscopic detection of traumatic brain injury etc. The limiting factor for phone attachment-based systems is the rapid evolution of phone designs, which change every two to three years for any phone brand. It is envisioned that with the development of additive manufacturing techniques such as 3D printing technology, different innovative and universal opto-mechanical designs can be created that are suited for any phone brand. Another key limitation is the challenge of implementing high-end machine learning (ML) or artificial intelligence (AI) algorithms in lower-end smartphones. For these applications, data need to be processed on a remote server and then transferred back to the smartphone for display. The ability to effectively perform this transfer depends on several factors such as connectivity and bandwidth. The recent development of 5G technology enables high-speed data transmission in an affordable way; therefore, it is envisioned that low-end, 5G-connected phones may be able to incorporate processing from sophisticated algorithms a combination of using cloud-based processing and high-speed transfers. For now, widespread adaptation of smartphone-based spectroscopic systems may seem unlikely, but we expect that given the utility of spectroscopic analysis, this technology will eventually become ubiquitous, similar to the adaptability of billions of smartphones currently blanketing the world.