The acoustically modulated laser speckle contrast technique has been employed to quantify and classify 25 colors (made up by different percentages of the two base colors cyan and magenta) hidden behind a 5 mm thick opaque layer with 0.24% transmittance. The main components included two He-Ne lasers (543 and 633 nm), a consumer grade digital camera (Nikon 1 J1), focusing optics and a loudspeaker. The camera captured the laser speckle patterns with the sound on and off, respectively, from which the speckle contrast differences were calculated and used in a nearest neighbor classification algorithm. The classification accuracy was between 55% and 88% depending on the underlying reflectance of all the colors to be classified.
© 2013 OSA
When coherent light interacts with moving particles in a scattering medium, a time-varying speckle pattern is resulted which indicates the dynamics and properties of the medium. In recent years, techniques based on laser speckle have been widely adopted in different areas of biomedical imaging including laser speckle contrast imaging (LSCI) for blood flow imaging , ultrasound modulated optical tomography (UOT) for tissue and shear wave imaging [2, 3] and acousto-optical elastography for skin lesion imaging .
Apart from imaging, we have demonstrated recently that laser speckle can also be used for spectroscopic measurement . In this previous study, we investigated the acoustic modulation of visible light (633 nm) with audible sound (200 Hz), a technique known as acoustically modulated laser speckle sensing . We employed a 4-phase stroboscopic detection scheme  with one He-Ne laser to detect the acoustically modulated optical signal and showed that this technique can measure the reflectance of a colored object hidden behind an opaque layer. The technique can potentially be applied to measure tissue oxygenation below a superficial layer, e.g. non-invasive cerebral oxygenation measurement beneath the skull.
In the present paper, we investigate the problem further with two He-Ne lasers (543 nm and 633 nm) and a different detection scheme based on speckle contrast , which is a more suitable technique to be translated into clinical practice because it is less susceptible to inaccuracy caused by the short correlation time of the tissue than the 4-phase stroboscopic detection scheme .
The aim of the present study was to quantify and classify a total of 25 color objects hidden behind a 5 mm thick opaque layer with 0.24% transmittance. By slightly vibrating the hidden colored object at 200 Hz and capturing the time-varying speckle pattern on the layer surface with a consumer grade digital camera, the speckle contrast difference measurements were shown to be indicative of the color of the hidden object. A classifier based on the nearest neighbor technique  was used to classify the 25 colors based on the speckle contrast differences measured at two wavelengths with accuracy higher than that based on intensity measurements.
2.1 The speckle contrast technique
In this work, we have employed the speckle contrast technique [1, 2] to quantify the dynamics of the acoustically modulated laser speckle pattern. This technique exploits a digital camera with a long exposure time (relative to the speed of the particle movement) to capture an image of the time-varying speckle pattern. Because of the movement artifacts, the resulting image becomes blurry. The blurriness can be quantified by calculating the speckle contrast C = σ/< Ī >, where σ and < Ī > are the standard deviation and mean of the intensity of all the pixels in the region of interest. The faster the moving particles, the more blurry the image becomes and the smaller the speckle contrast. When the medium is acoustically modulated leading to an increase in the dynamics of scatterers in the medium, the speckle contrast will decrease. The speckle contrast difference ΔC is defined as:
2.2 The colored objects
Two base colors cyan and magenta have been used to form a total of 25 colors, each with different percentages of cyan and magenta, as shown in Fig. 1. Forming a variety of colors with two base colors in this way is analogous to the situation in blood that consists of different percentages of the two chromophores oxy-hemoglobin and deoxy-hemoglobin which determine the “redness” of blood. The 25 colors were produced with the open source software Scribus 1.4.1 and printed as squares (40 × 40 mm) on a white paper with a laser color printer (EPSON AcuLaser C1750N). Each color in Fig. 1 has been given a color ID number to facilitate discussion in the later sections. Note that conventional printers can only print a combination of base colors cyan, magenta, yellow and black (CMYK) which is the reason why cyan and magenta have been chosen as the base colors, instead of other colors such as green and blue.
2.3 The experiment
The experimental setup is depicted in Fig. 2 which is similar to the one in a previous study based on the 4-phase stroboscopic technique . An opaque scattering layer, which was made of epoxy resin with a thickness of 5 mm, reduced scattering coefficient µs’(633 nm) = 1.8 mm−1, absorption coefficient µa(780 nm) = 0.01 mm−1 and 0.24% transmittance, was securely fixed to the optical table. A colored paper (40 mm × 40 mm), which acted as a light reflector, was placed behind the layer with a 5 mm air gap in between. The colored paper was held by a connecting rod, the other end of which was attached to the center of the loudspeaker’s diaphragm. The loudspeaker was driven by a function generator (33210A, Agilent) at 200Hz with vibration amplitude of 0.5 µm. Tests have shown that increasing the vibration amplitude further will not enhance the speckle contrast difference.
In the present experiment, two He-Ne lasers, i.e. green (wavelength λ = 543 nm, 1 mW, Melles Griot) and red (λ = 633 nm, 10 mW, Melles Griot), were employed as the light sources and a consumer grade digital camera (Nikon 1 J1) as the detector. A neutral density filter was placed in front of the red laser to reduce its power to 1 mW. The detachable lens of the digital camera was taken off, exposing the CMOS sensor (dimension = 13.2 × 8.8 mm = 3,872 × 2,592 pixels, and pixel size = 3.4 µm) in the air and a lens tube was placed in front of it to reduce ambient light. The digital camera was operated with a remote control (ML-L3, Nikon) to avoid any physical contacts with it during the experiment. A lens with a 100 mm focal length and a 10 mm diameter iris gave an f number (f/#) of 10. A magnification factor (M) of 2x was chosen to produce a field of view of 6.6 × 4.4 mm. The distance between the entry point of the laser beam and the center of the sensor’s field of view was 8 mm.
The speckle size can be calculated using an equation in : speckle size, S = 2.44λ(1 + M)f/# which resulted in S = 46 µm for the red laser speckle and S = 40 µm for the green laser speckle. For optimal speckle contrast, it has been suggested that the speckle size has to be at least twice the size of the detector pixel . The speckle size here was up to 13 times larger than the pixel size, mainly because the light source and detector here were configured in the reflection mode, resulting in an exponentially decaying light distribution on the opaque layer. A smaller field of view (larger magnification and speckle size) can therefore allow a more even distribution of light on the sensor. For the digital camera, the monochrome mode was selected, the exposure time and ISO value were set to 100 ms and 100 respectively, and all the automatic adjustments such as white balance were disabled. To avoid any automatic adjustment introduced by the image compression algorithm (e.g. JPEG), the images were captured in the raw format (NEF) and later converted into uncompressed TIF format using the Nikon proprietary software “View NX2”.
With the red laser as the light source (without mirror M1 in Fig. 1), each of the 25 colored papers was placed behind the opaque layer one after another. For each color measurement, two sets of 5 images were again taken when the loudspeaker was switched on and off, respectively. The measurement of 25 colors was repeated four times in total to provide four independent data sets. The whole process was then repeated again with the green laser as the light source (with mirror M1). For comparison, the reflectance of the 25 colored objects without the layer barrier was also measured by a spectrometer with a broadband light source (USB2000, Ocean Optics).
2.4 A model to explain the relationship between ΔC and the reflectance of the hidden object
To understand how ΔC in Eq. (1) can be used to measure the reflectance of the hidden object, we have adapted a model previously used by Li, Ku and Wang [2, 9] who investigated the speckle contrast technique for the UOT imaging applications. The full derivation, including Eqs. (2) and (3) to be introduced below, is provided in the appendix. In this model, each detector pixel in the digital camera detects two kinds of light intensity: the first kind, Ib has only interacted with the opaque layer and the second kind of light, Im has interacted with both the hidden colored object and the opaque layer. (In the case of Li, Ku and Wang’s work, Im has interacted with regions both inside and outside the ultrasound focus [2, 9].) Note that only Im is modulated by the acoustic vibration of the colored paper. It follows that the time- and spatially-averaged intensity becomes:
The variable M, i.e., the intensity ratio of <Im> to <Ib>, has been previously defined as the modulation depth by Li, Ku and Wang (2002) . When the acoustic modulation is fixed, as in our case, M will depend on Im, a value determined by the absorption (or color) of the hidden object. Figure 3 depicts the relationship between M and ΔC for a range of values of Cb. Due to the blocking of light by the opaque layer, Im << Ib and M << 1. Both Cb and Ib are determined by the optical properties of the opaque layer and are fixed in our experiment. It can be seen that ΔC increases with M, (or Im with a constant Ib) which explains why ΔC is indicative of the color of the hidden object.
2.5 Data analysis
The speckle contrast technique requires the light distribution of the image to be uniform so that any intensity variation on the image is due to the interferences of the speckle pattern only. Since the image in this experiment was taken in a reflection mode, the light distribution is not uniform but had a decreasing intensity profile, i.e., higher intensity for the part nearer to the beam spot. In order to provide a more uniform intensity distribution for the subsequent analysis, the original image (2,592 × 3,872 pixels) was divided into 16 × 16 blocks of smaller images, each has a dimension of 162 × 242 pixels (0.275 × 0.413 mm). The speckle contrast values Coff and Con were calculated for each block using Eq. (1) and then averaged across the 256 ( = 16 × 16) blocks. As mentioned in section 2.3, five images were taken when the loudspeaker was switched off and on respectively. The block-averaged speckle contrast values Coff and Con were again averaged over the five images. Finally, the speckle contrast difference was calculated based on the averaged speckle contrast values, ΔC = <Coff> - <Con>.
2.6 Nearest neighbor classifier
Four independent data sets were collected. The averaged ΔC of the four data sets were calculated and used as a training data set for the nearest neighbor algorithm , which classifies an unknown data point based on the nearest training data point using the following Euclidean distance measure:
3.1 Color quantification
Three main measurements of the 25 colored objects were performed using the green (Fig. 4) and red (Fig. 5) lasers as the light sources, respectively. The first measurement ΔC was measured with the layer barrier and sound on, using a digital camera as described in section 2.3. The second measurement, the obstructed reflectance, was measured with the same setup but with the sound off. The third measurement, the direct reflectance, was measured with a spectrometer and a broadband light source with no barrier and no sound, and the results have been extracted from the wavelengths 543 nm and 633 nm.
A colored object reflects (or absorbs) certain wavelengths of light more than the other wavelengths. For example, a red object appears red because it reflects more (or absorbs less) red light than other colors of light. It can be seen in Fig. 4 that with a green laser source, the magenta object (color ID = 5) has a direct reflectance of ~0.9%, indicating that the magenta object absorbs a fair amount of green light. On the other hand, with a red laser source as shown in Fig. 5, the magenta object (color ID = 5) has a relatively larger direct reflectance of ~3.1%, showing relatively less red light is absorbed by the magenta object. This explains why magenta appears closer to the color red than green. The colors in Figs. 4 and 5 have been arranged in such a way that the direct reflectance of the 25 colors is in ascending order. As mentioned before, since the color objects absorb red and green light differently, the order of color in Figs. 4 and 5 is also different. The other two measurements, i.e., ΔC and obstructed reflectance, follow the same color order. All the three measurements have been normalized with respect to the color with the highest direct reflectance value, i.e., color ID 1 for the green laser measurements and color ID 5 for the red laser measurements. Displaying the results in this way allow easy comparison among the three measurements. The standard deviations were calculated using the four independently measured data sets as described in section 2.3. The x-axis of Figs. 4 and 5 corresponds to the absolute direct reflectance which quantify the absorption characteristic of the colored objects. In Figs. 4(a) and 5(a), the color IDs are shown next to the colored data points along the normalized direct reflectance line.
Without the layer barrier, the direct reflectance lines (dotted) show the normalized (y-axis) and absolute (x-axis) reflectance of the 25 colors at 543 nm and 633 nm. When the layer barrier is present, the obstructed reflectance curve is mainly flat across all colors, indicating that it is insensitive to the color of the hidden object. The ΔC curves in Figs. 4(b) and 5(b), however, exhibit a similar characteristic as the direct reflectance line although certain variability is also present. One advantage of the ΔC is that the value is inherently normalized against the mean light intensity and is therefore less susceptible to the laser intensity fluctuation. However, the ΔC is subject to certain variability due to the noise in the CMOS sensor of the digital camera. This can be alleviated by using the average of a larger number of images to calculate ΔC.
3.2 Classification using the nearest neighbor algorithm
Before the performance of the classification is discussed, the underlying property of the 25 colors is first described. Figure 6 shows the direct reflectance (no barrier) of the 25 colors measured at 543 and 633 nm with a spectrometer as described in section 2.3.2. In this color space, it is evident that some colors are closer together than the others, e.g., color IDs 19, 24 and 25 are more closely spaced than color IDs 1, 2 and 3. It is understandable that closely spaced colors are more difficult to be differentiated from each other even without any barrier. To quantify the closeness of a color to another color, a Euclidean measure known as “inter-reflectance distance” (IRD) is defined here:Fig. 6, color ID 1 has an IRD of 0.69% and its nearest neighbor is color ID 2. Note that color ID 2’s nearest neighbor is not necessarily color ID 1 but in this case color ID 3, and the IRD of color ID 2 is 0.54%. In comparison, color ID 25 has a much lower value of IRD at 0.06% showing the closeness of this color to another one in this color space. Table 1 summarizes six groups of color (determined by their IRDs), their color IDs and their mean IRDs.
Figure 7 is a scatter plot of the ΔC of the 25 colors measured at 543 nm & 633 nm. Four independent series of experiments were performed with the symbols square, diamond, triangle and inverted triangle representing experiments 1, 2, 3 and 4. The symbol circle is the averaged value of the four measurements. Twenty five clusters can be seen in Fig. 7, each corresponds to one of the 25 colors. Lines were drawn linking the four measurements and the averaged value together for easy inspection. In general, the distribution of colors in Fig. 7 is similar to that in Fig. 6 (direct reflectance). For example, in both Figs. 6 and 7, color ID 1 (white) is at the top right hand corner and the bluish colors are at the bottom left hand corner.
The nearest neighbor algorithm was then employed to classify each data point of the four sets of experiments using the averaged values (symbol circle) as the training data. The algorithm calculates all the distances using Eq. (4) between one unknown data point (symbols square, diamond, triangle and inverted triangle) and each of the 25 training data points (symbol circle). The unknown data point was then classified as the color of the nearest training data point. The accuracy values were calculated as the averages over the four independent data sets.
Figure 8 shows the classification performance in terms of the mean accuracy for each color group as defined in Table 1. As expected, the obstructed reflectance measurement has a much lower accuracy than its ΔC counterpart because of the blocking of light by the barrier. The ΔC measurement will be the main focus in the following discussion. It can be seen that when a color group, e.g., color group 1, has a small underlying mean IRD, their mean classification accuracy based on the ΔC measurement is also lower. Figure 7 also shows that the colors in group 1 (color IDs 16, 17, 18, 21, 25) are located very close to each other in the color space based on ΔC, making them very difficult to be differentiated from each other and therefore the low classification accuracy. The colors in groups 5 and 6 (color IDs 1, 2, 6 and 7), on the other hand, have larger IRDs and they are also well separated in Fig. 7, leading to higher classification accuracy. In general, as the underlying IRD increases, the classification accuracy using the ΔC measurements also increases, ranging from accuracy = 55% (mean IRD = 0.13%) to 88% (mean IRD = 0.61%).
4.1 Potential Biomedical applications
In many non-invasive tissue oxygenation measurements, the region of interest is often beneath a superficial layer, e.g., cerebral oxygenation measurement beneath the scalp and skull, and muscle oxygenation measurement beneath the skin and fat layers. A measurement technique that can increase the sensitivity of the spectroscopic measurement in the deeper layer, such as the acoustically modulated speckle contrast technique, can therefore potentially improve the accuracy of the measurement.
Although outperforming obstructed reflectance measurement, the results shown in Fig. 8 indicates that the current system still may not have sufficient sensitivity to detect small changes in cerebral oxygenation (small IRD) which requires a higher degree of sensitivity to the blood color changes in the brain tissue. The current system, however, can be improved by using more lasers with carefully chosen wavelengths. For example, Fig. 7 shows that the red laser at 633 nm is not good at differentiating color IDs 2, 3 and 4 (this can be seen by projecting the colors on to the x-axis) but the green laser at 543 nm can separate these colors well (projecting the colors on to the y-axis). Similarly, color IDs 18, 24 and 25 may not be well separated (small IRD) at these wavelengths but may become more separable at other wavelengths in a higher dimension color space.
One other potential biomedical application is to detect subarachnoid hemorrhage, which is often accompanied by blood bleeding to the cerebrospinal fluid (CSF), changing the normally crystal clear color to red (or other colors such as yellow, orange, green and brown due to the breakdown products of blood) . (The CSF surrounds the brain and can be found just beneath the skull.) This application, in the first instance, only needs to differentiate the color of CSF between red (or other colors) and transparent, both of which have very different absorption coefficients and the current system may have the necessary sensitivity to accomplish the task.
Another major challenge here is to identify a way to acoustically modulate only the region of interest. To this end, focused ultrasound has been proposed to provide acoustic modulation to diffuse light predominately in the region of interest for imaging purpose [2, 3]. We have also shown that such technique can be employed to increase the sensitivity of optical absorption measurement deeper into a turbid medium [11, 12].
4.2 Consumer grade digital camera
One feature of this experiment was its simplicity. The main components included only two visible lasers, a loudspeaker and a consumer grade digital camera. The feasibility of using consumer grade digital cameras for laser speckle contrast measurement has also been established in a recent blood flow imaging study . The applicability of consumer products allows the instrumentation cost to be lowered. However, the digital camera used in our study had a high noise level and lacked the capability of software control, which not only lengthened the acquisition time but also limited the number of images that can practically be taken for each color measurement. The large standard deviations of ΔC shown in Figs. 4 and 5 can be reduced by averaging a larger number of images of ΔC which would be much easier with a digital camera having software control capability.
4.3 Comparison with the 4-phase stroboscopic detection scheme
We have previously investigated a 4-phase stroboscopic detection scheme to measure the reflectance of the hidden colored object , which is essentially the same problem presented here. Previously, an opaque layer as thick as 10 mm was used in the 4-phase stroboscopic detection scheme. Using the speckle contrast technique, however, it has been found that the ΔC signal is too low for classification when a 10 mm opaque layer was used (results not shown here) but a 5 mm opaque layer allowed classification to be performed as shown in this paper. The 4-phase stroboscopic detection scheme is therefore considered a more sensitive scheme but it suffers from the requirement of the medium having a long correlation time which makes it difficult to be applied to tissue sensing and imaging application.
In the past few decades, the application of laser speckle contrast has been focusing on the area of blood flow estimation . In the 1970s, Briers pointed out that speckle contrast was also related to “the ratio of the mean intensity of the light from the moving scatterers to the total intensity of the scattered light” . One implication of this realization is that speckle contrast also contains information about the “amount” of light that moving particles scatter, in addition to the “speed” information (or correlation time). In other words, speckle contrast has the potential to measure the optical absorption (or reflectance) of the moving scatterers.
The experiment described here has been designed to investigate this rarely explored area of speckle contrast, with the vibrating colored paper taking the role of moving scatterers. Our results have shown that with the speed of the moving scatterers kept constant, the speckle contrast difference ΔC is indeed indicative of the reflectance of the hidden colored paper. With ΔC measured at two wavelengths and a classification algorithm, twenty five colors hidden behind an opaque barrier can be classified with an accuracy ranging from 55% to 88%, depending on the underlying direct reflectance and the color’s IRD. The classification performance is much higher than those achievable by the obstructed reflectance (light intensity only) measurements.
The main purpose of this appendix is to provide a full derivation of Eq. (2) and (3). Following Li, Ku and Wang [2, 9], the electric fields that account for the unmodulated light Eb and modulated light Em are defined as:
The CMOS chip on the digital camera has an exposure time T and the time-averaged intensity is:Eq. (9) over the speckle field with the statistical property , the spatially-averaged intensity in Eq. (2) can be derived:2, 9]:Eq. (14) is reduced to:Eq. (14) becomes 1 and Eq. (14) is reduced to:Eq. (17) can be neglected, resulting inEq. (17), the property has been employed [2, 9]. The justification is that Eb and Em have different optical pathlengths, i.e., Em has longer pathlength since it has to travel an additional distance to reach the colored object and back. The speckle pattern is determined by the phases (or optical pathlengths) of the electric fields when they reach the detectors. The speckle patterns formed by Eb and Em are very different because of the random pathlength difference and therefore uncorrelated, leading to the property .
The study was funded by the Engineering and Physical Science Research Council (Grant Code EP/G005036/1).
References and links
3. R. Li, D. S. Elson, C. Dunsby, R. Eckersley, and M. X. Tang, “Effects of acoustic radiation force and shear waves for absorption and stiffness sensing in ultrasound modulated optical tomography,” Opt. Express 19(8), 7299–7311 (2011). [CrossRef] [PubMed]
4. S. J. Kirkpatrick, R. K. Wang, D. D. Duncan, M. Kulesz-Martin, and K. Lee, “Imaging the mechanical stiffness of skin lesions by in vivo acousto-optical elastography,” Opt. Express 14(21), 9770–9779 (2006). [CrossRef] [PubMed]
7. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification and Scene Analysis, 2nd ed. (Wiley-Interscience, 1995).
8. S. J. Kirkpatrick, D. D. Duncan, and E. M. Wells-Gray, “Detrimental effects of speckle-pixel size matching in laser speckle contrast imaging,” Opt. Lett. 33(24), 2886–2888 (2008). [CrossRef] [PubMed]
9. J. Li, “Ultrasound-Modulated Optical Tomography for Biomedical Applications,” (Texas A&M University, 2004).
10. D. A. Seehusen, M. M. Reeves, and D. A. Fomin, “Cerebrospinal fluid analysis,” Am. Fam. Physician 68(6), 1103–1108 (2003). [PubMed]
11. T. S. Leung and S. Gunadi, “The sensitivity of acousto-optic sensing measurements to absorption changes in superficial and deep layers,” Proc. SPIE 8223, 822326 (2012). [CrossRef]
14. J. D. Briers, “A note on the statistics of laser speckle patterns added to coherent and incoherent uniform background fields, and a possible application for the case of incoherent addition,” Opt. Quantum Electron. 7(5), 422–424 (1975). [CrossRef]