## Abstract

We propose a novel approach for measurements of two-dimensional distribution of the reflection spectra with high spatial resolution. It is based on a subspace vector model of surface reflections and includes sequential illumination of the object by basis functions preliminary calculated with principal component analysis. A simple optical system consisting of a computer controlled set of light-emitting diodes and a photo-receiver operating in integration regime is used to acquire spatial distribution of reflection spectra in compressed form. The compressed data can be directly used for accurate color classification or recognition. The system’s ability to distinguish metameric samples with extremely small hue difference is experimentally demonstrated.

©2007 Optical Society of America

## 1. Introduction

Beside size and shape, color is an important characteristic for recognition and identification of objects. In most of practical cases, visual discrimination of objects is carried out by their illumination and observation of the reflected light. In that sense we can speak about surface color, which is a result of interaction between surface, illumination source, and observer. Objectively, the building blocks of color are wavelengths of light. Conventionally color is usually presented through a standard trichromatic color-coordinate system. Consequently, in most cases color information is introduced in the recognition process by means of three spectral channels in addition to the channels bearing spatial information. Multi-channel joint transfer correlation with filter matching in each channel is frequently used to recognize the object [1–6]. For successful recognition of color pattern, the spectral power distribution (SPD) of the illuminant and spectral sensitivity of the imaging device are preferably to be already known. Several studies were devoted to the problem of finding pattern-recognition architecture capable to remove the effects of varying illumination conditions, i.e. finding color constancy algorithms [7–10]. Common feature for all these techniques is the recovery of the reflection spectrum from the responses of a trichromatic two-dimensional imaging system such as color charge-coupled device camera. In addition to the problem of susceptibility to changes in the illuminant, the problem of metamerism sometimes challenges the accuracy of recognition. If two or more surfaces possessing different reflection spectra have same coordinates in trichromatic presentation under illumination by a particular light source, then they are called metamers [11]. For some applications it is desirable to distinguish metameric samples.

Evidently the measurement of reflectance spectra overcomes both problems since it generates a specific signature of the material, which becomes a basis for accurate classification and recognition. However, a spatially resolved spectral measurement with consequent calculation of the correlation function of two images with a large number of spectral channels is a difficult and time-consuming task. Conventional spectrophotometers allow measuring reflectance with high spectral resolution but they are typically characterized by low spatial resolution. These spectrometers are generally designed for one-spatial-point spectral measurements. There are commercially available systems [12] capable of measuring one-dimensional distribution of the reflectance spectra along the illumination stripe. To obtain two-dimensional spectral image one should scan the object in direction orthogonal to the illumination line.

Two-dimensional spatially resolved spectral information is usually obtained by multi-spectral imaging of the object surface [13–18]. It is performed by using five or more color filters for multiband filtering of the reflected/transmitted light. Typically the rotated filters are mounted in front of a monochrome type of CCD camera [14–16,19]. Nevertheless, applications of liquid-crystal [13,20], acousto-optic [21] or electro-optic [22] tunable filters in multispectral imaging have been also reported. Both acousto-optic and electro-optic filters are fast-switching devices allowing fast acquisition of 2D-spectral data but they are characterized by relatively small field of view. The reflection spectra can be also recovered from responses of trichromatic camera-systems after proper multispectral imaging [18,23,24]. The aim of multispectral imaging is the reconstruction of the reflection spectrum of each pixel of CCD camera that can be achieved by using various algorithms [13,17,23].

In this paper we propose a novel approach for measurements of two-dimensional distribution of the reflection spectra with high spatial resolution. Our approach based on a vector subspace model of color representation saves time in the process of color pattern recognition. The spectral data are compressed already on the stage of measurements, while the reflection spectrum can be readily reconstructed by use of simple linear relationships. The compressed data can be used directly for the purpose of chromatic discrimination.

## 2. Vector-subspace model

As known, many of natural and artificial materials have rather smooth dependence of the reflectance on light wavelength in the visible range of the spectrum. In addition, color classification/recognition is typically performed within the limited number of samples with different reflection spectra. For such limited set of smooth reflectance spectra one can always find [13,25,26] a few basis functions *S*
_{i} (λ) so that any particular reflection spectrum *R*(λ) is a linear combination of these basis functions:

Here the coefficients, *σ _{i}*, are real numbers, which unambiguously represent particular reflectance spectrum within given set of samples.

The number *Q* of the basis functions depends on the database size, the complexity of the reflectance spectra, and the required accuracy of spectrum recognition. The results of previous researches [24,25,27,28] show that surface-spectral reflectances of both natural and artificial objects can be represented with the use of 5 to 7 basis functions. The determination of these basis functions is typically performed by statistical analysis of the measured set of the reflectance spectra, which is referred to as principle component analysis (PCA). Recently many researchers apply PCA for both reconstruction of the reflectance spectra from multispectral images [13–17,19,23] and for recognition of color patterns [29,30].

Let us suppose that we have defined the set of reflectance spectra within which recognition/classification of different colors is to be carried out. Consequently, the shape and number of the basis functions *S*
_{i} (λ) is as well determined. For example, if a color surface belongs to a set of artificial objects, one can use the basis functions calculated and published in [25]. The basis functions used in this work are shown in Fig. 1. Since the basis functions are known, the coefficients *σ _{i}* associated with particular reflectance spectrum,

*R*(λ), are defined as

where λ_{1} and λ_{2} are the boundary wavelengths of the spectral band used. Considering that in practice the spectral reflectance is measured in a discrete form (typical sampling rate is 1 -10 nm), the integral in Eq.2 is transformed into a sum:

where *M* is the number of wavelength intervals defined by the sampling rate.

It follows from Eqs.2,3 that by providing illumination of the object surface by a light source, which spectral power distribution (SPD) corresponds to *S*
_{i}(*λ*) we could directly measure the coefficients *σ _{i}*. However, the basis functions

*S*

_{i}(

*λ*) take both positive and negative values because these are mutually orthogonal eigenfuctions of the chosen set of color samples. Therefore, straightforward design of light source with SPD of

*S*

_{i}(

*λ*) is hardly possible. Some efforts were applied to search for approximate functions with non-negative values that can accurately enough represent any spectral reflectance from the given set [20,29]. Design of such kind of functions was optically implemented by using iterative feedback [29] or neural network [20] methods. In both cases an optical system with liquid-crystal spatial light modulator was used to synthesize light with desirable spectral power distribution for further illumination of the object surface. It has been shown [31] that the optimal set of nonnegative color filters can be obtained by using an invertible linear transformation of the basis functions

*S*

_{i}(

*λ*). The use of the set of nonnegative functions calculated in [31] may lead to simplification and acceleration of the data processing. Here we describe a technique for optical implementation of the basis functions containing both positive and negative parts.

## 3. System description

#### 3.1. Principle of the operation

To synthesize light possessing preliminary determined spectral power distribution we propose to use a set of light-emitting diodes (LED). Recent progress in LED technology makes them very attractive as a light source in spectral measuring systems. LEDs are cheap and robust; they possess high efficiency and spectral luminousity. It is not hard to collect a set of LEDs generating light at different wavelengths and covering whole visible diapason. Switching among different LEDs can be accomplished with high speed thus making possible to generate any arbitrary sequence of spectral lines.

Let the output optical power of a spectral line with the central wavelength of *λ _{k}* be of

*P*(

*λ*). First, we illuminate the object surface by a sequence of spectral lines at which the basis function

_{k}*S*(

_{i}*λ*) has positive values. The illumination of each spectral line is carried out during the exposure time of

_{k}*t*(

_{i}^{P}*λ*) so that

_{k}*P*(

*λ*)

_{k}*t*(

_{i}^{P}*λ*) =

_{k}*S*(

_{i}*λ*) for all

_{k}*λ*at which

_{k}*S*(

_{i}*λ*) > 0. The light reflected from the surface is collected into a photo-receiver operating in the integration regime. The integration of the reflected light flux during the total exposure to the sequence of spectral lines results in the output signal from the photo-receiver:

_{k}where *λ _{k}* is the peak wavelength of the spectral line illuminating the object and

*κ*(

*λ*) is the calibration coefficient, which takes into account the spectral dependence of sensitivity of photo-receiver.

_{k}Second, we illuminate the object by another sequence of spectral lines emitted with
exposure time of *t _{i}^{N}* (

*λ*) chosen to satisfy the condition

_{k}*P*(

*λ*)

_{k}*t*(

_{i}^{N}*λ*) =

_{k}*S*(

_{i}*λ*) when

_{k}*S*(

_{i}*λ*) < 0. After illumination by this sequence, the output signal of the photo-receiver is

_{k}Numbers *Mp* and *Mn* satisfy the condition of *Mn* + *Mp* = *M*. Evidently, by calculating the difference *J _{i}^{P}* -

*J*one can obtain the coefficient

_{i}^{N}*σ*. Repeating the measuring procedure for all basis functions we obtain all coordinates in the

_{i}*n*-dimensional subspace, which unambiguously represent the spectral reflectance of the object surface. Note that the calibration coefficients

*κ*(

*λ*) in Eqs.4,5 are independent from the shape of basis functions, while the output power

_{k}*P*(

*λ*) may be adjusted during the switching from one function to another for optimal fitting of dynamic range of the photo-receiver. This adjustment can be done by changing the injection current of LED.

_{k}Since the integration regime of photo-receiving is typical for a CCD matrix, the proposed approach provides fast and precise measurements of two-dimensional distribution of the reflectance spectra with high spatial resolution. In such case a monochrome CCD camera is to be used as a photo-receiver. Light reflected from a small area of the object is collected into respective pixel of the CCD matrix. Under illumination of the whole object surface by proper sequence of the spectral lines, measurements of coefficients *σ _{i}* are done in parallel for all small areas of the surface. The difference between two frames grabbed during illumination by sequences of the spectral lines corresponding to the positive and negative parts of the basis function

*S*(

_{i}*λ*) gives two-dimensional distribution of the weighting coefficients

_{k}*σ*. After illuminating the object by the complete set of the basis functions, we get

_{i}*n*matrices that can be used for accurate reconstruction of two-dimensional distribution of object reflectance with high spatial resolution. The spatial resolution is defined by the illuminated area divided by the total number of pixels in CCD matrix (which is typically larger than 10

^{6}).

#### 3.2. Computer controlled light source

The first prototype of light source capable to generate fast switchable sequence of spectral lines has been fabricated by Optoinspection Ltd. in accordance with design developed in our laboratory. Figure 2 shows principal layout of optical part of the light source without correct scaling and angular position of elements to simplify the drawing. Fifteen conventionally available light-emitting diodes were used to cover a wide band of wavelengths (390 – 710 nm). The spectral bandwidth of each individual LED varies from 20 to 150 nm. Sharpening of the emitted spectral line was achieved by means of tunable acousto-optic filter (TAOF). As known, acoustic wave excited in acousto-optic crystal acts as volume grating, the light diffraction from which is allowed only within the limited spectral band. So, the bandwidth of spectral lines is reduced to the magnitude from 4 nm in violet to 8 nm in the red. By changing the frequency of the excited acoustic wave one can readily vary the wavelength of the filtered light. In comparison with other tunable spectral filters, TAOF is characterized by fast switching time. The switching time of our particular TAOF between two arbitrary chosen wavelengths is less than 20 μs. The minimal time required to implement all seven basis functions *S _{i}*(

*λ*) is 12 ms.

For the efficient performance of TAOF it is required that the light from different LEDs is incident on the filter at the same angle. Light beams from different LEDs were combined by means of conventional diffraction grating. It is a challenging task to design an optical scheme for combining the light so as to provide propagation of all beams in the same direction. The output light power achieved in our first prototype is shown in Fig. 3 as a function of the generated wavelength. In the following version currently under design in our laboratory, we expect to get 10 – 50 folds increasing of the light power.

The electronic unit of the light source was designed in order to provide control of injection current of LEDs and both frequency and amplitude of the acoustic wave. It also includes a microprocessor responsible for data exchange with personal computer. The device
performance is fully controlled by a personal computer so that the central wavelength of the spectral line, duration of each illumination period, and the output power of such emission can be introduced into the computer, stored in its memory, and then retrieved in any required order. Particularly, sequences corresponding to either positive or negative part of any basis functions *S _{i}*(

*λ*) can be generated.

## 4. Experimental results

Feasibility of the proposed approach was demonstrated by checking ability of the system to distinguish samples with small hue difference. Four color samples of the Macbeth Daylight metamerism kit #3 from the GretagMacbeth Company were chosen as test object. A photograph of these samples is shown in Fig. 4. It was taken by a conventional digital still camera (Sony DSC-III) under day-light illumination. One can see from Fig. 4 that hue difference between samples is really small so that the border between samples B and C is hardly visible.

We used seven basis functions *S*
_{i}(λ) calculated by Parkkinen et al. for the set of 1257 Munsell color chips [25] (see Fig. 1). Sequences *P*(*λ _{k}*)

*t*(

_{i}^{P}*λ*) and

_{k}*P*(

*λ*)

_{k}*t*(

_{i}^{N}*λ*) proportional to the functions

_{k}*S*

_{i}(λ) were stored in a computer connected with the light-source prototype. Each sample of the metamerism kit was illuminated under normal incidence and light reflected from the surface at average angle of 45° was collected into a photodiode. Electrical signal from the photodiode was integrated during the total time of surface exposure by the sequence of spectral lines as we described in Sect. 3.1. Repeating the measurements for positive and negative parts of all 7 basis functions and performing respective subtraction, we get components of a 7-dimensional (7D) vector,

*σ*⃗, which characterizes the spectral reflectance of the sample. Measurements were carried out several times in different parts of samples. To estimate how these 7D-vectors vary in each part and from one sample to another, we calculate the dot product,

*R*

_{mn}of normalized vectors σ⃗:

Here symbols *m* and *n* identify the metameric samples shown in Fig. 4: both symbols may run values of A, B, C, or D. Three-dimensional diagram in Fig. 5 shows the difference Δ*R _{mn}* = 1 -

*R*between all possible combinations of samples. Cylinders in the diagonal (Δ

_{mn}*R*when

_{mm}*m*=

*n*) relate to difference in dot products of 7D-vectors corresponding to different measurements of the same sample. They reflect repeatability of the measurement of σ-vector for each sample. We plot in Fig. 5 the maximal values of Δ

*R*

_{mn}for each diagonal element. Non-diagonal elements (

*m*≠

*n*) show the difference between 7D-vectors of different metameric samples. In contrast with the diagonal elements, minimal values of Δ

*R*

_{mn}are plotted in Fig. 5 for each non-diagonal element.

Comparison of the absolute maximum for diagonal elements (Δ*R _{BB}* = 0.0007) with the absolute minimum for non-diagonal elements (Δ

*R*= 0.0019) clearly indicates that the metameric samples are surely distinguishable.

_{AB}## 5. Conclusion

In this paper we described a novel technique for fast measurements of two-dimensional distribution of the reflection spectra in visible diapason. Projections of a reflection spectrum on a preliminary defined set of basis functions are directly and simultaneously measured at all spatial points of the object surface. This allowes us to acquire spatial distribution of the reflection spectra in compressed form at the stage of measurements. The compressed data can be directly used for accurate color classification or recognition without reconstruction of the reflection spectra. The key-system for implementation of the proposed approach is computer-controlled light source capable to generate fast switchable sequence of spectral lines. The first prototype of such light source was fabricated and used to demonstrate ability of the system to distinguish samples with extremely small hue difference.

Owing to parallel processing of the spatial information, fast switching time, and high sensitivity to the reflectance spectra difference, the proposed approach may be very useful for quality control of various industrial processes through on-line color monitoring. Fast and accurate measurement of 2D-distribution of the reflection spectra is also an important instrument for medicine and cosmetic since it can provide assessment and telecommunication of the objective information concerning conditions of human skin. The proposed approach of color recognition can also have applications in security document authentication systems where highly precise distinguishing of colors is a crucial factor.

## References and links

**1. **C. Warde, H. J. Caulfield, F. T. S. Yu, and J. E. Ludman, “Real-time joint spectral-spatial matched filtering,” Opt. Commun. **49**, 241–244 (1984). [CrossRef]

**2. **E. Badiqué, Y. Komiya, N. Ohyama, J. Tsujiuchi, and T. Honda, “Color Image correlation,” Opt. Commun. **61**, 181–186 (1987). [CrossRef]

**3. **Z. Q. Wang, C. M. Cartwright, C. Soutar, and W. A. Gillespie, “Real-time color image correlation with a color liquid-crystal television and a Fresnel holographic filter,” Appl. Opt. **32**, 715–717 (1993). [CrossRef] [PubMed]

**4. **M. S. Millán, M. Corbalán, J. Romero, and M. J. Yzuel, “Optical pattern recognition based on color vision model,” Opt. Lett. **20**, 1722–1724 (1995). [CrossRef] [PubMed]

**5. **A. Fares, P. García-Martínez, C. Ferreira, M. Hamdi, and A. Bouzid, “Multi-channel chromatic transformations for nonlinear color pattern recognition,” Opt. Commun. **203**, 255–261 (2002). [CrossRef]

**6. **J. Nicolás, C. Iemmi, J. Campos, and M. J. Yzuel, “Encoding 3D correlation in an optical processor,” Opt. Commun. **256**, 279–287 (2005). [CrossRef]

**7. **M. D’Zmura and G. Iversion, “Color constancy. I. Basic theory of two-stage linear recovery of spectral descriptions for lights and surfaces,” J. Opt. Soc. Am. A **10**, 2148–2165 (1993). [CrossRef]

**8. **B. V. Funt and G. D. Finlayson, “Color constant color indexing,” IEEE Trans. Pattern Anal. Mach. Intell. **17**, 522–529 (1995). [CrossRef]

**9. **G. D. Finlayson, S. D. Hordley, and P. M. Hubel, “Color by correlation: a simple, unifying framework for color constancy,” IEEE Trans. Pattern Anal. Mach. Intell. **23**, 1209–1221 (2001). [CrossRef]

**10. **M. Corbalán, M. S. Millán, and M. J. Yzuel, “Color pattern recognition with CIELAB coordinates,” Opt. Eng. **41**, 130–138 (2002). [CrossRef]

**11. **G. D. Finlayson and P. Morovic, “Metamer sets,” J. Opt. Soc. Am. A **22**, 810–819 (2005). [CrossRef]

**12. **http://www.specim.fi/products/spetral-imaging-products/imaging-spectrographs.html.

**13. **J. Y. Hardeberg, F. Schmitt, and H. Brettel, “Multispectral color image capture using a liquid crystal tunable filter,” Opt. Eng. **41**, 2532–2548 (2002). [CrossRef]

**14. **S. Tominaga, “Multichannel vision system for estimating surface and illumination functions,” J. Opt. Soc. Am. A **13**, 2163–2173 (1996). [CrossRef]

**15. **S. Baronti, F. Casini, F. Lotti, and S. Porcinai, “Multispectral imaging system for the mapping of pigments in works of art by use of principal-component analysis,” Appl. Opt. **37**, 1299–1309 (1998). [CrossRef]

**16. **N. Tsumura, “Appearance reproduction and multispectral imaging,” Color Res. Appl. **31**, 270–277 (2006). [CrossRef]

**17. **D.-Y. Ng and J. P. Allebach, “A subspace matching color filter design methodology for a multispectral imaging system,” IEEE Trans. Image Process. **15**, 2631–2643 (2006). [CrossRef] [PubMed]

**18. **R. S. Berns, L. A. Taplin, M. Nezamabadi, and Y. Zhao, “Methods of spectral reflectance reconstruction for a sinarback 54 digital camera,” in *Munsell Color Science Lab Technical Report* (2004).

**19. **H. Haneishi, T. Hasegawa, A. Hosoi, Y. Yokoyama, N. Tsumura, and Y. Miyake, “System design for accurately estimating the spectral reflectance of art paintings,” Appl. Opt. **39**, 6621–6632 (2000). [CrossRef]

**20. **M. Hauta-Kasari, Miyazawa K., S. Toyooka, and J. P. S. Parkkinen, “Spectral vision system for measuring color images,” J. Opt. Soc. Am. A **16**, 2352–2362 (1999). [CrossRef]

**21. **J. Calpe-Maravilla, J. Vila-Frances, E. Ribes-Gomez, V. Duran-Bosch, J. Munoz-Mari, J. Amoros-Lopez, L. Gomez-Chova, and E. Tajahuerce-Romera, “400-to 1000-nm imaging spectrometer based on acousto-optic tunable filters,” J. Electron. Imaging **15**, 023001 (2006). [CrossRef]

**22. **S. M. Nascimento, F. P. Ferreira, and D. H. Foster, “Statistics of spatial cone-excitation ratios in natural scenes,” J. Opt. Soc. Am. A **19**, 1484–1490 (2002). [CrossRef]

**23. **V. Cheung, S. Westland, C. Li, J. Y. Hardeberg, and D. Connah, “Characterization of trichromatic color cameras by using a new multispectral imaging technique,” J. Opt. Soc. Am. A **22**, 1231–1240 (2005). [CrossRef]

**24. **L. T. Maloney and B. A. Wandell, “Color constancy: a method for recovering surface spectral reflectance,” J. Opt. Soc. Am. A **3**, 29–33 (1986). [CrossRef] [PubMed]

**25. **J. P. Parkkinen, J. Hallikainen, and T. Jaaskelainen, “Characteristic spectra of Munsell colors,” J. Opt. Soc. Am. A **6**, 318–322 (1989). [CrossRef]

**26. **L. T. Maloney, “Evaluation of linear models of surface spectral reflectance with small number of parameters,” J. Opt. Soc. Am. A **3**, 1673–1683 (1986). [CrossRef] [PubMed]

**27. **T. Jaaskelainen, J. P. S. Parkkinen, and S. Toyooka, “Vector-subspace model for color representation,” J. Opt. Soc. Am. A **7**, 725–730 (1990). [CrossRef]

**28. **M. J. Vrhel, R. Gershon, and L. S. Iwan, “Measurements and analysis of object reflectance spectra,” Color Res. Appl. **19**, 4–9 (1994).

**29. **N. Hayasaka, S. Toyooka, and T. Jaaskelainen, “Iterative feedback method to make a spatial filter on a liquid crystal spatial light modulator for 2D spectroscopic pattern recognition,” Opt. Commun. **119**, 643–651 (1995). [CrossRef]

**30. **J. L. Nieves, J. Hernández-Andrés, E. Valero, and J. Romero, “Spectral-reflectance linear models for optical color-pattern recognition,” Appl. Opt. **43**, 1880–1891 (2004). [CrossRef] [PubMed]

**31. **R. Piché, “Nonnegative color spectrum analysis filters from principal component analysis characteristic spectra,” J. Opt. Soc. Am. A **19**, 1946–1950 (2002). [CrossRef]