A specially designed phase mask embedded in the lens assembly of an imaging system is shown to provide different response in the three major color bands, R, G and B of a detector array. Each channel provides optimal performance for different depth of field regions, such that the three channels jointly provide an imaging system with wide depth of field. The approach is useful in particular for Barcode imagers.
© 2010 OSA
Barcode readers generally consist of two main types: flying spot scanners  and electronic imagers . Scanning barcode readers use a beam that runs across a barcode, whereas imaging barcode readers analyze and process the entire barcode image captured as a whole. Both types rely on the fact that the barcode is black and white mostly and color information is irrelevant. The depth of field of such systems is determined by resolving the highest spatial frequency of the barcode.
Several methods have been presented recently, having the goal to extend the depth of field (DOF) of imaging systems. These methods modify the optical system and thus change its behavior, usually by introducing additional elements, masks, etc or by altering conventional lens elements.
Systems with extended depth of field for imaging have been achieved by fabricating a lens whose focal distance varies continuously within the lens aperture, such as a logarithmic aspheric [3–5]. For each distance, there is an annular portion of the lens, which provides a sharp image. The remaining part of the lens gives a blurred image; the captured image is corrected by post processing.
Coding masks have been used in order to provide an extended DOF; extensive post-processing allowed the reconstruction of high resolution images [6,7]. However the image contrast in the acquisition stage was very low and as a result of that, the sensitivity to noise was large and the quality of the reconstruction suffered.
Still another method [8,9] exploits phase masks with one or several annular rings creating a phase difference of π. Those masks can greatly extend the depth of field of an optical system and keep the contrast above a minimum level, without the use of post processing. However, the contrast exhibited with those masks was found to be too low for some practical applications.
The utilization of the three color channels (RGB) of a CMOS or CCD detector array as different image sources has been suggested previously  as a source of visual information that can be useful for auto focus and depth estimation, since the defocus generated in each color channel image is different. Super resolution of video images has been achieved via the use of three color channels  as different sources with different spatial resolution. Similar effect occurred while using color barcodes imaged by an uncorrected lens and separated into 3 channels. The chromatic aberration has complementary spectral characteristics and thus increases the capacity in comparison with encoding methods based on a single color channel .
The present paper describes a novel method that utilizes the three color channels of a standard digital imaging system as three independent channels that jointly contribute to a system exhibiting an extended depth of field. A phase mask, to be located in the pupil of an optical system, is designed to deliberately provide different responses for the RGB channels. Thereafter, by jointly using the three channels, one can achieve a system with an extended congruent depth of field. Since barcode readers should only recognize valid codes, identification achieved by one of the color channels will suffice.
The paper is organized as follows: Section 2 presents the theory as well as design and optimization considerations. Then, optimization results are presented in section 3, followed by simulations and experimental results, presented in section 4.
2.1 Out-of-focus effects
It is well known that in imaging theory [8,13] defocus manifests itself by multiplying the inherent pupil function of the imaging system by a quadratic phase, resulting in a generalized pupil function
D is the exit pupil diameter, λis the wavelength of the incoherent quasi-monochromatic illumination, f is the effective focal length of the imaging system, and are the longitudinal distances between the object and the image to the lens' principal planes and are the Cartesian coordinates at the exit pupil plane normalized by a factor . With ψ = 0 the sharp (focused) image is formed at the image sensor plane with longitudinal position.
With the quadratic defocusing term of Eq. (1) prevents the object to be imaged exactly at the plane of the image sensor with longitudinal position. Therefore, the detector acquires a degraded defocused image whose intensity is a convolution of the sharped focused image with a defocusing spot.
In order to evaluate the optical transfer function (OTF) of the defocused imaging system, we rely on the cut off spatial frequency (COF) of a diffraction limited coherent imaging system :Eq. (3) individually for each design wavelength. Accordingly, we introduce new dimensionless spatial frequency variables
The normalized OTF of the defocused incoherent imaging optical system is related by the auto-correlation operation [8,13] to the generalized pupil function. Exploiting Eq. (1) yield the OTF expressed directly via the inherent pupil function of the imaging system and the defocus parameter ψ
2.2 Chromatic aspects of the Depth of Field
Although the focal distance fof a simple lens is a function of the wavelength, most imaging systems use lens assemblies that are chromatically corrected. We will thus assume that fin the imaging system under consideration is a constant for all wavelengths in the visible spectrum. As indicated in Eq. (3), the phase error in case of misfocus is an inverse function of the wavelength:
For polychromatic illumination, the parameter ψ varies with the wavelength so that:
The phase shift is achieved by etching (or depositing) a layer of depth into a glass plate with a refractive index of. As taught in  the phase shift is related to the layer parameters as:
Accordingly the phase shift of at a given design wavelength is achieved with the layer depth:
With changing the wavelength in Eq. (9), mechanical depth parameter hremains constant. Therefore depends onλ as . Let's assume that the layer with depth hin is designed with the aid of Eq. (10) for a wavelength, but exposed to light with different wavelength. Equation (11) applied sequentially for different wavelengths andyields the ratio between the phase shifts at two different wavelengths as:
The last approximation in Eq. (11) can be applied because of the fact that the refractive index n slowly varies with the wavelength. Nevertheless we used in our design the exact form of Eq. (11) which takes into account the variations of n with λ. Another quantity that is influenced by the wavelength is the diffraction cut off frequency (COF). According to Eq. (3) the variation of the cut off frequency with the wavelength is:
It is clear that wavelengths variations should be taken into account when a phase mask is designed. The goal of our approach is to design a mask that provides different responses for the three main wavelengths RGB, such that the DOFs exhibited by each one of these three wavelengths, complement each other, with perhaps some overlap. As a result of that, the “global” DOF of a system that analyzes independently the response of each wavelength has the potential to provide a system with a combined DOF of superior performance.
The search for the solution has been carried over the full visible spectrum (400-700nm). Since a clear aperture has the best MTF for small values ofψ, i.e., the region surrounding the in-focus position, one should design a mask that behaves as a clear aperture for one of the primary wavelengths, while the DOF provided by the two other primary wavelengths should cover different regions when out-of-focus conditions are encountered, i.e., higher values ofψ.
2.3 Bayer Matrix
The three different color channels of a standard sensor are usually organized in the form of the Bayer Matrix . We have chosen the primary wavelengths = 450nm, = 550nm and = 650nm according to typical RGB color filters. A Bayer Matrix mosaic is a color filter array that distributes the RGB color filters on a square grid of photo sensors. Such arrangement of color filters is used in most single-chip digital image sensors used in digital cameras, camcorders, and scanners to create a color image. The filter pattern is 50% green, 25% red and 25% blue, hence it is called GRGB or in another permutation RGGB. Sub-pixels should be decoded in order to fill the matrix of each color (R, G and B). The process of decoding colors (debayering or demosaicing) is done by the CPU in the digital camera. However, one can select and analyze each channel separately, as we propose in this study. In Fig. 1 , a typical Bayer Matrix is shown.
2.4 Mask design
Before engaging in the design itself, the rationale behind the new approach should be described.
Since barcode information is in most cases encoded as black and white stripes, the existence of three color channels in the detector array can be advantageously utilized if the phase mask intended to be inserted in the optical train will provide different response at each channel.
By combining these two observations, one can now design masks that consist of circular phase rings , but by etching (or depositing) thicker layers, the phase (modulus) will be significantly different at the three main wavelength. Design according to this provision has now an additional degree of freedom.
The procedure followed in the design of the mask can now be summarized:
- a) A mask composed of several concentric phase rings exhibits at one wavelength (say blue, B) a phase of π (modulus), at another one (say Red, R) a phase of 0 (modulus), while at an intermediate wavelength (say Green, G) the phase will be approximately π/2 (modulus). Etching (or depositing) a layer providing a phase difference of for Blue (450 nm) will exhibit phase for Green (550 nm) and phase for Red (650 nm), thus approaching the desired goal. The design will search for the proper ring radii, such that in view of the above mentioned phase values, the resulting mask will exhibit extended complementary DOF regions, if information acquired in each one of the three channels is handled independently. By this we mean that the DOF for one wavelength will be located next to the DOF of another wavelength (gaps are not allowed, but overlapping is not forbidden) so that the system as a whole will exhibit a large extended DOF. It has been observed that the higher the number of rings, the larger is the extension of the DOF, albeit the minimum contrast is being reduced as well. Since high contrast (above 25%) is required for electronic processing of the signal in many applications, only masks having one or two phase rings have been considered in this study.
- b) The goal is set to design an imaging system that provides reliable operation with an MTF exhibiting a contrast of at least 25% for the highest possible DOF, defined by a certain |ψ| max. One then finds out that a clear full aperture provides acceptable performance in terms of MTF contrast up to a cut-off frequency of 0.6 (diffraction limit is 2.0) if the maximum out-of-focus position corresponds to ψ = 3.3. One can achieve such response via information provided by the Red channel, since the mask at Red wavelength exhibits a phase of 2.077π and thus barely alters the pupil function. Thus, the position of the ring(s) is insignificant, as far as the Red channel response is concerned. One should now search for the ring distribution that will provide acceptable performance for ψ out-of-focus condition extending from 3 to the maximum achievableψ. This can be achieved when operating in the Green and Blue channels.
- c) In the search for the best choice of a mask exhibiting two phase rings  and , if 2 phase rings are considered, these radii are allowed to get all possible values between 0 (center of the aperture) and 1(normalized aperture size), such that . Due to the limited number of variables, a direct “brute force” approach was used that is faster than other optimization techniques, such as simulated annealing; the radii were allowed to change by increments of 0.01 for each iteration. Following this design, the system was able to provide imaging up to |ψ| = 6, where Green provided best image when ψ varied between 2 and 5, and Blue provided best image when ψ was between 4 and 6. For each set of values, the MTF of the system for all values of ψ in the range of 3 to 6 is calculated. When carrying he calculations, ψ was let to vary with increments of 0.1. For each MTF curve, the highest frequency for which the contrast was above 25% was recorded. Acceptable performance in such design, is to obtain an MTF with a contrast of at least 25% up to a normalized cut-off spatial frequency of 0.6, whereby the diffraction limit spatial frequency is defined as 2.0
- d) One now associates with each set of a single cut-off spatial frequency value. The highest of these cut-off spatial frequencies corresponds to the best obtainable mask that consists of two phase rings  and . The mask was tested according to the general description of the RGB method, thus MTF of each mask was calculated for the Red channel, between 600 to 700nm with and increment of The same was done for the Green channel, for the 500-600nm and for the Blue channel for the 400-500nm. If the mask doesn't behave properly for each of the 3 zones, it cannot be chosen.
Carrying out the above procedure, one finds out that for the set goal of providing acceptable performance with a contrast level of 25%, and a maximum value of the out of focus parameter of +/−6, a single ring is sufficient. Such a ring extends from the pupil edge outside radius up to a normalized radius of 0.7. The contrast achieved with such a mask is above 25% up to a normalized spatial frequency of 0.6, whereby 2.0 is the cut off frequency of a diffraction limited system. A similar search was carried for masks with 2 and 3 rings as well. Masks with 2 and 3 phase rings that provide a contrast of 25% as well, do not allow the extension of the DOF beyond. On the other hand, if a contrast level of 15% is allowed, then a mask with 2 or 3 rings can provide a working range till ψ = 7, and for a contrast level of 10%, a mask with 2 or 3 rings can provide a working range till ψ = 8. However such levels of contrast may be insufficient for the requirements of a barcode decoder. Because the resulting masks with more than a single ring don't provide better performances than the single ring mask, the latter was chosen. Figure 2 shows the effective phase level of each color channel:
The decoding algorithm searches for the presence of a barcode and its identification in each color channel, independent of the other channels. It is possible that a barcode will be deciphered in more than one channel when overlaps occurs; this will only increase the robustness of the system
Computer simulations, executed on the basis of Eqs. (8), (11) and (12), have shown that the entire depth of field, from the in-focus position up to the end of the DOF (ψ = 6.25) is covered by at least one of the channels. Thus, at least one of the channels provides enough contrast to allow decoding. The green channel overlaps parts of the red channel response towards the center and the blue channel response towards the extremes of the DOF and allows some flexibility in the determination of the limits provided by the red and blue channels. The DOF is continuous with no gaps.
The working range (DOF) of the Barcode imager is covered by the three color channels as sketched in Fig. 3 :
Figure 4 shows MTF curves of the separate RGB color channels for different values ofψ. The vertical axis shows which is the image contrast for a sinusoidal object, as customary for the MTF. The scale of the horizontal axis is the normalized spatial frequencyξ. The plots of Fig. 4 were evaluated for the wavelength of = 450 nm, = 550 nm, = 650 nm accordingly. Thus each one has spatial frequencies extending from the range. One should note that a spatial frequency in the input object will thus appear at different locations along the horizontal axis in view of the different wavelengths.
4. Simulations and experimental Results
Simulation results and experiments have been carried in order to verify the performance of the mask designed in accordance to the procedure described in section 3 and its utilization in a RGB camera. The simulations are presented in section 4.1. The requirements of the imaging systems as well as the characteristics of the optimal mask that has been used are presented in section 4.2. Section 4.3 describes the experimental apparatus as well as the experimental results.
A simulation program has been written in Matlab in order to verify the ability to acquire an image, separate it into 3 sub-images by virtue of the Bayer Matrix and analyze the influence of the phase mask on each channel. The investigated object was a “rosette” that allows easy evaluation since it exhibits simultaneously a wide range of spatial frequencies and orientations. According to theoretical predictions, the Red channel should be best for the range of ψ = 0 to 3, the Green channel should be best for the range of ψ = 2.5 to 5 and the Blue channel should be best for the range of 5 to 6.25. The values of ψ (the scale of the ordinate axis in Fig. 4) have been calculated for a wavelength of 500nm.
The rosette object consisted of 100 spokes equally spaced. Indeed, the red channel provides the best image when ψ < 3, hence little defocus. Thereafter, the green channel provides the best image in the mid-range of the DOF and the blue channel provides the best image towards the DOF limit. The images of the spoke target acquired by the 3 channels, for different values ofψ, are shown in Fig. 5 :
4.2 Experimental set-up
A commercial RGB camera [uEYE® 1225], equipped with a CMOS detector with a resolution of 752x480 pixels and a pixel dimension of 6 was utilized. It was equipped with a lens having an effective focal length of 16mm [Computar® M1614W], and a field of view (FOV) of 45 degrees along the sensor diagonal. The target was incoherently illuminated by ambient room light.
The imaging system was set to operate from a distance of 18 cm to 50 and provide a minimal contrast value of 25%, whereby the nominal distance has been set at 23cm. A pupil with a 2mm diameter was chosen, compatible with the apertures of imaging systems used by commercial barcode readers. Such a pupil has thus been inserted in the Computar lens as a clear aperture when experimental results for an open aperture were collected; likewise the fabricated mask inserted in the pupil plane had such a diameter as well. Based on the simulation results presented earlier, a binary phase mask with a single phase ring was fabricated by wet etching of the fused silica substrate which has refractive indices of 1.46557, 1.45991 and 1.45653 for the RGB wavelengths = 450nm, = 550nm and = 650nm respectively.
We investigated objects with feature sizes corresponding to values up to 0.6 of the normalized spatial frequency which is compatible with the Nyquist frequency of the detector array. Note that a value ξ = 2, corresponds to the diffraction limited cutoff spatial frequency of the blue light (wavelength 450 nm). The experimental set-up is sketched in Fig. 6 :
4.3 Experimental results
In the experiment, barcodes as well as a rosette spoke target were imaged by the camera, with and without the mask. The object position was allowed to vary over the entire DOF region. In Fig. 7 the images of a standard 7.5 mil Barcode (corresponding to a spatial frequency of 2.6 line pairs/mm) positioned at three different distances corresponding to ψ = 0 (in focus position, 23cm from the lens), ψ≈ 4 (19.8cm away from the lens) and ψ≈ 6 (18.5cm away). If going in the other direction, the ψvalues of- 4 and −6 correspond to distances of 27.4cm and 30.3cm respectively. At each position, images obtained by the RGB channels are provided for a clear aperture as well as for a mask-equipped pupil. There is at least one channel in which the barcode can be deciphered when the novel mask described in this work was included in the optical train. However, when the mask wasn't included and a clear aperture was used, barcodes could be identified only in a narrow region around the in-focus position.
In order to better visualize the results obtained with the Barcode imaged in Fig. 7, the images have been hard clipped, using the non-linear hard clipping function provided in Fig. 8 . For machine vision algorithms such hard clipping is not necessary.
The experiment was repeated for a binary Rosette spoke target. Each radius of the rosette corresponds to a different spatial frequency. A Matlab code was written in order to find the fulcrum of the spokes in each image. Then, by a linear manipulation, each circle representing a certain spatial frequency was converted into a vector and the resulting image contrast was calculated. The experimental graphs for images captured at ψ≈0 (in focus position), ψ≈ 4 and ψ≈ 6 are presented in Fig. 9 . The vertical axis shows values of image contrast received from the binary object with defocusingψ.
For low ψ values the red channel provides the best results. Those are even slightly better than the results obtainable with a clear aperture. The reason is that the results of the clear aperture case are affected by the presence of all three channels, R, G and B, since the processor averages information from all color channels. The red channel in the camera is much more sensitive than the green and blue channels and provides higher contrast . Moreover, when in a defocus position, the Red channel is less sensitive since the ψ variations are smaller than for the other channels.
A commercial Metrologic Barcode imager, MS-4980 having a resolution of 1280X960 pixels and a pixel size of 4  was tested. First, we used the imager as is and managed to identify a 7.5 mil Barcode within a range of 90-180mm. Then, we equipped the imager with the phase mask attached to the outer lens and captured the same image sequentially with one of the color filters, Red, Green or Blue. Since the imager was equipped with a monochrome detector array, at each distance we placed one color filter at a time in front of the imager and checked whether there is a “beep” with one color filter at least. In this fashion it was possible to detect barcodes within a range of 70-241mm. For a 13 mil Barcode, the DOF with a clear aperture was 55-250mm and with the phase mask we could decipher the Barcode in the range of 43-340mm. It would have been possible to extend the low limit (43 mm) to even lower numbers, but the field of view was limiting the Barcode capture capabilities
A novel approach for detecting Barcodes over an extended DOF based on information gathered by the RGB channels of a commercial imager has been described and demonstrated. At the pupil of the imaging system we added a simple circular symmetric binary phase structure with about 3π phase shift at the blue wavelength. Even with a phase structure composed of a single phase ring we achieved acquisition of images with a contrast above 25%, which is well above the noise levels and is suitable for Barcode readers with extended depth of field. This exceeds the capabilities of existing Barcode reader imaging systems by far. There is room for further trade-offs, such as allowing operation at a lower contrast level for obtaining larger depths of field or at a higher contrast level when the required depth of field is being reduced. Computer simulations as well as experiments have proven the capability of this method.
1. E. Barkan and J. Swartz, “System design considerations in bar-code laser scanning,” Opt. Eng. 23, 413–420 (1984).
3. W. Chi and N. George, “Electronic imaging using a logarithmic asphere,” Opt. Lett. 26(12), 875–877 (2001). [CrossRef]
4. N. George and W. Chi, “Computational imaging with the logarithmic asphere: theory,” J. Opt.Soc 20(12), 2260–2273 (2003). [CrossRef]
6. A. Levin, R. Fergus, F. Durand, and T. W. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph. 26(3 Issue 3), 70 (2007) (TOG). [CrossRef]
8. E. Ben-Eliezer, N. Konforti, B. Milgrom, and E. Marom, “An optimal binary amplitude-phase mask for hybrid imaging systems that exhibit high resolution and extended depth of field,” Opt. Express 16(25), 20540–20561 (2008). [CrossRef] [PubMed]
9. B. Milgrom, N. Konforti, M. A. Golub, and E. Marom, “Pupil coding masks for imaging polychromatic scenes with high resolution and extended depth of field,” Opt. Express 15, 15569-15584 (2010) [CrossRef]
10. J. Garcia, J. M. Sanchez, X. Orriols, and X. Binefa, “Chromatic aberration and depth extraction,” Proc. IEEE 1, 762–765 (2000).
11. B. Fishbain, I. A. Ideses, G. Shabat, B. G. Salomon, and L. P. Yaroslavsky, “Superresolution in color videos acquired through turbulent media,” Opt. Lett. 34(5), 587–589 (2009). [CrossRef] [PubMed]
12. O. Bulana, V. Mongab, and G. Sharma, “High capacity color barcodes using dot orientation and color separability,” Proc. SPIE 7254, (2009).
13. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, Second Edition, New York 1996), pp. 126–151.
14. B. E. Bayer, “Color imaging array,” US Patent 3,971,065, Issued: 20.07.1976.
15. User Manuel of uEYE camera, V. 3.33.0 pp.122–124 (2009), from www.ids-imagers.com
16. Data sheet of Metrologic Model MS-4980 Barcode reader.