Open Access
Add to BibSonomy
Abstract
The imaging depth of field (DOF) of white-light illuminated objects is extended by carefully integrating two image-processing techniques, one optical and one digital. The optical technique makes use of a tailored phase mask positioned at the pupil of the imaging system to cause different color channels to have different focal lengths; accordingly, the phase-mask equipped imaging system acquires a high resolution and reasonably focused image in at least one of the three, red, green, blue (RGB), color channels at any location within the specified DOF. The digital processing comprises fusing the separate RGB images with an original technique that implements principal component analysis to deliver the overall sharpest grayscale composite image throughout the DOF region. The obtained experimental results agree well with the theoretical predictions and demonstrate the capability of the integrated technique to extend the DOF.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Although imaging is now generally performed in full color, there are many instances in which color is not a necessity, and the essential information for decoding is monochromatic. One such case is barcode imaging since most barcodes used at present are monochrome, mostly black and white. In the newly expanding area of face recognition, for instance, most algorithms [1–4] are designed to process grayscale images since color has been found to confound the recognition algorithms. For such applications, color images are converted to grayscale before processing and attempting recognition. For this and similar applications, it is important to extend the depth of field (DOF) of imaging systems that acquire greyscale images.
Image fusion became popular in recent years in order to analyze and restructure the large amounts of data acquired by multi-sensor or multi-spectral systems. In the area of multi-focus imaging and super-resolution, several fusion algorithms have been proposed to handle the situation [5–7]. The advantage of the fusion concept is in allowing corroboration of information that exists in various subset images via an automatic, relatively simple algorithm. Image Fusion has been used for extending the DOF of optical systems, for example by making use of spatial features [8–12]. An in-focus image contains higher spatial frequencies with better contrast than an out of focus image. Thus, by selecting the regions with the highest spatial frequencies, one can get an image with more information. Another method achieves extended DOF for range images by spatial super resolution [13]. Range images are low resolution images, created by laser range scanners; those are followed by pseudo-coloring to associate each pixel with a measured range. The range image contains data corresponding to a certain distance in the scanned area. The laser range scanner scans a specific area from different points of view and creates a few images for this specific zone. A range image is iteratively up-sampled to the same size as a camera image with higher resolution; by overlapping images from different zones, a 3D in-focus image is generated. Mathematical morphology can also be used as the basis of fusion algorithms, both for grayscale and color images [14], where the effective DOF of the sensor may be enhanced considerably without compromising the quality of the image by fusing images captured with different focused regions. Morphological filters select sharply focused regions from various images by selecting regions that have higher information content, since features of an image that are sharply focused contain more data, i.e., higher spatial frequencies, as mentioned earlier. There are other methods using linear mixing models for multi resolution image fusion such as those in [15,16]. Image Fusion can also be useful for systems with controllable cameras [17], where each operation mode creates a different image.
In many applications, image fusion algorithms must be robust and simple enough to provide the fused image almost instantly. Often, image processing algorithms must be generic and require no input from the user. In this work, we employ an image fusion technique to improve the performance of imaging systems with extended DOF. Many methods for extending DOF have been presented previously. In [18], a method is proposed for extending the range over contrast, and in [19] it is suggested that polarization birefringent plate be used for concurrently focusing the near and far fields. Dowski and Cathey [20] suggested using a cubic phase mask– a method with superior theoretical performance, but concurrently, one which is difficult to execute considering the fabrication demands. Complex Wavelets can extend microscopic elements DOF, though the required illumination is approximately constant and specimens should be nonreflective [21]. Color coding can also improve the DOF using sharpness transport across color channels [22] or the combination of color coding and wavefront coding [23]. These are elegant methods which do not require making substantial changes to the optical system. The method proposed in this article using the phase mask is aimed to significantly improve the DOF, beyond the concept of color separation alone, for conditions in which conventional imaging systems produces distorted results.
The paper is organized as follows: Section 2 describes the theory of the phase mask that provides different characteristics to various wavelength range bands, followed by the fusion algorithm that we implement. It is followed by section 3 in which we present experimental results of plane objects imaged at different depths and 3D real-life scene. Section 4 is a first attempt to apply these techniques to the subject of face recognition.
2. Theory
2.1 Phase mask for extended depth of field
Color effects should be taken into consideration when a phase mask is designed, even if the objects handled are monochrome or black and white. This is important when one attempts to create an imaging system that has a large DOF, such that in-focus imaging cannot be maintained. In a previous publication [24], the problems of high resolution imaging even in out-of-focus conditions were resolved by making use of a special pupil mask. The goal was to achieve high resolution acquisition of grayscale objects over a large DOF by taking advantage of the fact that each color band can best provide high resolution images over some portion of the DOF. The regions in which images of a certain color are in sharp focus are designed to complement each other, and perhaps to overlap somewhat, so that altogether the system as a whole has a large DOF. This method can be readily applied to applications where the color content in the resulting fused image is not to be preserved after the processing. The final display is “color blind” but exhibits superior performance in terms of working range and resolution.
A detailed description of the misfocus condition inherent to systems with extended DOF is presented in [25]. In brief, due to [26] the defocus is manifested in the generalized pupil function $\widehat {p}(u,v)$ by a quadratic-phase exponent multiplier of the natural geometrical-optics pupil function of the system $\widetilde {p}(u,v)$, as follows
where $(u,v)$ are the exit plane Cartesian coordinates normalized to half of the exit pupil diameter $D/2$, and $\psi$ is the defocus parameter defined by:The technique presented here relies on handling interim images obtained for various $\lambda$’s in the whole visible spectrum ($400$ nm – $700$ nm), as explained in what follows. Since a clear aperture provides the best modulation transfer function (MTF) for the region surrounding the in-focus position, defined by small values of $\psi$ (see [24]), one should design a mask that behaves as a clear aperture for one of the primary wavelengths, while the DOF provided by the two other primary wavelengths should cover different working ranges corresponding to out-of-focus conditions represented by higher values of $\psi$. In [24], the chosen central wavelengths for blue, green and red are $450$ nm, $550$ nm and $650$ nm, respectively.
We found that high resolution performance requirements – a MTF contrast level exceeding 25% and an out of focus parameter (at the highest $\lambda$) in the range [-6,+6] – can be met by a mask consisting of a single phase ring. Such a ring extends from the outer radius of the pupil up to a normalized radius of 0.7 as seen in Fig. 1(a). The contrast achieved with such a mask is above 25% up to a normalized spatial frequency of 0.6, whereby 2.0 is the cut off frequency of a diffraction limited system. Requiring a thick phase layer to be deposited (or etched) on the mask substrate, the phase exhibited at the three main wavelengths, is sketched in Fig. 1(b). Considering that the applied phases are effectively modulo $2\pi$, the phase at red wavelength is essentially 0, the one at green is $\pi /2$ and the one for the blue wavelength is $\pi$. Therefore at red-wavelength illumination, the pupil mask is transparent, providing zero phases everywhere. As such, the full aperture is available for imaging objects when in-focus at near focus distances. The response at the other two wavelengths will be shown to best cover regions away from focus.
2.2 Principal components analysis
Principal components analysis (PCA) is a common method for splitting a signal into uncorrelated component parts [27]. Following Stathaki [5], we summarize the salient properties of the method which transforms a set of correlated variables into a (possibly) smaller sets of uncorrelated variables, called principal components, in order to achieve two objectives: 1. Reduce the size of the dataset, while retaining the variability in the data; 2. Identify hidden patterns in the data and classify them according to how much of the information, stored in the data, they account for.
When mining a dataset comprised of numerous measurements’ results, it is often the case that some, or all, of the measurements are highly correlated with one another. Given a high correlation between two or more measurements, some of the measurements may be measuring the same properties in different ways. PCA is used to try to take such redundant measurements/data sets and splits them into uncorrelated data sets – where the hope is that each data set is associated with a limited number (perhaps even a single) parameter of interest.
When using PCA, one generally considers one’s dataset as a matrix, $X$. Each column of $X$ consist of the samples of a single measurement set. If each measurement set is an image, then each image is converted into a column vector, the vectors become the columns of the matrix $X$, and each column of the matrix is associated with a single image.
In order for PCA to succeed, one must make sure that the vectors are all organized in the same way. In the case of images, one must make sure that the $n^\textrm {th}$ element of each vector – of each column – represents the same pixel in each image. In addition before making use of PCA, one generally forces each column of $X$ to have a mean value of zero.
Assuming that all necessary pre-processing has been performed, when working with correlated images, each column of $X$ represents one image whose mean value has been forced to zero (by subtracting the mean value of the image from each element). In our case, where we have three images, one each for the red, green, and blue channels – the matrix $X$ is an $N \times 3$ matrix, where $N$ is the number of pixels in the image. Since all of the images are acquired by the same optical system under the same imaging conditions, the magnification is (nearly) identical for all three data sets (no color based rescaling is performed).
Let the elements of the covariance matrix of $X$, $C\left ( X \right )$, be given by
where through our preprocessing we have guaranteed that $N^{-1}\mathop{\sum}\limits_{k = 1}^N {X_i(k)} = 0,\;\;i = 1,2,3$. The diagonal element, $C_{ii}(X)$, is the (empirical) variance of ${X_i}$ while $C_{ij}(X),\;i \ne j$ is the (empirical) covariance. The covariance, like the correlation coefficient to which it is closely related, is a way of measuring the degree of similarity of two random variables. (A covariance of zero, for example, indicates that the random variables are uncorrelated.)After calculating the covariance matrix, one calculates its eigenvalues and their corresponding eigenvectors:
where ${\vec a_i}$ is the $i^\textrm {th}$ eigenvector and ${\alpha _i}$ is the corresponding eigenvalue. As the real matrix $C\left ( X \right )$ is symmetric and positive semi-definite, its eigenvalues are real, non-negative ($\alpha _i\geq 0$) and the eigenvectors can be chosen to be orthogonal and of unit norm. Without loss of generality, we assume that the eignvalues and eigenvectors are indexed to force $\alpha _1\geq \alpha _2\geq \alpha _3$.Next, one uses the eigenvectors to produce a set of derived measurements. The measuremnts are given in matrix form by the matrix $Y$ which is defined by the matrix product $Y = XA$, where $A$ is the $3 \times 3$ matrix whose columns are the eigenvectors ${\vec a_i},\;i = 1,2,3$ and the columns of $Y$ are the derived measurements. The columns of $Y$ are uncorrelated, and the variance of its $i^\textrm {th}$ column is ${\alpha _i}$. Since the three channels are highly pairwise correlated, it can be assumed that ${C_{ij}} > 0,\; i\neq j$, and ${C_{ii}}>0$ by construction. It follows from the Perron-Frobenius theorem [28] that the eigenvector $\vec {a}_1= \left (\rm {W}_1,\rm {W}_2,\rm {W}_3\right )^\textrm {T}$ corresponding to the largest eigenvalue will have all positive components. The vectorized final PCA assessed image $S$, is given by
where the vectors $X_{i,{\rm s}}$ are those for the source RGB channels image (whose means have not been forced to zero). Since $\alpha _1$ is the largest eigenvalue, and it is the variance of the first column of $Y$, $S$, which is the first column of $Y$ plus a constant, is (in its matrix form) the image with the highest variance, i.e. with the most “activity.” Because a blurred image has less “activity,” we speculate that by using the first principal component, which separates out an uncorrelated component with large variance, we will obtain a relatively unblurred final image.Using the phase mask described in section 2.1, one acquires three images, one from each of the three color channels. Usually these images are spatially multiplexed on the output sensor using a Bayer color filter array (CFA). The phase mask that we designed provides extended focus images for the three colors at three different distances. It is this property that allows us to create an imaging system with a large DOF; at any position within the extended DOF one has a good image from at least one of the color channels. Since the necessary information from the target is assumed to be monochrome, one is allowed to fuse the three color images into a final image and display it as a greyscale image. In general, if one of the three images is much better than the others and contains high frequency features (high activity) it will have a weight close to 1, and the other two images will not affect the resulting image very much. PCA decorrelates the input data using second-order statistics and thereby generates compressed data with minimum mean-squared reprojection error. It is intimately related to the blind source separation problem [29], where the goal is to decompose an observed signal into a linear combination of unknown independent signals.
3. Experimental results
A low-cost RGB camera [uEYE 1225], equipped with a CMOS detector with a resolution of $752\times 480$ pixels and a pixel dimension of $6$ microns was utilized. It was equipped with a Computar M1614W achromatic lens having an effective focal length of 16mm and a field of view (FOV) of 45 degrees along the sensor diagonal. A pupil with a 3mm diameter was chosen. Such a stop has been inserted in the Computar lens to define a clear aperture pupil size when experimental results for an open aperture were collected; likewise the fabricated mask inserted in the pupil plane had such a diameter as well. Based on the simulation results in [1], a binary phase mask with a single phase ring was fabricated using quartz with a refractive index of $n_\textrm {{quartz}}(450nm,550nm,650nm)=(1.4656,1.4599,1.4565)$. The normalized cut-off spatial frequency (COF) was $0.6$, whereby $2.0$ is the diffraction limit cutoff. This COF value is compatible with the Nyquist frequency of the detector array.
3.1 Plane objects imaged in varying depths
The imaging system was set to operate from a distance of $30$ cm to $100$ cm and provides a minimal contrast value of 25%, whereby the nominal distance has been set at $50$ cm. The experimental set-up is sketched in Fig. 2. A $184 \times 184$ pixel "Albert Einstein" image was printed and was incoherently illuminated by ambient room light. Images were collected by the camera at distances of 30 to 100cm with an increment of 5cm both with and without the mask. For the mask case, each distance provides 3 sub-images, one from each color channel. The three sub-images were processed using a MATLAB implementation of the PCA algorithm. The resulting images were compared to the clear aperture image at the same range (by casting an RGB image into a grayscale image using a built-in command of MATLAB). The same process was performed with a vehicle license plate" of size $72 \times 136$ pixels. The minimum magnification value, when an image is set at 100cm, was 0.0165 (${d_{img}}$= 16.5mm). Thus, according to the CMOS array technical specification, the horizontal image size should be at least 5cm for a 136-pixel image (the maximum number of pixels in the three images). The physical horizontal size of the 2 printed images was 10 cm, so the images of the various objects were detectable by the sensor array.
In Fig. 3, the sub-images of Albert Einstein, each taken with a different color sensor, at three different distances are shown. Note that for the in-focus condition, the red image is the best since the phase mask behaves like a clear aperture. Under misfocus conditions, the green and blue images become better since the mask now presents a different phase in the designated ring. To produce the fused image, the phase mask acquired images were fused using the PCA algorithm, as described by the block diagram in Fig. 4.
Fig. 3. Sub-images of Einstein taken at three different positions using the RGB camera with the binary phase pupil mask.
In Fig. 5 we present the fused images created using MATLAB and the clear aperture images (acquired without a mask) side by side. The clear aperture image was cast into a greyscale image. In Fig. 6, the sub-images of a vehicle license plate acquired by the three-color channels of a detection focal plane array are shown. The resulting images after applying the fusion method are shown in Fig. 7.
Fig. 5. Three images of Einstein taken at different distances. The images on the left were taken with a clear aperture and those on the right were derived from pictures taken with the mask by applying the fusion algorithm to the images.
Fig. 6. Sub-images of a vehicles license plate taken at three different positions using the RGB camera with the binary phase pupil mask.
Fig. 7. Three images of a license plate taken at different distances. The images on the left were taken with a clear aperture, and those on the right were derived from pictures taken with the mask by applying the fusion algorithm to the images.
In Figs. 5 and 7, one sees that the fused image for $\psi$=4 is better than for $\psi$=6. The reason is the better performance of the blue channel at $\psi$=6 than the blue or the green channel for $\psi$=4 [24]. In any case, the contrast level is above 25%. It can easily be seen that the resulting images using the mask and the fusion process provide better contrast and more details.
3.2 Three-dimensional scene
The imaged targets in the first two experiments were 3D objects printed on a paper – a two dimensional surface. Thus, when imaged at different depths, they possessed a single defocus parameter. To further demonstrate the effectiveness of the suggested method, we imaged cars, illuminated by sunlight, as a case study of real-life 3D objects and conditions. The camera was placed outdoors and a snapshot of two cars – one white and one green, at distances of approximately 1.5m and 10m, respectively, was taken. The focus was adjusted to approximately 1m for simultaneous minor and major defocusing of the front and rear cars, respectively. The blue, green, red, fused and clear aperture images are shown in Fig. 8, with roughly the same FOV excluding minor deviations caused by the insertion or extraction of the phase mask. We follow a similar process to the lab experiments by comparing the fused image to each of the color channels and to clear aperture imaging. In Fig. 8, it is seen clearly that the fused image provides a superior combination of the sharpest areas imaged by each of the three color channels. The license plate of the white vehicle (seen most clearly in the red channel) and the license plate of the green vehicle and its brand logo (seen most clearly in the blue channel) are simultaneously decipherable in the fused image. Also, the area behind the cars with its different details and frequencies appears in the fused image with minimum distortion.
In addition to the qualitative visualization in Fig. 8, a quantitative illustration of the corresponding channels and the fused image contrast improvement is shown in Fig. 9. The close-ups of the near and far licence plates, along with their matching gray scale levels, sampled across the horizontal mid-lines of each plate, clearly demonstrate the beneficial effects of the suggested method. In particular, examining the near and far plates gray scale graphs, the best contrast between the dark numbers to the brighter background is seen for the red and blue channels channels, respectively. Unsurprisingly, the fused image denoted by doted black lines in both graphs, shows performance that is quite similar to that of the noted channels. Note that in the graph corresponding to the close plate, the red and black lines are almost identical, whereas some minor deviations between the blue and black lines are visible in the graph corresponding to the more distant plate. The technique’s performance is due to the weights assigned to each channel by the PCA algorithm. The weights are given by the first eigenvector whose values are ${\vec a}\sim (0.53,0.1,0.37)^\textrm {T}$, and a normalization step has been added, performed after acquiring the PCA coefficients, using the maximum gray level for each color channel to avoid bias. The improvements brought about by the proposed technique, which uses a phase mask and PCA, have been demonstrated in both the 2D and 3D cases. In the 3D scenario in which different objects are located at varying distances, the method provides satisfactory images (with the parameters detailed above) and extends the DOF continuously.This improvement is seen even when the objects are not located at the precise points at which any of the RGB channels is in perfect focus.
Fig. 9. Zoomed in close, upper, and far lower, licence plates, and their matching red, green, blue and fused gray scale levels, sampled across the horizontal mid-lines of each plate.
4. Facial recognition
Facial Recognition (FR) is a classic application of machine vision and is used for security and identification. There are many algorithms for FR; most of them work on grayscale images, and color image inputs are converted to grayscale for processing.
The best-known methods for FR make use of pixel or region correlation [16,17,27,30]. Fusion can help these techniques by providing a "better quality" image. FR algorithms are based on the existence of source images, collected and stored before use, and on our ability to compare newly acquired images with the source images. In addition to problems inherent in the FR process, such as haircut/hairdo issues, different face positions and orientations, and different face expressions, the misfocus condition can lead to a low contrast acquired image that cannot be identified at all.
By using easily available MATLAB open source code, downloaded from [31], we tested our system using images of Albert Einstein. The resulting images acquired by a clear aperture imaging system and thereafter also by a mask-equipped system were tested to see when the software would identify them correctly. The software succeeded in identifying the fused image in each misfocus condition until $\psi$ = 6. However, the software did not identify the two faces when acquisition was done with a clear aperture system if the misfocus parameter $\psi$ was 2.5 in case of Einstein. This test was carried out again with two other FR algorithms [32] with similar results.
Next, a database of 100 faces was tested as well. Each face in the database is a $96\times 96$ pixel image. The effects of misfocus were then simulated and sets of images suffering from misfocus (where $\psi$ varied from 0, when the image is in focus, to 6 in increments of 1) were produced both for the clear aperture and the mask-equipped aperture case. Each simulated image was entered as an input source to the same FR code. The percentages of identification were high when $\psi \leq 3$ for images with a “clear aperture”. However, the fused image was clear enough for the FR algorithm to produce good results until $\psi \sim 6$. In Fig. 10, a summary of the FR test results is shown.
Fig. 10. Percentages of correct identification for a database of 100 images. Blue Bars: Clear Aperture, Red Bars: Phase Mask with Fusion.
This procedure was repeated for negative values of $\psi$ down to $-6$. The results were the same as summarized in Fig. 10. Since the face that needs to be recognized is usually located in an out-of-focus position with respect to the field of the lens without the color mask, the fusion algorithm chooses the appropriate channel automatically. It is interesting to note that for a misfocus condition of $\psi = 4$, when both the green channel and blue channels provide high contrast images (as described in [24]), the weights of the PCA algorithm were found to be $\sim 0.05$, $0.47$ and $0.47$, for the R, G, B channels, respectively. Thus, the fused image provided the final result by combining the images acquired by the G and B channels, and this was better than using either one of those channels by itself.
5. Conclusions
This paper describes some basic image fusion algorithms and provides a proof-of-concept for their ability to provide extended DOF imaging systems with high resolution. A simple circular symmetric binary phase structure used as a pupil mask, complete with a PCA based fusion algorithm, provides acquisition of high resolution grayscale images without introducing dramatic changes in the optical system. Experiments demonstrated the ability to acquire high contrast images (above 0.25) within a substantially extended DOF. This enhanced performance can benefit many machine vision applications such as barcode readers, FR systems, and other applications. In this paper, we limited ourselves to the discussion of the applications, where the information to be deciphered from the fused image is monochrome. An interesting extension for future work is using the color information in the three images to “recolor” the grayscale image. In this work, PCA is used to find a single set of coefficients that is used to fuse the images. The work can be extended by segmenting the image and using PCA to derive a separate set of coefficients for each segment. This will necessitate “re-stitching” the segments – which will provide a further challenge when extending this technique.
Acknowledgments
B.M. and R.A. acknowledge helpful, enlightening discussions with Prof. Mark Auslender of the School of Electrical and Computer Engineering, Ben-Gurion University of the Negev
Disclosures
The authors declare no conflicts of interest.
References
1. A. Samal and P. A. Iyengar, “Automatic recognition and analysis of human faces and facial expressions: a survey,” Pattern Recognit. 25(1), 65–77 (1992). [CrossRef]
2. Q. Wang, A. Alfalou, and C. Brosseau, “New perspectives in face correlation research: a tutorial,” Adv. Opt. Photonics 9(1), 1–78 (2017). [CrossRef]
3. L.-C. Huang, M. A. Neifeld, and A. Ashok, “Face recognition with non-greedy information-optimal adaptive compressive imaging,” Appl. Opt. 55(34), 9744–9755 (2016). [CrossRef]
4. W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld, “Face recognition: A literature survey,” ACM Comput. Surv. 35(4), 399–458 (2003). [CrossRef]
5. A. Katartzis and M. Petrou, “Chapters 1-4,” in Image Fusion, T. Stathaki, ed. (Academic Press, Oxford, 2008), pp. 1–26.
6. H. A. Eltoukhy and S. Kavusi, “Computationally efficient algorithm for multifocus image reconstruction,” in Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications IV, vol. 5017N. Sampat, R. J. Motta, M. M. Blouke, N. Sampat, and R. J. Motta, eds., International Society for Optics and Photonics (SPIE, 2003), pp. 332–341.
7. R. Maruthi, “Multi focus image fusion based on the information level in the regions of the images,” J. Theor. Appl. Inf. Technol. 3, 80–85 (2007).
8. S. Li and Y. Wang, “Multifocus image fusion using spatial features and support vector machine,” in Advances in Neural Networks – ISNN 2005, J. Wang, X.-F. Liao, and Z. Yi, eds. (Springer Berlin Heidelberg, Berlin, Heidelberg, 2005), pp. 753–758.
9. J. Kong, K. Zheng, J. Zhang, and X. Feng, “Multi-focus image fusion using spatial frequency and genetic algorithm,” Int. J. Comput. Sci. Netw. Secur. 8, 220–224 (2008).
10. X. Qu and J. Yan, “Multi-focus image fusion algorithm based on regional firing characteristic of pulse coupled neural networks,” in 2007 Second International Conference on Bio-Inspired Computing: Theories and Applications, (2007), pp. 62–66.
11. M. Zhang, C. Wei, Y. Piao, and J. Liu, “Depth-of-field extension in integral imaging using multi-focus elemental images,” Appl. Opt. 56(22), 6059–6064 (2017). [CrossRef]
12. L. R. Liang and C. G. Looney, “Image fusion with spatial frequency,” in Proceedings of the 15th International Conference on Computer Applications in Industry and Engineering, (2002), pp. 98–100.
13. Q. Yang, R. Yang, J. Davis, and D. Nister,“ Spatial-depth super resolution for range images,” in 2007 IEEE Conference on Computer Vision and Pattern Recognition, (2007), pp. 1–8.
14. I. De, B. Chanda, and B. Chattopadhyay, “Enhancing effective depth-of-field by image fusion using mathematical morphology,” Image Vis. Comput. 24(12), 1278–1287 (2006). [CrossRef]
15. R. H. Pérez, C. Toxqui-Quitl, and A. Padilla-Vivanco, “Image fusion of color microscopic images for extended the depth of field,” in Frontiers in Optics 2015, (Optical Society of America, 2015), p. FTh1F.7..
16. H. Wang, S. Yan, L. Yang, H. Wang, J. Feng, H. Wang, Q. Liu, A. Yan, J. Liao, and G. Liu, “A new image fusion algorithm based on bayer format,” in 2017 14th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), (2017), pp. 158–161.
17. W. B. Seales and S. Dutta, “Everywhere-in-focus image fusion using controlablle cameras,” in Sensor Fusion and Distributed Robotic Agents, vol. 2905P. S. Schenker and G. T. McKee, eds., International Society for Optics and Photonics (SPIE, 1996), 227–234.
18. E. Ben-Eliezer, Z. Zalevsky, E. Marom, and N. Konforti, “All-optical extended depth of field imaging system,” J. Opt. A: Pure Appl. Opt. 5(5), S164–S169 (2003). [CrossRef]
19. Z. Zalevsky and S. Ben-Yaish, “Extended depth of focus imaging with birefringent plate,” Opt. Express 15(12), 7202–7210 (2007). [CrossRef]
20. E. R. Dowski and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34(11), 1859–1866 (1995). [CrossRef]
21. B. Forster, D. Van De Ville, J. Berent, D. Sage, and M. Unser, “Complex wavelets for extended depth-of-field: A new method for the fusion of multichannel microscopy images,” Microsc. Res. Tech. 65(1-2), 33–42 (2004). [CrossRef]
22. F. Guichard, H.-P. Nguyen, R. Tessières, M. Pyanet, I. Tarchouna, and F. Cao, “Extended depth-of-field using sharpness transport across color channels,” in Digital Photography V, vol. 7250B. G. Rodricks and S. E. Süsstrunk, eds., International Society for Optics and Photonics (SPIE, 2009), pp. 192–203.
23. Z. Cao, C. Zhai, J. Li, F. Xian, and S. Pei, “Combination of color coding and wavefront coding for extended depth of field,” Opt. Commun. 392, 252–257 (2017). [CrossRef]
24. B. Milgrom, N. Konforti, M. A. Golub, and E. Marom, “Novel approach for extending the depth of field of barcode decoders by using rgb channels of information,” Opt. Express 18(16), 17027–17039 (2010). [CrossRef]
25. B. Milgrom, N. Konforti, M. A. Golub, and E. Marom, “Pupil coding masks for imaging polychromatic scenes with high resolution and extended depth of field,” Opt. Express 18(15), 15569–15584 (2010). [CrossRef]
26. J. Goodman, Introduction to Fourier Optics, McGraw-Hill Series in Electrical and Computer Engineering: Communications and Signal Processing (McGraw-Hill, 1996).
27. K. Pearson, “Liii. on lines and planes of closest fit to systems of points in space,” The London, Edinburgh, Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901). [CrossRef]
28. C. R. MacCluer, “The many proofs and applications of perron’s theorem,” SIAM Rev. 42(3), 487–498 (2000). [CrossRef]
29. P. Comon, C. Jutten, and J. Herault, “Blind separation of sources, part ii: Problems statement,” Signal Process. 24(1), 11–20 (1991). [CrossRef]
30. C. Eckart and G. Young, “The approximation of one matrix by another of lower rank,” Psychometrika 1(3), 211–218 (1936). [CrossRef]
31. “The image fusion server,” (2019). http://www.imagefusion.org.
32. “Face recognition homepage,” (2019). http://www.face-rec.org.
