Open Access
Add to BibSonomy
Abstract
The normalized cross-correlation (NCC) is widely used for image registration due to its simple geometrical interpretation and being feature-agnostic. Here, after reviewing NCC definitions for images with an arbitrary number of dimensions and channels, we propose a generalization in which each pixel value of each channel can be individually weighted using real non-negative numbers. This generalized normalized weighted cross-correlation (NWCC) and its zero-mean equivalent (ZNWCC) can be used, for example, to prioritize pixels based on signal-to-noise ratio. Like a previously defined NWCC with binary weights, the proposed generalizations enable the registration of uniformly, but not necessarily isotropically, sampled images with irregular boundaries and/or sparse sampling. All NCC definitions discussed here are provided with discrete Fourier transform (DFT) formulations for fast computation. Practical aspects of NCC computational implementation are briefly discussed, and a convenient function to calculate the overlap of uniformly, but not necessarily isotropically, sampled images with irregular boundaries and/or sparse sampling is introduced, together with its DFT formulation. Finally, examples illustrate the benefit of the proposed normalized cross-correlation functions.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
The relative alignment or registration of images with common features is important for applications including: image analysis [1–7], computer vision [8–10], wavefront sensing [11–14], medical diagnostics and biomedical research [15–24]. Image registration is often achieved by estimating a coordinate transformation that maximizes or minimizes a function of the images to be registered [25–27]. One such function, and the focus of this work, is the normalized cross-correlation (NCC), which can be maximized to estimate geometrical transformations such as translation, rotation, and/or scaling [16,18,28–30]. Despite the development of ever more sophisticated registration algorithms, the NCC remains widely used because: it is image feature-agnostic, it requires few parameters, it uses the information in all pixels in the overlapping image area, and it has a simple geometric interpretation that will be discussed later. In contrast, registration through the matching of segmented landmarks only uses a fraction of the image information, requires prior knowledge of the features to segment, and often requires understanding of the segmentation algorithm itself for the tuning of relevant parameters [26].
The manuscript is structured as follows. Sections 2 and 3 introduce definitions of the NCC for images with an arbitrary number of dimensions and channels, respectively. In Section 4 we define a generalized normalized weighted cross-correlation (NWCC) in which the contribution of each image pixel can be individually weighted by a real non-negative number, which for the particular case of binary values reduces to the NWCC proposed by Padfield [30,31]. We then define a zero-mean equivalent of the novel NWCC, also for images with arbitrary number of dimensions and channels. Section 6 touches on practical aspects of the computational implementation of all these NCC variations, including calculation times and numerical errors. A convenient overlap function for images with arbitrary boundaries (e.g., irregular, non-convex, non-connected, sparsely sampled) and its fast implementation formulation are presented in section 7, before illustrative examples and a final summary section. Although the motivation for this work is the registration of images, all the functions discussed here are applicable to experimental or simulated samples of any function of interest.
2. Normalized cross-correlation
The NCC between D-dimensional uniformly, although not necessarily isotropically or continuous, sampled images ${I_1}$ and ${I_2}$ with pixel indices $({{i_1}, \ldots ,{i_D}} )$, can be defined as
If the overlapping portions of the normalized cross-correlated images are interpreted as vectors, with each pixel corresponding to an orthogonal dimension in a vector space and the pixel value being the vector amplitude along that dimension, then, the NCC can be interpreted as the cosine of the angle between these vectors. That is, because the NCC is, effectively, the inner product of two vectors divided by their Frobenius norm. Hence, the more similar the overlapping portions of the images, the smaller the angle and the higher the NCC, up to a maximum value of 1. In the extreme situation in which the image overlap is a single pixel (e.g., each of the NCC domain corners when the images are rectangular), the NCC is exactly 1, provided neither pixel value is zero, in which case, the denominator of the NCC is not defined. Therefore, such single pixel overlap scenarios should be excluded from the image registration, as they are of no practical utility. In less extreme situations, when images overlap is only a small fraction of their total number of pixels (i.e., near the NCC domain boundaries), NCC values can be higher than that of NCC peak value that provides the correct image registration, as illustrated by the image pairs shown in Fig. 1, from a custom pupil tracker [32] (top) and a custom ophthalmoscope equipped with adaptive optics [33] (bottom). In the pupil image pairs, a large number of the NCC values (315,144, ∼14% of all NCC values) are equal to or higher than that of the desired peak towards the center of the NCC domain. Because selecting any of these high values would lead to erroneous registration, NCC-based image registration methods often ignore the NCC values in the vicinity of the domain boundary, by restricting the region over which the maximum can be found. Of course, the NCC area with high values that must be ignored can vary substantially with input images, as the second example in Fig. 1 shows, in which a much smaller fraction of NCC values are equal or larger than the correct central peak (56, ∼0.06% of all NCC values).
Fig. 1. Image pairs from a pupil tracker (top) and an adaptive optics ophthalmoscope (bottom, photoreceptor mosaic) the corresponding normalized cross-correlation (NCC, third column) and the thresholded NCC (fourth column), with the last two showing correct but undesired high values near the domain boundary.
The calculation of the NCC for all possible relative image shifts using Eq. (1) can be computationally intensive for large images, while calculation of just a few values using an iterative maximization routine could return an undesired local maximum, rather than the global maximum over the region of interest. Alternative algorithms based on the cross-correlation theorem of discrete Fourier analysis and the availability of fast discrete Fourier transform (DFT) algorithms, are commonly used to reduce calculation times [1,34–37]. In D-dimensional images with N samples along each dimension, fast DFT algorithms can reduce the number of multiplications from $\mathrm{{\cal O}}({{N^{2D}}} )$ to $\mathrm{{\cal O}}({{N^D}{{\log }_2}{N^D}} )$. Such formulation for a matrix formed by the NCC values for all potential shifts along all image dimensions as per the definition in Eq. (1), is,
Fig. 2. Depiction of normalized cross-correlation (NCC) calculated using the discrete Fourier transform (DFT) and its inverse (IDFT), with * denoting complex conjugation and the array multiplication and squaring being performed on an element by element basis. Each rectangle represents a data array, with the grayed areas label ${I_1}$ and ${I_2}$ representing the NCC input images and ${W_1}$ and ${W_2}$ being arrays with unit values and the dimensions of the corresponding images.
It is worth noting that the indices along the D-dimensions can indicate Cartesian, polar, cylindrical, or spherical coordinates to estimate image translation, rotation and/or scaling. In fact, one of the dimensions could even correspond to time.
3. Normalized cross-correlation of multi-channel images
The NCC can be generalized for images in which multiple values are assigned to each pixel. Such multi-channel images are common in spectroscopy, with each channel capturing waves with a different frequency range. Perhaps, the most ubiquitous of such instruments is the color camera with red, green, and blue channels [40–42]. Also, multi-channel images can be captured as combinations of reflectance, fluorescence and/or scattering channels [43–46].
Multiple NCCs have been proposed for working with multi-channel images. Some define the NCC of all the combined channels as the average of the NCC of each channel [47,48], while others first define the cross-correlation between images as the average of the different channels, and then normalize this value [49,50]. We adopt the latter definition because it preserves the geometrical interpretation of the NCC as the inner product of two vectors (images).
Let us now consider images with M channels described by secondary pixel indices m that are not varied as NCC indices. For such images the NCC can be defined as,
This multi-channel NCC can be written in terms of the N-dimensional discrete Fourier transform for fast computation as,
Note that the IDFT and summation order across channels was chosen to avoid the calculation of one IDFT per channel term. Importantly, both the definition in Eq. (3) and the DFT formulation in Eq. (4) reduce to their corresponding single channel versions when $M = 1$.
4. Normalized weighted cross-correlation
An NCC generalization, often referred to as weighted NCC or WNCC [51,52] has been proposed to weight the contribution of each image pixel through the use of a mask. Here we adopt the term normalized weighted cross-correlation (NWCC) instead, to better reflect the order of the mathematical operations, that is first weighting, and then, normalization.
This generalization requires the introduction of the discrete weighting functions ${W_1}$ and ${W_2}$, corresponding to ${I_1}$ and ${I_2}$, respectively, such that,
The use of binary weights, that is, 0 or 1, is equivalent to excluding or including pixels from the NCC calculations, respectively, facilitating the calculation of NCC values using the DFT formulation for non-rectangular images or over irregular regions of interest, as proposed by Padfield [30,31]. Here, we propose to use real, rather than binary values, to implement a more general pixel prioritization criterion. These weights could, for example, be monotonically related to pixel signal-to-noise ratio, selected to prioritize the contribution of pixels to the NCC associated to moving objects over stationary backgrounds, or to mitigate the potentially detrimental impact of saturated pixel values.
This generalized NWCC, which reduces to that by Padfield when weight values are 0 or 1, can be expressed in terms of the DFT as,
A depiction of this algorithm using the naming conventions from Fig. 2 can be seen in Fig. 3.
Fig. 3. Depiction of normalized weighted cross-correlation (NWCC) calculation using the discrete Fourier transform (DFT) and its inverse DFT (IDFT), with * denoting complex conjugation and the array multiplication and squaring being performed on an element by element basis. Each rectangle represents a data array, with the grayed areas label ${I_1}$ and ${I_2}$ representing the NCC input images and ${W_1}$ and ${W_2}$ being arrays with real non-negative values and the dimensions of the corresponding images. Note that this diagram becomes the one in Fig. 2 when the values in ${W_1}$ and ${W_2}$ are 1.
It is important to note from the denominator terms in the formulae and the algorithm depictions, that the NWCC is not equivalent to the NCC with ${I_1}{W_1}$ and ${I_2}{W_2}$ as inputs. Also, the proposed NWCC is not a trivial modification of that proposed by Padfield with binary values, as it becomes apparent when comparing the role of the weighting masks in the denominator.
The NWCC can be naturally extended to multi-channel images by redefining the terms in Eq. (5),
The corresponding DFT formulation for the NWCC is,
5. Zero-mean normalized weighted cross-correlation
The final variation of the NCC introduced here is the zero-mean normalized weighted cross-correlation (ZNWCC), in which for every relative image shift, the mean of the overlapping portion of each image is subtracted before calculating the NWCC. In this way, the ZNWCC would be,
The ZNCC (i.e., the ZNWCC with unit weights) can appear more desirable than the NCC, because it often results in lower number of high values near the NCC domain boundary, as can be seen by comparing Fig. 4 against Fig. 1. In the ZNCC figure, the fraction of values equal to or larger than that of the central ZNCC peak are 0.07 and 0.03%, for the pupil and photoreceptor mosaic images, respectively. This suggests that the ZNCC provides accurate registration over larger displacement ranges than the NCC. This, however, is not always the case [53–57]. Hence, the choice of ZNCC vs. NCC, or ZNWCC vs. NWCC might need to be informed by testing with images representative of those to be registered.
Fig. 4. Image pairs illustrating how high values near the zero-mean normalized cross-correlation (ZNCC) domain boundary poses less of a challenge for image registration than in the NCC (see Fig. 1). The top row shows eye pupil images, and the bottom row shows human photoreceptor mosaic images. The corresponding NCC array values are shown in the third column, with the values equal or greater than the peak near the matrix center, are highlighted in red on the rightmost panels. These red pixels, which are much fewer than those in the NCC, correspond to relative image translations would result in incorrect image registration, and thus, must be excluded from the NCC maximum search.
Just as the NCC and the NWCC, the ZNWCC can also be implemented in the Fourier domain, with the numerator in Eq. (15) becoming,
6. Computational implementation
Comparison of NCC calculation times using its definition against the DFT implementation was performed using a central processing unit (CPU) and a graphics processing unit (GPU). Three algorithms were implemented in the interpreted computing language Python 3.10 on a desktop computer (Intel Core i7-6850 CPU with 32 GB of RAM and an NVIDIA GeForce GTX 1050 Ti GPU). The DFT implementations of the NCC used the library Scipy version 1.9.1 for the CPU-based calculation and the library Cupy version 10.6.0 for the GPU-based calculation. For this test, 2-dimensional images with sizes ranging from 10 × 10 to 1,000 × 1,000 pixels with random values were used, with Fig. 5 showing calculation times. The images were zero-padded just enough to avoid aliasing, as mentioned earlier. As expected, the direct NCC calculation requires execution times that are orders of magnitude larger than for the CPU- and GPU-based DFT implementations.
Fig. 5. Normalized cross-correlation (NCC) computation time for various 2-dimensional image sizes, using three implementations: as per definition running in a central processing unit (CPU) (Eq. (1)), DFT-based running in a CPU, and DFT-based running in a graphics processing unit (GPU). Each point represents an average value, and the vertical bars indicate plus/minus one standard deviation.
It is important to recognize that the NCC formulae using the DFT presented above lend themselves to various optimizations, including reusing memory buffers, choice of DFT algorithm, keeping intermediate calculation arrays in memory, and parallelization of DFT calculations for the two input images. These should be selected based on available computational resources and prioritization of computation time, power consumption, or memory requirements.
An important aspect of implementing NCC, NWCC, ZNCC or ZNWCC functions is dealing with computation errors such as those due to limited precision [58]. The magnitude of these numerical errors is illustrated in Fig. 6, by comparing the NCC between the two photoreceptor images in Fig. 4, calculated with different precisions in the CPU and GPU. A visual examination of the panels on the right side of this figure shows that the DFT implementations in CPU and GPU are slightly different
Fig. 6. Normalized cross-correlation (NCC) computation differences between processors (Intel CPU and Nvidia GPU) and single vs. double precision (SP and DP, respectively) for the photoreceptor image pair in Fig. 4 above. Note that because the difference intensity plots are shown in a logarithmic color scale, the numbers in the color bars are orders of magnitude.
Calculation errors are most impactful when the denominator takes low values and can result in output values with modulus greater than one, which is both incorrect and undesirable. Values greater than 1 are easy to detect, and thus, discard. However, detecting the values that are high due to computation error, yet below one, is more challenging. Here, we propose the provision of an input parameter to NCC functions in addition to the actual pair of images to register. This parameter would be a threshold below which the denominator can be considered negligible, as for example, 0.1% of the Frobenius norm of the images. For the NCC, NWCC, ZNCC or ZNWCC function values with denominator below this threshold, the returned value should be what in most programming languages is referred to as not-a-number (NaN). The rationale for this choice stems from the interpretation of the NCC as the angle between two (image) vectors: if at least one of the two vectors has zero modulus, then the angle between the vectors is not defined.
7. Image overlap function
Image registration using any of the NCC forms presented above requires finding the global maximum within a region of interest of the NCC domain. One could define the region of interest based on a maximum acceptable relative shift between the images. An alternative approach would be to calculate this region of interest so that it corresponds to an image overlap greater than a desired minimum number of pixels, or a fraction of image area. This requires the calculation of a binary mask with the same dimensions as the NCC that defines which values should be used to find the NCC maximum that corresponds to the relative shift between images. For that, one can first calculate the image overlap using weight functions with binary values for each relative shift as the cross-correlation between two binary masks, that is,
8. Examples
8.1 Multi-channel normalized cross-correlation
Let us now illustrate the registration of image pairs with red and green channels using the NCC through the panels in Fig. 7. Each channel of the first image contains two identical letters, one of which is only partially contained within the image boundaries, and a second image containing a single letter. Because the letters in both channels are identical, each of the single-channel NCC for the red and green channels (rightmost columns of the first and second rows) shows two peaks that reach the maximum possible NCC value of 1. These two peaks indicate that the second single-channel images match the first single-channel images perfectly for two different relative shifts, and thus, neither channel on its own provides a unique registration signal. The use of both channels in the NCC (bottom panel), on the other hand, shows three peaks, with only one being the global maximum, thus providing the desired registration shifts. This is, of course, an artificially simple example, but it exemplifies how the combination of multiple channels can reduce the possibility of ambiguous registration.
Fig. 7. Simulated images and the corresponding 1- and 2-channel normalized cross-correlations (NCCs). The NCC of the red and green image pairs show two equal-amplitude peaks, corresponding to perfect image matching over the overlapping areas, while the NCC of the 2-channel images shows three peaks, but only one corresponds to a perfect image match, thus resolving the registration ambiguity.
It should be noted that there is an infinite number of operations that one could use to generate a single array from the pair of single-channel NCC arrays (e.g., average or product), effectively combining the information from both channels. In fact, some might provide superior registration than the NCCs discussed here, but each of them would carry a different mathematical or geometrical interpretation.
The next example, shown in Fig. 8, shows the single- and dual-channel NCC of two-channel human photoreceptor mosaic images captured using a custom ophthalmoscope equipped with adaptive optics [33,45]. In these image pairs, one channel captures multiply scattered light, while the other captures singly scattered light. In the corresponding NCCs, the maxima were found by masking the values that corresponded to less than 4% image area overlap. Here, we can see that the maxima within the search region of the single channel NCCs clearly do not match, indicating that at least one of the two is incorrect. The two-channel NCC shows a maximum that coincides with that of the singly scattered light channel, which is the correct one. The practical challenge with the single-channel NCC is that a person would have to manually decide which of the two channels to use, based on experience (i.e., subjective decision), while in this case, the use of the two-channel NCC can eliminate the need for such a decision.
Fig. 8. Example illustrating how the individual 1-channel normalized cross-correlation (NCC) of multiple scattering (ms) and single scattering (ss) human retinal images is more ambiguous, than the 2-channel NCC, calculated here with equal channel weights. The arrows indicate maxima within the search area, defined by the minimum acceptable image overlap (non-white portion of the panel). The 1-channel NCC peaks do not match, highlighting the potential challenge of using a single channel for image registration.
8.2 Multi-channel normalized weighted cross-correlation with binary weights
Let us now apply the NWCC to a pair of images with irregular boundaries defined using binary weighting functions as shown Fig. 9. It is important to note that the NWCC weight functions need not have convex or even contiguous non-zero regions. In fact, it can even be used to register the sparsely sampled images which are becoming more common due to advances in compressive sensing/imaging [59–61]. In this example, the NCC shows a central vertical region with high values that reduce towards the left and right due to the poor correlation between the zero and non-zero areas of the images, and a maximum that is not the desired peak (i.e., local maximum). The NWCC, on the other hand, looks quite different, with a low contrast but sharp and unique peak that corresponds to the correct registration of the two images.
Fig. 9. NCC and NWCC of human photoreceptor mosaic images with irregular boundaries defined by binary weight functions ${W_1}$ and ${W_2}$. The white NCC and NWCC areas correspond to zero denominator values, and thus, forced to be not-a-number (NaN) because they are undefined. The white arrows indicate the NCC and NWCC maxima, which correspond to incorrect and correct registration, respectively.
8.3 Multi-channel normalized weighted cross-correlation with non-binary weights
The NWCC can also be used to register images using non-binary weights, which might be chosen to represent a function of signal-to-noise ratio. In shot-noise limited images this could be the square root of the image intensity. Alternatively, weights can be used to mitigate image regions known to contain objects that could confound the registration, such as moving or shape-changing features.
Here, we provide an example in which the contribution to the NCC of image pixels known to be affected by noise are either removed or mitigated in the NWCC by using 0.0 and 0.5 weights, respectively. The test images consist of a letter E with pixel values 1.0 over a 0.5 background (see Fig. 10). Both images have additive Gaussian noise with 0.5 standard deviation clipped to between 0 and 2 added to either the left or the right side of the image. This is of course, a carefully chosen example, which shows that completely removing the vertical beam of the letter E using the NWCC with binary weight is better than using the NCC, as the image registration error is reduced from 80 pixels to just 32 when considering a minimum image overlap of 40%. Using the NWCC with partial weight 0.5, however, provides the correct NWCC maximum location, and thus, the desired image registration.
Fig. 10. Normalized cross correlation (NCC, top row) and normalized weighted cross-correlation (NWCC, middle and bottom rows) of simulated images using binary and non-binary (real-valued) weight masks ${W_1}$ and ${W_2}$. The image letter and background values are 1.0 and 0.5, respectively, with additive noise having a 0.5 standard deviation and zero mean. The NCC and NWCC with binary and non-binary masks maxima (red arrows), searched over an area corresponding to a minimum image overlap of 40%, show horizontal image registration errors of -80, 32 and 0 pixels.
9. Conclusions
After reviewing NCC definitions for registration of uniformly sampled multi-channel images with any number of dimensions, we introduced a generalization of Padfield’s binary-weighted NCC to allow real non-negative pixel weights that can be different for each channel. This weighting is of practical importance, as it could be used for addressing pixel signal-to-noise ratio, for example, due to photon noise or non-uniform illumination, prioritization of features of interest, irregular image or data array boundaries, and sparse sampling.
DFT-based formulations were provided for the NCC, NWCC and their zero-mean equivalent to allow fast computation using fast DFT algorithms, known as FFTs, that are widely available for most compiled and interpreted programming languages. A brief mention of numerical errors was made, suggesting the use of not-a-number values as NCC output when the denominator of any of the NCC variations discussed here is zero or close to it. A convenient image overlap function and its DFT formulation were also introduced to facilitate the definition of the NCC maximum search region as requiring a minimum image overlap.
Examples presented here illustrate how the simultaneous use of multiple imaging channels, binary weights, and non-binary weights can improve image registration through disambiguation. This work was motivated by image registration, but all the definitions here can be used on data arrays with samples of any experimental, theoretical, or simulated functions of interest.
Funding
National Eye Institute (P30EY026877, R01EY031360, R01EY032147, R01EY032669, R01EY033475); Research to Prevent Blindness (Departmental award).
Disclosures
The authors declare no conflicts of interest.
Data availability
Software and Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. P.E. Anuta, “Spatial registration of multispectral and multitemporal digital imagery using fast Fourier transform techniques,” IEEE Trans. Geosci. Electron. 8(4), 353–368 (1970). [CrossRef]
2. A. Goshtasby, S.H. Gage, and J.F. Bartholic, “A two-stage cross correlation approach to template matching,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(3), 374–378 (1984). [CrossRef]
3. M. Irani and S. Peleg, “Improving resolution by image registration,” CVGIP: Graphical models and image processing 53(3), 231–239 (1991).
4. K. Briechle and U. D. Hanebeck, “Template matching using fast normalized cross correlation.,” Proc. SPIE 4387, 421129 (2001). [CrossRef]
5. T. Kim and Y.J. Im, “Automatic satellite image registration by combination of matching and random sample consensus,” IEEE Trans. Geosci. Remote Sensing 41(5), 1111–1117 (2003). [CrossRef]
6. Y. Bentoutou, Y. Bentoutou, N. Taleb, et al., “An automatic image registration for applications in remote sensing,” IEEE Trans. Geosci. Remote Sensing 43(9), 2127–2137 (2005). [CrossRef]
7. F. Zhao, Q. Huang, and W. Gao, “Image matching by normalized cross-correlation,” in IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, II729–732 (2006). [CrossRef]
8. B.D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proc. 7th Intl. Joint Conf. on Artificial Intelligence (81) 674–679 (1981).
9. Q. Zheng and R. Chellappa, “A computational vision approach to image registration,” IEEE Trans. on Image Process. 2(3), 311–326 (1993). [CrossRef]
10. S. Baker and I. Matthews, “Lucas-Kanade 20 years on: a unifying framework,” International journal of computer vision 56(3), 221–255 (2004). [CrossRef]
11. V. Michau, G. Rousset, and J. Fontanella, “Wavefront sensing from extended sources,” in Real Time and Post Facto Solar Image Correction, Proc. of the 13th National Solar Observatory, 124 (1992).
12. T.R. Rimmele and R.R. Radick, “Solar adaptive optics at the National Solar Observatory,” Proc. SPIE 3353, 321734 (1998). [CrossRef]
13. P.A. Knutsson, M. Owner-Petersen, and C. Dainty, “Extended object wavefront sensing based on the correlation spectrum phase,” Opt. Express 13(23), 9527–9536 (2005). [CrossRef]
14. V. Michau, J.-M. Conan, T. Fusco, et al., “Shack-Hartmann wavefront sensing with extended sources,” Proc. SPIE 6303, 682728 (2006). [CrossRef]
15. J.A. Maintz and M.A. Viergever, “A survey of medical image registration,” Med. Image Anal. 2(1), 1–36 (1998). [CrossRef]
16. S.B. Stevenson and A. Roorda, “Correcting for miniature eye movements in high resolution scanning laser ophthalmoscopy,” Ophthalmic Technologies Xv 5688, 12–151 (2005). [CrossRef]
17. S. Yang, S. Yang, D. Kohler, et al., “Nonrigid registration of 3-D multichannel microscopy images of cell nuclei,” IEEE Trans. on Image Process. 17(4), 493–499 (2008). [CrossRef]
18. A. Dubra and Z. Harvey, “Registration of 2D images from fast scanning ophthalmic instruments,” Biomedical Image Registration 6204, 60–71 (2010). [CrossRef]
19. F.P. Oliveira and J.M.R. Tavares, “Medical image registration: a review,” Computer methods in biomechanics and biomedical engineering 17(2), 73–93 (2014). [CrossRef]
20. M. Chen, M. Chen, R. F. Cooper, et al., “Multi-modal automatic montaging of adaptive optics retinal images,” Biomed. Opt. Express 7(12), 4899–4918 (2016). [CrossRef]
21. A.E. Salmon, A. E. Salmon, R. F. Cooper, et al., “An Automated Reference Frame Selection (ARFS) Algorithm for Cone Imaging with Adaptive Optics Scanning Light Ophthalmoscopy,” Trans. Vis. Sci. Tech. 6(2), 9 (2017). [CrossRef]
22. M. Azimipour, M. Azimipour, R. J. Zawadzki, et al., “Intraframe motion correction for raster-scanned adaptive optics images using strip- based cross-correlation lag biases,” Plos One 13(10), e0206052 (2018). [CrossRef]
23. M. Chen, M. Chen, R. F. Cooper, et al., “Automatic longitudinal montaging of adaptive optics retinal images using constellation matching,” Biomed. Opt. Express 10(12), 6476–6496 (2019). [CrossRef]
24. M. Zhang, M. Zhang, E. Gofas-Salas, et al., “Strip-based digital image registration for distortion minimization and robust eye motion measurement from scanned ophthalmic imaging systems,” Biomed. Opt. Express 12(4), 2353–2372 (2021). [CrossRef]
25. L.G. Brown, “A survey of image registration techniques,” ACM Comput. Surv. 24(4), 325–376 (1992). [CrossRef]
26. B. Zitova and J. Flusser, “Image registration methods: a survey,” Image and Vision Computing 21(11), 977–1000 (2003). [CrossRef]
27. X.Y. Zhang, X. Zhang, C. Leng, et al., “Multimodal remote sensing image registration methods and advancements: a survey,” Remote Sens. 13(24), 5128 (2021). [CrossRef]
28. B.S. Reddy and B.N. Chatterji, “An FFT-based technique for translation, rotation, and scale-invariant image registration,” IEEE Trans. on Image Process. 5(8), 1266–1271 (1996). [CrossRef]
29. L. Lucchese, G. Doretto, and G.M. Cortelazzo, “A frequency domain technique for range data registration,” IEEE Trans. Pattern Anal. Machine Intell. 24(11), 1468–1484 (2002). [CrossRef]
30. D. Padfield, “Masked object registration in the Fourier domain,” IEEE Trans. on Image Process. 21(5), 2706–2718 (2012). [CrossRef]
31. D. Padfield, “Masked FFT registration,” Proc. IEEE CVPR 15008, 5540032 (2010). [CrossRef]
32. B. Kowalski, B. Kowalski, X. Huang, et al., “Hybrid FPGA-CPU pupil tracker,” Biomed. Opt. Express 12(10), 6496–6513 (2021). [CrossRef]
33. A. Dubra and Y. Sulai, “Reflective afocal broadband adaptive optics scanning ophthalmoscope,” Biomed. Opt. Express 2(6), 1757 (2011). [CrossRef]
34. J.P. Lewis, “Fast template matching. Vision Interface ‘95,” Proceedings 120–123 (1994).
35. D.M. Tsai and C.T. Lin, “Fast normalized cross correlation for defect detection,” Pattern Recognition Letters 24(15), 2625–2631 (2003). [CrossRef]
36. J. W. Luo and E. E. Konofagou, “A fast normalized cross-correlation calculation method for motion estimation,” IEEE Transactions on Ultrasonics Ferroelectrics and Frequency Control 57(6), 1347–1357 (2010). [CrossRef]
37. A. Kaso, “Computation of the normalized cross-correlation by fast Fourier transform,” Plos One 13(9), e0203434 (2018). [CrossRef]
38. R. N. Bracewell, The Fourier Transform and Its Applications, 3rd. ed. (McGraw Hill, 2000).
39. A. V. Oppenheim and R. W. Schafer, Digital Signal Processing (Prentice-Hall, 1975).
40. G.L. Lu and B.W. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. 19(1), 010901 (2014). [CrossRef]
41. B. Lu, B. Lu, P. Dao, et al., “Recent advances of hyperspectral imaging technology and applications in agriculture,” Remote Sens. 12(16), 2659 (2020). [CrossRef]
42. Ul Rehman and S. A. Qureshi, “A review of the medical hyperspectral imaging systems and unmixing algorithmsin biological tissues,” Photodiagn. Photodyn. Ther. 33, 102165 (2021). [CrossRef]
43. D. C. Gray, D. C. Gray, W. Merigan, et al., “In vivo fluorescence imaging of primate retinal ganglion cells and retinal pigment epithelial cells,” Opt. Express 14(16), 7144–7158 (2006). [CrossRef]
44. T. Y. P. Chui, D. A. VanNasdale, and S. A. Burns, “The use of forward scatter to improve retinal vascular imaging with an adaptive optics scanning laser ophthalmoscope,” Biomed. Opt. Express 3(10), 2537–2549 (2012). [CrossRef]
45. D. Scoles, Y. N. Sulai, and A. Dubra, “In vivo dark-field imaging of the retinal pigment epithelium cell mosaic,” Biomed. Opt. Express 4(9), 1710–1723 (2013). [CrossRef]
46. D. Scoles, D. Scoles, Y. N. Sulai, et al., “In Vivo Imaging of Human Cone Photoreceptor Inner Segments,” Invest. Ophthalmol. Vis. Sci. 55(7), 4244–4251 (2014). [CrossRef]
47. R.B. Fisher and P. Oliver, “Multi-variate cross-correlation and image matching,” Proc. 6th British conference on Machine vision (2):623–632 (1995).
48. B. Kong, B. Kong, J. Supancčič, et al., “Cross-domain image matching with deep feature maps,” International Journal of Computer Vision 127(11-12), 1738–1750 (2019). [CrossRef]
49. D.-M. Tsai, C.-T. Lin, and J.-F. Chen, “The evaluation of normalized cross correlations for defect detection,” Pattern Recognition Letters 24(15), 2525–2535 (2003). [CrossRef]
50. S. Mattoccia, F. Tombari, and L. Di Stefano, “Efficient template matching for multi-channel images,” Pattern Recognition Letters 32(5), 694–700 (2011). [CrossRef]
51. T. Liu, L. Qiao, and X. Peng, “Local stereo matching using binary weighted normalized cross-correlation,” Proc. SPIE 9067, 2051674 (2013). [CrossRef]
52. H. Zhang, Z. Tang, Y. Xie, et al., “A similarity-based burst bubble recognition using weighted normalized cross correlation and chamfer distance,” IEEE Trans. Ind. Inf. 16(6), 4077–4089 (2020). [CrossRef]
53. O. Faugeras, J. Martin, J.L. Crowley, et al., “Real time correlation-based stereo: algorithm, implementations and applications,” (Inria, 1993).
54. J. Martin and J.L. Crowley, “Experimental comparison of correlation techniques,” Intelligent Autonomous Systems – IAS 4, 86–93 (1995).
55. N. Roma, J. Santos-Victor, and J. Tomé, “A comparative analysis of cross-correlation matching algorithms using a pyramidal resolution approach,” in Empirical Evaluation Methods in Computer Vision, 117–142 (2002). [CrossRef]
56. U. R. Makkithaya and K. K. Ak, “Qualitative and quantitative evaluation of correlation based stereo matching algorithms,” Lecture Notes in Computer Science 7135 (2011).
57. N. S. Bindu and H. S. Sheshadri, “A Comparative Study of Correlation Based Stereo Matching Algorithms: Illumination and Exposure. Intelligent Computing,” Communication and Devices 309, 191–201 (2015). [CrossRef]
58. J.C Schatzman, “Accuracy of the discrete Fourier transform and the fast Fourier transform,” SIAM J. Sci. Comput. 17(5), 1150–1166 (1996). [CrossRef]
59. D. Takhar, M.A. Neifeld, J. Ke, et al., “A new compressive imaging camera architecture using optical-domain compression,” Proc. SPIE 6065, 659602 (2006). [CrossRef]
60. M.A. Neifeld and J. Ke, “Optical architectures for compressive imaging,” Appl. Opt. 46(22), 5293–5303 (2007). [CrossRef]
61. X. Yuan, D.J. Brady, and A.K. Katsaggelos, “Snapshot compressive imaging: theory, algorithms, and applications,” IEEE Signal Process. Mag. 38(2), 65–88 (2021). [CrossRef]
