Abstract

Accurate and automatic registration of multimodal retinal images such as fluorescein angiography (FA) and optical coherence tomography (OCT) enables utilization of supplementary information. FA is a gold standard imaging modality that depicts neurovascular structure of retina and is used for diagnosing neurovascular-related diseases such as diabetic retinopathy (DR). Unlike FA, OCT is non-invasive retinal imaging modality that provides cross-sectional data of retina. Due to differences in contrast, resolution and brightness of multimodal retinal images, the images resulted from vessel extraction of image pairs are not exactly the same. Also, prevalent feature detection, extraction and matching schemes do not result in perfect matches. In addition, the relationships between retinal image pairs are usually modeled by affine transformation, which cannot generate accurate alignments due to the non-planar retina surface. In this paper, a precise registration scheme is proposed to align FA and OCT images via scanning laser ophthalmoscopy (SLO) photographs as intermediate images. For this purpose, first a retinal vessel segmentation is applied to extract main blood vessels from the FA and SLO images. Next, a novel global registration is proposed based on the Gaussian model for curved surface of retina. For doing so, first a global rigid transformation is applied to FA vessel-map image using a new feature-based method to align it with SLO vessel-map photograph, in a way that outlier matched features resulted from not-perfect vessel segmentation are completely eliminated. After that, the transformed image is globally registered again considering Gaussian model for curved surface of retina to improve the precision of the previous step. Eventually a local non-rigid transformation is exploited to register two images perfectly. The experimental results indicate the presented scheme is more precise compared to other registration methods.

© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Nowadays, many researches have been done in medical image processing to present systems that can detect and analyze various diseases automatically. Minimizing expenditures, increasing detection speed, reducing subjective mistakes and enhancing accuracy are the chief goals of these systems. Particularly in ophthalmology, a lot of researches have been conducted to propose and apply systems for diagnosis and analysis of several retinal diseases including Diabetic Retinopathy (DR) and Age-related Macular Degeneration (AMD) [1].

Image registration is the procedure of aligning two or more images into a unique common coordinate system. Applying registration on multimodal images leads to integration of information, thus benefitting from complementary nature of images. There are various ocular imaging modalities such as Fluorescein Angiography (FA) and Optical Coherence Tomography (OCT), each of which shows some features of retina better than the others. FA is an invasive imaging technique that clearly illustrates retina vasculature through injection of fluorescein dye. FA is known as the gold standard ocular imaging for presenting neurovascular structure and assessing vasculature diseases such as DR and Diabetic Macular Edema (DME) [2]. On the other hand, OCT is non-invasive, prevalent ocular imaging that works based on the Michelson interferometry rule using near-infrared light instead of sound in ultrasound and presents a cross-sectional view of retinal structure. OCT can be used to identify macular edema, macular cysts, vitreomacular traction, sub-retinal fluid, pigment epithelial detachment, choroidal neovascularization and measuring retinal thickness [3].

As mentioned before, FA is used to diagnose DR as a common retinal disease with hyperfluorescent spots as hallmarks of Microaneurysm (MA) [4]. The greater number of MA spots indicates the DR progression. Albeit FA depicts neurovascular structure of retina exactly, it utilizes intravenous injection that is harmful for human body and requires an expert photographer with the patient remaining strictly still. Unlike FA, OCT is non-invasive with retinal in-depth data. Therefore, clinicians usually use information of both imaging modalities for better diagnosis and evaluation of retinal diseases. Great number of papers have evaluated MA visual properties via intuitive comparison of FA and OCT images simultaneously [48] whereas information of both images can be added together using automatic and precise retinal image registration. Furthermore, due to reduction in subject-dependent errors, this can be more helpful for clinical purposes.

So far, many retinal image registration techniques have been proposed which align images of various modalities into a unique coordinate system. From a technical point of view, these techniques can be classified into two categories: segmentation-based methods and non-segmentation-based ones [9]. In the former, the retinal image is partitioned into meaningful segments such as vessels and optic disc making the analysis of the image simpler [10]. In the latter, the full image contents are processed using image processing techniques. A comprehensive review, analysis, and classification of the retinal vessel segmentation algorithms and methodologies is provided by [11], also a survey of state-of-the-art techniques for extracting retinal vessels is presented in [12]. In [13], a fast and accurate registration algorithm is proposed based on Salient Feature Regions (SFR) followed by local rigid transformation. The alignment process is then augmented with a global second order polynomial transformation. This method is only applicable to mono-modal fundus images. In [14], a registration algorithm for poor quality multimodal retinal images using Harris corner detector and Partially Intensity Invariant Feature Descriptor (PIIFD) is presented. A multimodal registration of Spectral Domain OCT (SD-OCT) volumes and fundus photographs is suggested in [15] which uses Features from Accelerated Segment Test (FAST) feature detector and Histogram of Oriented Gradients (HOG) descriptor. In this method, the affine transformation is computed using RANdom SAmple Consensus (RANSAC) [16]. A multi-resolution difference of Gaussian pyramid with Saddle detector (D-Saddle) is proposed in [17], to detect feature points on the low-quality region that consists of vessels with varying contrasts and sizes. In [18] a two-step framework is proposed to register multimodal retinal images. In the first step, HOG descriptor matching is applied to the mean phase image for transformation computation and in the second step, the deformation registration method based on Modality Independent Neighborhood Descriptor (MIND) is used to improve the registration accuracy. In [19], reliable feature point sets are extracted based on SURF detector and PIIFD descriptor from model and target retinal images. Then Mixture Feature and Structure Preservation (MFSP) is used to map feature point sets for finding exact point correspondences. An Adaptive RANSAC (A-RANSAC) method is proposed in [20] for multimodal retinal image registration. In this method, the threshold value is chosen so that the Root Mean Square Error (RMSE) and the number of removed matches are optimized simultaneously. It should be noted that all of the aforementioned methods and techniques are included in none segmentation-based category. The following methods are from the segmentation-based one.

In [21] a registration of OCT projection images with color fundus photographs is proposed based on a hierarchical local feature matching that utilizes the search of local maximization of the similarity function. Reference [22] registers pairs of FA frames for the sake of segmenting fluorescein leakage in patients with DME where two-step registration method performs global and local registration in turn. In global registration step, after vessel extraction using exploratory dijkstra forest algorithm, Speeded Up Robust Features (SURF) features are detected, extracted, and matched, then M-estimator Sample Consensus (MSAC) [23] algorithm is applied for transformation computation. Thereafter in local registration step, intensity multi-resolution registration is performed on local patches to obtain optimal results. In [24], a joint vessel segmentation and deformable registration model is proposed based on Convolutional Neural Network (CNN). In vessel segmentation, a style loss guides the model to generate segmentation maps and helps transform images of different modalities into consistent representations. In deformable registration, a content loss helps finding dense correspondence for multi-modal images. An image registration algorithm is proposed in [25] to trace changes in the retina structure across modalities using vessel segmentation and automatic landmark detection. The segmentation of the vessels is done using a U-Net and the detection of the vessel junctions is achieved with Mask R-CNN. In [26] a new method is presented to register SD-OCT and FA images. For this purpose SLO images captured by Heidelberg Spectrally HRA2/OCT device are used as intermediate images. To the best of our knowledge, only [26] registers OCT images with FA photographs via corresponding SLO images.

As a result of differences in multimodal retinal images, when common feature-based registration methods are applied in RANSAC or MSAC, the outlier matches are not removed completely. State-of-the-art mismatch removal techniques are proposed in [2730]. An efficient approach is designed in [27] based on maintaining the local neighborhood structures of the potential true matches. The problem is formulated into a mathematical model, and a closed-form solution with linearithmic time and linear space complexities is derived. In [28], the mismatch removal is converted to a two-class classification problem, and learning a general classifier to determine the correctness of an arbitrary putative match. Method in [29] adaptively clusters the putative matches into several motion consistent clusters together with an outlier/mismatch cluster. The classic Density Based Spatial Clustering method of Applications with Noise (DBSCAN) is customized in the context of feature matching to implement spatial clustering. An iterative clustering strategy is designed to promote the matching performance in case of severely degraded data. In [30], a feature guided Gaussian Mixture Model (GMM) is proposed for the non-rigid registration of retinal images. The problem is formulated as an estimation of a feature guided mixture of densities: a GMM is fitted to one point set in which the centers of the Gaussian densities characterized by spatial positions associated with local appearance descriptors are constrained to coincide with the other point set.

Due to differences in contrast, resolution and brightness of the multimodal retinal images, prevalent feature detection, extraction and matching schemes do not result in perfect matches. Also, the images resulted from vessel extraction of image pairs are not exactly the same and the accuracy of retinal image registration methods that use vessel segmentation is largely dependent on the accuracy of vessel segmentation. In addition, the relationships between retinal image pairs are usually modeled by affine transformation, which cannot generate accurate alignments due to the non-planar retina surface.

In this paper, a novel method is proposed for registering FA and SD-OCT images using SLO photographs. For doing so, after applying some preprocessing techniques on FA and SLO images, the vessels are extracted in both images and the gray level images are converted to new black and white vessel-map ones. Next, a new feature detector-descriptor method is applied on these vessel-map photographs to remove outlier matches completely. For this goal, first, SURF features are detected and extracted from both images. Likewise, FAST-HOG framework is applied on both images to detect and extract new feature set. Applying only one of these feature detector-descriptors separately in RANSAC or MSAC, does not result in perfect matches. So, SURF feature descriptors from two images are matched using approximate nearest neighbors [31]. The same procedure is iterated with HOG feature descriptors. Since this matching method may include some erroneous matches, the MSAC method is applied separately on matched SURF and HOG features to refine them, but because of vessel segmentation errors and different modalities, some outliers still remained. After that the two vectors of matched featuresF resulted from the previous steps are concatenated to enhance matched features. After applying MSAC again on concatenated features and obtaining perfect matches, global rigid transformation is calculated and applied to the vessel-map and the original images. Combining these features and applying the MSAC for three times, guarantee complete elimination of outlier matches. In addition, to consider non-planar surface of retina, the transformed images are globally aligned again applying a Gaussian model for the surface of retina which increases the accuracy of the previous step. This reduces the amount of deformation effects resulted from the subsequent local registration step. High image deformations in multimodal image registration, could affect the primary image, especially in parts of image that contain no vessels. Eventually displacement field between transformed and reference original images are estimated using method presented in [32] to register two images perfectly. Our dataset contains 36 image pairs of 21 subjects with diabetic retinopathy. Using accurate registration of FA and SD-OCT images, enables us to exactly detect location and morphological appearance of diabetic retinopathy symptoms such as MAs and leakages in OCT B-scans.

The proposed methods are explained in section 2. section 3 presents experimental results and finally, this paper is concluded in section 4.

2. Methods

Because of the different nature of two modalities, direct registration of OCT images and FA photographs is not possible; one presenting a depth view of retina and the other showing the surface of it. On the other hand, Heidelberg Spectralis HRA2/OCT device provides the capability of simultaneous SLO/OCT imaging by means of a single light source that guarantees exact correspondence between SLO image of retina surface and cross sectional B-scans of OCT. In this paper precise automatic registration of FA images with OCT photographs is proposed using registration of FA images with SLO photographs. Comprehensive block diagram of the proposed four-step method is demonstrated in Fig. 1. As shown, overall process is divided into four main sections: data acquisition, preprocessing, global registration, and local registration. In the following, the steps are explained in detail.

 figure: Fig. 1.

Fig. 1. Block diagram of overall process of registration.

Download Full Size | PPT Slide | PDF

2.1 Data acquisition

In this research, dataset is comprised of 36 pairs of FA and SLO images of 21 subjects with diabetic retinopathy, where SLO image pixels are perfectly in correspondence with OCT B-scans (Fig. 2 and Fig. 3). The FA, OCT, and SLO images are captured via Heidelberg Spectralis HRA2/OCT device in Didavaran Eye Clinic, Isfahan. Moreover the FA and SLO images are the same size as 768×768 pixels and FA images were captured with two different fields of views (30 and 55 degrees). Here, the FA and SLO images are considered as moving (source) and fixed (target) images, respectively.

 figure: Fig. 2.

Fig. 2. (a) SLO image. (b) the OCT B-scan corresponds to the yellow line in the SLO image.

Download Full Size | PPT Slide | PDF

 figure: Fig. 3.

Fig. 3. Our data set includes 36 pairs of FA and SLO images. Odd columns illustrate FA images of different field of view (30 and 55 degrees), even columns depict the SLO images corresponding to the left side column.

Download Full Size | PPT Slide | PDF

2.2 Preprocessing

In this step, first the contrast of FA and SLO images are enhanced utilizing the Contrast Limited Adaptive Histogram Equalization (CLAHE) method [33]. Then FA image pixels intensities are complemented to become similar to SLO image. Next, the vessels of both moving and fixed photographs are extracted. In order to do so, the method presented in [34] is applied to precisely segment the main vessel structure of the smoothed image from original one, then binarization using Otsu's threshold selection method [35] and elimination of small objects from binary image are performed. Due to differences in contrast, resolution and brightness of the multimodal retinal images, the binary images resulted from vessel extraction of FA and SLO images are not exactly the same (Fig. 4).

 figure: Fig. 4.

Fig. 4. Vessel Extraction. (a) FA image. (b) vessels extracted from FA. (c) SLO image. (d) vessels extracted from SLO.

Download Full Size | PPT Slide | PDF

2.3 Global registration

Global registration process consists of bringing FA and SLO images into spatial alignment. Here, FA image is mapped into the space of SLO photograph. For the registration scheme, it should be taken into account that the retina surface is not flat [1,36,37]. The registration schemes in [36,37] use the quadratic model for the surface of retina. Since these methods consider only the small shifts in eye and then model them as translations and rotations, they cannot be applied directly to multimodal and multi-scale images. Also, quadratic model does not adequately describe the retina surface for field of view of our image data set and leads to inaccurate results. On the other hand, studies in [1] describes the retina as a Gaussian surface, but they did not use this theory in related applications. Therefore, a new two-step global registration is proposed applying Gaussian model to improve the global registration accuracy. In our proposed method, the first step aligns two images globally using a new feature-based method and making them of the same scale, and then Gaussian model is applied to the system to calculate displacements to decrease the effects of deformations in the consequent local registration step. Rigid global registration is obtained using the following steps:

2.3.1 Control points detection

From another point of view, image registration techniques can be classified into two categories: intensity based techniques and feature based ones. In the former, whole image or sub- images are registered whereas the latter uses distinct image features as control points and matches corresponding features of two images for registration. Control points are well-defined image pixels that can be detected robustly and are distributed all over the image. In the proposed method, features are detected, extracted and matched to be used as input of transformation computation stage. Therefore, after vessel extraction, the control points should be detected in vessel-map images. As mentioned before, due to inevitable errors in vessel segmentation algorithms, using a feature detector-descriptor and MSAC alone does not lead to perfect matches. To address this problem, two feature detection algorithms are utilized: Features from Accelerated Segment Test (FAST) corner points [38] and Speeded Up Robust Features (SURF) [39].

2.3.1.1 FAST corner detector

FAST corner detector utilizes a Bresenham circle of radius 3 containing 16 pixels around the candidate pixel (Fig. 6(a)). If the pixel satisfies either of the two following conditions, it can be determined as the corner point:

$$\begin{array}{l} 1:\forall x \in S,{I_x} > {I_p} + T\\ 2:\forall x \in S,{I_x} < {I_p} - T \end{array}$$
where $S$ and ${I_x}$ denote a set of $N$ contiguous pixels of Bresenham circle and the intensity of pixel $x$, respectively. Also ${I_p}$ stands for the intensity of candidate pixel and $T$ for the threshold. In these conditions the parameters T and $N$ are selected in a manner that there is a compromise between number of corner points and computational complexity; here $N$ can be 9, 10, 11 and 12. Also a high speed test is used to reject non-corner pixels quickly in a way that if at least three pixels out of four specific pixels of Bresenham circle, namely pixel 1, 5, 9, and 13 are not less than ${I_p} - T$ or greater than ${I_p} + T$, the evaluated pixel is not a corner, otherwise all 16 pixels should be examined [38] (Fig. 5(a)). Fig. 5(b) and Fig. 5(c) indicate FAST features detected in vessel maps of SLO and FA images.

2.3.1.2 SURF (Fast-Hessian) blob detector

Blobs are image regions which differ from surrounding regions in some image properties like color, brightness, etc. The principles behind the SURF blob detector are based on that of Scale-Invariant Feature Transform (SIFT) [40], however they differ in details. SURF works on integral images instead of original ones and uses box filters to increase the speed of filtering. Utilizing integral image, the summation of values within a rectangle of a different size in image can be computed just through evaluating four corners of that rectangle. SURF employs determinant of hessian matrix Eq. (1) to specify the location and scale of feature and scale of feature points.

$$H(x,\sigma ) = \left[ {\begin{array}{{cc}} {{L_{xx}}(x,\sigma )}&{{L_{xy}}(x,\sigma )}\\ {{L_{xy}}(x,\sigma )}&{{L_{yy}}(x,\sigma )} \end{array}} \right]$$
Here, ${L_{xx}}(x,\sigma )$ denotes the convolution of Gaussian second order derivatives with the image point $x$ at scale $\sigma$, similar to ${L_{xy}}(x,\sigma )$ and ${L_{yy}}(x,\sigma )$. For speeding up, the approximations of second order derivatives are used instead. The point could be considered as blob where $\det ({H_{approx}})$ of Eq. (2) becomes maximum in that point.
$$\det ({H_{approx}}) = {D_{xx}}.{D_{yy}} - {(0.9{D_{xy}})^2}$$

 figure: Fig. 5.

Fig. 5. (a) Fast corner detector and Bresenham circle. (b) FAST corners in FA vessel map (green dots). (c) FAST corners in SLO vessel map (green dots).

Download Full Size | PPT Slide | PDF

 figure: Fig. 6.

Fig. 6. SURF features (a) 150 strongest SURF features on SLO vessel map (green circles). (b) 150 strongest SURF features on FA vessel map (green circles).

Download Full Size | PPT Slide | PDF

In Eq. (2), $\det ({H_{approx}})$, ${D_{xx}}$, ${D_{yy}}$ and ${D_{xy}}$ are approximations for $\det (H)$, ${L_{xx}}$, ${L_{yy}}$ and ${L_{xy}}$, respectively [38]. Figure 6 indicates the 150 strongest SURF features in both SLO and FA vessel maps.

2.3.2 Feature extraction

When features have been detected, the local patch around each feature can be described as a compact representation named feature vector. In proposed method, the SURF (fast-hessian) and FAST detected features are extracted using SURF and HOG feature descriptors.

2.3.2.1 SURF feature descriptor

To achieve rotation-invariant descriptor, main orientation is assigned to the previously-detected SURF features. For doing so, Haar wavelet responses are calculated in vertical and horizontal directions in circular neighborhood of radius 6×s around detected feature, where s denotes the image scale of the feature. After that, appropriate Gaussian weights are applied. The main orientation is appraised via computing the sum of all wavelet responses within a sliding orientation window of 60 degrees angle. Then for a $(4 \times 4)$ sub-region aligned to the estimated orientation around detected feature, horizontal and vertical wavelet responses are taken and a vector $v = (\sum {{d_x},\sum {{d_y}} ,\sum {|{{d_x}} |} } ,\sum {|{{d_y}} |} )$ is formed for each pixel leading to $64$-dimension $(4 \times 4 \times 4)$ feature vector [39].

2.3.2.2 HOG feature descriptor

Here HOG descriptor is used to describe FAST features [41]. Individual gradients may be affected by noise. This is why histogram of gradients is utilized to represent image patches that makes descriptor robust against noise. For this aim, first image is divided to blocks and each block is partitioned into cells. Thereafter gradient value and orientation are calculated for each pixel within the cell. $N$ bins are considered, each of which represents an angle starting from 0 degree, ending by $180 - ({\raise0.7ex\hbox{${180}$} \!\mathord{\left/ {\vphantom {{180} N}} \right.}\!\lower0.7ex\hbox{$N$}})$ degree with steps of ${\raise0.7ex\hbox{${180}$} \!\mathord{\left/ {\vphantom {{180} N}} \right.}\!\lower0.7ex\hbox{$N$}}$ degree for unsigned gradients. Within the cell, the gradient values corresponding to the same orientation are summed leading to $N$-dimension vector where each dimension contains aggregate of corresponding gradient values. Taking into account that if the gradient orientation is between two bins, the corresponding value should be split proportionally into both adjacent bins. Finally, feature vector is normalized to be not affected by lighting variations. In the proposed method, $N$=9, cell size = [8,8] pixels, and block size = [2,2] cells, so HOG vector has 36 dimensions (9×2×2). Figure 7 visualizes HOG descriptors on vessel map images.

 figure: Fig. 7.

Fig. 7. HOG visualization. (a) SLO vessel map. (b) HOG visualization of (a). (c) FA vessel map. (d) HOG visualization of (c).

Download Full Size | PPT Slide | PDF

2.3.3 Feature matching

After FAST and SURF features are detected and extracted utilizing HOG and SURF descriptors, HOG descriptors of both binary images containing vessel maps should be matched. This procedure is applied for SURF features in the same way. To do this, the method in [31] is used which presents an automatic algorithm for searching approximate nearest neighbor considering data sets and desired precision to speed up the matching process. According to [31], depending on the data set and required precision, two methods namely searching hierarchical k-means trees with a priority search order [31] and multiple randomized kd-trees [42] can perform perfectly. Afterwards, since the abovementioned matching methods may include some erroneous matches, the MSAC method is applied separately on matched SURF and HOG features to refine them and obtain better matches. This results in more precise inlier set of matched features, but as can be seen in Fig. 8(a) some outliers still remained. MSAC method is a variant of RANSAC. For more details, the thresholding in RANSAC can be understood as a top-hat cost function whereas MSAC uses truncated quadratic cost function [43]. The steps of MSAC are as follows:

  • a. Select a subset of two pairs from matched features (may contain incorrect matches) and compute the similarity transformation from this subset.
  • b. Apply the transformation to the remaining matched features from moving image and calculate the distances between transformed feature points and corresponding feature points from fixed image. The set of transformed features of moving image with their distances from corresponding features of fixed image are less than the previously-mentioned threshold are considered as inliers and denoted by ${S_i}$.
  • c. Repeat the steps a and b for $N$ times ($N$=1000) and determine the largest number of members for ${S_i}$ as inliers. In this step, transformation computation is used only to refine inlier matches and overall transformation estimation is done in the next step.

 figure: Fig. 8.

Fig. 8. Final MSAC step refines the previously-matched features. (a) matched features after joining separately refined SURF and HOG matches (still contains few outliers). (b) obtaining perfect matches after final MSAC step.

Download Full Size | PPT Slide | PDF

2.3.4 Transformation computation

To compute the similarity transformation which is registration objective, the matched FAST and SURF features of fixed and moving images are joined. Thereafter to calculate transformation, the MSAC method is utilized again and final transformation is re-computed utilizing all inliers associated with the largest number of ${S_i}$. Using SURF and FAST features together in this manner has two main advantages: first, it benefits from properties of both feature detection-description frameworks and secondly, refining matched features using the MSAC for three times, causes the erroneous matches to be removed completely. Figure 8 shows how final MSAC step refines the previously-matched features and perfects them. Then the computed transformation is applied to the original images. The proposed feature detection, description and matching scheme is flexible and can be generalized. In fact, other feature detection, extraction and matching techniques can be added to the scheme to increase the number of inlier matches under the penalty of longer execution time. In general, for $k$ number of separate feature detectors, m-sac should be applied for $k + 1$ times.

2.3.5 Applying the Gaussian model for retina surface

As mentioned before, the retina surface is not flat and in the proposed method, Gaussian model is applied to improve the global registration accuracy. Figure 9 illustrates a point on retina surface that is expressed in two different 3D coordinate systems related to two different cameras namely ${C_p}$ and ${C_q}$. These points are represented as ${P^T} = (X,Y,Z)$ and ${Q^T} = (X^{\prime},Y^{\prime},Z^{\prime})$ in coordinate systems of ${C_p}$ and ${C_q}$. Assume the transformed FA and SLO images are shown by ${I_p}$ and ${I_q}$ corresponding to $P$ and $Q$ coordinate systems. The Gaussian equation which models the retina surface and relates $X,Y$ and $Z$ in the coordinate system of ${C_p}$ is:

$$Z = {A_1}{e^{ - {{(\frac{{(X - {A_2})}}{{{A_3}}})}^2}}}{e^{ - {{(\frac{{(Y - {A_4})}}{{{A_5}}})}^2}}}$$
where ${A_1},{A_2},{A_3},\ldots ,{A_5}$ are unknown parameters. Here, the aim is to derive the transformation that maps the image coordinates $p$ to $q$, using the transformation that relates P and $Q$. Because of the global registration of previous step, the rigid transformation relates $P$ and Q (Eq. (4)):
$${\boldsymbol Q} = {\boldsymbol RP} + {\boldsymbol t}$$
where Q, $P$, $R$ and $t$ are ${C_q}$ and ${C_p}$ coordinate systems, orthonormal rotation and translation matrices.
$$\left[ {\begin{array}{{c}} {X^{\prime}}\\ {Y^{\prime}}\\ {Z^{\prime}} \end{array}} \right] = \left[ {\begin{array}{{ccc}} {{r_{11}}}&{{r_{12}}}&{{r_{13}}}\\ {{r_{21}}}&{{r_{22}}}&{{r_{23}}}\\ {{r_{31}}}&{{r_{32}}}&{{r_{33}}} \end{array}} \right]\left[ {\begin{array}{{c}} X\\ Y\\ Z \end{array}} \right] + \left[ {\begin{array}{{c}} {{t_x}}\\ {{t_y}}\\ {{t_z}} \end{array}} \right]$$

 figure: Fig. 9.

Fig. 9. Two different camera viewpoints of the retina.

Download Full Size | PPT Slide | PDF

Here, weak perspective projection model adequately projects the 3D retina surface to 2D image plane [36,37]. The projections of P and $Q$ in images ${I_p}$ and ${I_q}$ considering homogenous coordinates is as follows:

$$\left[ {\begin{array}{{c}} {wx}\\ {wy}\\ w \end{array}} \right] \cong {M_p}\left[ {\begin{array}{{c}} X\\ Y\\ Z\\ 1 \end{array}} \right]\quad and\quad \left[ {\begin{array}{{c}} {w^{\prime}x^{\prime}}\\ {w^{\prime}y^{\prime}}\\ {w^{\prime}} \end{array}} \right] \cong {M_q}\left[ {\begin{array}{{c}} {X^{\prime}}\\ {Y^{\prime}}\\ {Z^{\prime}}\\ 1 \end{array}} \right]$$
where ${M_p}$ and ${M_q}$ are weak perspective camera projection matrices:
$${M_p} = \left[ {\begin{array}{cccc} {{\alpha_x}}&0&0&{{c_x}}\\ 0&{{\alpha_y}}&0&{{c_y}}\\ 0&0&0&s \end{array}} \right]\quad and\quad {M_q} = \left[ {\begin{array}{cccc} {{{\alpha^{\prime}}_x}}&0&0&{{{c^{\prime}}_x}}\\ 0&{{{\alpha^{\prime}}_y}}&0&{{{c^{\prime}}_y}}\\ 0&0&0&{s^{\prime}} \end{array}} \right]$$
In Eq. (7), ${\alpha _x},{\alpha _y},{\alpha ^{\prime}_x},$ and ${\alpha ^{\prime}_y}$ are pixel dimension parameters, ${c_x},{c_y},{c^{\prime}_x},$ and ${c^{\prime}_y}$ denote the centers of projection in ${C_p}$ and ${C_q}$, while $s$ and $s^{\prime}$ are scaling parameters of the weak perspective projection.

From Eq. (6):

$$p=\left[\begin{array}{l} x \\ y \end{array}\right]=\left[\begin{array}{l} \frac{\alpha_{x} X+c_{x}}{s} \\ \frac{\alpha_{y} Y+c_{y}}{s} \end{array}\right] \rightarrow\left[\begin{array}{l} X \\ Y \end{array}\right]=\left[\begin{array}{c} \frac{s x-c_{x}}{\alpha_{x}} \\ \frac{s . y-c_{y}}{\alpha_{y}} \end{array}\right] \text { and } q=\left[\begin{array}{l} x^{\prime} \\ y^{\prime} \end{array}\right]=\left[\begin{array}{c} \frac{\alpha_{x}^{\prime} X^{\prime}+c_{x}^{\prime}}{s^{\prime}} \\ \frac{\alpha_{y}^{\prime} Y^{\prime}+c_{y}^{\prime}}{s^{\prime}} \end{array}\right]$$
Applying Eq. (8) to Eq. (3) results in:
$$Z = {a_1}{e^{ - {{(\frac{{(x - {a_2})}}{{{a_3}}})}^2}}}{e^{ - {{(\frac{{(y - {a_4})}}{{{a_5}}})}^2}}}$$
where ${a_1} = {A_1}$, ${a_2} = \frac{{{c_x} + ({\alpha _x} \times {A_2})}}{s}$ and so on.

Now from Eq. (5), Eq. (8), and Eq. (9), the $X^{\prime}$ and $Y^{\prime}$, are calculated in terms of $x$ and $y$. Finally $x^{\prime}$ and $y^{\prime}$ can be obtained from Eq. (8) as follows:

$$\left\{ {\begin{array}{{c}} {x^{\prime} = {d_{11}} + {d_{12}}x + {d_{13}}y + {d_{14}}{e^{ - {{(\frac{{x - {d_{15}}}}{{{d_{16}}}})}^2}}}{e^{ - {{(\frac{{y - {d_{17}}}}{{{d_{18}}}})}^2}}}}\\ {y^{\prime} = {d_{21}} + {d_{22}}x + {d_{23}}y + {d_{24}}{e^{ - {{(\frac{{x - {d_{25}}}}{{{d_{26}}}})}^2}}}{e^{ - {{(\frac{{y - {d_{27}}}}{{{d_{28}}}})}^2}}}} \end{array}} \right.$$
The unknown parameters ${d_{11}},{d_{12}},\ldots ,{d_{18}},{d_{21}},\ldots ,{d_{28}}$ are calculated using robust least squares method. For doing so, the first step of global registration is applied again to the transformed images and the matched features are utilized as the points to find parameters ${d_{11}}$ through ${d_{28}}$. As shown in Fig. 10 (also in Fig. 13 and Table 1) the proposed Gaussian model significantly improves the result of first step compared to quadratic model and decreases the deformations effects of the consequent local registration step.

 figure: Fig. 10.

Fig. 10. Qualitative evaluation of three global registration schemes (Yellow rectangles highlight the differences). (a) global registration using first step(false color). (b) global registration exploiting quadratic model for retina surface(false color). (c) global registration using Gaussian model for retina surface(false color). In false color method, first and second images are shown in green and magenta, and the overlapped regions are depicted in white.

Download Full Size | PPT Slide | PDF

2.4 Local registration

Since global registration aligns images approximately (Fig. 11(a), Fig. 11(b), Fig. 11(d) and Fig. 11(e)), local registration is required to estimate displacement field and register images accurately (Fig. 11(c) and Fig. 11(f)). For local registration, the method proposed in [32] is applied. To achieve this, one image called diffusing model should be locally deformed to become similar to the other fixed one, named scene. In this approach, diffusing model (transformed FA image) is assumed as a grid model that must be deformed. This model contains vertices that are particles. On the other hand, contours of the scene objects (SLO image objects) are assumed to be membrane with effectors named demons. These demons are placed throughout the membrane and the presumptive vectors which are perpendicular to this contour, oriented from inside of image objects to the outside. In grid model, vertices polarity are labeled as either inside or outside and accordingly in the other image, demons locally apply forces to push the model inside the other image objects or vice versa [44]. For example, the demon corresponding to vertex with inside polarity, pushes that vertex of model toward inside of the image object. This method applies transformation iteratively. Magnitudes of forces applied from demons are constant within an iteration, but decreases between iterations for convergence. On the other words, local registration using diffusing model can be divided into two steps:

 figure: Fig. 11.

Fig. 11. Vessel maps and images after different steps of the proposed registration. (a) vessel maps after first step of global registration (false color). (b) vessel maps after applying Gaussian model to the first step of global registration (false color). (c) vessel maps after local registration step (false color) (arrows point out the differences in registration accuracy and yellow circles show the key points). (d-f) images correspond to the vessel maps (checker board). (g-i) zoomed images correspond to the ROI specified via red rectangle in second row (checker board). (j-l) zoomed images correspond to the ROI specified via magenta rectangle in second row (checker board). (m-o) zoomed images correspond to the ROI specified via yellow rectangle in second row (checker board). In checker board method, image is composed of alternating rectangular regions from two images.

Download Full Size | PPT Slide | PDF

2.4.1 Pre-computation of demons set

The demons should be extracted from the scene (SLO). They can be elicited from all image pixels, image contours or edges. Each demon includes information of its spatial position, direction from the inside to the outside, current displacement from the corresponding point of the model.

2.4.2 Iterative process

After applying an initial transformation to the model, the iterative process is started. Each iteration is composed of two steps:

  • a. Computing associated elementary force for each demon which depends on the demon direction and the polarity of grid model at the point that corresponds to the demon.
  • b. Computing next transformation using the previous one and all elementary demon forces computed in step a.
In the proposed method, to increase robustness and speed of local registration process, the number of iterations in each multi-resolution pyramid level equals 100 and three pyramid levels is considered. Figure 12 indicates data set images after local registration which are zoomed around Region of Interest (ROI).

 figure: Fig. 12.

Fig. 12. All data set images (zoomed ROI) after local registration step (checker board). Since registration of pair of images 36 is failed, this image is not shown.

Download Full Size | PPT Slide | PDF

3. Results

In this paper, three standard and prevalent parameters have been considered to assess the performance of proposed registration method: Root-Mean-Square (RMS) error, success rate and speed of registration.

3.1 RMS error

In mono-modal image registration, the pixel intensity values of two images can be utilized to determine distance between corresponding pixels. However in this case, for multimodal images, distances are calculated according to Eq. (11) where n and $||{{p_i}} ||$ stand for the number of landmarks and Euclidean distances of $i$-th corresponding landmarks.

$$RMS_{error} = \sqrt {\displaystyle{1 \over n}\sum\limits_{i = 1}^n {{\left\| {p_i} \right\|}^2} } $$

Since vessel branches and crossovers are easy to locate even in multimodal retinal images, these points have been determined as landmarks in each image. Here $n$ equals 15. The mean and standard deviation of RMS error of proposed method after global and local registration steps is indicated in Table 1, also the box-plot of RMS errors of global and local registration steps are demonstrated in Fig. 13. The process of measuring RMS errors is done by two observers, and the differences between mean and standard deviation of RMS errors measured by them is negligible. Nevertheless, the maximum values are reported in Table 1. Table 2 compares mean RMS error of proposed method with other methods. It can be seen from this table that due to combining SURF and FAST features, applying Gaussian model for retina surface, and final local registration, in the proposed method the RMS error have been dramatically reduced.

Tables Icon

Table 1. Mean and maximum RMS errors of global and local registration

 figure: Fig. 13.

Fig. 13. The box-plot of RMS errors of global and local registration steps.

Download Full Size | PPT Slide | PDF

Tables Icon

Table 2. Quantitative evaluation of mean and maximum RMS errors of different methods

3.2 Success rate

Success rate determines what percentage of images are aligned correctly. Registration success or failure, is directly observable. The success rate of different methods is compared in Table 3. Figure 14 indicates qualitative evaluation of the proposed method.

 figure: Fig. 14.

Fig. 14. Qualitative evaluation of proposed method. (a) matching features using SURF on image case 3. (b) matching features using proposed method. (c) registration failure utilizing SURF features.(in color) (d) registration using proposed method. (e) matching features using FAST-HOG on image case 8. (f) matching features using proposed method. (g) registration failure utilizing FAST-HOG. (in color) (h) registration using proposed method.

Download Full Size | PPT Slide | PDF

Tables Icon

Table 3. Success rate of different methods on the dataset

3.3 Speed

Running time of the proposed method is compared with other methods in Table 4. It is worth mentioning that, our experiments were done using a PC with Windows 7 64-bit operating system, 4 GB RAM, and Intel Core i5 CPU 2.5 GHz whereas other reported results were obtained utilizing a PC with Windows 7 64-bit operating system, 64 GB RAM, and Intel Xeon i5 CPU 3.7 GHz.

Tables Icon

Table 4. Running time of different methods

4. Discussions and conclusions

In this paper, a novel method is proposed to register OCT images with FA photographs in an accurate manner. This method uses SLO images as intermediate, and includes four main steps namely image acquisition, preprocessing, global registration, and local registration. Because of using SURF and FAST features together and refining the matches for three times, the proposed method has inherited the advantages of both methods resulting in enhanced success rate. On the other hand, applying Gaussian model for retina surface and local registration after global registration leads to dramatic reduction in RMS error. Precise registration of FA and OCT images via SLO photographs enables us to exactly detect location and morphological appearance of diabetic retinopathy symptoms such as MAs and leakages in OCT B-scans. In this research, after registration of FA and OCT images using proposed method, morphological appearance and location of MAs in the OCT b-scans are estimated exploiting the same properties in the corresponding FA images. Using the fact that MAs in the FA images are indicated by hyper-fluorescent dots, and utilizing the position of these dots in FA photographs, we analyzed location and morphological appearance of MAs in OCT b-scans (Fig. 15). For this purpose, 38 MAs which are located in 20 images are analyzed. In 8 out of 38 cases, MAs had complete or incomplete ring sign, whereas remaining MAs have no ring sign. Some leakages in OCT B-scans were observed as blob signs whereas other leakages did not show any visual sign. Figure 16 illustrates blob sign of a leakage in FA and corresponding B-scan of OCT image. Also, many researches have been conducted to analyze appearance of MAs in OCT B-scans [5,46].

 figure: Fig. 15.

Fig. 15. MAs in B-scans of diabetic retinopathy. (a) MA in FA image. (b) corresponding B-scan of the yellow line in (a). (c) zoomed complete ring sign in image (b). (d) MA in an FA image. (e) corresponding B-scan of the yellow line in (d). (f) zoomed incomplete ring sign in image (e). (g) MA in an FA image.(h) corresponding B-scan of the yellow line in (g). (i) MA with no sign.

Download Full Size | PPT Slide | PDF

 figure: Fig. 16.

Fig. 16. leakage in B-scan of diabetic retinopathy (a) FA image with leakage. Red arrow and circle show part of leakage. (b) corresponding B-scan of the yellow line in (a). yellow arrows indicate the blob shaped (cystoid) area corresponds to red circle in (a). (c) zoomed image of the blob shaped (cystoid) area in (b).

Download Full Size | PPT Slide | PDF

Funding

Office of Vice Chancellor for Research and Technology, University of Isfahan (1390734).

Acknowledgments

Authors would like thank Prof. Sina Farsiu from Duke Eye Center for his valuable ideas and comments in starting and development of this study.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

References

1. R. Kafieh, H. Rabbani, and S. Kermani, “A review of algorithms for segmentation of optical coherence tomography from retina,” J. Med. Signals Sens. 3(1), 45 (2013). [CrossRef]  

2. K. Khaderi, K. Ahmed, J. Berry, L. Labriola, and R. Cornwell, “Retinal imaging modalities: advantages and limitations for clinical practice,” Retinal Physician 8 (2011).

3. H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009). [CrossRef]  

4. H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012). [CrossRef]  

5. T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010). [CrossRef]  

6. S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013). [CrossRef]  

7. Y. Yamada, K. Suzuma, A. Fujikawa, T. Kumagami, and T. Kitaoka, “Imaging of laser-photocoagulated diabetic microaneurysm with spectral domain optical coherence tomography,” Retina 33(4), 726–731 (2013). [CrossRef]  

8. L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009). [CrossRef]  

9. M. A. Viergever, J. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, “A survey of medical image registration–under review,” (Elsevier, 2016).

10. L. Barghout and L. Lee, “Perceptual information processing system,” (2004).

11. M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012). [CrossRef]  

12. C. L. Srinidhi, P. Aparna, and J. Rajan, “Recent advancements in retinal vessel segmentation,” J. Med. Syst. 41(4), 70 (2017). [CrossRef]  

13. J. Zheng, J. Tian, Y. Dai, K. Deng, and J. Chen, “Retinal image registration based on salient feature regions," in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 2009), 102–105.

14. J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010). [CrossRef]  

15. M. S. Miri, M. D. Abràmoff, Y. H. Kwon, and M. K. Garvin, “Multimodal registration of SD-OCT volumes and fundus photographs using histograms of oriented gradients,” Biomed. Opt. Express 7(12), 5252–5267 (2016). [CrossRef]  

16. M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981). [CrossRef]  

17. R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017). [CrossRef]  

18. Z. Li, F. Huang, J. Zhang, B. Dashtbozorg, S. Abbasi-Sureshjani, Y. Sun, X. Long, Q. Yu, B. t, H. Romeny, and T. Tan, “Multi-modal and multi-vendor retina image registration,” Biomed. Opt. Express 9(2), 410–422 (2018). [CrossRef]  

19. H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018). [CrossRef]  

20. Z. Hossein-Nejad and M. Nasri, “A-RANSAC: Adaptive random sample consensus method in multimodal retinal image registration,” Biomed. Signal Process. Control 45, 325–338 (2018). [CrossRef]  

21. S. Niu, Q. Chen, H. Shen, L. de Sisternes, and D. L. Rubin, “Registration of SD-OCT en-face images with color fundus photographs based on local patch matching,” (2014).

22. H. Rabbani, M. J. Allingham, P. S. Mettu, S. W. Cousins, and S. Farsiu, “Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema,” Invest. Ophthalmol. Visual Sci. 56(3), 1482–1492 (2015). [CrossRef]  

23. P. Torr and A. Zisserman, “Robust computation and parametrization of multiple view relations,” in Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271), (IEEE, 1998), 727–732.

24. J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

25. M. Arikan, A. Sadeghipour, B. Gerendas, R. Told, and U. Schmidt-Erfurt, “Deep Learning Based Multi-modal Registration for Retinal Imaging,” in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support (Springer, 2019), pp. 75–82.

26. Z. G. Kamasi, M. Mokhtari, and H. Rabbani, “Non-rigid registration of Fluorescein Angiography and Optical Coherence Tomography via scanning laser ophthalmoscope imaging,” in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (IEEE, 2017), 4415–4418.

27. J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality preserving matching,” Intl. J. Comput. Vis. 127(5), 512–531 (2019). [CrossRef]  

28. J. Ma, X. Jiang, J. Jiang, J. Zhao, and X. Guo, “LMR: Learning a two-class classifier for mismatch removal,” IEEE Trans. on Image Process. 28(8), 4045–4059 (2019). [CrossRef]  

29. X. Jiang, J. Ma, J. Jiang, and X. Guo, “Robust feature matching using spatial clustering with heavy outliers,” IEEE Trans. on Image Process. 29, 736–746 (2020). [CrossRef]  

30. J. Ma, J. Jiang, C. Liu, and Y. Li, “Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration,” Inf. Sci. 417, 128–142 (2017). [CrossRef]  

31. M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” VISAPP 2(1), 2 (2009).

32. J.-P. Thirion, “Image matching as a diffusion process: an analogy with Maxwell's demons,” Med. Image Anal. 2(3), 243–260 (1998). [CrossRef]  

33. S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987). [CrossRef]  

34. T. Coye, “A novel retinal blood vessel segmentation algorithm for fundus images,” MATLAB Central File Exchange (Jan 2017) (2015).

35. N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern. 9(1), 62–66 (1979). [CrossRef]  

36. A. Can, C. V. Stewart, B. Roysam, and H. L. Tanenbaum, “A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina,” IEEE Trans. Pattern Anal. Machine Intell. 24(3), 347–364 (2002). [CrossRef]  

37. M. Golabbakhsh and H. Rabbani, “Vessel-based registration of fundus and optical coherence tomography projection images of retina using a quadratic registration model,” IET Image Processing 7(8), 768–776 (2013). [CrossRef]  

38. E. Rosten and T. Drummond, “Fusing points and lines for high performance tracking,” in Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, (Ieee, 2005), 1508–1515.

39. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Understanding 110(3), 346–359 (2008). [CrossRef]  

40. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Intl. J. Comput. Vis. 60(2), 91–110 (2004). [CrossRef]  

41. N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), (IEEE, 2005), 886–893.

42. C. Silpa-Anan and R. Hartley, “Optimised KD-trees for fast image descriptor matching,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2008), 1–8.

43. K. Lebeda, “Robust Sampling Consensus,” (2013).

44. T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, “Diffeomorphic demons: Efficient non-parametric image registration,” NeuroImage 45(1), S61–S72 (2009). [CrossRef]  

45. M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015). [CrossRef]  

46. M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009). [CrossRef]  

References

  • View by:

  1. R. Kafieh, H. Rabbani, and S. Kermani, “A review of algorithms for segmentation of optical coherence tomography from retina,” J. Med. Signals Sens. 3(1), 45 (2013).
    [Crossref]
  2. K. Khaderi, K. Ahmed, J. Berry, L. Labriola, and R. Cornwell, “Retinal imaging modalities: advantages and limitations for clinical practice,” Retinal Physician 8 (2011).
  3. H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009).
    [Crossref]
  4. H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
    [Crossref]
  5. T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010).
    [Crossref]
  6. S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
    [Crossref]
  7. Y. Yamada, K. Suzuma, A. Fujikawa, T. Kumagami, and T. Kitaoka, “Imaging of laser-photocoagulated diabetic microaneurysm with spectral domain optical coherence tomography,” Retina 33(4), 726–731 (2013).
    [Crossref]
  8. L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009).
    [Crossref]
  9. M. A. Viergever, J. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, “A survey of medical image registration–under review,” (Elsevier, 2016).
  10. L. Barghout and L. Lee, “Perceptual information processing system,” (2004).
  11. M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
    [Crossref]
  12. C. L. Srinidhi, P. Aparna, and J. Rajan, “Recent advancements in retinal vessel segmentation,” J. Med. Syst. 41(4), 70 (2017).
    [Crossref]
  13. J. Zheng, J. Tian, Y. Dai, K. Deng, and J. Chen, “Retinal image registration based on salient feature regions," in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 2009), 102–105.
  14. J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010).
    [Crossref]
  15. M. S. Miri, M. D. Abràmoff, Y. H. Kwon, and M. K. Garvin, “Multimodal registration of SD-OCT volumes and fundus photographs using histograms of oriented gradients,” Biomed. Opt. Express 7(12), 5252–5267 (2016).
    [Crossref]
  16. M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
    [Crossref]
  17. R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
    [Crossref]
  18. Z. Li, F. Huang, J. Zhang, B. Dashtbozorg, S. Abbasi-Sureshjani, Y. Sun, X. Long, Q. Yu, B. t, H. Romeny, and T. Tan, “Multi-modal and multi-vendor retina image registration,” Biomed. Opt. Express 9(2), 410–422 (2018).
    [Crossref]
  19. H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
    [Crossref]
  20. Z. Hossein-Nejad and M. Nasri, “A-RANSAC: Adaptive random sample consensus method in multimodal retinal image registration,” Biomed. Signal Process. Control 45, 325–338 (2018).
    [Crossref]
  21. S. Niu, Q. Chen, H. Shen, L. de Sisternes, and D. L. Rubin, “Registration of SD-OCT en-face images with color fundus photographs based on local patch matching,” (2014).
  22. H. Rabbani, M. J. Allingham, P. S. Mettu, S. W. Cousins, and S. Farsiu, “Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema,” Invest. Ophthalmol. Visual Sci. 56(3), 1482–1492 (2015).
    [Crossref]
  23. P. Torr and A. Zisserman, “Robust computation and parametrization of multiple view relations,” in Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271), (IEEE, 1998), 727–732.
  24. J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.
  25. M. Arikan, A. Sadeghipour, B. Gerendas, R. Told, and U. Schmidt-Erfurt, “Deep Learning Based Multi-modal Registration for Retinal Imaging,” in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support (Springer, 2019), pp. 75–82.
  26. Z. G. Kamasi, M. Mokhtari, and H. Rabbani, “Non-rigid registration of Fluorescein Angiography and Optical Coherence Tomography via scanning laser ophthalmoscope imaging,” in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (IEEE, 2017), 4415–4418.
  27. J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality preserving matching,” Intl. J. Comput. Vis. 127(5), 512–531 (2019).
    [Crossref]
  28. J. Ma, X. Jiang, J. Jiang, J. Zhao, and X. Guo, “LMR: Learning a two-class classifier for mismatch removal,” IEEE Trans. on Image Process. 28(8), 4045–4059 (2019).
    [Crossref]
  29. X. Jiang, J. Ma, J. Jiang, and X. Guo, “Robust feature matching using spatial clustering with heavy outliers,” IEEE Trans. on Image Process. 29, 736–746 (2020).
    [Crossref]
  30. J. Ma, J. Jiang, C. Liu, and Y. Li, “Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration,” Inf. Sci. 417, 128–142 (2017).
    [Crossref]
  31. M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” VISAPP 2(1), 2 (2009).
  32. J.-P. Thirion, “Image matching as a diffusion process: an analogy with Maxwell's demons,” Med. Image Anal. 2(3), 243–260 (1998).
    [Crossref]
  33. S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987).
    [Crossref]
  34. T. Coye, “A novel retinal blood vessel segmentation algorithm for fundus images,” MATLAB Central File Exchange (Jan 2017) (2015).
  35. N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern. 9(1), 62–66 (1979).
    [Crossref]
  36. A. Can, C. V. Stewart, B. Roysam, and H. L. Tanenbaum, “A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina,” IEEE Trans. Pattern Anal. Machine Intell. 24(3), 347–364 (2002).
    [Crossref]
  37. M. Golabbakhsh and H. Rabbani, “Vessel-based registration of fundus and optical coherence tomography projection images of retina using a quadratic registration model,” IET Image Processing 7(8), 768–776 (2013).
    [Crossref]
  38. E. Rosten and T. Drummond, “Fusing points and lines for high performance tracking,” in Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, (Ieee, 2005), 1508–1515.
  39. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Understanding 110(3), 346–359 (2008).
    [Crossref]
  40. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Intl. J. Comput. Vis. 60(2), 91–110 (2004).
    [Crossref]
  41. N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), (IEEE, 2005), 886–893.
  42. C. Silpa-Anan and R. Hartley, “Optimised KD-trees for fast image descriptor matching,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2008), 1–8.
  43. K. Lebeda, “Robust Sampling Consensus,” (2013).
  44. T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, “Diffeomorphic demons: Efficient non-parametric image registration,” NeuroImage 45(1), S61–S72 (2009).
    [Crossref]
  45. M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
    [Crossref]
  46. M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
    [Crossref]

2020 (1)

X. Jiang, J. Ma, J. Jiang, and X. Guo, “Robust feature matching using spatial clustering with heavy outliers,” IEEE Trans. on Image Process. 29, 736–746 (2020).
[Crossref]

2019 (2)

J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality preserving matching,” Intl. J. Comput. Vis. 127(5), 512–531 (2019).
[Crossref]

J. Ma, X. Jiang, J. Jiang, J. Zhao, and X. Guo, “LMR: Learning a two-class classifier for mismatch removal,” IEEE Trans. on Image Process. 28(8), 4045–4059 (2019).
[Crossref]

2018 (3)

Z. Li, F. Huang, J. Zhang, B. Dashtbozorg, S. Abbasi-Sureshjani, Y. Sun, X. Long, Q. Yu, B. t, H. Romeny, and T. Tan, “Multi-modal and multi-vendor retina image registration,” Biomed. Opt. Express 9(2), 410–422 (2018).
[Crossref]

H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
[Crossref]

Z. Hossein-Nejad and M. Nasri, “A-RANSAC: Adaptive random sample consensus method in multimodal retinal image registration,” Biomed. Signal Process. Control 45, 325–338 (2018).
[Crossref]

2017 (3)

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

C. L. Srinidhi, P. Aparna, and J. Rajan, “Recent advancements in retinal vessel segmentation,” J. Med. Syst. 41(4), 70 (2017).
[Crossref]

J. Ma, J. Jiang, C. Liu, and Y. Li, “Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration,” Inf. Sci. 417, 128–142 (2017).
[Crossref]

2016 (1)

2015 (2)

H. Rabbani, M. J. Allingham, P. S. Mettu, S. W. Cousins, and S. Farsiu, “Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema,” Invest. Ophthalmol. Visual Sci. 56(3), 1482–1492 (2015).
[Crossref]

M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
[Crossref]

2013 (4)

M. Golabbakhsh and H. Rabbani, “Vessel-based registration of fundus and optical coherence tomography projection images of retina using a quadratic registration model,” IET Image Processing 7(8), 768–776 (2013).
[Crossref]

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

Y. Yamada, K. Suzuma, A. Fujikawa, T. Kumagami, and T. Kitaoka, “Imaging of laser-photocoagulated diabetic microaneurysm with spectral domain optical coherence tomography,” Retina 33(4), 726–731 (2013).
[Crossref]

R. Kafieh, H. Rabbani, and S. Kermani, “A review of algorithms for segmentation of optical coherence tomography from retina,” J. Med. Signals Sens. 3(1), 45 (2013).
[Crossref]

2012 (2)

H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
[Crossref]

M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
[Crossref]

2010 (2)

T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010).
[Crossref]

J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010).
[Crossref]

2009 (5)

L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009).
[Crossref]

H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009).
[Crossref]

M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” VISAPP 2(1), 2 (2009).

M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
[Crossref]

T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, “Diffeomorphic demons: Efficient non-parametric image registration,” NeuroImage 45(1), S61–S72 (2009).
[Crossref]

2008 (1)

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Understanding 110(3), 346–359 (2008).
[Crossref]

2004 (1)

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Intl. J. Comput. Vis. 60(2), 91–110 (2004).
[Crossref]

2002 (1)

A. Can, C. V. Stewart, B. Roysam, and H. L. Tanenbaum, “A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina,” IEEE Trans. Pattern Anal. Machine Intell. 24(3), 347–364 (2002).
[Crossref]

1998 (1)

J.-P. Thirion, “Image matching as a diffusion process: an analogy with Maxwell's demons,” Med. Image Anal. 2(3), 243–260 (1998).
[Crossref]

1987 (1)

S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987).
[Crossref]

1981 (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[Crossref]

1979 (1)

N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern. 9(1), 62–66 (1979).
[Crossref]

Abbasi-Sureshjani, S.

Abdul Wahab, A. W.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Abràmoff, M. D.

M. S. Miri, M. D. Abràmoff, Y. H. Kwon, and M. K. Garvin, “Multimodal registration of SD-OCT volumes and fundus photographs using histograms of oriented gradients,” Biomed. Opt. Express 7(12), 5252–5267 (2016).
[Crossref]

M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
[Crossref]

Ahmed, K.

K. Khaderi, K. Ahmed, J. Berry, L. Labriola, and R. Cornwell, “Retinal imaging modalities: advantages and limitations for clinical practice,” Retinal Physician 8 (2011).

Ahmedy, F.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Ahmedy, I.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Allingham, M. J.

H. Rabbani, M. J. Allingham, P. S. Mettu, S. W. Cousins, and S. Farsiu, “Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema,” Invest. Ophthalmol. Visual Sci. 56(3), 1482–1492 (2015).
[Crossref]

Amador, M.

J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

Amburn, E. P.

S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987).
[Crossref]

An, C.

J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

Aparna, P.

C. L. Srinidhi, P. Aparna, and J. Rajan, “Recent advancements in retinal vessel segmentation,” J. Med. Syst. 41(4), 70 (2017).
[Crossref]

Arikan, M.

M. Arikan, A. Sadeghipour, B. Gerendas, R. Told, and U. Schmidt-Erfurt, “Deep Learning Based Multi-modal Registration for Retinal Imaging,” in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support (Springer, 2019), pp. 75–82.

Arof, H.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Austin, J. D.

S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987).
[Crossref]

Ayache, N.

T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, “Diffeomorphic demons: Efficient non-parametric image registration,” NeuroImage 45(1), S61–S72 (2009).
[Crossref]

Barghout, L.

L. Barghout and L. Lee, “Perceptual information processing system,” (2004).

Barman, S. A.

M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
[Crossref]

Barteselli, G.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

Bartsch, D.-U.

H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
[Crossref]

J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

Bay, H.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Understanding 110(3), 346–359 (2008).
[Crossref]

Berry, J.

K. Khaderi, K. Ahmed, J. Berry, L. Labriola, and R. Cornwell, “Retinal imaging modalities: advantages and limitations for clinical practice,” Retinal Physician 8 (2011).

Bezerra, H. G.

H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009).
[Crossref]

Bolles, R. C.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[Crossref]

Bolz, M.

M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
[Crossref]

Borooah, S.

J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

Can, A.

A. Can, C. V. Stewart, B. Roysam, and H. L. Tanenbaum, “A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina,” IEEE Trans. Pattern Anal. Machine Intell. 24(3), 347–364 (2002).
[Crossref]

Chan, C. K.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
[Crossref]

Chen, J.

J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010).
[Crossref]

J. Zheng, J. Tian, Y. Dai, K. Deng, and J. Chen, “Retinal image registration based on salient feature regions," in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 2009), 102–105.

Chen, Q.

S. Niu, Q. Chen, H. Shen, L. de Sisternes, and D. L. Rubin, “Registration of SD-OCT en-face images with color fundus photographs based on local patch matching,” (2014).

Cheng, L.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
[Crossref]

Chhablani, J.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
[Crossref]

Cornwell, R.

K. Khaderi, K. Ahmed, J. Berry, L. Labriola, and R. Cornwell, “Retinal imaging modalities: advantages and limitations for clinical practice,” Retinal Physician 8 (2011).

Costa, M. A.

H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009).
[Crossref]

Cousins, S. W.

H. Rabbani, M. J. Allingham, P. S. Mettu, S. W. Cousins, and S. Farsiu, “Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema,” Invest. Ophthalmol. Visual Sci. 56(3), 1482–1492 (2015).
[Crossref]

Coye, T.

T. Coye, “A novel retinal blood vessel segmentation algorithm for fundus images,” MATLAB Central File Exchange (Jan 2017) (2015).

Cromartie, R.

S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987).
[Crossref]

Dai, J.

J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

Dai, Y.

J. Zheng, J. Tian, Y. Dai, K. Deng, and J. Chen, “Retinal image registration based on salient feature regions," in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 2009), 102–105.

Dalal, N.

N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), (IEEE, 2005), 886–893.

Dashtbozorg, B.

de Sisternes, L.

S. Niu, Q. Chen, H. Shen, L. de Sisternes, and D. L. Rubin, “Registration of SD-OCT en-face images with color fundus photographs based on local patch matching,” (2014).

Deak, G.

M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
[Crossref]

Deng, K.

J. Zheng, J. Tian, Y. Dai, K. Deng, and J. Chen, “Retinal image registration based on salient feature regions," in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 2009), 102–105.

Drummond, T.

E. Rosten and T. Drummond, “Fusing points and lines for high performance tracking,” in Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, (Ieee, 2005), 1508–1515.

El-Emam, S.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

Ess, A.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Understanding 110(3), 346–359 (2008).
[Crossref]

Farsiu, S.

H. Rabbani, M. J. Allingham, P. S. Mettu, S. W. Cousins, and S. Farsiu, “Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema,” Invest. Ophthalmol. Visual Sci. 56(3), 1482–1492 (2015).
[Crossref]

Fischler, M. A.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[Crossref]

Fraz, M. M.

M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
[Crossref]

Freeman, W. R.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
[Crossref]

J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

Fujikawa, A.

Y. Yamada, K. Suzuma, A. Fujikawa, T. Kumagami, and T. Kitaoka, “Imaging of laser-photocoagulated diabetic microaneurysm with spectral domain optical coherence tomography,” Retina 33(4), 726–731 (2013).
[Crossref]

Garcia, P.

L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009).
[Crossref]

Garvin, M. K.

M. S. Miri, M. D. Abràmoff, Y. H. Kwon, and M. K. Garvin, “Multimodal registration of SD-OCT volumes and fundus photographs using histograms of oriented gradients,” Biomed. Opt. Express 7(12), 5252–5267 (2016).
[Crossref]

M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
[Crossref]

Gerendas, B.

M. Arikan, A. Sadeghipour, B. Gerendas, R. Told, and U. Schmidt-Erfurt, “Deep Learning Based Multi-modal Registration for Retinal Imaging,” in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support (Springer, 2019), pp. 75–82.

Geselowitz, A.

S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987).
[Crossref]

Golabbakhsh, M.

M. Golabbakhsh and H. Rabbani, “Vessel-based registration of fundus and optical coherence tomography projection images of retina using a quadratic registration model,” IET Image Processing 7(8), 768–776 (2013).
[Crossref]

Gomez, M. L.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

Greer, T.

S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987).
[Crossref]

Guagliumi, G.

H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009).
[Crossref]

Guo, X.

X. Jiang, J. Ma, J. Jiang, and X. Guo, “Robust feature matching using spatial clustering with heavy outliers,” IEEE Trans. on Image Process. 29, 736–746 (2020).
[Crossref]

J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality preserving matching,” Intl. J. Comput. Vis. 127(5), 512–531 (2019).
[Crossref]

J. Ma, X. Jiang, J. Jiang, J. Zhao, and X. Guo, “LMR: Learning a two-class classifier for mismatch removal,” IEEE Trans. on Image Process. 28(8), 4045–4059 (2019).
[Crossref]

Hartley, R.

C. Silpa-Anan and R. Hartley, “Optimised KD-trees for fast image descriptor matching,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2008), 1–8.

Hasikin, K.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Hoppe, A.

M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
[Crossref]

Horii, T.

T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010).
[Crossref]

Hossein-Nejad, Z.

Z. Hossein-Nejad and M. Nasri, “A-RANSAC: Adaptive random sample consensus method in multimodal retinal image registration,” Biomed. Signal Process. Control 45, 325–338 (2018).
[Crossref]

Huang, F.

Idris, M. Y. I.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Jiang, J.

X. Jiang, J. Ma, J. Jiang, and X. Guo, “Robust feature matching using spatial clustering with heavy outliers,” IEEE Trans. on Image Process. 29, 736–746 (2020).
[Crossref]

J. Ma, X. Jiang, J. Jiang, J. Zhao, and X. Guo, “LMR: Learning a two-class classifier for mismatch removal,” IEEE Trans. on Image Process. 28(8), 4045–4059 (2019).
[Crossref]

J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality preserving matching,” Intl. J. Comput. Vis. 127(5), 512–531 (2019).
[Crossref]

J. Ma, J. Jiang, C. Liu, and Y. Li, “Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration,” Inf. Sci. 417, 128–142 (2017).
[Crossref]

Jiang, X.

X. Jiang, J. Ma, J. Jiang, and X. Guo, “Robust feature matching using spatial clustering with heavy outliers,” IEEE Trans. on Image Process. 29, 736–746 (2020).
[Crossref]

J. Ma, X. Jiang, J. Jiang, J. Zhao, and X. Guo, “LMR: Learning a two-class classifier for mismatch removal,” IEEE Trans. on Image Process. 28(8), 4045–4059 (2019).
[Crossref]

Kadri, N. A.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Kafieh, R.

R. Kafieh, H. Rabbani, and S. Kermani, “A review of algorithms for segmentation of optical coherence tomography from retina,” J. Med. Signals Sens. 3(1), 45 (2013).
[Crossref]

Kamasi, Z. G.

Z. G. Kamasi, M. Mokhtari, and H. Rabbani, “Non-rigid registration of Fluorescein Angiography and Optical Coherence Tomography via scanning laser ophthalmoscope imaging,” in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (IEEE, 2017), 4415–4418.

Karim, A.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Kermani, S.

R. Kafieh, H. Rabbani, and S. Kermani, “A review of algorithms for segmentation of optical coherence tomography from retina,” J. Med. Signals Sens. 3(1), 45 (2013).
[Crossref]

Khaderi, K.

K. Khaderi, K. Ahmed, J. Berry, L. Labriola, and R. Cornwell, “Retinal imaging modalities: advantages and limitations for clinical practice,” Retinal Physician 8 (2011).

Khairiah, N.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Kitaoka, T.

Y. Yamada, K. Suzuma, A. Fujikawa, T. Kumagami, and T. Kitaoka, “Imaging of laser-photocoagulated diabetic microaneurysm with spectral domain optical coherence tomography,” Retina 33(4), 726–731 (2013).
[Crossref]

Klein, S.

M. A. Viergever, J. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, “A survey of medical image registration–under review,” (Elsevier, 2016).

Kozak, I.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
[Crossref]

Kriechbaum, K.

M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
[Crossref]

Kumagami, T.

Y. Yamada, K. Suzuma, A. Fujikawa, T. Kumagami, and T. Kitaoka, “Imaging of laser-photocoagulated diabetic microaneurysm with spectral domain optical coherence tomography,” Retina 33(4), 726–731 (2013).
[Crossref]

Kwon, Y. H.

M. S. Miri, M. D. Abràmoff, Y. H. Kwon, and M. K. Garvin, “Multimodal registration of SD-OCT volumes and fundus photographs using histograms of oriented gradients,” Biomed. Opt. Express 7(12), 5252–5267 (2016).
[Crossref]

M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
[Crossref]

Labriola, L.

K. Khaderi, K. Ahmed, J. Berry, L. Labriola, and R. Cornwell, “Retinal imaging modalities: advantages and limitations for clinical practice,” Retinal Physician 8 (2011).

Laine, A. F.

J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010).
[Crossref]

Landa, G.

L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009).
[Crossref]

Lebeda, K.

K. Lebeda, “Robust Sampling Consensus,” (2013).

Lee, K.

M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
[Crossref]

Lee, L.

L. Barghout and L. Lee, “Perceptual information processing system,” (2004).

Lee, N.

J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010).
[Crossref]

Lee, S. N.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

Li, Y.

J. Ma, J. Jiang, C. Liu, and Y. Li, “Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration,” Inf. Sci. 417, 128–142 (2017).
[Crossref]

Li, Z.

Lima, V. C.

L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009).
[Crossref]

Liu, C.

J. Ma, J. Jiang, C. Liu, and Y. Li, “Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration,” Inf. Sci. 417, 128–142 (2017).
[Crossref]

Long, X.

Lowe, D. G.

M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” VISAPP 2(1), 2 (2009).

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Intl. J. Comput. Vis. 60(2), 91–110 (2004).
[Crossref]

Luo, Y.

H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
[Crossref]

Ma, J.

X. Jiang, J. Ma, J. Jiang, and X. Guo, “Robust feature matching using spatial clustering with heavy outliers,” IEEE Trans. on Image Process. 29, 736–746 (2020).
[Crossref]

J. Ma, X. Jiang, J. Jiang, J. Zhao, and X. Guo, “LMR: Learning a two-class classifier for mismatch removal,” IEEE Trans. on Image Process. 28(8), 4045–4059 (2019).
[Crossref]

J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality preserving matching,” Intl. J. Comput. Vis. 127(5), 512–531 (2019).
[Crossref]

J. Ma, J. Jiang, C. Liu, and Y. Li, “Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration,” Inf. Sci. 417, 128–142 (2017).
[Crossref]

Maintz, J. A.

M. A. Viergever, J. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, “A survey of medical image registration–under review,” (Elsevier, 2016).

Mettu, P. S.

H. Rabbani, M. J. Allingham, P. S. Mettu, S. W. Cousins, and S. Farsiu, “Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema,” Invest. Ophthalmol. Visual Sci. 56(3), 1482–1492 (2015).
[Crossref]

Miri, M. S.

M. S. Miri, M. D. Abràmoff, Y. H. Kwon, and M. K. Garvin, “Multimodal registration of SD-OCT volumes and fundus photographs using histograms of oriented gradients,” Biomed. Opt. Express 7(12), 5252–5267 (2016).
[Crossref]

M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
[Crossref]

Mokhtari, M.

Z. G. Kamasi, M. Mokhtari, and H. Rabbani, “Non-rigid registration of Fluorescein Angiography and Optical Coherence Tomography via scanning laser ophthalmoscope imaging,” in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (IEEE, 2017), 4415–4418.

Muja, M.

M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” VISAPP 2(1), 2 (2009).

Murakami, T.

T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010).
[Crossref]

Murphy, K.

M. A. Viergever, J. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, “A survey of medical image registration–under review,” (Elsevier, 2016).

Mylonas, G.

M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
[Crossref]

Nasri, M.

Z. Hossein-Nejad and M. Nasri, “A-RANSAC: Adaptive random sample consensus method in multimodal retinal image registration,” Biomed. Signal Process. Control 45, 325–338 (2018).
[Crossref]

Nguyen, T. Q.

J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

Niemeijer, M.

M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
[Crossref]

Nishijima, K.

T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010).
[Crossref]

Niu, S.

S. Niu, Q. Chen, H. Shen, L. de Sisternes, and D. L. Rubin, “Registration of SD-OCT en-face images with color fundus photographs based on local patch matching,” (2014).

Ong, S. H.

H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
[Crossref]

Ota, M.

T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010).
[Crossref]

Otsu, N.

N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern. 9(1), 62–66 (1979).
[Crossref]

Owen, C. G.

M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
[Crossref]

Pan, A.

H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
[Crossref]

Pennec, X.

T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, “Diffeomorphic demons: Efficient non-parametric image registration,” NeuroImage 45(1), S61–S72 (2009).
[Crossref]

Perchant, A.

T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, “Diffeomorphic demons: Efficient non-parametric image registration,” NeuroImage 45(1), S61–S72 (2009).
[Crossref]

Pizer, S. M.

S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987).
[Crossref]

Pluim, J. P.

M. A. Viergever, J. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, “A survey of medical image registration–under review,” (Elsevier, 2016).

Rabbani, H.

H. Rabbani, M. J. Allingham, P. S. Mettu, S. W. Cousins, and S. Farsiu, “Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema,” Invest. Ophthalmol. Visual Sci. 56(3), 1482–1492 (2015).
[Crossref]

R. Kafieh, H. Rabbani, and S. Kermani, “A review of algorithms for segmentation of optical coherence tomography from retina,” J. Med. Signals Sens. 3(1), 45 (2013).
[Crossref]

M. Golabbakhsh and H. Rabbani, “Vessel-based registration of fundus and optical coherence tomography projection images of retina using a quadratic registration model,” IET Image Processing 7(8), 768–776 (2013).
[Crossref]

Z. G. Kamasi, M. Mokhtari, and H. Rabbani, “Non-rigid registration of Fluorescein Angiography and Optical Coherence Tomography via scanning laser ophthalmoscope imaging,” in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (IEEE, 2017), 4415–4418.

Rajan, J.

C. L. Srinidhi, P. Aparna, and J. Rajan, “Recent advancements in retinal vessel segmentation,” J. Med. Syst. 41(4), 70 (2017).
[Crossref]

Ramli, R.

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

Remagnino, P.

M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
[Crossref]

Rollins, A. M.

H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009).
[Crossref]

Romeny, H.

Rosen, R. B.

L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009).
[Crossref]

Rosten, E.

E. Rosten and T. Drummond, “Fusing points and lines for high performance tracking,” in Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, (Ieee, 2005), 1508–1515.

Roysam, B.

A. Can, C. V. Stewart, B. Roysam, and H. L. Tanenbaum, “A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina,” IEEE Trans. Pattern Anal. Machine Intell. 24(3), 347–364 (2002).
[Crossref]

Rubin, D. L.

S. Niu, Q. Chen, H. Shen, L. de Sisternes, and D. L. Rubin, “Registration of SD-OCT en-face images with color fundus photographs based on local patch matching,” (2014).

Rudnicka, A. R.

M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
[Crossref]

Sadeghipour, A.

M. Arikan, A. Sadeghipour, B. Gerendas, R. Told, and U. Schmidt-Erfurt, “Deep Learning Based Multi-modal Registration for Retinal Imaging,” in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support (Springer, 2019), pp. 75–82.

Sakamoto, A.

T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010).
[Crossref]

Schmidt-Erfurt, U.

M. Arikan, A. Sadeghipour, B. Gerendas, R. Told, and U. Schmidt-Erfurt, “Deep Learning Based Multi-modal Registration for Retinal Imaging,” in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support (Springer, 2019), pp. 75–82.

Schmidt-Erfurth, U.

M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
[Crossref]

Scholda, C.

M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
[Crossref]

Shen, H.

S. Niu, Q. Chen, H. Shen, L. de Sisternes, and D. L. Rubin, “Registration of SD-OCT en-face images with color fundus photographs based on local patch matching,” (2014).

Silpa-Anan, C.

C. Silpa-Anan and R. Hartley, “Optimised KD-trees for fast image descriptor matching,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2008), 1–8.

Simon, D. I.

H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009).
[Crossref]

Smith, R. T.

J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010).
[Crossref]

Srinidhi, C. L.

C. L. Srinidhi, P. Aparna, and J. Rajan, “Recent advancements in retinal vessel segmentation,” J. Med. Syst. 41(4), 70 (2017).
[Crossref]

Staring, M.

M. A. Viergever, J. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, “A survey of medical image registration–under review,” (Elsevier, 2016).

Stewart, C. V.

A. Can, C. V. Stewart, B. Roysam, and H. L. Tanenbaum, “A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina,” IEEE Trans. Pattern Anal. Machine Intell. 24(3), 347–364 (2002).
[Crossref]

Sun, Y.

Suzuma, K.

Y. Yamada, K. Suzuma, A. Fujikawa, T. Kumagami, and T. Kitaoka, “Imaging of laser-photocoagulated diabetic microaneurysm with spectral domain optical coherence tomography,” Retina 33(4), 726–731 (2013).
[Crossref]

t, B.

Tan, T.

Tanenbaum, H. L.

A. Can, C. V. Stewart, B. Roysam, and H. L. Tanenbaum, “A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina,” IEEE Trans. Pattern Anal. Machine Intell. 24(3), 347–364 (2002).
[Crossref]

Tang, H.

H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
[Crossref]

Thirion, J.-P.

J.-P. Thirion, “Image matching as a diffusion process: an analogy with Maxwell's demons,” Med. Image Anal. 2(3), 243–260 (1998).
[Crossref]

Tian, J.

J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010).
[Crossref]

J. Zheng, J. Tian, Y. Dai, K. Deng, and J. Chen, “Retinal image registration based on salient feature regions," in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 2009), 102–105.

Told, R.

M. Arikan, A. Sadeghipour, B. Gerendas, R. Told, and U. Schmidt-Erfurt, “Deep Learning Based Multi-modal Registration for Retinal Imaging,” in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support (Springer, 2019), pp. 75–82.

Torr, P.

P. Torr and A. Zisserman, “Robust computation and parametrization of multiple view relations,” in Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271), (IEEE, 1998), 727–732.

Triggs, B.

N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), (IEEE, 2005), 886–893.

Tuytelaars, T.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Understanding 110(3), 346–359 (2008).
[Crossref]

Uyyanonvara, B.

M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
[Crossref]

Van Gool, L.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Understanding 110(3), 346–359 (2008).
[Crossref]

Vercauteren, T.

T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, “Diffeomorphic demons: Efficient non-parametric image registration,” NeuroImage 45(1), S61–S72 (2009).
[Crossref]

Vienna, D. R. R. G.

M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
[Crossref]

Viergever, M. A.

M. A. Viergever, J. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, “A survey of medical image registration–under review,” (Elsevier, 2016).

Wang, H.

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
[Crossref]

Wang, J.-K.

M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
[Crossref]

Yamada, Y.

Y. Yamada, K. Suzuma, A. Fujikawa, T. Kumagami, and T. Kitaoka, “Imaging of laser-photocoagulated diabetic microaneurysm with spectral domain optical coherence tomography,” Retina 33(4), 726–731 (2013).
[Crossref]

Yang, K.

H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
[Crossref]

Yang, Y.

H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
[Crossref]

Yeung, L.

L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009).
[Crossref]

Yoshimura, N.

T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010).
[Crossref]

Yu, Q.

Zhang, J.

Z. Li, F. Huang, J. Zhang, B. Dashtbozorg, S. Abbasi-Sureshjani, Y. Sun, X. Long, Q. Yu, B. t, H. Romeny, and T. Tan, “Multi-modal and multi-vendor retina image registration,” Biomed. Opt. Express 9(2), 410–422 (2018).
[Crossref]

J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

Zhang, S.

H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
[Crossref]

Zhao, J.

J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality preserving matching,” Intl. J. Comput. Vis. 127(5), 512–531 (2019).
[Crossref]

J. Ma, X. Jiang, J. Jiang, J. Zhao, and X. Guo, “LMR: Learning a two-class classifier for mismatch removal,” IEEE Trans. on Image Process. 28(8), 4045–4059 (2019).
[Crossref]

Zheng, J.

J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010).
[Crossref]

J. Zheng, J. Tian, Y. Dai, K. Deng, and J. Chen, “Retinal image registration based on salient feature regions," in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 2009), 102–105.

Zhou, H.

J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality preserving matching,” Intl. J. Comput. Vis. 127(5), 512–531 (2019).
[Crossref]

Zisserman, A.

P. Torr and A. Zisserman, “Robust computation and parametrization of multiple view relations,” in Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271), (IEEE, 1998), 727–732.

Am. J. Ophthalmol. (3)

H. Wang, J. Chhablani, W. R. Freeman, C. K. Chan, I. Kozak, D.-U. Bartsch, and L. Cheng, “Characterization of diabetic microaneurysms by simultaneous fluorescein angiography and spectral-domain optical coherence tomography,” Am. J. Ophthalmol. 153(5), 861–867.e1 (2012).
[Crossref]

T. Horii, T. Murakami, K. Nishijima, A. Sakamoto, M. Ota, and N. Yoshimura, “Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy,” Am. J. Ophthalmol. 150(6), 840–848.e1 (2010).
[Crossref]

S. N. Lee, J. Chhablani, C. K. Chan, H. Wang, G. Barteselli, S. El-Emam, M. L. Gomez, I. Kozak, L. Cheng, and W. R. Freeman, “Characterization of microaneurysm closure after focal laser photocoagulation in diabetic macular edema,” Am. J. Ophthalmol. 155(5), 905–912.e2 (2013).
[Crossref]

Biomed. Opt. Express (2)

Biomed. Signal Process. Control (1)

Z. Hossein-Nejad and M. Nasri, “A-RANSAC: Adaptive random sample consensus method in multimodal retinal image registration,” Biomed. Signal Process. Control 45, 325–338 (2018).
[Crossref]

Commun. ACM (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[Crossref]

Comput. Meth. Prog. Bio. (1)

M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman, “Blood vessel segmentation methodologies in retinal images–a survey,” Comput. Meth. Prog. Bio. 108(1), 407–433 (2012).
[Crossref]

Comput. Vis. Graph. Image Process. (1)

S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, and T. Greer, “B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987).
[Crossref]

Comput. Vis. Image Understanding (1)

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Understanding 110(3), 346–359 (2008).
[Crossref]

IEEE Trans. Biomed. Eng. (1)

J. Chen, J. Tian, N. Lee, J. Zheng, R. T. Smith, and A. F. Laine, “A partial intensity invariant feature descriptor for multimodal retinal image registration,” IEEE Trans. Biomed. Eng. 57(7), 1707–1718 (2010).
[Crossref]

IEEE Trans. Med. Imaging (1)

M. S. Miri, M. D. Abràmoff, K. Lee, M. Niemeijer, J.-K. Wang, Y. H. Kwon, and M. K. Garvin, “Multimodal segmentation of optic disc and cup from SD-OCT and color fundus photographs using a machine-learning graph-based approach,” IEEE Trans. Med. Imaging 34(9), 1854–1866 (2015).
[Crossref]

IEEE Trans. on Image Process. (2)

J. Ma, X. Jiang, J. Jiang, J. Zhao, and X. Guo, “LMR: Learning a two-class classifier for mismatch removal,” IEEE Trans. on Image Process. 28(8), 4045–4059 (2019).
[Crossref]

X. Jiang, J. Ma, J. Jiang, and X. Guo, “Robust feature matching using spatial clustering with heavy outliers,” IEEE Trans. on Image Process. 29, 736–746 (2020).
[Crossref]

IEEE Trans. Pattern Anal. Machine Intell. (1)

A. Can, C. V. Stewart, B. Roysam, and H. L. Tanenbaum, “A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina,” IEEE Trans. Pattern Anal. Machine Intell. 24(3), 347–364 (2002).
[Crossref]

IEEE Trans. Syst., Man, Cybern. (1)

N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern. 9(1), 62–66 (1979).
[Crossref]

IET Image Processing (1)

M. Golabbakhsh and H. Rabbani, “Vessel-based registration of fundus and optical coherence tomography projection images of retina using a quadratic registration model,” IET Image Processing 7(8), 768–776 (2013).
[Crossref]

Inf. Sci. (1)

J. Ma, J. Jiang, C. Liu, and Y. Li, “Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration,” Inf. Sci. 417, 128–142 (2017).
[Crossref]

Intl. J. Comput. Vis. (2)

J. Ma, J. Zhao, J. Jiang, H. Zhou, and X. Guo, “Locality preserving matching,” Intl. J. Comput. Vis. 127(5), 512–531 (2019).
[Crossref]

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Intl. J. Comput. Vis. 60(2), 91–110 (2004).
[Crossref]

Invest. Ophthalmol. Visual Sci. (1)

H. Rabbani, M. J. Allingham, P. S. Mettu, S. W. Cousins, and S. Farsiu, “Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema,” Invest. Ophthalmol. Visual Sci. 56(3), 1482–1492 (2015).
[Crossref]

J. Healthc. Eng. (1)

R. Ramli, M. Y. I. Idris, K. Hasikin, A. Karim, N. Khairiah, A. W. Abdul Wahab, I. Ahmedy, F. Ahmedy, N. A. Kadri, and H. Arof, “Feature-based retinal image registration using D-Saddle feature,” J. Healthc. Eng. 2017, 1–15 (2017).
[Crossref]

J. Med. Imaging Hlth. Inform. (1)

H. Tang, A. Pan, Y. Yang, K. Yang, Y. Luo, S. Zhang, and S. H. Ong, “Retinal image registration based on robust non-rigid point matching method,” J. Med. Imaging Hlth. Inform. 8(2), 240–249 (2018).
[Crossref]

J. Med. Signals Sens. (1)

R. Kafieh, H. Rabbani, and S. Kermani, “A review of algorithms for segmentation of optical coherence tomography from retina,” J. Med. Signals Sens. 3(1), 45 (2013).
[Crossref]

J. Med. Syst. (1)

C. L. Srinidhi, P. Aparna, and J. Rajan, “Recent advancements in retinal vessel segmentation,” J. Med. Syst. 41(4), 70 (2017).
[Crossref]

JACC: Cardiovasc. Interv. (1)

H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009).
[Crossref]

Med. Image Anal. (1)

J.-P. Thirion, “Image matching as a diffusion process: an analogy with Maxwell's demons,” Med. Image Anal. 2(3), 243–260 (1998).
[Crossref]

NeuroImage (1)

T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, “Diffeomorphic demons: Efficient non-parametric image registration,” NeuroImage 45(1), S61–S72 (2009).
[Crossref]

Ophthalmology (2)

M. Bolz, U. Schmidt-Erfurth, G. Deak, G. Mylonas, K. Kriechbaum, C. Scholda, and D. R. R. G. Vienna, “Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema,” Ophthalmology 116(5), 914–920 (2009).
[Crossref]

L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009).
[Crossref]

Retina (1)

Y. Yamada, K. Suzuma, A. Fujikawa, T. Kumagami, and T. Kitaoka, “Imaging of laser-photocoagulated diabetic microaneurysm with spectral domain optical coherence tomography,” Retina 33(4), 726–731 (2013).
[Crossref]

VISAPP (1)

M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” VISAPP 2(1), 2 (2009).

Other (14)

P. Torr and A. Zisserman, “Robust computation and parametrization of multiple view relations,” in Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271), (IEEE, 1998), 727–732.

J. Zhang, C. An, J. Dai, M. Amador, D.-U. Bartsch, S. Borooah, W. R. Freeman, and T. Q. Nguyen, “Joint Vessel Segmentation and Deformable Registration on Multi-Modal Retinal Images Based on Style Transfer,” in 2019 IEEE International Conference on Image Processing (ICIP), (IEEE, 2019), 839–843.

M. Arikan, A. Sadeghipour, B. Gerendas, R. Told, and U. Schmidt-Erfurt, “Deep Learning Based Multi-modal Registration for Retinal Imaging,” in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support (Springer, 2019), pp. 75–82.

Z. G. Kamasi, M. Mokhtari, and H. Rabbani, “Non-rigid registration of Fluorescein Angiography and Optical Coherence Tomography via scanning laser ophthalmoscope imaging,” in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (IEEE, 2017), 4415–4418.

S. Niu, Q. Chen, H. Shen, L. de Sisternes, and D. L. Rubin, “Registration of SD-OCT en-face images with color fundus photographs based on local patch matching,” (2014).

E. Rosten and T. Drummond, “Fusing points and lines for high performance tracking,” in Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, (Ieee, 2005), 1508–1515.

T. Coye, “A novel retinal blood vessel segmentation algorithm for fundus images,” MATLAB Central File Exchange (Jan 2017) (2015).

M. A. Viergever, J. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, “A survey of medical image registration–under review,” (Elsevier, 2016).

L. Barghout and L. Lee, “Perceptual information processing system,” (2004).

K. Khaderi, K. Ahmed, J. Berry, L. Labriola, and R. Cornwell, “Retinal imaging modalities: advantages and limitations for clinical practice,” Retinal Physician 8 (2011).

J. Zheng, J. Tian, Y. Dai, K. Deng, and J. Chen, “Retinal image registration based on salient feature regions," in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 2009), 102–105.

N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), (IEEE, 2005), 886–893.

C. Silpa-Anan and R. Hartley, “Optimised KD-trees for fast image descriptor matching,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2008), 1–8.

K. Lebeda, “Robust Sampling Consensus,” (2013).

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (16)

Fig. 1.
Fig. 1. Block diagram of overall process of registration.
Fig. 2.
Fig. 2. (a) SLO image. (b) the OCT B-scan corresponds to the yellow line in the SLO image.
Fig. 3.
Fig. 3. Our data set includes 36 pairs of FA and SLO images. Odd columns illustrate FA images of different field of view (30 and 55 degrees), even columns depict the SLO images corresponding to the left side column.
Fig. 4.
Fig. 4. Vessel Extraction. (a) FA image. (b) vessels extracted from FA. (c) SLO image. (d) vessels extracted from SLO.
Fig. 5.
Fig. 5. (a) Fast corner detector and Bresenham circle. (b) FAST corners in FA vessel map (green dots). (c) FAST corners in SLO vessel map (green dots).
Fig. 6.
Fig. 6. SURF features (a) 150 strongest SURF features on SLO vessel map (green circles). (b) 150 strongest SURF features on FA vessel map (green circles).
Fig. 7.
Fig. 7. HOG visualization. (a) SLO vessel map. (b) HOG visualization of (a). (c) FA vessel map. (d) HOG visualization of (c).
Fig. 8.
Fig. 8. Final MSAC step refines the previously-matched features. (a) matched features after joining separately refined SURF and HOG matches (still contains few outliers). (b) obtaining perfect matches after final MSAC step.
Fig. 9.
Fig. 9. Two different camera viewpoints of the retina.
Fig. 10.
Fig. 10. Qualitative evaluation of three global registration schemes (Yellow rectangles highlight the differences). (a) global registration using first step(false color). (b) global registration exploiting quadratic model for retina surface(false color). (c) global registration using Gaussian model for retina surface(false color). In false color method, first and second images are shown in green and magenta, and the overlapped regions are depicted in white.
Fig. 11.
Fig. 11. Vessel maps and images after different steps of the proposed registration. (a) vessel maps after first step of global registration (false color). (b) vessel maps after applying Gaussian model to the first step of global registration (false color). (c) vessel maps after local registration step (false color) (arrows point out the differences in registration accuracy and yellow circles show the key points). (d-f) images correspond to the vessel maps (checker board). (g-i) zoomed images correspond to the ROI specified via red rectangle in second row (checker board). (j-l) zoomed images correspond to the ROI specified via magenta rectangle in second row (checker board). (m-o) zoomed images correspond to the ROI specified via yellow rectangle in second row (checker board). In checker board method, image is composed of alternating rectangular regions from two images.
Fig. 12.
Fig. 12. All data set images (zoomed ROI) after local registration step (checker board). Since registration of pair of images 36 is failed, this image is not shown.
Fig. 13.
Fig. 13. The box-plot of RMS errors of global and local registration steps.
Fig. 14.
Fig. 14. Qualitative evaluation of proposed method. (a) matching features using SURF on image case 3. (b) matching features using proposed method. (c) registration failure utilizing SURF features.(in color) (d) registration using proposed method. (e) matching features using FAST-HOG on image case 8. (f) matching features using proposed method. (g) registration failure utilizing FAST-HOG. (in color) (h) registration using proposed method.
Fig. 15.
Fig. 15. MAs in B-scans of diabetic retinopathy. (a) MA in FA image. (b) corresponding B-scan of the yellow line in (a). (c) zoomed complete ring sign in image (b). (d) MA in an FA image. (e) corresponding B-scan of the yellow line in (d). (f) zoomed incomplete ring sign in image (e). (g) MA in an FA image.(h) corresponding B-scan of the yellow line in (g). (i) MA with no sign.
Fig. 16.
Fig. 16. leakage in B-scan of diabetic retinopathy (a) FA image with leakage. Red arrow and circle show part of leakage. (b) corresponding B-scan of the yellow line in (a). yellow arrows indicate the blob shaped (cystoid) area corresponds to red circle in (a). (c) zoomed image of the blob shaped (cystoid) area in (b).

Tables (4)

Tables Icon

Table 1. Mean and maximum RMS errors of global and local registration

Tables Icon

Table 2. Quantitative evaluation of mean and maximum RMS errors of different methods

Tables Icon

Table 3. Success rate of different methods on the dataset

Tables Icon

Table 4. Running time of different methods

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

1 : x S , I x > I p + T 2 : x S , I x < I p T
H ( x , σ ) = [ L x x ( x , σ ) L x y ( x , σ ) L x y ( x , σ ) L y y ( x , σ ) ]
det ( H a p p r o x ) = D x x . D y y ( 0.9 D x y ) 2
Z = A 1 e ( ( X A 2 ) A 3 ) 2 e ( ( Y A 4 ) A 5 ) 2
Q = R P + t
[ X Y Z ] = [ r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] [ X Y Z ] + [ t x t y t z ]
[ w x w y w ] M p [ X Y Z 1 ] a n d [ w x w y w ] M q [ X Y Z 1 ]
M p = [ α x 0 0 c x 0 α y 0 c y 0 0 0 s ] a n d M q = [ α x 0 0 c x 0 α y 0 c y 0 0 0 s ]
p = [ x y ] = [ α x X + c x s α y Y + c y s ] [ X Y ] = [ s x c x α x s . y c y α y ]  and  q = [ x y ] = [ α x X + c x s α y Y + c y s ]
Z = a 1 e ( ( x a 2 ) a 3 ) 2 e ( ( y a 4 ) a 5 ) 2
{ x = d 11 + d 12 x + d 13 y + d 14 e ( x d 15 d 16 ) 2 e ( y d 17 d 18 ) 2 y = d 21 + d 22 x + d 23 y + d 24 e ( x d 25 d 26 ) 2 e ( y d 27 d 28 ) 2
R M S e r r o r = 1 n i = 1 n p i 2

Metrics