A large number of human retinal diseases are characterized by a progressive loss of cones, the photoreceptors critical for visual acuity and color perception. Adaptive Optics (AO) imaging presents a potential method to study these cells in vivo. However, AO imaging in ophthalmology is a relatively new phenomenon and quantitative analysis of these images remains difficult and tedious using manual methods. This paper illustrates a novel semi-automated quantitative technique enabling registration of AO images to macular landmarks, cone counting and its radius quantification at specified distances from the foveal center. The new cone counting approach employs the circle Hough transform (cHT) and is compared to automated counting methods, as well as arbitrated manual cone identification. We explore the impact of varying the circle detection parameter on the validity of cHT cone counting and discuss the potential role of using this algorithm in detecting both cones and rods separately.
© 2015 Optical Society of America
Retinal diseases are a significant cause of progressive deterioration of visual function and collectively, they blind millions of people every year . Accurate diagnosis and monitoring of retinal diseases is now heavily reliant on high-resolution retinal imaging, in addition to standard visual acuity testing, examination and other functional assessments including microperimetry and electroretinography.
Significant advances have been made in the capability of retinal imaging over the past several decades through the introduction of digital retinal cameras and fluorescein angiography for visualizing retinal vasculature. The development of scanning laser ophthalmoscopes and optical coherence tomography systems that can capture en face, three-dimensional images have led to enhanced contrast in image details and the ability to analyze cross-sections of the retina, respectively [2–5]. Adaptive optics (AO) is the most recently adopted optical technology used to improve the performance of retinal photography by compensating for the effects of optical aberrations within the eye. Borrowed from astrophysics, AO technology enables the visualization of individual retinal photoreceptor cells, retinal blood vessel capillaries and bundles of ganglion cell axons within the living human retina [6–8].
Although recognized as a powerful research tool, AO retinal imaging is not yet in widespread clinical use. Central to the realization of the clinical potential of AO imaging is the development of robust, automated techniques to process and analyze the image outputs. Assessment of the cone density, spacing and packing arrangements at certain spatial locations within the central region of the retina (i.e. the macula) may be useful in determining whether the photoreceptor mosaic of a particular individual has changed over time, or whether it differs from normal. Our ability to draw these conclusions ultimately depends on the reliability and repeatability of the cone metrics that are used as clinical trials end points.
Recently, several research groups have developed software that semi-automates montaging of AO cone images [9–13]. Automatic photoreceptor detection algorithms have also been proposed [9–19]. In 2007, Li et al. introduced a procedure of automated cone counting based on the detection of local maxima in the image. This is the most widely-used algorithm . In the first step, the image is filtered using a Gaussian low-pass filter and then the local maxima are found using the inbuilt “maxima” function in Matlab. If multiple maxima are closer than the minimum cone separation their centroid is taken as the final location. In the same year, Xue et al. implemented the cone detection formula based on an image histogram analysis . Here, the background is first subtracted from the original image, enhancing linear brightness. Then, the image is divided into intensity ranges. The algorithm searches the connected regions of pixels for intensity values within a specific range. The centroids of the connected regions are defined as the cone coordinates. This process is repeated for each intensity range, from highest to lowest. If two or more coordinates occur closer than the minimum cone separation, their centroid is taken as the final location. Turpin et al. had proposed the use of multi-scale modelling and normalized cross-correlation to identify retinal cones in AO images . Briefly, using a Gaussian-based model they initially modelled the size and shape of retinal cones. Normalized cross-correlation is then performed generating an image where all the regions that are similar to the shape of the Gaussian are highlighted. Then by applying local maxima detection, the cones are counted. An alternative technique for segmenting and detecting cones was introduced by Chiu et al. . The authors used a graph theory and dynamic programing (GTDP) to segment the AO images to detect cones. This method relies on a transform that maps closed-contour features in the Cartesian domain onto lines in the quasi-polar domain. Features of interest are then segmented as layers using GTDP. More recently, Cooper et al. proposed a fully automated algorithm for estimating photoreceptor density based on the radius of Yellott’s ring . The authors inspected the image power spectrum and extracted features that corresponded to cell packing. Although this technique is accurate in measuring density of cones it is not possible to derive information on packing geometry of cones.
The available methods vary in the repeatability of the results and the degree of automation. In some of these, manual correction of the counting process is still required and a large amount of time is invested in creating a montage to enable cone counting in regions of the retina that are remote from the fovea. Since these functionalities are not yet available in commercial devices, there remains a need for further improvement of cone detection procedures.
In this paper, we describe a method of processing AO image frames that enables: 1, co-registration the small field AO frame with a wide field macular image so that we know the precise coordinate of any region of interest in the AO frame relative to the foveal center; 2, creation of a montage of the cone images and the corresponding density map. This provides an overview of cone distribution in the macular region and allows correlation between structure and function by overlaying these montages on macular images derived from other imaging modalities; and 3, development of a database of AO images for future comparison.
In addition to the above functions, we demonstrate that our customized software is able to identify cones reliably using the circle Hough transform (cHT) algorithm [21–25] implemented in Matlab; the cHT searches for circular formations of a given radius in the image. We also evaluate the effectiveness of the algorithm in analyzing human AO retinal images based on known parameters related to the cone size, density and spacing across the macular region. The cone counting software was validated for different-sized cones, from small foveal cones with compact arrangement to large perifoveal cones that were sparsely distributed. A set of images from an adaptive optics scanning laser ophthalmoscope (AO-SLO) was obtained from an outside source for analysis of cones at approximately 0.65° eccentricity from the center of the fovea (within the foveal region). At this location, the radius of the cones is approximately 1 µm and easily resolvable by the SLO system . Images from the adaptive optics flood illumination ophthalmoscope (AO-FIO) at our center were used to optimize the parameters of the cHT at 3° to 9° eccentricities (perifoveal region), where the radius of cones increases from 2 to 3.5 µm . Cones in this range of eccentricities are easily imaged by the FIO system because of the large field of view (4° x 4°) compared to the SLO system (1° x 1°). We report the impact of image quality enhancement on inter-observer agreement and how the circle detection parameters in cHT are chosen. Finally, we report the performance of our optimized cHT approach in AO-SLO and AO-FIO images and compare this with the performance of other automated methods against manual cone identification.
2. Materials and methods
2.1. Adaptive optics instrumentation
In this study we used images from two sources to develop and test our customized cone counting software and to validate the cHT algorithm. Images taken from our AO-FIO (rtx1, Imagine Eyes, Orsay, France) were used to identify cones in the perifoveal region between 3° to 9° from the center of the fovea. We also applied the algorithm to the AO-SLO cone images from the Chiu and Garrioch study [14,15], since the AO-FIO is unable to resolve cones from the center of the fovea to about 2.5°. The images are publicly available at the website: http://www.duke.edu/~sf59/Chiu_BOE_2013_dataset.htm .
The rtx1 AO-FIO instrument is comprised of two parts. The first is a non-contact, en face reflectance retinal imaging device, employing a non-coherent flood illumination light source, with a central wavelength of 850 nm and a low-noise CCD camera. The second part is the AO control loop that measures and corrects the ocular aberrations. The apparatus directs a small beam of light generated by a superluminescent diode with a central wavelength of 750 nm into the eye, which then backscatters off the retina. The scattered light leaving the eye succumbs to spherical aberration of the eye’s optics before being recorded by the imaging components of the Shack-Hartmann wavefront sensor. In addition, a corrective element (the deformable mirror) and a control system are used to correct the eye's wave aberrations and control the interaction between the wavefront sensor and the corrector element respectively; it interprets the wavefront sensor data and computes the appropriate wavefront corrector drive signals in real time.
During a single measurement, 40 images are acquired over 4 seconds. This number of frames results in increased signal-to-noise ratio and therefore improved visibility of cones. This is achieved through co-registration of frames using a cross-correlation method (registration of X/Y and rotation) and finally computing the average of the images. The raw images that show artefacts due to eye blinking and saccades are automatically eliminated before averaging. The final image corresponds to a 4° × 4° (750 pixels × 750 pixels oversampled to 1,500 pixels × 1,500 pixels) region in the retina. In linear dimensions, this is approximately 1.2 mm × 1.2 mm. The resolution of the system is 250 line pairs per millimeter. This limits the ability to distinguish cones in close proximity to the fovea. Essentially, images obtained at retinal eccentricities within 2.5° from the foveal center are not reliable for performing cone metrics in this system .
The methods for image acquisition and pre-processing of AO-SLO images are described in detail in . Briefly, the authors used their system to image the central foveal cone mosaic of the right eye. The wavelength of the superluminescent diode used for retinal imaging was 775 nm. Separate image sequences of 150 frames each were collected at four retinal locations (bottom left, bottom right, top left, and top right), each at approximately 0.65° from the center of fixation. The intra-frame distortions are corrected within the frames and the forty frames with the highest normalized cross correlation to the reference are averaged. The final image corresponded to a 0.96° × 0.96° (~650 pixels x 650 pixels) region in the retina. In linear dimensions, this is approximately 260 µm × 260 µm.
2.2. Study subjects
AO-FIO images used for algorithm validation were obtained from 7 healthy adult subjects (3 females and 4 males) aged 23-35 years old. These subjects underwent a single imaging session in which 12 AO image frames were acquired from both eyes of neighboring regions, with a 2° x 2° region of overlap between each image. These images covered the area from 7° nasal to 7° temporal (−3° + 3° vertical) to the fovea. AO imaging on each healthy volunteer took approximately 30 minutes. Each patient also underwent complete eye examination including non-contact biometry (IOL Master, Carl Zeiss Meditect Inc, Germany) and retinal imaging using a near-infrared scanning laser ophthalmoscope/ spectral-domain optical coherence tomography (Spectralis, Heidelberg Engineering GmbH, Germany). All research procedures described in this work followed the tenets of the Declaration of Helsinki. The ethical protocol was approved by The University of Western Australia Human Research Ethics Committee (RA/4/1/7662) and all participants of the study provided written consent.
2.3. Semi-automated AO image analysis and workflow
A pre-requisite for analysis of cone density distribution in AO image frames derived from the rtx1 camera, is the ability to assign a coordinate relative to the foveal center in all AO image frames acquired. This requires a combination of frame registration and mapping in addition to cone signal enhancement for accurate cone detection. Our cone counter suite of programs for semi-automatic analysis and visualization of cones on AO imaging is composed of several sub-programs and has been implemented in Matlab 2014 and executed in the Matlab Image Processing and Computer Vision Toolboxes (Mathworks; Nattick, MA, USA). A graphic user interface (GUI) is designed to allow users to interact with the software and to assist data analysis. The core steps are outlined in Fig. 1 and are explained in detail in paragraphs: 2.3.1– 2.3.4.
2.3.1. Conversion from angular to metric coordinates
In the first step, we use a program provided by the producer of the AO-FIO device to correct for distortion within frames of the raw AO image sequence and to average frames to enhance the signal-to-noise ratio of the final image. Then, we employ Littmann’s formula (Eq. (1)) and the Gullstrand schematic model of the eye to convert each final image from angular coordinates (degrees of visual angle) to metric coordinates (in millimeters) on the retina.
2.3.2. Image pre-processing
Poor image quality has been proposed as a major contributor in the high test-retest variability of cone density measurement in AO images [15,29,30]. Therefore, we incorporated an image pre-processing function into the workflow of AO automated image analysis. Optimizing cone visibility is also critical to reduce inter-observer variation in the cone count since manual identification of cones is used as the gold standard in our analyses of the validity of various automated cone segmentation algorithms. The procedures applied here are: 1, contrast-limited adaptive histogram equalization using a full range and Rayleigh distribution to map the grayscale pixel values of the cones into the full range of the grayscale histogram (0-255); and 2, unsharp masking to enhance edge features resulting in a sharpened AO image.
In addition to the above image processing procedures, we used a band-pass filter available in ImageJ software to enhance cone reflexes. This filter processes an image in the frequency domain and attenuates very low and very high frequencies. It enhances the edges (suppressing low frequencies) whilst reducing the noise (attenuating high frequencies). Through manual adjustment of the filter setting, we subjectively chose the one that produced the most apparent improvement in the quality of these images. We filtered large structures down to 10 pixels and small structures up to 5 pixels. A logarithmic operation was subsequently applied to the images to enhance details contained in the low pixel grey values.
From the 106 cropped AO images that were used for developing our own cone counting software (see below), a subset of 20 were used for determining the optimal image processing parameters. These images were 50 µm × 50 µm in size chosen from different locations in the retina with a range of image qualities. The amount of unsharp mask defines the strength of the sharpening effect and is expressed as a scalar value ranging from 0 to 1. A higher value leads to an increase in the contrast of sharpened pixels. The unsharp threshold is also expressed as a scalar value ranging from 0 to 1 and it describes the minimum contrast required for a pixel to be considered as an edge pixel. Higher values (closer to 1) allow sharpening only in high-contrast regions whilst leaving low-contrast regions unaffected. Lower values will also allow sharpening in relatively smoother regions of the image. We found that the most suitable values for unsharp amount and unsharp threshold were 0.99 and 0.10 respectively. This set of parameters was then applied to the entire data set of 106 cropped AO images derived from the rtx1 camera.
2.3.3. Image registration and montage
Loss of perspective of the AO image frame in relation to the foveal center is a potential source of measurement error in cone density because of significant topographic variation in this parameter. Accurate localization of the area of interest (from 3° to 9° eccentricity) for cone counting is facilitated through image registration of AO frames (4° × 4°) with a wide field photograph of the macular region (30° × 30°). We manually perform control point registration between the macular photograph and the AO images using blood vessel landmarks within the macula (Figs. 2(a) and 2(b)).
After loading the macular photograph (30° × 30°) and individual AO frames (4° × 4°) the user first marks the foveal center on the macular image. At least three corresponding landmarks are then marked on the macular photograph and AO frame for image co-registration. The procedure is repeated for each AO frame taken and it takes approximately 30 minutes to complete for the 12 AO frames. These are saved as a set of configuration files (one per AO frame) that link the macular photograph and AO frames, recording their location and the affine 2D transformation that maps the AO frames onto the macular photograph.
On completion of AO frame overlay onto the macular photograph, these individual frames are stitched together. The program stores all the AO frames and affine transformation matrices in order to determine the number of images that overlap at any one sampling point in the stitched image. When a region of interest is chosen for cone density measurement, the AO frame that provides the highest cone density from the corresponding region is used for calculation of the density.
2.3.4. Image analysis: the circle Hough transform
Identification of cones and measurement of density, is performed using the two-step Hough transform available in the Matlab function ‘imfindcircles’ .
The Hough transform (HT) was initially designed for analysis of curves in 1962. This method is able to detect any shape that can undergo geometric transformation in an image . Detection of lines, circles and other structures is possible if their parametric equation is known. A circle with radius ri and center (ai,bi) can be described with parametric equations as:22]. The first stage involves a two-parameter HT to find the center of the circles (ai,bi). An edge detection process is carried out to identify significant edge points. Then, a cubic polynomial curve-fitting method is employed to estimate the surface normal and the concavity of the fitted curve at each boundary point. Based on the normal direction and concavity information, the line segment upon which the circle center may lie is determined. The votes are collected in a parameter plane based on the coordinates of each point on the resulting line segment. At the end of it, those array elements containing large numbers of votes indicate the presence of circle centers . After detecting possible centers, the histogram of the distances of all feature points from the centers is used to verify the existence of circles and extract their radii. The user sets the radius detection range to detect circle formations of a particular set of radii in the image. A sharp maximum appears in the graph, presenting the number of votes within the specified radius detection range, indicating the presence of discrete circle centers.
We propose two methods to analyze cone images. First, the cones can be counted globally. The configuration files that link the macular photograph with the AO frames are used to create a montage of the AO frames. This montage is used for automated detection of cones. A small sampling window specified by the operator is then shifted at steps equal to the window size to cover the entire montage. Circular structures are identified within each window and a density map that matches the montage is generated (Fig. 3). The average total time for creating the montage and density map covering a 14° × 6° region (requires 6 × 2 overlapping AO frames) centered on the fovea is approximately 4 minutes on a laptop computer with a 64-bit operating system, Intel Core i7-3667U, 8GB RAM.
In the second method, the user may select an interactive user interface to analyze individual areas in the AO frames. It loads an individual AO frame so that the user can sample areas of interest that are then saved as individual images for output and further analysis. This cone counting procedure takes less than 1 second. A summary of the cones counted in each sampled area is then saved in Microsoft Excel. In both methods the user can also define a sensitivity factor (between 0 and 1); increasing the factor results in more circular objects being detected, including dim and partially obscured circles.
2.4. Image selection for manual and automated cone counting
Images from both AO-FIO and AO-SLO systems were chosen for analysis because these devices are complementary in their resolution (AO-SLO has a higher resolution than AO-FIO) and field of view (AO-FIO has a wider field of view than AO-SLO).
We analyzed the image set delivered by the AO-FIO system using 50 µm × 50 µm sampling windows. This window size was chosen based on the common approach currently used to derive density estimates and reported previously by other groups . 106 cropped images (65 pixels × 65 pixels, approximately equivalent to a 50 µm × 50 µm region or 0.0025 mm2) selected from 24 AO frames of 7 subjects were used for validation of our software. The images have been grouped into two categories according to their location on the retina from the center of fovea: 3° to 5° eccentricity (52 images) and 7° to 9° eccentricity (54 images). According to Curcio et al.  the cone radii at these locations are approximately 2 µm (~2-3 pixels in AO-FIO images) and 3.5 µm (~4-5 pixels in AO-FIO images) respectively. Cropped images from the regions of interest were chosen randomly from our image database. Each AO frame (1,500 pixels × 1,500 pixels) was sorted based on the relative position within the macular regions. Information regarding eccentricity is included in each AO frame. Results of cone counting from manual cone identification (our gold standard), AOdetect (onboard software of the rtx1, already used by other research groups for cone identification) and our cHT algorithm (presented in this paper) were compared.
We also randomly selected 60 images obtained from the AO-SLO system for analysis. The data set of high quality images and the corresponding Garrioch et al. and Chiu et al. segmentation results are publicly available at the: http://www.duke.edu/~sf59/Chiu_BOE_2013_dataset.htm . These images were taken at approximately 0.65° from the center of fixation (the cone radius at the fovea is approximately 1µm according to Curcio et al. , corresponding to 2-3 pixels in their AO-SLO images). The images were cropped to 55 µm × 55 µm in the image center, which is approximately 137 pixels × 137 pixels (the resolution of this system is higher than the AO-FIO instrument, therefore µm/pixel value is different from AO-FIO system). This window size was selected to compare the results of cHT cone detection with the results reported in the Garrioch and Chiu studies. The same image set was cropped later to 50 µm × 50 µm to analyze a similar metric-sized image as the rtx1 images and to optimize the cHT algorithm. Results of cone counting from manual cone identification (our gold standard), Garrioch’s semi-automated method (Garrioch’s gold standard), Chiu’s GTDP method and our cHT algorithm (presented in this paper) were compared.
2.4.1. Manual cone identification and inter-observer variation
Manual counting was performed by three independent observers using software implemented in Labview and developed by our group. The observers had varying levels of familiarity with AO images, ranging from a complete novice to an expert counter. The same instructions were delivered to each observer along with the images to be analyzed. The observers were required to mark circular bright signals in the AO image. This was done on unfiltered images initially and then repeated in filtered images 1 month later to minimize learning effect in cone visualization. If the bright signal was truncated by the edge of the image frame, the observer was instructed to only mark those with more than 50% of its area visible in the image frame. This program allowed rapid storage and retrieval of a collection of landmarks. The user identified cones in an AO image simply by clicking on the image so that a colored dot was overlaid at the clicked position. The maps of colored dots from each user were collated and simultaneously overlaid on the analyzed image and saved to an Excel spreadsheet and .png file respectively. The manual counts were arbitrated by an independent observer. The cones marked by at least two observers were included without validation and the cones marked by only one of the three observers were either dismissed or confirmed by an arbitrator.
2.4.2. Cone counting of AO-FIO images using the AOdetect software
Cone photoreceptors within the AO-FIO images are identified in the AOdetect software by automatically detecting the central coordinates of small circular spots whose brightness are higher than the surrounding background level. The spatial distribution of these point coordinates are analyzed in terms of cone density and inter-cone spacing, as well as Voronoi analysis using the Delauney triangulation method. Results are provided in the form of comprehensive statistics and graphics, all of which can be exported to easily interpretable file formats. When the cones are detected and the Voronoi diagram is prepared, the Voronoi polygons are considered to be the cells’ surfaces. From this diagram, it becomes relatively simple to estimate the density for each cell (1/surface of the cell). Based on this information the min, max, mean and standard deviation of cone density can be calculated. The processing time for each 4° × 4° AO-FIO frame is approximately 1 minute.
2.4.3. Cone counting of AO-SLO images reported by Garrioch et al. and Chiu et al
Cones in the AO-SLO images were identified by a semi-automated method described by Garrioch et al  and also the graph theory and dynamic programing (GTDP) method reported by Chiu et al. . In the semi-automated method, local maxima are identified as cones and missed cones are counted manually and added to the automated count to derive the total cone count. Chiu et al. use a graph theory and dynamic programing (GTDP) method to segment the AO images and detect cones. Briefly, local maxima are also detected in the pre-processed image frame. The quasi-polar transform is then used to map the closed contour cone estimates from the Cartesian domain into layers in the quasi-polar domain. These structures are then segmented using GTDP method.
2.4.4. Cone counting using the circle Hough transform
The circle Hough transform (cHT) counting technique was applied to both AO-SLO and AO-FIO. We empirically chose a sensitivity factor (between 0 and 1) of 0.99 for the cHT algorithm. The higher the factor, the greater the number of circle-like objects that are detected, including those which are dim or partially obscured by the image frame boundary. The range of radii of the circular objects to be identified in the AO image is optimized by varying two parameters: the minimum radius and its range. We systematically tested 22 combinations of radius parameters. The smallest circle radius we aimed to detect was 1 pixel and the largest was 10 pixels. The size of the radius range was varied between 3 and 9 pixels. We hypothesized that cones of different sizes (pixel dimensions) may require different radius settings in cHT for optimal cone detection as compared to manual count.
2.5. Statistical analysis
Agreement between two measurement methods were evaluated by using Bland-Altman plots and limits of agreement. These were implemented in Matlab [31–33]. In the first step a scatter of differences in cone counts between the methods against the average counts is plotted to confirm that there is no relationship between the differences and the mean. If this condition is preserved then the bias (mean difference), standard deviation of the differences (SD of diff.) and confidence limits for the bias (called the limits of agreement, LoA, and defined as: mean difference ± 1.96 SD of diff.) are calculated. The bias and LoA are then displayed as solid and dashed lines respectively on the graphs (see Section 3). The plotted differences represent the new method minus the established method, such that the bias quantifies how much higher (positive bias) or lower (negative bias) average values are with the new method compared with the established one . The standard deviation of the differences measures the random fluctuation around the mean. The LoA represents the range of values for the differences between the methods that can be expected 95% of the time. It gives an indication of how well the measurements agree between methods. Paired t-tests were used to examine the significance of the bias. The two way repeated measure analysis of variance (ANOVA) test was used to determine whether there was a significant difference in cone count before and after filtering and between the 3 observers. Spearman, Pearson and Kendall correlation coefficients were calculated to determine the relationship between inter-observer variation and magnitude of the cone count. Coefficient of repeatability was calculated as described by Bland and Altman  and the relationship between inter-observer differences and mean cone count was examined through Spearman and Pearson correlation.
The effect of image processing on inter-observer variability in manual cone counting is reported in Section 3.1. Comparisons of the performance of cHT against arbitrated manual cone identification in AO-SLO and AO-FIO images are detailed in section 3.2. Assessment of cone detection within AO-SLO and AO-FIO images by the circle Hough transform and how they compare to the existing automated methods are reported in Sections 3.3 and 3.4.
3.1. Manual cone identification–effect of image processing on inter-observer variability
Figure 4(a) (top row) illustrates AO-FIO images selected from our database before filtering. Background noise was reduced and dim structures that resemble cones became more obvious after filtering (Fig. 4(a) bottom row). We also applied the same image processing procedures to AO-SLO images to check that our image filter did not diminish features of readily identifiable cones whilst reducing noise (Fig. 4(b)).
Two-way repeated measures ANOVA test on cone counts of the 3°-5° images demonstrated a statistically significant main effect of the observer, F(2,51) = 59.85, p < 0.001, η2 = 0.540 but not of image filtering. The interaction between observer and filtering was also statistically significant, F(1,51) = 22.88, p < 0.001, η2 = 0.310. For cone counts of 7°-9° images, the main effect of observer was statistically significant, F(2,53) = 204.11, p < 0.001, η2 = 0.790 as was the effect of image filtering, F(2,53) = 10.37, p = 0.002, η2 = 0.164. The interaction between observer and filtering was also significant, F(1,53) = 91.65, p < 0.001, η2 = 0.634. High inter-observer variation in cone counts was encountered in unfiltered images (Fig. 5). Coefficients of repeatability (95% confidence interval) between the 3 observers were 25.10 (21.68-28.51) and 39.62 (34.34-44.91) for cropped images at 3°-5° and 7°-9° from the fovea respectively. This reduced to 10.82 (9.35-12.29) and 14.10 (12.22-15.98) respectively when filtered images were counted by the same observers.
Filtering reduced the mean inter-observer variance of cone counts by 81% and 87% for the 3°-5° and 7°-9° images respectively. There was a weak relationship between standard deviation and mean cone counts except for unfiltered images at 3°-5° where a strong negative association existed: the inter-observer variation reduced as cone density increased (Fig. 5). Figure 6 illustrates the change in inter-observer agreement between two independent observers.
The above results support the importance of image filtering in enhancing consistency of cone identification. This is a pre-requisite for further analysis and comparison between automated and manual counting methods. However, it is important to note that cone segmentation using AOdetect can only be done with original unfiltered images.
3.2. Cone identification–optimization of the circle Hough transform parameters
We systematically tested the performance of cHT against arbitrated manual cone counts on the entire set of 106 AO-FIO and 60 AO-SLO filtered images (50 µm × 50 µm) using different radius ranges and minimum radii for the circular objects that we wish cHT to detect. The counts for different ranges are presented in Table 1. These radius ranges have been selected on the basis that the cone radius at 0.65° is approximately 2-3 pixels in the AO-SLO image, while the cone radii at 3°-5° and 7°-9° are approximately 2-3 and 4-5 pixels respectively.
The numbers of detected cones varied with the circle detection parameters chosen. The radius range of 1 to 4 pixels resulted in the tightest agreement between manual and automated cone identification for images obtained with AO-SLO. For AO-FIO images the tightest agreement between manual and automated cone identification at 3° to 5° eccentricity was observed when the radius range is set at 2 to 4 pixels. At 7° to 9° eccentricity, the tightest agreement between these counting methods occurred when the pixel range is set at 2 to 9 pixels although 2 to 8 and 2 to 10 pixels also showed good agreement.
Our results indicate that cone identification using a cHT is dependent on the radius range selected. This is in keeping with the knowledge that cone diameter increases with eccentricity. Setting a narrow radius range may risk underestimation of cone numbers. On the other hand, if the radius range is too wide, clusters of closely packed cones may be identified as a single large circular structure as well as individual circular structures leading to overestimation of cone numbers (Fig. 7(c)).
3.3. Cone identification in AO-SLO images: comparison with other automated method
The performance of optimized cHT for cone identification within AO-SLO images (55 µm × 55 µm) was compared to manual cone counting, semi-automated counting by Garrioch et al.  and automated counting by Chiu et al.  using Bland-Altman approach. The results are presented in Fig. 8. While the Garrioch study defines the gold standard as the semi-automated identification of cones where the automatic identification of cones was carefully reviewed and corrected, Chiu et al. used a fully automated method based on a graph theory and dynamic programming (GTDP) to detect cones.
The Bland-Altman and scatter plots demonstrate high agreement between all the methods of counting. The mean difference between our optimized cHT method and manual counting is - 0.2449 cones, while that between our optimized cHT and Garrioch’s gold standard is −0.4898 cones. Similar results were achieved for comparisons of GTDP versus manual counting and Garrioch’s gold standard (0.7959 cones and 0.5510 cones respectively). Comparing cHT directly with GTDP a trend for slightly lower cone counts with our method was noted. This bias is likely due to variation in cone identification criteria at the edges of the image frames.
3.4. Cone identification in AO-FIO images–comparison with other automated method
The performances of cHT, AOdetect software and manual method for cone identification are presented in Fig. 9. The estimates of the mean difference between the automated counting methods and arbitrated manual counting tended to be lower for cHT as compared to AOdetect software. The mean differences between AOdetect method and arbitrated manual cone counting were −3 cones and −2 cones for the 3° to 5° and 7° to 9° eccentricities respectively. The underestimation of cone number by AOdetect was more pronounced in images with lower numbers of cones (30-50 per cropped region). In contrast, the mean differences between optimized cHT and arbitrated manual cone count were estimated to be −1 cone and 0 cones for the 3-5 and 7-9 degrees respectively. There was a trend for underestimation of cone count in frames with higher number of cones (50-70 per cropped region).
In this paper, we describe an AO image analysis software that is designed to facilitate clinical use of AO image frames by 1, allowing co-registration and overlay of any AO image frames onto a wide field macular photograph and 2, creating a montage of cone images (with a field of up to 18° x 18°) using overlapping 4° x 4° field AO image frames. An overview of cone mosaic and density can thus be appreciated and compared across clinic visits and between eyes and patients. These features are not available on the current AOdetect software that comes with the rtx1 camera. Additionally, we describe the implementation of an image filtering procedure to enhance the visibility of cones to improve inter-observer agreement in the cone counts. Filtering does not degrade the high quality AO-SLO images, and enhances AO-FIO image quality, enabling more reliable manual cone counts.
Using arbitrated manual counts as the gold standard, we optimized cHT parameters to detect cones in equivalent-sized AO-SLO and AO-FIO cropped images with varying cone densities and dimensions. Our optimized cHT method outperforms AOdetect in identification of cones in AO-FIO images and is equivalent to Chiu’s GDTP method for cone detection in AO-SLO images.
Testing 840 cropped AO-SLO images of 55 µm x 55 µm size, Garrioch et al. reported an average coefficient of repeatability of 1,967 cones/mm2 in 4 equivalent regions of interest (at 0.65° eccentricity) that had a mean density of 72,528 cones/mm2 based on their semi-automated cone counting method. Garnier et al also reported on inter-grader agreement of cone counts from 60 cropped AO-FIO images (90 µm x 90 µm sampling windows). They showed a bias of 169 cones/mm2 and limits of agreement of −1,409 to + 1,747 cones/mm2. We reported that before and after filtering of AO images at 3°to5° eccentricity, coefficients of repeatability of manual counting by 3 observers were 10,038 and 4,329 respectively. The bias and limits of agreement between observers 1 and 2 were + 1,792 (−1,629 to + 5,213) before and + 315 (−2,046 to + 2,677) after filtering of AO images at 3° to 5° eccentricity. The results obtained using our cHT method on filtered images were comparable to those reported by Garrioch and Garnier et al. The poor inter-observer repeatability seen in our analysis are not unexpected given the varied familiarity of the observers with AO images. However, a reduction of over 80% in the average within-subject variance indicates the strong effect of filtering on image quality. We also show that inter-observer variation in cone counting does not increase with increasing number of cones as imaged by the rtx1 camera with the exception of unfiltered images at 3° to 5°. Random error may explain the similarity in the absolute number of ambiguous cones in the cropped image between different regions of the retina, irrespective of whether there are 30 or 60 cones in the frame. The lack of association (e.g. increased error with increasing number of cones) suggests that the occurrence of a poorly visualized cone (i.e. those that are identified by one observer or algorithm but not the other) is unrelated to intrinsic features of the cone or to a fixed area of the image affected by poor quality.
Optimized cHT parameters were determined using arbitrated cone counts from 60 AO-SLO and 106 AO-FIO cropped AO images as the gold standard. For closely packed cones with a radius of around 2-3 pixels (AO-SLO images), we showed that manual cone counting is most closely matched by cHT algorithm if the radius setting is 1 to 4 pixels. Similarly, for moderately packed, small cones of radius around 2-3 pixels (AO-FIO images) at 3°to 5° eccentricity, a radius setting of 2 to 4 pixels gives the optimal cHT result compared to manual counting. For loosely packed, large cones of 4-5 pixels (AO-FIO images) at 7° to9° eccentricity, a radius setting of 2 to 9 pixels gives the optimal cHT result compared to manual counting. It is noteworthy that even by altering the lower bound of the radius range by 1 pixel (i.e. 1 to 9 or 3 to 9 pixels), the performance of cHT is severely compromised. Increasing or decreasing the upper boundary by 1 pixel (i.e. 2 to 8 or 2 to 10 pixels) had less effect on agreement with manual cone counts. We used the entire set of 166 cropped images to optimize these parameters rather than splitting the sample into training and validation sets. These radius settings will require further validation in an independent study to confirm their generalizability to counting cones in other regions of the macula or subjects with retinal diseases. We did not analyze AO-SLO images from regions where both rods and cones can be visualized. It may be possible to use cHT to count these structures separately through defining cone-specific and rod-specific radius ranges as these photoreceptors have very different diameters in the perifoveal region.
The numbers of cones detected by our optimized cHT algorithm were similar to those obtained by the automated GDTP method described by Chiu et al. and semi-automated methods described by Garrioch et al. This excellent agreement held for densely packed cones of densities ranging from 60,000 to 100,000 cones/mm2 or 150 to 250 cones in each cropped 50 µm x 50 µm AO-SLO image. We showed that AOdetect had moderate agreement with arbitrated manual counts with a significant bias due to underestimation of cone counts at lower cone densities (30 to 50 cones per 50 µm x 50 µm image or regions with equivalent density of 12,000-20,000 cones/mm2). The most likely cause for this is low image quality because agreement at this range is much improved when cropped images were filtered and counted manually. Interestingly, there was a trend for cHT to underestimate cone counts in cropped frames with larger numbers of cones (60-70 cones per 50 µm x 50 µm image or regions with equivalent densities of 24,000-28,000 cones/mm2). This bias may be due to the small number of cropped images with this range of cone density or to the failure of cHT to detect truncated cones at the edges of the frames. Given this potential limitation of cHT, further work is still required to enhance image quality and optimize the circle detection algorithm so that truncated cones at the edges of AO images are also identified. An advantage of the wider field of view in the AO-FIO image is the ability to acquire overlapping frames so that cone signals can be further enhanced by averaging regions of interest that can be visualized in 2-4 adjacent AO frames. Future studies are also required to automate the process of selecting the optimal radius range for regions of interest at various eccentricities from the fovea.
There are several limitations in this study. First, we have optimized the cHT algorithm parameter without validation in an independent set of AO images. This can create over-fit and the performance of cHT algorithm may be overestimated. We made adjustments to the image filtering settings based on subjective impression of whether cones are more or less visible. Second, we did not address the questions of (1) how much image adjustment is required for a clinically meaningful improvement in repeatability of cone counts and (2) whether the parameters can be varied semi-automatically depending on eccentricity (distance from fovea), average overall image brightness and other quantitative measures of image quality.
The relative versatility of our software compared with AOdetect will also allow the establishment of a database of cone-related parameters in association with other measures of retinal structure and function as they can be readily co-registered using familiar retinal landmarks. We anticipate this software will be most useful in the measurement of changes in cone density over time and the study of the relationship between cone mosaic parameters, retinal sublayer thicknesses and retinal sensitivity on microperimetry. Future work is needed to investigate automation of radius settings in relation to eccentricity from the fovea and whether cHT may also be useful in quantifying the number of rods and cones separately on AO-SLO images as these structures have very different diameters in the perifoveal region.
The authors would like to kindly acknowledge Sina Farsiu, Ph.D and Stephanie J. Chiu, Ph.D for making adaptive optics images of cones accessible on-line at http://www.duke.edu/~sf59/Chiu_BOE_2013_dataset.htm and allowing us to use them in this study. This work was support by the NH and MRC Early Career Fellowship and Bayer HealthCare - Global Ophthalmology Awards Program (Fred K. Chen), a New Investigator research grant through the Ophthalmic Research Institute of Australia (Danuta M. Bukowska), and Novartis Medical Retina Fellowship (Sue Ling Wan).
References and links
1. M. Zacharria, B. Lamory, and N. Chateau, “Biomedical imaging: New view of the eye,” Nat. Photonics 5(1), 24–26 (2011). [CrossRef]
3. L. Yeung, V. C. Lima, P. Garcia, G. Landa, and R. B. Rosen, “Correlation between spectral domain optical coherence tomography findings and fluorescein angiography patterns in diabetic macular edema,” Ophthalmology 116(6), 1158–1167 (2009). [CrossRef] [PubMed]
5. V. J. Srinivasan, R. Huber, I. Gorczynska, J. G. Fujimoto, J. Y. Jiang, P. Reisen, and A. E. Cable, “High-speed, high-resolution optical coherence tomography retinal imaging with a frequency-swept laser at 850 nm,” Opt. Lett. 32(4), 361–363 (2007). [CrossRef] [PubMed]
6. A. Uji, S. Ooto, M. Hangai, S. Arichika, and N. Yoshimura, “Image quality improvement in adaptive optics scanning laser ophthalmoscopy assisted capillary visualization using B-spline-based elastic image registration,” PLoS One 8(11), e80106 (2013). [CrossRef] [PubMed]
7. A. Pinhas, M. Dubow, N. Shah, T. Y. Chui, D. Scoles, Y. N. Sulai, R. Weitz, J. B. Walsh, J. Carroll, A. Dubra, and R. B. Rosen, “In vivo imaging of human retinal microvasculature using adaptive optics scanning light ophthalmoscope fluorescein angiography,” Biomed. Opt. Express 4(8), 1305–1317 (2013). [CrossRef] [PubMed]
8. S. Zayit-Soudry, J. L. Duncan, R. Syed, M. Menghini, and A. J. Roorda, “Cone structure imaged with adaptive optics scanning laser ophthalmoscopy in eyes with nonneovascular age-related macular degeneration,” Invest. Ophthalmol. Vis. Sci. 54(12), 7498–7509 (2013). [CrossRef] [PubMed]
9. D. H. Wojtas, B. Wu, P. K. Ahnelt, P. J. Bones, and R. P. Millane, “Automated analysis of differential interference contrast microscopy images of the foveal cone mosaic,” J. Opt. Soc. Am. A 25(5), 1181–1189 (2008). [CrossRef] [PubMed]
11. B. Xue, S. S. Choi, N. Doble, and J. S. Werner, “Photoreceptor counting and montaging of en-face retinal images from an adaptive optics fundus camera,” J. Opt. Soc. Am. A 24(5), 1364–1372 (2007). [CrossRef] [PubMed]
12. B. S. Liu, S. Tarima, A. Visotcky, A. Pechauer, R. F. Cooper, L. Landsem, M. A. Wilk, P. Godara, V. Makhijani, Y. N. Sulai, N. Syed, G. Yasumura, A. K. Garg, M. E. Pennesi, B. J. Lujan, A. Dubra, J. L. Duncan, and J. Carroll, “The reliability of parafoveal cone density measurements,” Br. J. Ophthalmol. 98(8), 1126–1131 (2014). [CrossRef] [PubMed]
14. S. J. Chiu, Y. Lokhnygina, A. M. Dubis, A. Dubra, J. Carroll, J. A. Izatt, and S. Farsiu, “Automatic cone photoreceptor segmentation using graph theory and dynamic programming,” Biomed. Opt. Express 4(6), 924–937 (2013). [CrossRef] [PubMed]
15. R. Garrioch, C. Langlo, A. M. Dubis, R. F. Cooper, A. Dubra, and J. Carroll, “Repeatability of in vivo parafoveal cone density and spacing measurements,” Optom. Vis. Sci. 89(5), 632–643 (2012). [CrossRef] [PubMed]
16. Q. Wang, O. P. Kocaoglu, B. Cense, J. Bruestle, R. S. Jonnal, W. Gao, and D. T. Miller, “Imaging retinal capillaries using ultrahigh-resolution optical coherence tomography and adaptive optics,” Invest. Ophthalmol. Vis. Sci. 52(9), 6292–6299 (2011). [CrossRef] [PubMed]
17. A. Turpin, P. Morrow, B. Scotney, R. Anderson, and C. Wolsley, “Automated identi cation of photoreceptor cones using multi-scale mod- elling and normalized cross-correlation,” Image Analysis and Processing 6978, 494–503 (2011).
18. K. Loquin, I. Bloch, K. Nakashima, F. Rossant, P. Y. Boelle, and M. Paques, “Photoreceptor detection in in-vivo Adaptive Optics images of the retina: Towards a simple interactive tool for the physicians,” Proc. Int. Symp. Biomed. Imaging, 191–194 (2011). [CrossRef]
19. R. S. Jonnal, O. P. Kocaoglu, Q. Wang, S. Lee, and D. T. Miller, “Phase-sensitive imaging of the outer retina using optical coherence tomography and adaptive optics,” Biomed. Opt. Express 3(1), 104–124 (2012). [CrossRef] [PubMed]
20. R. F. Cooper, C. S. Langlo, A. Dubra, and J. Carroll, “Automatic detection of modal spacing (Yellott’s ring) in adaptive optics scanning light ophthalmoscope images,” Ophthalmic Physiol. Opt. 33(4), 540–549 (2013). [CrossRef] [PubMed]
21. H. K. Yuen, J. Princen, J. Illingworth, and J. Kittler, “Comparative study of Hough Transform methods for circle finding,” Image Vis. Comput. 8(1), 71–77 (1990). [CrossRef]
23. C. Kimme, D. Ballard, and J. Sklansky, “Finding circles by an array of accumulators,” Commun. ACM 18(2), 120–122 (1975). [CrossRef]
24. O. Ecabert and J. P. Thiran, “Adaptive Hough transform for the detection of natural shapes under weak affine transformations,” Pattern Recognit. Lett. 25(12), 1411–1419 (2004). [CrossRef]
25. D. Ioannou, W. Huda, and A. F. Laine, “Circle recognition through a 2D Hough Transform and radius histogramming,” Image Vis. Comput. 17(1), 15–26 (1999). [CrossRef]
27. M. N. Muthiah, C. Gias, F. K. Chen, J. Zhong, Z. McClelland, F. B. Sallo, T. Peto, P. J. Coffey, and L. da Cruz, “Cone photoreceptor definition on adaptive optics retinal imaging,” Br. J. Ophthalmol. 98(8), 1073–1079 (2014). [CrossRef] [PubMed]
28. D. M. Tsai, “A machine vision approach for detecting and inspecting circular parts,” Int. J. Adv. Manuf. Technol. 15(3), 217–221 (1999). [CrossRef]
29. M. Lombardo, S. Serrao, P. Ducoli, and G. Lombardo, “Influence of sampling window size and orientation on parafoveal cone packing density,” Biomed. Opt. Express 4(8), 1318–1331 (2013). [CrossRef] [PubMed]
30. M. Bidaut Garnier, M. M. Flores, G. Debellemanière, M. Puyraveau, P. Tumahai, M. Meillat, C. Schwartz, M. Montard, B. Delbosc, and M. Saleh, “Reliability of cone counts using an adaptive optics retinal camera,” Clin. Experiment. Ophthalmol. 42(9), 830–840 (2014).