1. Introduction

The first stage of vision is the capture of photons by the visual pigment within the outer segment of photoreceptor cells in the retina. Light capture initiates a structural change in the photopigment whereby its affinity for visible light is dramatically reduced – the activated pigment is “bleached” [1]. Activation of the visual pigment in turn elicits an amplified cascade of biochemical reactions, which ultimately result in hyperpolarization of the cell and relay of neural information to the visual system.

Both the bleaching of pigment and associated phototransduction result in distinct optical signals, which have been used to assess the extent to which each cell has responded to a light stimulus. Such work was first undertaken over 20 years ago [2,3], following the advent of adaptive optics to achieve cellular resolution in vivo. Flood illumination was used to image the trichromatic photoreceptor mosaic in humans and primates with visible light under 3 adaptation conditions: in one condition the eye was dark-adapted to replenish the photopigment; here long- and middle-wavelength sensitive cones (“L” and “M” respectively) strongly absorbed the 550 nm imaging light and hence appeared dark, whilst the short-wavelength sensitive “S” cones did not and appeared bright. In the other two conditions, after dark adaptation the retina was exposed to an adaptation light that preferentially bleached either “L” or “M” cones, causing the bleached cones to appear brighter when imaged at 550 nm. These experiments allowed the cells to be classified as S, M or L type by clustering the observed responses, with classification uncertainty of a few percent. However, the experimental procedure required dozens of repeats to achieve sufficient signal-to-noise ratio [3]. Despite this, the method has shown great utility in elucidating the arrangement of cone cells tiling both healthy and abnormal retina [3–5].

One way to improve the accuracy of cell-resolved densitometry is to acquire video data for each cell, observing a cell as it transitions from the pre-adaptation condition to the fully bleached state. This helps to control for natural variations in the amount of pigment and degree of waveguiding between neighboring cones [6–8]. Using this dynamic approach and tracking the bleach over several seconds, similar accuracy of cone classification has been reported from approximately 4 times fewer experimental repetitions; however, experimental sessions still required up to a full day [8]. Further reductions in experimental time required could be gained by imaging at a different wavelength to further enhance the differential signal between M and L cones, although the extent to which this improves accuracy is not yet known [9].

Reasons for low signal-to-noise ratio for cellular resolved retinal densitometry include variations in pigment between neighbouring cones [6,7], as well as non-pigment based variations whereby cone intensity varies markedly over time even when the cells are fully bleached [10]. One major cause for the latter kind of variations are the effects of optical interference, whereby light reflected from and between the anterior and posterior bounds of the photoreceptor outer segment interferes to modulate the observed intensity of each photoreceptor [11]. Wavelength-scale changes in the optical path length of the photoreceptor outer segment, therefore, cause sinusoidal oscillations in cellular intensity. Although these oscillations are a confound for densitometry, they did enable the first cell-resolved imaging of processes associated with phototransduction in the living eye [11]. The direction (brighter or darker) of these changes is essentially random due to the sub-micrometer differences in the resting length of any given cell, however the repetition of many experimental runs reveals a consistent signal [12].

Although initially observed via intensity-based imaging, light-evoked changes in optical path length (OPL) through the photoreceptors have been best characterized by phase-sensitive optical coherence tomography [13], particularly when equipped with adaptive optics (AO-OCT) [14,15]. This technique affords nanometer-scale tracking of changes in length and refractive index of the photoreceptor outer segments [15,16]. After a flash of light, the outer segments undergo reduction in OPL on the order of tens of nanometres, detectable by ∼1 ms from light onset and peaking at ∼5-7 ms [15–17]. The time scale implicates rapid changes in osmolarity and/or refractive index shortly following bleaching of the visual pigment, which has a time constant of ∼0.25–0.50 ms [1]. Following this rapid reduction is a slower, and larger, expansionary phase during which OPL increases by hundreds of nanometres, peaking on the order of hundreds of milliseconds from light onset [15,16]. These changes in OPL can be used to improve the accuracy of cone classification, with error rates below 1% [15,18]. In addition to low error rates, this technique does not require full dark adaptation as the signal is generally able to be elicited using lights that bleach only a few percent of the available photopigment [14–20]. Beyond the classification of photoreceptor type, the sensitivity of this technique has been exemplified by the detection of cellular resolved functional deficits in retinitis pigmentosa prior to the development of structural loss [20], and by measurement of the functional response in the much smaller rod cells which exhibit a more sensitive, albeit slower, response [19].

Functional signals afforded by both cell-resolved densitometry and changes in the optical path length through the photoreceptor outer segment offer complementary information: the former reveals the presence of functioning photopigment and the efficiency of optical coupling of light into photoreceptors, whilst the latter confirms the directly ensuing processes of phototransduction. There is therefore good reason to revisit the efficiency of cell-resolved densitometry, whose performance has not been improved for almost a decade [8] and currently lags behind regarding signal-to-noise ratio and the lengthy experimental times required. Here, we aimed to improve the accuracy of cellular-resolved densitometry by mitigating the influence of phototransduction related interference on the measurement of pigment bleaching. This was achieved using a flood illumination adaptive optics ophthalmoscope imaging at a full-field rate of 4000 fps, together with a broadband light source. The problem of classifying the human trichromatic cone mosaic was used to evaluate the utility of this approach.

2. Methods

2.1 Subjects

Three healthy subjects were recruited, two male (age 39 and 29 years) and one female (age 28 years). Two of the subjects are authors on this manuscript (PB and ACBJ). Subjects had best corrected visual acuity of 20/20 or better, wore no habitual refractive correction, and had no notable ocular pathology on routine clinical fundus examination by a qualified optometrist. Subjects reported normal color vision and passed Ishihara plate testing. The project was approved by the University of Melbourne Human Research Ethics Committee (Project #21022). All subjects gave written informed consent, and the study proceeded in accordance with the tenets of the Declaration of Helsinki.

2.2 Adaptive optics imaging system

Our flood-based system for cell-resolved densitometry has been reported previously [6,7]. In brief, a small beacon is projected onto the retina using 835 nm light from a superluminescent diode (Hamamatsu, Japan) at 9 µW. Light returned from a pharmacologically dilated pupil (tropicamide, 1%) is relayed through a mirror-based telescope to form optical conjugate planes between the eye’s pupil, the deformable mirror (HiSpeed DM97-15, Alpao, France) and the Hartmann-Shack lenslet array. A single lens is used to form a retinal image on the science camera. The system pupil is set by the deformable mirror aperture which corresponds to 7.6 mm diameter at the eye’s pupil. A CCD camera (Pike, Allied Vision Technologies, Stadtroda, Germany) with overlying lenslet array (focal length 24 mm, pitch 0.4 mm, Adaptive Optics Associates, Cambridge MA) produces a Hartmann-Shack image that is processed by custom Matlab software. The software relates found spot locations to apply the actuator command signals required to rectify them at 20 Hz. When the wavefront quality reaches a satisfactory level (typically < 0.06 µm RMS over a 7.0 mm pupil), the operator initiates a command to trigger an imaging sequence via a TTL signal yoked to the light source and science camera.

The light source is an 8W supercontinuum laser filtered through a tunable transmission filter (Fianium, Southampton, UK) to pass a band for imaging at 555 ± 50 nm (FWHM). This relatively wide band (double that in our previous work) was used to reduce coherence length to the order of ∼1 µm assuming a Gaussian spectral distribution. This step was taken following previous observations of interference phenomena in some cells despite the coherence length being significantly shorter than the cone outer segments which is suggestive of intracellular scattering [6,7]. After the transmission filter, light is relayed via 32 m of multimode step index optical fiber (core diameter 200 µm, 0.37 NA, length 32 m) to reduce spatial coherence and associated image speckle [21]. The illumination arm resulted in 1.39 mW at 555 nm entering the pupil, which was delivered over a circular field 1.25° in diameter. With a total exposure of 200 ms, light levels were approximately a factor of 5 below the maximum permissible exposure recommended by ANSI, limited by thermal as opposed to photochemical effects [22,23]. Safety calculations were made with a widely used spreadsheet tool that implements common calculations based on the ANSI standards [22].

The science camera is an ultrafast CMOS detector (Photron NOVA Fastcam S12). Images were captured at 4000 fps, with frame exposure at 1/4000^th sec. The pixels are 20 µm wide, corresponding to approximately 1.5 µm on the retina for an emmetropic eye.

2.3 Image processing

Acquired image sequences were saved in raw 12-bit format. A flat-field image was computed by averaging video data acquired whilst subjects shifted their fixation erratically over the central few degrees. Each raw video frame was divided by the flat field image to correct for non-uniformities in delivered illumination, then corrected for eye motion with a standard cross-correlation approach. Between video sequences it was necessary to correct for rotation, due to torsional motion of the eye and/or head. This was achieved prior to correction of translation between sequences, by computing the magnitude of the Fast Fourier Transform (FFT) for reference and acquired images (mixed rotation and translation act only to rotate the FFT magnitude so that rotation can be assessed independently of translation). Following these processing steps, a co-registered stack of video sequences was available for analysis.

2.4 Signal-to-noise considerations at high frame rate

The science camera used could in principle acquire full-field images at rates in excess of 100 kHz. The limiting factor is the light budget; high frame rates require short exposures which result in high shot noise. The read noise for this camera is higher than that of sCMOS camera technology that we used previously [7] (approx. 40 electrons RMS cf. 1-2 electrons), exacerbating the effects of a reduced electron count during each exposure. For these reasons we limited the frame rate to 4 kHz, filling approximately one quarter of the electron well for a bleached cone (∼4000 electrons per pixel in each frame). This appeared to produce raw data frames of comparable signal-to-noise ratio as our previous work (we calculate approx. 1100 electrons/pixel in a single frame exposure at 1000 Hz with our previous camera [7]; however those smaller pixels were binned 3 × 3 and the read noise is negligible for that camera, such that we expect around half the signal-to-noise ratio in the present work).

The pixels of this camera are larger than those in our previous work, corresponding to ∼1.5 µm on the retina (instead of 0.5 µm). This is comparable to the Airy radius at 555 nm over a 7.6 mm pupil, indicating pixel sampling of only half the fidelity required to achieve Nyquist sampling by the Rayleigh resolution criterion. This could be counteracted by increasing system magnification, but this would further lower signal-to-noise ratio (e.g. doubling effective pixel size on the retina would yield one quarter the light seen by each pixel in a single exposure; binning pixels post hoc would be of little use due to the high read noise).

Instead, to improve resolution and allow more precise location and tracking of each cone, we created a super-resolved representation of the data by exploiting the incessant motion of the eye using a “shift and add” procedure. This involves assembling a registered image stack by up-sampling raw frames by a factor of 3 with 2D interpolation and then shifting to correct for the change in the eye’s gaze position (again via 2D interpolation). This strategy stems from the astronomy literature where it is not uncommon for pixel sampling to be lower than optical resolution in order to assay a wider field [24]. For our data, the improvement of resolution afforded by this approach is demonstrated in Fig. 1. Cone centers were labeled in a montage generated as per Fig. 1 (c), giving 0.5 µm precision on cell positions.

 

Fig. 1. Super-resolution of acquired image data. a) Shows the raw average of 800 frames, after shifting each one using linear interpolation to correct for eye movements; b) shows the results of upsampling, by a factor of 3, the averaged image from (a) with linear interpolation; c) shows the result of first upscaling individual frames, then shifting to correct for eye movements, and then averaging. Scale bar = 25 µm.

Download Full Size | PDF

2.5 Experimental paradigm

To reduce experimental time we used two conditions, rather than three as in other work [8,15,16]. The first was a dark-adapted condition (6 min., to ensure full regeneration of cone pigment [25]), whilst in the second condition the same dark adaptation period was followed by a pre-bleach with 700 ± 10 nm light for 30 sec. (generated by the same source used for imaging). The pre-bleach light measured 17.5 µW at the pupil, bleaching approx. 95% of L pigment and 23% of M pigment. The predicted ratio of unbleached pigment is accordingly around 15-fold as per other densitometry work [8]. There was a delay of approx. 10 sec. between the pre-bleach stimulus ending and the acquisition of image data, during which we manually re-configured our light source and updated the adaptive optics correction. This protocol of two conditions was repeated 3-4 times, with 2-4 high quality data sequences contributing to analysis in each subject.

Compared to our previous work in cone-resolved densitometry [7], the present work employed imaging rates four times faster, imaged over the full illuminated field which allowed more cells to be tracked, used a wider bandwidth to reduce coherence effects, and used brighter illumination to bleach the photopigment more rapidly. In previous work we also did not attempt to classify the trichromatic cone mosaic, making the present work more directly comparable with other functional imaging approaches [2,8,9,15,17,19].

2.6 Area imaged and cone labelling

In each subject we selected a single high quality fixation locus centered between 0.75 and 1.1° from the foveal center. Data from all experimental conditions were co-registered and averaged to create a single high-quality montage for cone labelling. Only pixels that were within the illuminated field in all experimental runs were considered. Cone centers were labelled in the upsampled montage to the nearest pixel (0.5 µm) using Adobe Photoshop. Only cones that could be conclusively seen were labelled, leaving a small fraction of probable cone locations that were too dark to label. “Dark” cones have been observed in disease where the cell is lost [26], but are also routinely seen in healthy individuals where visual function is often preserved and the cell is termed “dysflective” [26–28].

We observed that the apparent cell centers in our data drifted by small amounts (on the order of 1 µm) within our functional images computed over 10 ms, as compared to the labelled cell montage which was an average over 200 ms. This may relate to changes in cell waveguide properties suggested to occur with bleaching [6,7,29]. To mitigate drifts in apparent cell location, functional images were automatically thresholded by Otsu’s method [30] to identify “activated” pixels. Cone locations corresponding to an activated pixel were allowed to drift by up to 1 µm to maximize the functional signal, whereas non-activated cones were allowed to drift to minimize the functional signal. A drift of this amount corresponds to two pixels in the upsampled data sequence, or less than one pixel in the raw data sequence. The drift is less than one half the minimum separation between cones, and we note that previous approaches [6–8] have integrated cone signals over slightly wider spatial scales, e.g. a central region ∼1.5 to 3.0 µm in diameter [6–8]. Here, we considered only the information from a single pixel for each cone (after allowing for drifting as described above). We found this approach (a single pixel allowed to drift) to give minor improvement to clustering performance, compared with integrating over a small region of interest (with no drift).

2.7 Cone classification by cluster analysis

Using data from the first 10 ms (40 frames) after light exposure, we generated three basic images for each condition: the first frame, the mean over time, and the standard deviation over time (Fig. 2). We further computed “interaction” terms, being the pairwise ratio between the three basic images. For example, the ratio of the standard deviation to the mean is shown in Fig. 3 for both conditions (Fig. 3(a) and Fig. 3(c)). This shows bright clumps of putative M cones embedded within a field of dim L cones (Fig. 3(c)). The information generated is very similar to the double pass optical density returned by fitting intensity over time with an exponential bleaching model (shown for comparison in Fig. 3(b) and 3(d)).

 

Fig. 2. Basic images input for cluster analysis. The first row (a-c) shows data from the dark-adapted condition (bleaching L and M cones) whilst the second row (d-f) shows data from the pre-bleached condition (bleaching largely M cones). From left to right are shown the first frame of the sequence (a, d), the mean over the first 10 ms (b, e) and the standard deviation over the same time period (c, f). Scale bar = 25 µm. Corresponding video data are shown in Visualization 1 for each experimental condition, both raw and normalized to the intensity of the first frame.

Download Full Size | PDF

 

Fig. 3. Example “interaction” images input to cluster analysis. The first row (a, b) shows data from the dark-adapted condition (bleaching L and M cones) whilst the second row (c, d) shows data from the pre-bleached condition (bleaching M cones more than L or S cones). On the left is the coefficient of variation over the first 10 ms (a, c); on the right is the double pass optical density returned from an exponential fit to each pixel over the same period. Scale bar = 25 µm.

Download Full Size | PDF

In total across the two conditions, there were 12 non-independent images input to cluster analysis. Data from one representative pixel for each cone were input to cluster analysis as described above. Data were automatically clustered in this 12-dimensional space by fitting Gaussian mixed models with the Matlab function “fitgmdist” (Mathworks, Natick, USA). Cluster analysis was seeded with a simple heuristic to guess the identity of each cone: outlier cells with a response below the median in the dark-adapted condition were seeded as S cones; of the remaining cells, those with a response below the median in the pre-bleach condition were seeded as M cones; all other cells were seeded as L cones.

2.8 Evaluating the accuracy of cone classification

The most common approach for classifying the cone photoreceptor mosaic is to fit the distribution of some outcome measure(s) with a mixed Gaussian function [2,3,8,15,17]. Raw data are typically first reduced in dimensionality to create a 1D or 2D representation, and individual cells are classified based on which of the fitted distributions has the highest probability at that location. The overlap in area between distributions is a dataset wide metric of error that has been referred to as “uncertainty”. The posterior probability of observing each datum, given the fitted clusters, gives a cell-specific metric of clustering reliability whose average is typically reported.

The standard metrics of uncertainty and average probability described above are not significantly affected by a handful of outlier cones, potentially leading to an under-estimate of the potential error rate (e.g. error as low as 0.02% has been claimed, corresponding to approximately one-fifth of one cell being misclassified [15]). In addition to reporting the above metrics, here we also report the proportion of cells achieving individual posterior probability of 95% or greater, which is more conservative with respect to outliers.

3. Results

3.1 Basic signal characteristics

Example data showing the change in cone intensity over time are plotted in Fig. 4, for 3 repeats of the dark-adapted condition. Note the log scale on the time axis. Data were normalized to the intensity of each cell in the first frame. On the average (black), the bleach proceeds with the expected exponential kinetics and is largely complete within approx. 10 ms. Individual cones show the same rapid initial rising phase during the bleach, but at later times some cones (example in red) oscillate markedly whilst others imaged at the same time remained relatively steady (example in blue).

 

Fig. 4. Intensity over time in the pre-bleach condition. The average of all cones over three experimental runs is shown in black. Two example cones with high coefficient of variation (CoV) following the bleach are plotted for each experimental run (both are L cones; the blue plots show a cone with low variance following the bleach, and the red plots show a cone with high variance due to interference phenomena). Note the log scale on the time axis.

Download Full Size | PDF

To determine the point at which these oscillations significantly impact the ability to predict cone intensity, we computed the autocovariance of intensity over time for each cone. The average autocovariance is plotted in Fig. 5 for both experimental conditions (note the logarithmic scale on the x-axis). Despite the bleach being largely complete by approx. 10 ms, the sharpest drop in the autocovariance occurs after this time, indicating marked changes in cone intensity which are not a direct result of bleaching. Based on analysis such as that shown in Fig. 4 and Fig. 5, to mitigate the influence of phototransduction-related changes on the assessment of cone type based on bleaching information, only the first 10 ms of data were used.

 

Fig. 5. Average autocovariance of cone intensity through time. At short time lags (< 10 ms), cone intensity is reasonably predictable despite rapid bleaching of pigment. Beginning around 10-20 ms is a precipitous drop in autocovariance due to oscillations in cone intensity.

Download Full Size | PDF

3.1.1 Cone classification

We labelled a total of 3,252 cones (1105, 1120 and 1027) across the 3 subjects. For each subject, all labelled cells were used for three-way cluster analysis. The posterior probability for the identified cluster, given the data from a particular cone, averaged 99.8, 99.5 and 99.3% respectively. The proportion of cones with individual probabilities exceeding 95% were 99.2, 97.6 and 97.0% respectively. Classification accuracy was strong within each cone sub-class, with individual probabilities exceeding 95% in a total of 1798 L cones (98.2%), 1248 M cones (97.7%) and 139 S cones (96.5%) across the 3 subjects. The proportion of S cones at the imaged eccentricities (0.75 to 1.1° from the foveal center) was 144 / 3252 = 4.4% across the 3 subjects, which accords with previous estimates of 4-5% at 1° eccentricity [2].

The outcomes of cone classification are shown in Fig. 6 for one subject (corresponding figures for the other two subjects are included in Fig. S1 and Fig. S2). Note that images have been cropped from the 1.25° imaged field to concentrate on the labelled cone region. Figure 6(a) shows the average intensity of the cone mosaic, and Fig. 6(b) the coefficient of variation over the first 10 ms following the pre-bleach condition. This latter image is the most useful single image in our dataset to decide the identity of each cone (bright cells correspond to M cones, darker cells to L cones, and especially dark cells to S cones). Figure 6(c) and Fig. 6(d) repeat these two images, overlaid by symbols colored (red, green, blue) according to cone identity (L, M, S respectively). Circles indicate individual classification probability exceeding 95%, whilst crosses indicate individual probability less than this threshold. Figure 6(e) plots 2 of the 12 clustering variables against one another, with red, green and blue colored symbols respectively indicating L, M or S cones classified with probability exceeding 95%. Figure 6(f) shows the same cone assignments plotted in a collapsed representation of the higher dimensional space: the distance between each datum and each cluster was computed in the higher dimensional space using a multi-dimensional analogue of the Z-score (known as the Mahalanobis distance). This resulted in three “scores” for each cell. The pairwise difference between these scores was computed to enable plotting in two dimensions, with the angle for the vector (S-L) plotted against the vector (M-L). Plotting the data in this space demonstrates the improvement in separation between distributions with the full set of clustering variables (generated from the first frame, average and standard deviation images from each condition), compared to that in Fig. 6(e).

 

Fig. 6. Cone classification results in one subject. a) Average intensity image, used to label cone centers. Scale bar = 25 µm. b) Coefficient of variation, computed over the first 10 ms in the pre-bleach condition. c) Classified mosaic overlaid on the image from (a). Red, green and blue indicate L, M and S cones respectively. Rings indicate cone assignments exceeding 95% individual probability, and crosses show those below this threshold. d) As for (c), but classifications are overlaid on the image from (b). e) Plots two of the clustering variables, with colored circles indicating cone assignments exceeding 95% individual probability and black crosses showing those below this threshold. f) Plots a representation of the data based on the separation of each datum from each of the clusters in the higher dimensional clustering space, showing better cluster separation. Colored circles and black crosses as in e).

Download Full Size | PDF

The mean intensity of each sub-group of classified cones is shown in Fig. 7. Red, green and blue plots correspond respectively to L, M and S cones. Error bars indicate 95% confidence intervals for the mean at each time point. There appears to be significant “bleaching” of S cones despite their known low optical density at 555 ± 50 nm (it is worth noting that, like in other work, S cones were identified by the absence of a measurable signal so cannot be differentiated from cones devoid of pigment). Although some fraction of the pigment in these cones is undoubtedly bleached by the imaging light, the bleach should proceed at a much slower rate whereas the rate constant appears comparable across all cell types in Fig. 7. Similarly, pixels maximally distant from a labelled cone (i.e. vertices in Voronoi space) also brightened significantly over time (not plotted). We attribute these observations of apparent bleaching to the influence of scattered light that has interacted with the visual pigment within neighboring cells.

 

Fig. 7. Mean intensity over time in the pre-bleach condition for classified cells. Each panel shows a different subject. Green shows M cones, red shows L cones and blue shows S cones. Shaded regions indicate 95% confidence on the mean for each group of cones.

Download Full Size | PDF

The locations and cluster probabilities of all classified cones in this work are included in Data File 1 [31].

2.2 Validation of analysis approach

Our analysis differs from some other work in the field in two important respects: a large number of cluster variables were input to analysis (a similar strategy has been applied successfully using AO-OCT data for cone classification, in which some 50 variables were input to cluster analysis [15]), and we discarded data beyond a certain time point (10 ms). To offer further validation for these decisions, we repeated classification using a simple, two-way cluster analysis which classified cones as either L or M type (cones classified as S type via the three-way scheme described above were omitted here). Only two cluster variables were used (the coefficient of variation in the two experimental conditions). The time included in the analysis was varied to gauge the effect on the quality of clustering. The results are shown in Fig. 8 for three different metrics: the mean probability of all individual cone assignments (blue), the proportion of cells exceeding 95% individual probability (yellow), and the agreement with the more advanced three-way cluster analysis (red). It can be seen that regardless of the metric used, beyond approx. 15-30 ms there was a significant reduction in the accuracy of cone assignment. It can also be seen that at the 10 ms time point, agreement with the more complex clustering was high (above 95%), which suggests that the latter did not produce spurious cell assignments (i.e. due to over-fitting of a large number of cluster variables).

 

Fig. 8. Performance of simple two-way classification as a function of time after light onset. Blue shows the average of individual cone probabilities, yellow shows the proportion of cells achieving probability exceeding 95%, and red shows the agreement with advanced three-way classification (within the first 10 ms). Overall, significant reductions in accuracy were encountered for data beyond approx. 15-30 ms.

Download Full Size | PDF

3. Discussion

By application of rapid imaging together with bright illumination of the retina, the bleaching of individual photoreceptor cells can be more reliably measured. Our approach was designed to avoid the well-known oscillations in cone intensity that occur with changes in cell size during phototransduction, beginning within milliseconds of light exposure. Using data from the first 10 ms following onset of light, corruption of the signal by these changes was largely avoided. The mean error rate for cone classification was estimated at 0.2 to 0.7% from 2-4 experimental runs acquired in less than one hour of laboratory time. This is a significant improvement over previous approaches in cell-resolved densitometry (e.g. error on the order of 4-5% from 15 runs over 3-9 hours of laboratory time [8]).

We observed the distribution of cone clustering probabilities to be highly skewed. This can be readily verified by inspection of Fig. 6(c) and Fig. 6(d), where the cluster membership is particularly ambiguous in a small proportion of cones. For this reason, we also reported the proportion of cones for which individual classification probability exceeded 95%, which may offer a more realistic estimate of the error rate rather than relying on the overall distribution shape. Estimated error from this metric ranged from 0.7 to 3.0%, confirming robust classification in the vast majority of cells.

There are three primary reasons that may have caused a small fraction of cells to be less reliably classified in this work. The first is that despite our use of broad bandwidth light, whose coherence length is far less than that of the cone outer segments [6,7], significant interference driven fluctuations were seen in some cones. This may occur due to the suspected presence of multiple closely spaced reflector sites at the bounds of the outer segment [32,33]. There may also be transient scattering boundaries formed within the cone outer segment, which could tend to happen more often with sudden and intense activation of the phototransduction cascade [7]. Discrete changes in refractive index occurring with strong bleaches have been directly observed using AO-OCT, and are expected to have complex effect on our acquired images as they appear to drift along the outer segment [14,34]. The second major cause could be related to real differences in the degree of pigment and/or degree of optical waveguiding efficiency between neighboring cones [6,7]. In the extreme case, a certain proportion of cones in the healthy retina are observed to be “dysflective” at any one time [26–28], indicating that they do not strongly waveguide light at the time of imaging. The prevalence of such cones must place an upper limit on the accuracy of cone classification from a single experimental window. Finally, the influence of scattered light from neighbouring cells, as shown by the apparent bleaching of S cones in Fig. 7, creates an implicit source of noise for flood-based densitometry approaches which lack confocality.

Despite the improvements to cone-resolved densitometry described here, recently published AO-OCT methods offer the lowest known error for in vivo cone classification [14–19]. By considering the area of overlap between the distribution of clustering variables, error has been reported as low as 0.02% using 7-10 videos acquired over ∼30 mins [15]. As mentioned above, we believe this method of estimating the error may not faithfully capture outliers (for example, it is equivalent to approximately one-fifth of one cell in the cited work). Despite this, other estimates of error are also very low (for example, test-retest repeatability of 0.37% from 7-10 videos, or 3% from a single run requiring only a few minutes of experimental time). It is worth noting that eye movements during each OCT scan necessarily corrupt the data from a certain portion of the cones within each video sequence; however at least 90% of cells can generally be classified (with high individual accuracy) following the acquisition of 2 videos [15]. Overall our method presents a viable alternative to the use of AO-OCT in cone classification, which may be more accessible given the lower system complexity.

Unlike previous work in cell-resolved densitometry, here we did not fit a model to the intensity vs time data of each cell to classify cones with parameters such as the optical density or rate constant of the bleach [6–8]. We eschewed this approach because of a noticeable worsening of the quality of functional images (e.g. compare left and right side of Fig. 3), together with a commensurate reduction in cluster separation. One reason for this reduction in quality may be that interference-driven fluctuations in cone intensity begin before bleaching of pigment is complete (which we estimate at approximately 10 ms based on the average cone response plotted in Fig. 4). This will necessarily increase the error in fitting of a simple exponential bleaching model, whereas a simple metric such as standard deviation could be unaffected, or even bolstered, by the additional variance. Interference-driven fluctuations are predicted to pose particular difficulty for cell-resolved densitometry beyond approximately 15-30 ms of light onset, as was shown in Fig. 5 and Fig. 8.

There are a number of ways in which our approach to cone classification could potentially be improved. Stronger illumination could be utilized, causing the bleach to be completed even earlier and also increasing the signal-to-noise ratio on each frame, which is a limiting factor at such short exposures. Of course, it is possible that brighter imaging light could prove more damaging to the retina in the presence of disease, such that a stronger safety factor than applied in normal eyes may be warranted. Nonetheless, according to present ANSI guidelines our illumination power at 555 nm could be as high as approx. 7.1 mW over 10 ms whilst remaining at the same safety factor relative to maximum permissible exposure [22,23]. An alternate improvement made possible by a brighter light source would be to provide sufficient signal to noise ratio at increased system magnification, allowing the wider pixels of the ultrafast camera to sample the full diffraction-limited resolution afforded by the adaptive optics correction. This would obviate the need for super-resolution of acquired video data, a process with unknown influence on the fidelity of measurements made through time (e.g. Figure 1(c) showed a high quality photoreceptor image on the average, but perhaps noise is injected at the level of individual frames). Beyond extra light, the method could also be improved by altering the wavelength of pre-bleaching and imaging lights (for example, pre-exposure to a shorter wavelength to bleach M pigment followed by imaging with a longer wavelength to which L cones should rapidly respond, increasing the differential signal between L and M cones [9]). Ultimately, it would be advantageous to be able to definitively classify large, contiguous portions of the photoreceptor mosaic in only a single experimental run following just a brief flash of light.

Beyond vision science applications such as cone classification, rapid measurement of visual pigment within individual cells offers a sensitive tool for the study of retinal disease. Here we have presented a new method that greatly enhances the potential of cell-resolved densitometry for clinical applications by significantly reducing the experimental time required. Ascertaining which cells both contain viable photopigment and waveguide light sufficiently could provide powerful diagnostic information and establish a new class of outcome measure to gauge the impact of interventions in clinical trials. For example, suitable patients for inclusion in many gene therapy trials require that the disease be sufficiently advanced (so that impact of the treatment can be demonstrated), but with many viable cells remaining (possessing the cellular machinery to be rescued by incorporation of the appropriate genetic information [35,36]). Adaptive optics functional imaging methods, such as that described here, and phototransduction related advances offer powerful new tools in this regard. Cell-resolved densitometry may be particularly useful in conditions characterised by a primary abnormality in the visual pigment, such as in Rho-associated retinitis pigmentosa in which there is a direct mutation to the gene coding for rhodopsin. Similarly, it may be beneficial where disruption of pigment recycling is suspected, for example in conditions affecting the retinal pigment epithelium (including age-related macular degeneration). Finally, this method also facilitates determination of the rate constant of the bleach and hence the quality of optical waveguiding, which is expected to alter upon significant structural change to a cell. When combined with study of the phototransduction cascade via optoretinography, our method may shed additional light on the study of mechanisms of disease; for example in conditions affecting cellular metabolism a cell may be observed to have a muted response by optoretinography, despite having viable pigment confirmed by cell-resolved densitometry.