For many decades fundus imaging has been one of the standard procedures in ophthalmology for examining the posterior eye. In fundus photography, back-reflected light from the flood-illuminated retina is recorded as a 2D image within a few milliseconds. However, with flood-illumination, the out-of-focus light is also recorded, causing features to blur. The invention of the confocal scanning laser ophthalmoscope (cSLO) by Webb et al. in 1987 provided a new tool in retinal imaging [1]. The confocal nature of the cSLO makes it less sensitive to scatter from out-of-focus tissue, and therefore, it provides high contrast, high resolution images from the retina.

The human eye is continuously in motion with a $1/f$ frequency spectrum up to frequencies of 100 Hz [2]. Currently, the state-of-the-art cSLOs have a typical frame rate of $\sim 30\mathrm{Hz}$, which makes them vulnerable to rapid eye movements.

To make the cSLO less susceptible to eye motion, the imaging speed should be improved. Increasing the physical speed of the cSLO scanning mirrors does increase the overall frame rate but it also reduces the signal-to-noise ratio (SNR). A large increase in speed is achieved with a limited reduction in confocality when line illumination is used instead of a spot namely the line scanning ophthalmoscope (LSO) as introduced by Hammer et al. [3]. The next step for faster imaging is to acquire multiple lines within the field of view (FOV) in parallel [4]. A digital micromirror device (DMD) is a spatial light modulator consisting of a 2D micromirror array for which the mirror elements can be individually controlled to create specific illumination patterns. The use of the DMD has been demonstrated for ophthalmic imaging to create an LSO by rapidly projecting a single line to the retina and recording the back-scattered light with a rolling shutter camera [5]. Combining the DMD with a high-speed full-field camera will provide further speed improvements.

In this Letter, a parallel LSO (PLSO) system is demonstrated that uses a digital light projection LightCrafter 4500 development module (Texas Instruments, Dallas, USA) containing a DMD with $912\times 1140$ micromirror elements (pitch 7.6 μm) in a 45° diagonal orientation which can be switched on/off over a mechanical tilt of $\pm 12°$ based on the digital binary pattern. A 624 nm LED provided the illumination and a total internal reflection (TIR) prism was placed in front of the DMD to guide the light propagation depending on the mechanical state of the DMD mirror element (Fig. 1(A)). When the mirror elements are switched on, the light is directed toward the projection optics. In the off state, the light is reflected back in the direction of the source. A set of parallel lines are imaged onto the retina by switching on two-element wide lines in the DMD. After a frame acquisition, mirrors are turned off and the mirrors for the next set of lines are turned on. This is repeated until the full image is acquired. The external optical layout of the PLSO is shown in Fig. 1(B). The lens pair L1 ($f=30\mathrm{mm}$) focuses the parallel output of the LightCrafter, lens L2 ($f=50\mathrm{mm}$) and lens pair L3 ($f=30\mathrm{mm}$) relay the image plane, and ophthalmic lens L4 ($f=30\mathrm{mm}$) collimates the beams before entering the eye through the pupil. To control the optical power and minimize stray light-induced artifacts, apertures (A) were placed in the beam path. To reduce the internal reflections in the system, linear polarizers, a polarizing beam splitter, and a quarter wave plate (QWP) were used. As shown in Fig. 1(B), the output of the projector is first horizontally linearly polarized with linear polarizer (P1) before it passes through the polarizing beam splitter (PBS) in transmission. The light becomes circularly polarized after passing through the quarter wave plate, reflects of the retina, and becomes vertically linearly polarized on the return path through the QWP. The vertically linear polarized light propagates through the PBS in reflection. The lens pairs L5 ($f=30\mathrm{mm}$) and L6 ($f=30\mathrm{mm}$) relay the image to a CMOS camera (acA2040-180kmNIR, Basler, Berlin, Germany). A linear polarizer (P2) is used between the lens pairs L5 and L6 to remove spurious light reflections from lenses in the optical system that could leak through the reflection path of the PBS. A full-field confocal image is constructed by selectively combining multiple acquired line images. The usable FOV for imaging was calculated to be $10°\times 10°$. The thickness of a single line was set to two DMD mirror elements, which is equivalent to a 3.6 pixel thick diagonal line on the camera. The camera operated in global shutter mode at a 100 Hz frame rate for a region of interest of $1400\times 1400$ pixels. The camera image acquisition was done with a high-speed frame grabber (NI PCIe-1473R, National Instruments, Austin, USA), which was triggered every time the projected pattern changed in the DMD to acquire a frame with 10 ms integration time.

 

Fig. 1. Optical layout of the PLSO system. (A) The DMD is illuminated with 624 nm light and the light propagates via the on-switched micromirror out of the TIR prism. (B) Polarizing optics are used to reduce the effects of reflections within the system. L: achromatic doublet lens; P: linear polarizer; A: aperture; PBS: polarizing beam splitter; QWP: quarter wave plate; TIR: total internal reflection prism; DMD: digital micromirror device; CMOS: global shutter camera.

Download Full Size | PDF

The PLSO system does not have physical pinholes in the detection path, and therefore, the acquired image is not confocal. The out-of-focus scattering was removed with a previously established image-processing method for structured illumination by Heintzmann [6]. After all the patterns from a sequence were recorded, the maximum and minimum intensity values for each pixel in the sequence were subtracted from each other to reconstruct a confocal image. The highest intensity values represent the signal in focus, whereas the lowest values are regarded as background signal (resulting from stray light or multiply scattered light). Values between the maximum and minimum intensities represent varying degrees of the out-of-focus signal. In a layered structure like the retina, the level of confocality is especially important for recording a high-contrast image and to reduce the effect of corneal scattering.

A model eye was designed and built to characterize the imaging performance. An uncoated N-BK7 ½” plano-convex lens with a 15 mm focal length and curvature radius of 7.7 mm was used in the model eye to simulate the human cornea with a similar refractive index ($n_{\mathrm{BK}7}=1.52$ versus $n_{\text{cornea}}=1.38$). In addition, the model eye was filled with water to mimic the optical properties of an in vivo human eye, which increased the focal length to 20 mm. The resolution performance was tested with a 1951 USAF resolution test target as the model retina, which was positioned in the focal plane of the model eye. Figure 2 shows an image of the target [Fig. 2(A)] and two cross sections from the areas indicated by yellow lines [Figs. 2(B) and 2(C)]. The smallest line pairs that could be resolved were group 7 element 1, which corresponds to a lateral resolution of 3.91 μm. This was found to be the same regardless of the amount of parallel lines in the illumination pattern. According to the well-known Rayleigh criterion, the optical resolution limit is defined as $r=0.61\lambda /\mathrm{NA}$, where $r$ is the minimum distance between resolvable points, $\lambda$ is the wavelength used, and NA is the numerical aperture of the (microscope) objective. The lateral resolution of the PLSO is limited by the camera sensor pixel size when the pupil size of the eye is $\ge 2\mathrm{mm}$, which has a theoretical resolution limit of 3.7 μm. This is in a good agreement with the obtained experimental data. If smaller pupil sizes are used, the resolution limit is determined by the numerical aperture of the (model) eye lens.

 

Fig. 2. Lateral resolution of the PLSO. The confocal image of the resolution target displayed on the left was produced using seven parallel lines. The lines in group 7 element 1 are visible with the naked eye (A), and when taking a cross section from the horizontal (B) and vertical line groups (C), three different peaks are resolved giving a resolution of 3.91 μm.

Download Full Size | PDF

The SNR performance with respect to the line pattern density was determined using the same model eye. An artificial retina was placed in the focal plane of the model eye, which consisted of a titanium dioxide (${\mathrm{TiO}}_2$) layer with a weight concentration of 0.5% and an attenuation coefficient of $10{\mathrm{mm}}^{-1}$ [7]. The ${\mathrm{TiO}}_2$ scattering properties were selected so that they mimic the scattering properties of the human eye. The artificial retina was imaged with different line pattern densities while keeping the total illumination power per line constant. A DMD fill factor for each scanning sequence was defined as the fraction of the DMD mirror elements in the on position when a line is projected (e.g., if 1/72 mirrors are on, the fill factor is 0.014, and this results in 7 parallel lines in the FOV). The fill factors were chosen in a way that there was no overlap between neighboring lines. Figure 3(A) shows an example of a single acquired frame (fill factor 0.021). Figure 3(B) shows cross sections of the back-scattered signals [illustrated as a purple dashed line in Fig. 3(A)] using different fill factors. The signal shape remained almost identical regardless of the fill factor used. The signal side lobes are caused by light scattering within the artificial retina, which broadens the captured image of the lines compared to their illumination line patterns. This broadening limits the maximum fill factor that can be used for which no overlap occurs between neighboring lines. From Fig. 3(B), the line width was estimated to be $\sim 30$ pixels at the base of the line, which results in a maximum fill factor that is given by the ratio of the theoretical line thickness and its broadened image as $3.6/30=0.12$.

 

Fig. 3. Determining the SNR of the PLSO. (A) An example frame (fill factor 0.021) showing how the background and signal were determined from the frames for ${\mathrm{SNR}}_e$ calculation. The spacing between neighboring signal lines is approximately 200 pixels. (B) From each fill factor sequence, one frame was taken and a cross section of the signal line was plotted. The signal peaks from different sequences were overlaid on top of each other to show the stability and shape of the signal slope. (C) Obtained mean signal (${\mu }_{e,\text{signal}}$) and mean background (${\mu }_{e,\text{backgr}\text{ound}}$) electrons plotted as a function of the DMD fill factor show a linear trend. The retinal signal is present on top of the background and has a constant value regardless of the DMD fill factor. (D) ${\mathrm{SNR}}_e$ plotted as a function of the DMD fill factor shows that the ${\mathrm{SNR}}_e$ of the system decreases when the fill factor increases (denser line pattern).

Download Full Size | PDF

To evaluate the SNR, values for the background and the retinal signal were determined for each fill factor using ten different frames. The arrows in Fig. 3(A) point out the signal coming back from the retina and the dashed boxes represent the areas from which the background signal was determined. These values are plotted for all fill factors in Fig. 3(C). The signal is plotted as the solid line while the dashed line displays the background signal. The separation between the two lines stays the same for all fill factors, which means that the retinal signal is constant, and only the background increases with the fill factor. More specifically, the retinal signal is present on top of the background signal, which is caused by the (corneal) scattering from the uncoated model eye lens. The background increases linearly as a function of the DMD fill factor, meaning that doubling the DMD fill factor (having a denser line pattern) doubles the amount of light entering the eye and thus doubles the background (corneal) haze. The SNR of the PLSO can be calculated using the number of electrons obtained from Fig. 3(B) in the signal and background regions and the definition of the SNR according to Eq. (11) of the European Machine Vision Association Standard 1288 [8]:

The use of the PSLO setup in vivo in humans was approved by the Institutional Review Board of the VU University Medical Center Amsterdam and adhered to the tenets of the Declaration of Helsinki. Informed consent was obtained from the participant.

To demonstrate the capabilities of the PLSO, images from a healthy patient were obtained. The time-averaged power at the cornea was measured to be 100 μW or lower. This value is well below the maximum permissible exposure (MPE) limit for 624 nm illumination indicated by the latest IEC standard 60825-1 [9]. To visualize the fovea and the optic nerve head (ONH), a fill factor of 0.014 was used, which consisted of seven parallel lines. The pattern projection speed was set to 100 Hz, resulting in a full image frame rate of 1.4 fps.

As a reference, Fig. 4(A) shows a standard fundus photo of the healthy participant. The locations that were imaged with the PLSO are shown as dashed boxes, centered at the fovea and the ONH. The acquired data were processed into nonconfocal images (by averaging all frames), as well as confocal images (by using the previously explained image-processing method) to show the advantage of the latter. In the nonconfocal images (Figs. 4(B) and 4(C)), the corneal scattering is dominant and makes the retinal structures covered in haze, rendering them almost invisible. In the foveal image (Fig. 4(B)), the light incident on the eye was not centered on the pupil, and at the right edge of the image (the dark portion), the corneal reflections were blocked by the apertures in the system. In Fig. 4(B), some of the blood vessels are seen as faint dark lines, while the strong scattering structures of the ONH (Fig. 4(C)) are more clearly seen, although they are still covered in the haze. In the confocal images (Figs. 4(D) and 4(E)), it is clear that confocality is drastically improved as well as the contrast when compared with the fundus photo or to the nonconfocal counterparts of the images. The foveal avascular zone and smaller blood vessels are visible in the fovea image (Fig. 4(D)). In Fig. 4(E), the quality of the ONH image is also drastically improved and structures can be distinguished. Some of the features are out of focus due to being at a different depth, which is caused by the central depression in ONH known as the optic cup. The visualization of the larger veins/arteries is enhanced greatly compared with the nonconfocal image.

 

Fig. 4. In vivo images of the retina from a healthy participant. (A) Fundus photo from the right eye, dashed boxes showing the areas imaged with the PLSO. (B) and (C) Nonconfocal PLSO images and (D) and (E) confocal PLSO images of the fovea/macula and optic nerve head, respectively. In (B) the light incident on the eye was not centered on the pupil, and at the right edge of the image (the dark portion), the corneal reflections were blocked by the apertures in the system. The images were acquired through a dark-adapted pupil without any dilatation. Scale bars are 2° in size.

Download Full Size | PDF

The main challenge is to increase the acquisition speed of the PLSO. The speed can be significantly increased by projecting more parallel lines, which increases the fill factor of the DMD. However, the corneal scattering increases as a function of the fill factor, as plotted in Fig. 3(C). To reduce the corneal haze, an annulus is a viable option [10]. By placing it in the conjugate image plane (the pupil plane), it would block the central portion of the illumination beam, thereby greatly reducing the corneal reflections and thus improve the SNR. Alternatively a higher camera frame rate could be used in combination with an annulus.

At the moment, the system is using visible light for illumination, namely a 624 nm LED. Even though the light levels are well below the MPE safety limit, the perceived brightness of the 624 nm wavelength can be worrisome to patients. Additionally, the pupil reacts strongly to bright visible light and constricts to limit the amount of light entering the eye. This may result in small pupil diameters and can cause beam clipping. This can be avoided by using a longer wavelength for which the eye is less sensitive.

In summary, the first results are shown from an experimental PLSO that uses a DMD to produce multiple line patterns to illuminate the retina. The acquired confocal images show better contrast and quality than traditional fundus photos without having to average multiple frames. An analysis was provided that describes the relation between the DMD fill factor and SNR of the retinal image. The PLSO is a very promising technology for visualization of the posterior part of the eye at high acquisition speeds with good resolution and contrast. This might be very helpful for the detection of pathological structural changes in the retina.