Ophthalmoscopes to image the retina are widely used diagnostic tools in ophthalmology and are vital for the early detection of many eye diseases. Although there are various effective optical implementations of ophthalmoscopes, new, robust systems may have a future practical importance in cases where ocular media present significant opacities. Here, we present, as a proof of concept, a novel approach for imaging the retina in real time using a single pixel detector combined with spatially coded illumination. Examples of retinal images in both artificial and real human eyes are presented for the first time to our knowledge.
© 2016 Optical Society of America
Modern flood-illuminated fundus cameras use a CCD or CMOS camera with an array of thousands of pixels to obtain an image, or video, of a part of the retina . Good image quality is achieved by illuminating the full area of interest homogenously. When aberrations are corrected with the use of adaptive optics, even detailed structures of the photoreceptor mosaic can be obtained [2,3]. On the other hand, a scanning laser ophthalmoscope (SLO) scans a light spot rapidly across the retina . The reflected intensity of each traversed point is recorded using a spot detector such as a photomultiplier tube (PMT)  or an avalanche photodiode (APD) . Subsequently, a high-resolution image of the retina is composed from the acquired data. Most recently, digital light ophthalmoscopes were developed using a digital micromirror device (DMD) to scan lines across the retina, which are imaged either on a line camera or on a two-dimensional (2D) detector array to obtain an image [7,8]. All the former described ophthalmoscopes are state-of-the-art devices and widely used as medical instruments for monitoring the condition and alteration of the eye fundus. Here, we introduce another imaging modality, the single pixel camera technique, to the field of ophthalmic imaging. The proposed system images the retina by combining a spatially coded illumination with a single pixel detector in a double-pass configuration. Single pixel imaging gained interest within the last decade and has been successfully applied in biological imaging and microscopy [9,10], infrared imaging [11–13], ultrasonic imaging , terahertz imaging , and three-dimensional (3D) imaging [16,17], and it has also been used in ghost imaging [18–20]. This type of system avoids the need of a scanning unit, which simplifies the optical system. The additional downscaling to a single pixel detector, instead of a pixel array, might lead to a lower illumination power, which benefits the patient’s comfort.
The system (Fig. 1) consists of a DMD (Vialux V7001, controller board V4395, chipset DLP 4100, pixel size: 13.68 μm, resolution: ). Each mirror pixel on the DMD can either guide light into the optical system or deflect it. The DMD is illuminated homogenously with a broadband xenon lamp (Hamamatsu L7810-02) with the UV part of the spectrum blocked. This light source guarantees a stable output over time, which is essential for discriminating minute intensity differences coming from the spatial coding of the later-described pattern.
The illumination is spatially coded by multiple 2D binary (black and white) patterns based on the Walsh–Hadamard transformation. These patterns are considered to have an optimum weighted design for spatial coding. The number of black and white pixels are equally distributed within each pattern and are preferred to others, e.g., pseudo-random binary patterns [18,21]. The so-called Hadamard patterns or masks are generated with an in-house developed software in C++ and sent to the DMD via a USB 3.0 connection into the DMD’s onboard memory. It is essential to pre-load all patterns before displaying them, as this is the only way to reach the maximum DMD frame rate of 22.727 kHz. Subsequently, the patterns sequence is initialized within an area of in the center of the DMD. The masks are scaled to match the size on the DMD, meaning e.g., for , each pixel of the pattern corresponds to 16 pixels on the DMD. Next, the patterns are imaged through the lenses L1, L2, L3, L4, and the eye’s optics onto the retina covering an area of around 15 deg of the visual field. The light intensity reflected from the retina is measured through the crystalline lenses, L5, L6, and L7 with a silicon photomultiplier (SIPM, Excelitas Lynx A-33-050-T1-A), the single pixel detector. This is synchronized with the DMD, which triggers the intensity measurement of each projected pattern individually. The diaphragm D1 is placed centrally on the back plane of lens L2, imaged via the telescope formed by lenses L3 and L4 onto the pupil plane, and it therefore defines the beams’ entry point. This configuration lowers the impact of aberrations on the image of the pattern and minimizes the area of possible backscattering and reflections. The data from the photomultiplier is transferred via an analog-to-digital converter (National Instruments PCIe-6361, sampling rate ) to a desktop PC (i5-4590, quad-core, 3.3 GHz, 8 GB RAM). The high sampling rate of the analog-to-digital converter (ADC) allows for 84 measurements per pattern. Not all data points are used for further processing, since data from the start of each pattern measurement is affected by minute mirror wiggling while the mirrors settle. The data received for each pattern are averaged and stored as per pattern. A full reconstruction of an image with the resolution needs the amount of patterns to be displayed, meaning for , . Since the mathematical description of the binary Hadamard pattern consists of negative and positive ones , and the DMD is only able to display zeros and positive ones , there is a need to display twice the number of patterns, resulting in (equals ) for . The patterns are displayed in the following order: a positive pattern (resembling patterns with 0 and ) and, subsequently, its inverted complement (patterns with and 0). Afterward, the data from the inverted masks are subtracted from the data of the positive masks, resulting in the final data corresponding to the correct mathematical model of the Hadamard patterns . With this displaying order, also used in other studies [9,10,22], noise is almost completely eliminated and the data complements the optimal reconstruction. One might argue that displaying only the positive patterns and subsequently subtracting the average of all intensity measurements would resemble the Hadamard patterns adequately, and, therefore, be sufficient to reconstruct the object, but that only works satisfactorily under optimum illumination conditions. In a low-light environment, noise induced by the photomultiplier, the light source, or the environment (Status LED, PC display) may have a severe impact on the image reconstruction . The reconstruction of the object is done by the use of the following equation:1) just transforms back to the spatial domain. Figure 2 shows the obtained intensity measurements using a model eye with a lens , , and an imprinted letter acting as a retinal surrogate after imaging with a resolution of and a total of 2048 patterns. Figure 2(a) shows the original object with a black square the size of and printed letters with four different grayscale values on a white letter. Figure 2(b) shows the image of the reconstructed object obtained with the maximum DMD frame rate and up-sampled to without interpolation for better viewing purposes. A detailed image reconstruction is already possible due to the object’s simple structure. Figure 2(c) displays the raw intensities, measured 84 times for each of the 2048 patterns. At this point, all the obtained measurements are positive. The final data are shown in Fig. 2(d), after the subtraction and averaging are done; consequently, the computed data contain negative values and vary around the zero baselines. Eventually, this set of data is used to reconstruct an image of the object, as depicted in Fig. 2(b), by performing Eq. (1).
Figure 3 shows additional results using a model eye but imaged with various resolutions, proofing the feasibility of the presented optical system. It is worth to mention that for , only all positive patterns (; all patterns would result in ) are shown due to the restrictions of the DMD’s memory, which cannot store more than 87380 binary patterns. In this case, the displaying time only doubles rather than quadrupling (compared to , see Table 1), but the reconstructed images are of lower quality, since the noise effects are not compensated effectively.
The images shown in Fig. 3 are reconstructed without any applied image post-processing except for , where contrast stretching is applied since the dynamic range was very narrow. The quality of the images could be further improved by lowering the frame rate of the DMD. That might be suitable for measurements with a static object, but during in vivo measurements, subtle eye movements would affect the image quality drastically, since the area of illumination needs to be constant. Furthermore, increasing the imaging time would be contrary to the ophthalmoscope’s desired real-time operation capability. One pixel in an reconstruction equals around 0.18 mm. Consequently, doubling results in halving the pixel size, meaning that when , one pixel equals 0.022 mm. The scale bar is indicated within Fig. 3.
We were able to record images and video streams with the proposed optical instrument. Table 1 shows an overview of the achieved frames per second (FPS) during the continuous acquiring and reconstruction of images when operating in real-time mode. The mathematical calculations to reconstruct an image are simple, but the frame rate drops fast with the increasing resolution not only because of the exponential growth of pattern’s size to be displayed, but also due to the vast amount of data that needs to be processed. Additionally, we compared our video rate to those obtained by other groups using the same imaging technique outside the field of retinal imaging.
The following results are obtained in living human eyes while using multiple frame rates of the DMD and the total amount of patterns. Figure 4(a) presents an initial unprocessed result taken in a young subject volunteer with an illumination time of 0.54 s and a resolution of . The images on the right in Fig. 4 show the average of 10 frames using the same settings as the left ones. Averaging was done without correcting for possible eye movements between each frame. Image post-processing is not applied, but the image is up-sampled to a size of . The main blood vessels, which join at the optical nerve head, can be distinguished clearly. Further details are not visible due to the lack of resolution and the presence of noise. Higher-resolution images obtained from the averaged video frames (taken from Visualization 1 for and Visualization 2 for ) are shown in Fig. 5.
Compared to the results shown in Fig. 4, more details of the vessel map around the optical nerve head are visible. The improvement is not as good as anticipated, which is due to the increased amount of time it needs to display the full set of patterns, and therefore, eye motions influence the outcome severely. If the subject moves, even subtlely, different areas of interest are illuminated, and this results in useless reflection intensity measurements. Furthermore, with the increase of resolution comes an increased level of noise. Images taken with a resolution of are not shown here due to the following reasons: the overall quality is low, as the imaging time for one single frame is around three seconds (at maximum speed) and, therefore, eye movements have a drastic effect on the reconstructed result. Second, with the current hardware, we are not able to project the inverted patterns, which are necessary to compensate for noise effects. As Figs. 4 and 5 show, an increase in resolution does not necessarily improve the image quality, but rather introduces high-frequency noise. This is mainly when finely structured patterns are not projected perfectly onto the retina (due to aberrations) and, therefore, introduce a light haze on the reconstructed image, as shown in Fig. 6. Nevertheless, the extended imaging time combined with a non-constant area of illumination, due to the locomotion of the eye, is the most challenging issue. To reduce the influence of motion artifacts, it is necessary to keep the imaging time as short as possible. For that reason, it is possible to remove masks from the display sequence, since some masks do not provide significant information. The technique of displaying less patterns is known as adaptive imaging or evolutionary compressive sensing [10,17]. To gain knowledge about the patterns that can be eliminated, a full set of masks needs to be displayed prior to classification. Subsequently, the obtained intensity data are sorted by means of highest responsivity and then loaded into the memory of the DMD to provide instant availability.
Afterward, a threshold is chosen to separate the patterns into useful and dispensable. From there, we are left with two options to gain an advantage of the lower amount of patterns:
- (i) Increasing the illumination time per pattern, , hence gaining a longer integration time for the detector area but maintaining the same overall illumination time, , which is not desired.
- (ii) Keeping the illumination time per pattern, , constant, results in a lower imaging time per frame.
Figure 6 shows results of the latter adaptive imaging configuration for 100, 75, 50, 30, 25, and 10% of patterns in the model eye setup and a resolution of . As can be seen, displaying less than 30% of the patterns is sufficient for a good reconstruction of the object but highly dependent on the object’s structure and its grade of detail. With fewer patterns displayed, the haze (high-frequency noise) in the reconstructed images disappears and the real-time video frame rate increases significantly. The imaging times per frame are indicated within the images. Another technique that is commonly used with a single pixel detector and a reduced amount of patterns is compressive sensing (CS) [12,16,21,24,25]. As it is based on a time-consuming statistical algorithm, we omit CS for the moment as we put our focus on a real-time optical instrument.
In conclusion, we described a novel instrument, based in a single-pixel detection, for imaging the fundus of an eye in vivo within an area of almost 15 visual degrees. Images and videos taken in a static artificial eye demonstrate its feasibility and capability, while imaging in vivo remains challenging mainly due to unintentional eye movements during extended illumination times. In terms of image quality and resolution, the obtained results cannot compete with comparable ophthalmic imaging instruments, since the main limitation is the maximum frame rate of the DMD. Future technology improvements and customized hardware might close that gap. In addition, one can expect that this type of single-pixel ophthalmoscope could operate under a larger range of eye conditions, such as increased ocular aberrations or scatter , as in the case of cataract patients . Other potential advantages would be to obtain retinal images in spectral bands beyond the current camera technology limits.
European Research Council (ERC) (ERC-2013-AdG-339228 (SEECAT)); Secretaría de Estado de Investigación, Desarrollo e Innovación (SEIDI), Spain (FIS2013-41237-R, FIS2013-40666-P).
1. R. D. Moore and G. W. Hopkins, “CCD camera and method for fundus imaging,” U.S. patent 5,140,352 (August 18, 1992).
2. J. Liang, D. R. Williams, and D. T. Miller, J. Opt. Soc. Am. A 14, 2884 (1997). [CrossRef]
3. S. S. Choi, N. Doble, J. L. Hardy, S. M. Jones, J. L. Keltner, S. S. Olivier, and J. S. Werner, Invest. Ophthalmol. Vis. Sci. 47, 2080 (2006). [CrossRef]
4. R. H. Webb, G. W. Hughes, and F. C. Delori, Appl. Opt. 26, 1492 (1987). [CrossRef]
5. A. Roorda, F. Romero-Borja, W. Donnelly III, H. Queener, T. J. Hebert, and M. C. W. Campbell, Opt. Express 10, 405 (2002). [CrossRef]
6. B. Vohnsen and D. Rativa, Biomed. Opt. Express 2, 1597 (2011). [CrossRef]
7. M. S. Muller, A. E. Elsner, and G. Y. Ozawa, Proc. SPIE 8567, 85670Y (2013). [CrossRef]
8. K. V. Vienola, M. Damodaran, B. Braaf, K. A. Vermeer, and J. F. de Boer, Opt. Lett. 40, 5335 (2015). [CrossRef]
9. E. Tajahuerce, V. Durán, P. Clemente, E. Irles, F. Soldevila, P. Andrés, and J. Lancis, Opt. Express 22, 16945 (2014). [CrossRef]
10. N. Radwell, K. J. Mitchell, G. M. Gibson, M. P. Edgar, R. Bowman, and M. J. Padgett, Optica 1, 285 (2014). [CrossRef]
11. M. P. Edgar, G. M. Gibson, R. W. Bowman, B. Sun, N. Radwell, K. J. Mitchell, S. S. Welsh, and M. J. Padgett, Sci. Rep. 5, 10669 (2015). [CrossRef]
12. L. McMackin, M. A. Herman, B. Chatterjee, and M. Weldon, Proc. SPIE 8353, 835303 (2012).
13. H. Peng, Z. Yang, D. Li, and L. Wu, Proc. SPIE 9795, 97952O (2015). [CrossRef]
14. N. Huynh, E. Zhang, M. Betcke, S. Arridge, P. Beard, and B. Cox, Optica 3, 26 (2016). [CrossRef]
15. R. I. Stantchev, B. Sun, S. M. Hornett, P. A. Hobson, G. M. Gibson, M. J. Padgett, and E. Hendry, Sci. Adv. 2, e1600190 (2016). [CrossRef]
16. W.-K. Yu, X.-R. Yao, X.-F. Liu, L.-Z. Li, and G.-J. Zhai, Appl. Opt. 54, 363 (2015). [CrossRef]
17. M.-J. Sun, M. P. Edgar, G. M. Gibson, B. Sun, N. Radwell, R. Lamb, and M. J. Padgett, Nat. Commun. 7, 12010 (2016). [CrossRef]
18. J. H. Shapiro, Phys. Rev. A 78, 1 (2008). [CrossRef]
19. Y. Bromberg, O. Katz, and Y. Silberberg, Phys. Rev. A 79, 053840 (2009). [CrossRef]
20. B. Sun, S. S. Welsh, M. P. Edgar, J. H. Shapiro, and M. J. Padgett, Opt. Express 20, 16892 (2012). [CrossRef]
21. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, IEEE Signal Process. Mag. 25(2), 83 (2008). [CrossRef]
22. M. Sun, M. P. Edgar, D. B. Phillips, G. M. Gibson, and M. J. Padgett, “Infrared single-pixel imaging utilising microscanning,” arXiv:1511.02637 (2015).
23. L. Streeter, G. R. Burling-Claridge, M. J. Cree, and R. Künnemeyer, Appl. Opt. 48, 2078 (2009). [CrossRef]
24. C. M. Watts, D. Shrekenhamer, J. Montoya, G. Lipworth, J. Hunt, T. Sleasman, S. Krishna, D. R. Smith, and W. J. Padilla, Nat. Photonics 8, 605 (2014). [CrossRef]
25. V. Duran, F. Soldevila, E. Irles, P. Clemente, E. Tajahuerce, P. Andres, and J. Lancis, Opt. Express 23, 14424 (2015). [CrossRef]
26. P. Artal, Adv. Opt. Photonics 6, 340 (2014). [CrossRef]
27. P. Artal, A. Benito, G. M. Pérez, E. Alcón, Á. de Casas, J. Pujol, and J. M. Marín, PLoS One 6, e1682 (2011). [CrossRef]