We introduce a new multi aperture system capable of capturing six identical images of the human fundus at six different spectral bands. The system is based on a lenslet array architecture and is well suited for spectroscopy application. The multi-aperture system was interfaced with a fundus camera to acquire spectroscopic sensitive images of the retina vessel and ultimately to calculate oxygen saturation in the retina in vivo. In vitro testing showed that the system is able to accurately reconstruct curves of partially oxygenated hemoglobin. In vivo testing on healthy volunteers was conducted and yielded results of oxygen saturation similar to the one reported in the literature, with arterial SO2~0.95 and venous SO2~0.5.
© 2008 Optical Society of America
Blood oxygenation levels in the retina can provide critical insight into early pathologic changes , as well as prove to be an important tool during critical care [2,3]. Venous oxygen saturation is related to the local oxygen metabolism while arterial oxygen saturation is connected to the patient respiratory function .
Noninvasive measurements of oxygen saturation in the retina are desirable for many different clinical applications; for example early changes in auto-regulation  and blood flow in the retina have been linked to the onset of diabetic retinopathy (DR). Various studies have suggested that detection and treatment of diabetic retinopathy and diabetic macular edema can significantly reduce the risk of visual loss . Lack of oxygenation and the resultant abnormal angiogenesis leads to loss of retinal tissue and resulting vision impairment . Although mechanisms for retinal damage in the late stages of detectable DR have been described, early changes, which lead to the onset of disease, are not well recognized.
Another useful application of retinal oxygen saturation is the measurement of central venous oxygen saturation as an index of oxygen delivery to vital organs such as the brain . Assessment of oxygen delivery cannot be gathered from blood pressure, pulse rate or other easily assessable vital signs. More invasive procedures for monitoring oxygen delivery, such as fiber optic sensors inserted into the heart and pulmonary artery , can be employed in a hospital environment but are not well suited for ambulatory medical care. The retina may prove to be the ideal site for monitoring blood oxygen levels since retinal vessels are easily accessible, provide a good central perfusion bed, and the perfusion levels are not sensitive to shock.
In the last forty years many groups have worked on instruments and algorithms targeted at oxygen saturation measurements in the retina. Unfortunately the layered structure of the eye, its many absorbing and scattering components, and its constant movement make this measurement particularly difficult. The first retinal oximeter was proposed by Hickam et al.  in 1963. Using a modified fundus camera, they imaged the retina at two different wavelengths and extracted the vessels optical density with the Beer-Lambert law. Pittman and Dulling  in 1975 showed that more accurate results of retinal oximetry could be achieved using three wavelengths instead of two. The scattering coefficient wavelength dependence was also taken into account in their model. Delori in 1988  used three wavelengths to calculate oxygen saturation; he used narrowly spaced wavelengths (558 nm, 569 nm, and 586 nm) so that the scattering coefficient could be considered constant. In 1995 Schweitzer et al.  built an instrument that could image the retina spectroscopically with selecting light source wavelengths from 400 nm to 700 nm in 2 nm intervals; an empirical scattering model was used in their calculations. Denninghoff et al.  used two diode lasers at 670 nm and 830 nm and Drewes and Smith  used four different lasers in their confocal system (629 nm, 679 nm, 821 nm, and 899 nm); their nonimaging technique scanned the retina vessels obtaining one-dimensional intensity transmission curves. From the four intensity curves the oxygen saturation was calculated. Recently Denninghoff et al.  have shown that their four-wavelength eye oximeter achieves accurate results. Their system called EOX (eye oximeter) was tested in a vitiligo swine during controlled hemorrhage and indicated a strong correlation between retinal venous oxygenation and total blood volume. Finally Johnson et al.  have used a hyperspectral imager combined with a fundus camera to obtain 50 retina images between 450 and 700 nm, this system like ours has no moving parts.
For an in depth review of instrumentation and models for retinal oximetry the reader is directed to reference 15.
In this paper we introduce a new type of multi spectral camera based on lenslet and filter arrays that is able to capture six spectrally sensitive images in one snapshot. The camera was interfaced with a fundus ophthalmoscope. The spatial division of the fundus image obtained with the lenslet array minimizes the effect of the eye movement since all the spectral data is collected at the same time, nevertheless saccades movement can still be an issue for long exposure times.
2. Material and methods
A schematic of the multi aperture system is shown in Fig. 1. The system is composed of a commercially available fundus ophthalmoscope (TRC-FET, Topcon Paramus, NJ) and a custom built multi-aperture camera. A 12 bit monochromatic digital camera (Lumenera, North Andover, MA, USA) was interfaced with a custom made lenslet array; the distance from the CCD to the lenslet array could be adjusted with a micro-positioning stage and was circa 0.5 mm. The CCD size was 10.2 mm×8.3 mm (1392 pixels×1040 pixels). A filter array was positioned in front of the lenslet array so that each image formed at a distinct wavelength. A focusing screen (Ritz Camera, Irvine, CA, USA) at 55 mm from the filter array was used to decrease the depth of field of the lenses and minimize the effect of the fundus camera light source. The focusing screen was also adjustable with a positioning stage and thus provided a crude magnification variation. The camera system was adapted to a fundus ophthalmoscope so that the image formed by the fundus was projected by the lenslet array into six identical images on the CCD, Fig. 2.
Although the focusing screen reduces the amount of light reaching the imager it has shown to be a very efficient way to reduce the depth of field of the multi-aperture camera without the addition of any other optical elements. This has the advantage of keeping the imager compact and reasonably light. The lenslet array was built with six plano-convex lenses (LightPath Optical Instrumentation, Shanghai, Cina) supported by a custom-made aluminum plate. The lenses were 2 mm in diameter and had a numerical aperture equal to 0.15 and an effective focal length of 5 mm. Lens to lens separation was 2.5 mm. The filters were 2.5×2.5 mm, narrow band filters (20 nm FWHM, Newport, Irvine, CA, USA); only the 575 nm filter was circular with a 3 mm diameter. Different combination of filter arrays were tested including the quintuplet [540, 560, 576, 600, 680] nm, and the sextuplets [560, 575, 600, 630, 650, 660] nm and [540, 560, 575, 600, 650, 660] nm. The choice of wavelengths was guided by several factors including previous publications [16, 17], Monte Carlo simulations of light travel into retinal tissue, and commercial availability (the typical 586 nm isosbestic wavelengths for example was not readily available in this small format). Several authors [11, 12, 13] have pointed out that the impact of melanin is lower for shorter wavelengths, and algorithms such as the one of Delori  works best in the 500–600 nm region. For this reason the multiaperture filters array included at least three wavelengths in this region. Longer wavelengths in the 600–700 nm range can be useful in establishing melanin concentration  hence some filters in this range were also included in the array. In the multi-aperture camera the filter arrays can be replaced easily, making the system suitable for evaluation of different models.
The filter array was positioned in front of the lens array so that the distance between each lens and corresponding filter was less than 0.2 mm. The only modification to the fundus ophthalmoscope apart from the connection to the multi aperture camera was the addition of two filters to the original light source; one filter was a high pass yellow filter with cutoff wavelength of 480 nm and the second filter was a low pass IR filter with cutoff wavelength 700 nm. A ray tracing of the multi aperture camera obtained with the software package Rayica (Optica Software, Champaign, IL) is shown in Fig. 2, the filters are simulated as square apertures. The distance between the lenslet array and the CCD (screen) is exaggerated here for clarity. The figure shows the projection of a letter F through the optical system.
2.1 System calibration
The multi aperture system was calibrated using three NIST traceable Spectralon standards (Labsphere, North Sutton, NH, USA) of different colors. During this test the reflection standards were located at the entrance pupil of the fundus system, in the same location where a patient would place his eye as noted in Fig. 1. Images of the standard were captured with the multi aperture system; the exposure time for all acquisition was 200 ms. A dark image was also captured keeping the light source off and with an exposure time of 200 ms; this image was subtracted from every image. The image of a white 80% reflectance standard was also captured (200 ms) and total reflectance was calculated as
Each image array contained 6 identical sub-images at 6 different wavelengths. In order to quantify the total reflectance from the color Spectralon, region of interest of 20×20 pixels were selected on each of the 6 sub-image. Results are shown in Fig. 3, and are compared with NIST traceable values. The correlation coefficients were R=0.997 for the red data, R=0.997 for the yellow data and R=0.983 for the green data.
The camera was also tested for linearity; in all the subsequent measures levels of camera exposure were kept within the linear range of the camera.
2.2 In vitro testing
All in vitro experiments were conducted on solutions of human hemoglobin (Sigma, St Louis, MO) and water. 10 mg of hemoglobin were diluted in 10 ml of DI water and thoroughly stirred. Two principal experimental layouts were considered. In the first transmission modality shown in Fig. 4, a 1 mm thick quartz cuvette filled with the hemoglobin solution was positioned at the exit pupil of the fundus ophthalmoscope inside the multi aperture camera.
A reflectance standard at the entrance pupil of the fundus camera reflected 99% of the incident light in the direction of the multi aperture camera. Reflected light traveled through the cuvette before reaching the lenslet array and the CCD. During this experiment the focusing screen was removed. Wavelength sensitive images of light transmission through the cuvette were captured and compared to measurement of absorption of the same cuvette obtained with a bench-top spectrophotometer (Ultrospec 3000, Pharmacia Biosystems, DK). Some results obtained with oxygenated hemoglobin (SO2=98%) are shown in Fig. 5.
This scenario is modeled with Beer’s Lambert law
Where T stands for transmission, s for oxygen saturation, L is the cuvette thickness, and µ aOxy and µ aDeOxy are the absorption coefficients for oxygenated and deoxygenated hemoglobin obtained from tabulated values  of hemoglobin.
The second experiment in reflection was conducted on a simplified eye model shown in Fig. 6. A similar eye model has been used previously by Drewes et al.  for calibration purposes. The eye model was constructed using a 5×5 cm black Delrin® case. A 150µm inner diameter micro-tube, that simulated a vessel in the retina, was positioned in front of a thick Spectralon slab (99% reflective) that represents the highly backscattering sclera. The dilated pupil was reproduced with a 6 mm hole in the front casing wall; finally a plano-convex lens on top of the pupil mimicked the crystalline lens of a real eye. The casing was filled with index matching fluid to minimize the eye to vessel interface and undesirable lensing effects and simulated the vitreous humor. The vessel was connected to a solenoid-actuated micro-pump.
The pump maintained a pulsed flow of 9 ml/min, a value close to human retinal flow. Oxygen content in vessel hemoglobin could be controlled through a small oxygen reservoir and flow-meter but was not used in these experiments. Instead, several reduced levels of hemoglobin were obtained adding different quantities of Sodium Hydrosulfite (Sigma, St. Louis, MO) to the hemoglobin solution; the experiment was repeated four times for each oxygen level. Calibration was achieved with a spectrometer connected to the vessels via fiber optic, that measured the absorbance of the re-circulating fluid. The eye model was located at the entrance pupil of the fundus system instead of a patient eye.
Images of light backscattered from the model eye were analyzed with the same process described above for the transmission experiment with the only difference that reflectance values on the vessel were normalized by reflectance of an area near the vessel. This particular normalization procedure is used in eye oximetry  to eliminate the effect of the background.
Two different models were used to obtain oxygen saturation values for both experiments. The first model was a three-wavelength algorithm proposed by Delori et al. .
Delori’s equation states that oxygen saturation in a retinal vessel can be calculated using tabulated values of extinction coefficient of oxygenated and deoxygenated hemoglobin as well as the experimentally obtained values of optical density in the retina at three different wavelengths. He introduces an operator RP relating the three optical densities Dλ1, Dλ2 and Dλ3 obtained at three different wavelengths (for Delori λ 1=569, λ 2=558 and λ 3=586).
Oxygen saturation is
D is the optical density obtained experimentally, and εHbO2 and εHb are tabulated values of extinction coefficient for oxygenated and deoxygenated hemoglobin . The table-top spectrophotometer results were also analyzed using this same model. Since Delori used two isobestic wavelengths (569 and 586) equation 4 simplifies as the second term in the denominator disappears, recently Smith has shown  that using isobestic wavelengths is not necessary mathematically; in our model we used two wavelength triplets λ 1=540, λ 2=560 and λ 3=577 nm and λ 1=540, λ 2=560 and λ 3=600. Wavelengths in the 500nm to 600nm range are often used in eye oximetry to minimize the effect of melanin absorption as we will show in the next section.
The second model was proposed originally by Schweitzer et al. ; this algorithm keeps into account not only the hemoglobin absorption but also the wavelength dependent scattering of erythrocytes. The optical density of light backscattered from a vessel is modeled as
where the first two terms B and nlog(1/λ) are used to simulate both wavelength independent and wavelength dependent scattering. The term b is an experimental geometry factor, ctot is the total hemoglobin concentration, l is the vessel thickness, and s is oxygen saturation.
A least-squares mechanism using the Nelder-Mead simplex method  and four fitting parameters (B, A=ctotlb, s, and n) was used to fit the model to the data. It is to be noted that this algorithm works best with large number of wavelengths. The only constraint to the model was that s and n had to be between 0 and 1. The spectra collected with the fiber-based spectrophotometer were also analyzed with the two models above. In Fig. 7 we show a comparison of the results obtained with the multi aperture camera and their respective calibration.
The set of experiments described in this section showed that the multi aperture camera is well suited for measuring oxygen saturation. We want to mention that a least square fit is not the best way to fit a non-linear model such as the one by Schweitzer et al. and that we are currently evaluating different techniques as suggested by Lompado . It is also important to note that this is a largely simplified model of a human eye since it does not consider the choroid and the RPE effect whose impact on the total remitted spectrum will be clarified in section 3. A study that includes this layers in underway .
2.3 Image registration
The registration of the 6 resulting images is achieved with the use of a IEEE full field resolution target (Edmund Optics Inc. Barrington, NJ). The target is illuminated with a white light source (Newport Stratford, CT) and images of the target are acquired with the multi-aperture camera. The camera to target distance is the same as the screen to camera distance shown in Fig. 1. Registration of the 6 resulting images is done manually, see Fig. 8, but more automatic models are currently being investigated.
Once the coordinates relative to the 6 registered images are obtained they can be used for registering the in vivo images of the eye.
3. Modeling considerations
The real difficulty of measuring oxygen saturation in the retina is in the choice of an algorithm or analyzing scheme that can capture the complex nature of this optical environment. The advantage of our system lies in its insensitivity to the eye movement artifacts and its capability of capturing images at six wavelengths. Generally published algorithms are either for 2 to 4 wavelengths such as the one by Delori  and Drewes  or use large data sets such as the one proposed by Schweitzer . Some authors  are also using Inverse Monte Carlo (IMC) models for this purpose. At present we have chosen to pursue algorithm type modeling instead of using IMC, that is computationally intensive and time consuming especially in an imaging environment such as ours but, we have used a forward Monte Carlo model quantify the effect of the various layers on the remitted light.
Peerce et al.  have shown that the critical parameters that impact measurement of oxygen saturation are, the hemoglobin concentration, the melanin variation in the choroids, and the density of melanin in the RPE so we restricted our investigation to these parameters too.
All Monte Carlo simulations were done using the program MCML ; the retinal layer absorption and scattering coefficient were sampled from the paper by Hammer et al.  using Matlab@. The Monte Carlo simulation was built with four layers: the neural retinal, the retinal pigmented epithelium (RPE), the choroids, and the sclera where the thicknesses of the layers were respectively 200µm, 10µm, 250 µm, and 700µm . The considered wavelengths were: 450, 480, 505, 515, 522, 540, 548, 560, 565, 569, 576, 586, 600, 610, 640, and 680 nm. The extinction coefficient for hemoglobin was reported by Takahani et al.  while the melanin absorption coefficient was from Sarna .
All the simulations were conducted with one million photons with an infinitesimally small beam. The effect of the choroids pigmentation and RPE thickness is shown Fig. 9. In this simulation the concentration of melanin in the choroids was increased in steps of 1, 10, and 50-fold, in different simulations the concentration of the RPE melanin was increased of the same amounts. The effect of melanin in the choroids is particularly visible at longer wavelengths and was reported by other authors [11, 15].
An increase in melanin concentration in the RPE has a uniform effect across the visible spectrum.
Other methods to minimize the effect of the melanin absorption were also investigated; for example it is a well know fact that the extinction coefficient of melanin decreases with increasing wavelength, and in the region between 550 nm and 600 nm this decay has a constant slope. Hemoglobin on the other hand has a high absorption up to 600 nm, while its effect in the 600 nm to 700 nm region is small. Some authors  have shown that when trying to calculate oxygen saturation in environments where melanin is present, the effect of melanin absorption can be reduced with a simple scheme. First the absorbance of the remitted light is calculated then a line is fit to the values of absorbance between 620 nm and 700 nm, then the corrected absorbance is calculated by subtracting the line from the original absorbance. Although this scheme is not appropriate for shorter wavelengths, since the extinction coefficient decreases more rapidly in different regions, it can be used in the ranges aforementioned. The oxygen saturation is finally measured with a two wavelengths algorithm. The implementation of this algorithm is shown in Fig. 10. Ideally, if all effect of the choroids and RPE melanin was eliminated, the curves should overlap at longer wavelengths. This is obviously not the case as shown in Fig. 10 for the choroids data. The same scheme was tested on the data obtained changing melanin concentration in the RPE; in this set of data we notice a higher overlap especially at longer wavelengths, this is to be expected since the RPE melanin has a more uniform effect on the total absorbance as shown in Fig. 9.
It is evident from these simulations that a simple layered model is not sufficient when describing the retina structure optically and that the cross-talk across layers has a strong impact on the final results.
The apparatus was tested on the eyes of healthy volunteers. A volunteer’s eye was dilated with Tropicamide Ophthalmic Solution (Akorn Inc, Buffalo Grove, IL), twenty minutes before testing. The imaging experiments were conducted in a very similar way to a clinical fundus exam; we did not use the fundus camera flash but simply used the system white light source as previously explained. Some typical results are shown in Fig. 11.
The camera is able to capture very large portions of the fundus, but due to the CCD size some of the images may be cropped when trying to maximize the imaging area. This issue can be easily solved with a larger CCD. Currently we are working with a 10.2×8.3mm size CCD, but 36.1×24.0mm CCD’s are already commercially available and we plan to use them in a phase 2 prototype.
Each of the images shows different features. Images collected in the green region of the spectrum (540, 560, 576) have high contrast in the vessel region due to the high hemoglobin absorbance, while in the image collected with the red filter (650 and 660 nm) the vessels are almost transparent. Differences in arterial and venous absorption are also noticeable. The white light image on the top-left corner is partially saturated. This image was used for image focusing and positioning and it was not used in the calculation of oxygen saturation.
After identifying arteries and veins on the images region of interest were selected on the vessel area (R vessel) and in regions just next to the vessel (R background), Figure 12 white lines. The pixel values captured on the vessel where then normalized by those next to it. Finally the optical density of the vessel region was calculated.
The resulting data were analyzed with the Delori’s model. The D image of the retina of a healthy volunteer is shown in the figure below. In the insert we show our calculation of oxygen saturation on a vein, values for this vessel seem to cluster between 45% to 50% oxygen saturation. In general SO2 values collected on arteries clustered around 95% while arteries values were close to 50%. Some errors in the assessment of SO2 in this particular vessel are clearly visible; this type of intra-vessel variability was present in most arteries and veins we analyzed. This variability is not uncommon  when dealing with reconstructed maps of SO2 due to lack of uniformity in the fundus layers and models limitations. The image shown in Fig. 12 is our first attempt in reconstructing maps of SO2 in the retina but further work remains to be done to generate more uniform maps as well as generalizing this concept to the complete fundus image. The map was built with images at 540 nm, 560 nm, and 580 nm.
We have introduced a new multi-aperture camera system for oxygen saturation measurement of the retinal vessel. The system is based on lenslet array architecture, has no moving parts, and can be interfaced with a fundus camera. 6 spectroscopic sensitive images are collected in a single snapshot so that the impact of the eye movement on the results is strongly reduced. The largest single image obtainable with the current system is 150 pixels×150 pixels but resolution could be improved by using a larger CCD. Image quality could also be improved by replacing the focusing screen currently used to reduce the depth of field of the lenslets, with a more appropriate optical layout.
The system can be easily modified to accommodate different sets of filters and other optical elements. For example polarizing optics could be included to reduce the effect of the eye melanin and to enhance the vessel to background contrast or to observe foveal birefringence.
In vitro tests have shown that the device is able to measure oxygen saturation in a simplified model eye. In vivo testing shows results similar to those published in the literature.
We gratefully acknowledge the support of the Wallace H. Coulter Foundation through the Early Career Award for Translational Research.
References and links
1. Q. D. Nguyen, S. M. Shah, E. van Anden, J. U. Sung, S. Vitale, and P. A. Campochiaro, “Supplemental oxygen improves diabetic macular edema: a pilot study,” Invest. Opthalmol. Visual Sci. 45, 617–624 (2004). [CrossRef]
2. K. R. Denninghoff, M. H. Smith, R. A. Chipman, L. W. Hillman, P. M. Jester, C. E. Hughes, F. Kuhn, and L. W. Rue, “Retinal large vessel oxygen saturation correlates with early blood loss and hypoxia in anesthetized swine,” J. Trauma. 43, 29–34 (1997). [CrossRef]
3. K. R. Denninghoff, M. H. Smith, A. Lompado, and L. W. Hillman, “Retinal venous oxygen saturation and cardiac output during controlled hemorrhage and resuscitation,” J. Appl. Phys. 94, 891–896 (2003).
4. A. Lompado, A confocal scanning laser ophthalmoscope for retinal vessel oximetry, PhD Thesis, University of Alabama, Huntsville (1999).
7. M. H. Smith, K. R. Denninghoff, L. W. Hillman, and R. A. Chipman, “Oxygen Saturation measurements of blood in retinal vessels during blood loss,” J. Biomed. Opt. 3, 296–303 (1998). [CrossRef]
8. J. C. Wo, W. C. Shoemaker, P. L. Appel, M. H. Bishop, H. B. Kram, and E. Hardin, “Unreliability of blood pressure and heart rate to evaluate cardiac output in emergency resuscitation and critical illness,” Crit. Care Med. 21, 218–223 (1993). [CrossRef]
9. J. B. Hickam, R. Frayser, and J. C. Ross, “A study of retinal venous blood oxygen saturation in human subjects by photographic means,” Circulation 27, 375–385 (1963). [PubMed]
10. R. N. Pittman and B. R. Duling, “A new method for the measurement of percent hemoglobin,” J. Appl. Phys. 38, 315–320 (1975).
12. D. Schweitzer, M. Hammer, J. Kraft, E. Thamm, E. Koenigsdoerffer, and J. Strobel, “Calibration-free measurement of the oxygen saturation in retinal vessel of men,” Proc. SPIE 2393, 210–218 (1995). [CrossRef]
13. J. J. Drewes, M. H. Smith, K. R. Denninghoff, and L. W. Hillman, “An instrument for the measurement of retinal vessel oxygen saturation,” Proc. SPIE 3591, 114–120 (1999). [CrossRef]
14. W. R. Johnson, D. W. Wilson, W. Fink, M. Humayun, and G. Bearman “Snapshot hyperspectral imaging in ophthalmology,” J. Biomed. Opt. 12, 14036–14043 (2007). [CrossRef]
15. A. Harris, L. Dinn, R. B. Kagemann, and E. Rechtman, “A review ofmethods for human retinal oximetry,” Ophtalmic Surg. Laser Imag. 34, 152–164 (2003).
16. M. Hammer, A. Roggan, D. Schweitzrt, and G. Muller, “Optical properties of ocular fundus tissues - an in vitro study using the double-integrating-sphere technique and inverse Monte Carlo simulation,” Phys. Med. Biol. 40, 963–978 (1995). [CrossRef]
18. J J. Drewes, Four-wavelength retinal vessel oximetry, PhD Thesis University of Alabama Huntsville.
19. M. Smith, “Optimum wavelength combinations for retina vessel oximetry,” Appl. Opt. 38, 258–267 (1999). [CrossRef]
20. S. Takatani and M. D. Graham, “Theoretical analysis of diffuse reflectance from a two-layer tissue model,” IEEE Trans. Biomed. Eng. 26, 656–664 (1987). [CrossRef]
21. J. A. Nelder and R. Mead, “A simplex method for function minimization,” Computer J. 7, 308–313 (1965).
22. A. Nabili, S. Mathews, and J. C. Ramella-Roman “Calibration of a retinal oximeter with a dynamic eye phantom,” Proc. SPIE (In preparation, 2008). [CrossRef]
24. H. M. Sarna, “The physical properties of melanins,” in The Pigmentary System, R. E. Nordlund, V. J. Hearing, R. A. King, and J. P. Ortonne, eds. (Oxford University Press, 1998), pp 439–450.