Open Access
Add to BibSonomy
Abstract
Optical coherence tomography angiography (OCTA) has been increasingly used in the analysis of ophthalmic diseases in recent years. Automatic vessel segmentation in 2D OCTA projection images is commonly used in clinical practice. However, OCTA provides a 3D volume of the retinal blood vessels with rich spatial distribution information, and it is incomplete to segment retinal vessels only in 2D projection images. Here, considering that it is difficult to manually label 3D vessels, we introduce a 3D vessel segmentation and reconstruction method for OCTA images with only 2D vessel labels. We implemented 3D vessel segmentation in the OCTA volume using a specially trained 2D vessel segmentation model. The 3D vessel segmentation results are further used to calculate 3D vessel parameters and perform 3D reconstruction. The experimental results on the public dataset OCTA-500 demonstrate that 3D vessel parameters have higher sensitivity to vascular alteration than 2D vessel parameters, which makes it meaningful for clinical analysis. The 3D vessel reconstruction provides vascular visualization in different retinal layers that can be used to monitor the development of retinal diseases. Finally, we also illustrate the use of 3D reconstruction results to determine the relationship between the location of arteries and veins.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Optical coherence tomography angiography (OCTA) is a noninvasive imaging modality that allows visualization of retinal vasculature with micron-level resolution, and it has been increasingly used in the analysis of various eye diseases, such as age-related macular degeneration (AMD), diabetic retinopathy (DR), and glaucoma [1–3]. Quantitative studies of OCTA images usually require the segmentation of retinal vessels. Several studies [4–11] have focused on segmenting retinal vessels in en face OCTA images (2D projection images, as shown in Fig. 1(a)), where supervised deep learning methods have achieved impressive segmentation performance. The recently released OCTA-500 [12] dataset provides sufficient samples and mature 2D vessel annotations, as shown in Fig. 1(b), allowing the training of deep learning models for 2D vessel segmentation tasks.
Fig. 1. A sample of (a) 2D en face OCTA image, (b) vessel annotation in OCTA-500 dataset, (c) 3D OCTA volume, (d) 3D vessel segmentation, (e) 3D vessel reconstruction.
2D projection images are commonly utilized in clinical evaluations due to their simplicity and readability. However, retinal blood vessels are inherently 3D structures with complex spatial and volumetric characteristics, and evaluating blood flow solely on two-dimensional projections is inadequate. OCTA provides 3D angiography volumes, as shown in Fig. 1(c), enabling a comprehensive three-dimensional analysis of blood flow. 3D vessel segmentation and reconstruction can be more intuitive for observing vascular morphology, as shown in Figs. 1(d)(e), and may be more sensitive to microvascular changes. Up to now, few studies have focused on 3D vessel segmentation in OCTA images, and the lack of 3D vessel annotation is one of the important reasons. Manual annotation of all vessels (including arterioles and capillaries) in 2D OCTA images is extremely time-consuming, while manual vessel annotation in 3D OCTA volumes is even more difficult to implement.
Here, we aim to share a simple and feasible 3D vessel segmentation and reconstruction method for OCTA images. The 3D vessel segmentation results are further used to calculate 3D vessel parameters and perform 3D reconstruction. The main contributions of this paper include: (1) We introduce a novel 3D vessel segmentation and reconstruction method that relies solely on 2D vessel annotations. (2) The 3D vessel parameters obtained from 3D vessel segmentation show higher sensitivity to vascular alteration than 2D vessel parameters, which makes it meaningful for clinical analysis. (3) The state-of-the-art 3D vessel reconstruction results can be used to distinguish the position relationship of arteries and veins at intersection points.
2. Materials and methods
The proposed method consists of three phases: 2D segmentation, 3D segmentation, 3D reconstruction, as shown in Fig. 2. First, a 2D vessel segmentation model is trained using 2D vessel annotation supervision. Then, the 2D vessel segmentation model is used for the OCTA C-scan images to obtain the serialized 2D vessel segmentation, and the 2D vessel segmentation sequences are merged to obtain the 3D vessel segmentation. Finally, the 3D vessel segmentations of the different retinal layers are reconstructed as 3D vessel models.
Fig. 2. Visualization process of retinal vessels in OCTA images. It comprises of three phases: 2D segmentation, 3D segmentation, 3D reconstruction.
2.1 2D vessel segmentation
We first need to train a 2D vessel segmentation model that can accurately segment the blood vessels in OCTA C-scan images. Many recent works focus on 2D vessel segmentation in OCTA projection images, including large vessel segmentation [8], capillary segmentation [9], and arteriovenous segmentation [7], etc. In this work, we trained the segmentation model under 2D full-vessel supervision (including large vessel and capillary). The trained model is like a high-performance vessel filter, which allows vessel segmentation in the adjusted C-scan images.
To train the required 2D segmentation model, we adopted the classical U-Net [13] as a 2D segmentation network. In the training phase, the inner-retinal projection images are taken as input and the full-vessel annotation is the supervision, and a cross-entropy loss function is used for supervised training. The projection images and the full-vessel annotations are provided by the OCTA-500 dataset [12]. The trained model can be used for the following 3D vessel segmentation.
2.2 3D vessel segmentation
A 3D OCTA volume can be sliced into a series of C-scan images along the projection direction. Since there is a significant brightness difference between the C-scan images and the projection images, a correction factor $\alpha $ ($\mathrm{\alpha \; =\; 2}$) is multiplied onto the C-scan images to keep their brightness close to that of the projection images. The adjusted C-scan images are used as the input of the trained 2D vessel segmentation model for testing, and the serialized 2D vessel segmentation results can be obtained. By merging the 2D vessel segmentation results, we can obtain 3D vessel segmentation results for the whole 3D OCTA volume.
2.3 3D Vessel reconstruction
The 3D vessel reconstruction process may be treated as a process of modeling 3D vessels from 3D vessel segmentation. 3D vessel segmentation results consist of inner retinal vessels (including superficial retina and deep retina), outer retinal vessel artifacts, and choroidal vessels. We extracted the vessels in superficial retina and deep retina via layer segmentation information. The superficial retina was set between the internal limiting membrane and the inner plexiform layer; The deep retina was set between the inner plexiform layer and the outer plexiform layer. Outer retinal vessel artifacts and indistinct choroidal vessels were excluded. Then, the volume rendering process in Amira software [14] is applied to render the superficial and deep retinal 3D vascular structures separately. Finally, joint visualization of superficial and deep retinal vasculature can be achieved by merging the 3D reconstruction results. The pseudo color helps distinguish vessels in different layers, purple for superficial vessels and green for deep vessels.
2.4 Evaluation Study
The publicly available dataset OCTA-500 [12] was used to implement the proposed vessel segmentation and reconstruction methods. It contains a total of 500 subjects divided into two subsets according to the field of view (FOV): OCTA-6 mm includes 300 subjects (NO. 10001 - NO. 10300), and its imaging range is 6 mm × 6 mm × 2 mm centered on the fovea, which corresponds to a volume size of 400 × 400 × 640 voxels; OCTA-3 mm includes 200 subjects (NO. 10301 - NO. 10500), and its imaging range is 3 mm × 3 mm × 2 mm centered on the fovea, which corresponds to a volume size of 304 × 304 × 640 voxels; The data were collected using a commercial 70 kHz SD-OCT (RTVue-XR, Optovue, CA). The 3D OCTA volumes, inner-retinal projection images, disease labels, vessel annotations, and retinal layer segmentation annotations provided by the OCTA-500 dataset were used in this paper. To more accurately segment vessels across different layers of the retina, we used the projection-resolved algorithm [23] to remove the projection artifact from the OCTA volumes.
Vessel density is one of the commonly used metrics to measure the retinal vascular plexus [15], and it is expressed as:
where A represents the pixels registered as vessel region, and N represents the number of pixels in the entire region. In this paper, we calculated the 2D and 3D vessel density of the inner retina based on 2D and 3D vessel segmentation, respectively. The linear regression to find the correlation between 2D vessel density and 3D vessel density. To compare the 2D vessel density and 3D vessel density in normal and disease eyes, the independent samples t-test was performed. It is considered statistically significant when a P value < 0.05. The smaller P value indicates a higher significance, which means that the parameter is more sensitive to the changes of disease.Due to the absence of 3D vessel annotation, we projected our 3D vessel segmentation results onto 2D and evaluated the segmentation performance using 2D vessel annotation from OCTA-500 dataset. We compared our results with several common vessel enhancement and segmentation methods, including the Otsu method [16], Frangi filter [17], Gabor filter [18], and SCIRD-TS [19]. To quantitatively assess the segmentation performance, we considered various metrics such as Dice coefficient, IOU (Intersection over Union), accuracy, and a CAL [20] (Connectivity, Area, and Length) indicator specifically used for evaluating vessel length, area, and connectivity.
3. Results
3.1 Statistical comparison between 2D vessel density and 3D vessel density
Totally 300 eyes in OCTA-6 mm with control (91) and disease (209) were included in this study. Eyes with AMD (43) and DR (35) in the disease group were also considered separately. The 2D vessel density and 3D vessel density in different groups are listed in Table 1, and it shows that the 3D vessel density is numerically smaller than the 2D vessel density. Figure 3 shows the correlation between 2D vessel density and 3D vessel density, which indicates that 2D vessel density and 3D vessel density (R = 0.8736, P < 0.001) has a positively linear correlation.
Table 1. Comparison of 3D vessel density and 2D vessel density in different groups
We also investigated which of the 2D vessel density and 3D vessel density could more sensitively distinguish between normal and disease, as shown in Table 1. The P-value order of magnitude for 2D vessel density is 10−3 in all comparison groups, whereas that for 3D vessel density is 10−5. It indicates that the 3D vessel density has a stronger significant difference than 2D vessel density.
3.2 Comparison of different vessel enhancement and segmentation methods
We quantitatively compared different vessel enhancement and segmentation methods using 2D vessel segmentation labels given in the OCTA-500 dataset. Table 2 presents the quantitative evaluation results of different methods, which show that our method has better segmentation performance than other threshold and filtering methods. Figure 4 shows examples of vessel enhancement and segmentation using those methods, which indicates that the proposed 3D segmentation can project a good 2D vessel map with less noise and clearer vascular morphology.
Fig. 4. Examples of vessel enhancement and segmentation using different methods. (a) OCT projection map. (b) Ostu. (c) Frangi filter. (d) Gabor filter. (e) SCIRD-TS. (f) Proposed (projection). (g) Ground truth.
Table 2. Quantitative comparison of segmentation performance on 2D vessel labels
3.3 3D visualization of retinal vessels in different diseases
Figure 5 shows two 3D reconstruction cases from normal people and DR patient, respectively. Figure 6 gives more examples of 3D full-vessel reconstruction results using the proposed method, which uses 2D full-vessel annotations in the OCTA-500 dataset as the supervision. Figure 7 shows examples of 3D reconstruction results of large vessels using the proposed method, which uses the 2D large vessel annotations in the OCTA-500 dataset as the supervision. Figure 8 shows two 3D reconstruction cases from normal people and DR patient, respectively. This AMD case has macular edema and the OCT volume shows arching of the retinal layers due to edema, while the shape of macular fovea in normal people is concave. The differences in spatial shape cannot be observed in the 2D projection image due to the loss of height information. From the 3D reconstruction result, it is easy to see that the vessels near the fovea bulged with macular edema.
Fig. 5. 3D retinal vessel reconstruction results from eyes of (a) normal population and (b) DR patient.
Fig. 6. More examples of 3D full-vessel reconstruction using the proposed method. Purple represents superficial retinal vessels, and green represents deep retinal vessels.
Fig. 7. 3D reconstruction results of large vessels using the proposed method. It uses the 2D large vessel annotations in OCTA-500 as supervision. AMD, age-related macular degeneration; ERM, epiretinal membrane; DR, diabetic retinopathy; RP, retinitis pigmentosa; RVO, retinal vein occlusion.
Fig. 8. Visualization of 3D vessel reconstruction results from eyes of (a) normal population and (b) AMD patient.
3.4 Application in differentiating arteriovenous locations
Several studies [7,21,22] have identified and segmented arteries and veins in 2D OCTA projection images. However, the lack of depth information in 2D projection images has led to a general oversight regarding the detailed examination at the intersections of arteries and veins in these works. As a result, the spatial relationship between arteries and veins has been rather arbitrarily defined, with assumptions such as arteries being consistently positioned above veins. This approach represents an oversimplification and does not match the actual situation. Figure 9 shows that our 3D vessel reconstruction can be used to guide the labeling of arteries and veins at the intersections. Figure 9(a) is a 2D OCTA projection image, and Fig. 9(b) is the artery and vein annotations based on Fig. 9(a), where there are mistakes at the artery and vein intersections. Figure 9(c) is the corresponding 3D vessel reconstruction results, where the two local regions are zoomed in.
Fig. 9. Using 3D vessel reconstruction to guide the labeling of arteries and veins at intersections: (a) 2D OCTA projection image. (b) Artery and vein annotations with red representing arteries and blue representing veins. (c) 3D vessel reconstruction results.
4. Discussion
In this study, we present a qualitatively and quantitatively analysis method of 3D retinal vessels in OCTA images by reconstructing 3D vessels from 2D labels. The 3D retinal vessel reconstruction results can visualize the retinal vasculature better than 2D OCTA projection. Table 1 shows that the 3D vessel density has a better distinguish ability for normal and disease than 2D vessel density, which is useful for the clinical diagnosis.
The proposed 3D vessel reconstruction allows for a more direct visualization of the superficial and deep retinal vasculature and can providing evidence for clinical disease analysis. By comparing the two cases in Fig. 5, we can intuitively see that the vessel density in the DR patient is significantly lower than that in the normal people. From the 3D reconstruction result of DR case, we can find that there are many aberrations at the ends of the vessels (Fig. 5, yellow box), and the 3D reconstruction result can clearly show the spatial shape of these aberrations. Figures 5–7 indicate that the proposed method is suitable for multiple types of 3D vessel reconstruction tasks. In addition, the proposed 3D vessel reconstruction can be used to observe the spatial position of retinal vessels, as shown in Fig. 8.
Recent studies [7,12] have focused on artery-vein segmentation in OCTA images. OCTA-500 [12] provides independent artery and vein annotation. However, when combining artery segmentation and vein segmentation, it is difficult to determine whether the artery is on top of the vein or the vein is on top of the artery only by 2D projection images, and the labeling of arteries and veins at intersections are usually inaccurate as shown in Fig. 9(b). This question can be solved based on the 3D reconstruction results in this paper, as shown in Fig. 9(c). Compared to 2D projection, 3D reconstruction results have depth information, allowing observation of the relative height of vessels at their intersections, thus accurately determining the relationship between arterial and venous locations.
5. Conclusion
In this paper, we introduced a 3D vessel segmentation and reconstruction scheme for OCTA images, using only 2D vessel annotations. The 3D vessel segmentation results are further used to calculate 3D vessel parameters and perform 3D reconstruction. The experimental results demonstrate that 3D parameters are more valuable for clinical analysis than 2D parameters. The 3D vessel reconstruction results provide vascular visualization in different retinal layers that can be used to monitor the development of retinal diseases. We also demonstrate the use of 3D reconstruction results to determine the relationship between the location of arteries and veins. We hope that the potential of 3D reconstruction in OCTA images will be further explored in the future.
Funding
National Natural Science Foundation of China (62172223, 61671242); Major Research Plan of the National Natural Science Foundation of China (92370109); Jiangsu Provincial Key Research and Development Program-Social Development (BE2023777); Key Medical Research Project of Jiangsu Commission of Health (H2022185).
Disclosures
The authors declare no conflicts of interest.
Data availability
The datasets used by the segmentation are publicly available: OCTA-500: [24].
References
1. R. Perrott-Reynolds, R. Cann, N. Cronbach, et al., “The diagnostic accuracy of OCT angiography in naive and treated neovascular age related macular degeneration: a review,” Eye 33(2), 274–282 (2019). [CrossRef]
2. Z. Sun, D. Yang, Z. Tang, et al., “Optical coherence tomography angiography in diabetic retinopathy: an updated review,” Eye 35(1), 149–161 (2021). [CrossRef]
3. L. V. Melkebeke, J. B. Breda, M. Huygens, et al., “Optical coherence tomography angiography in glaucoma: a review,” Ophthalmic Res. 60(3), 139–151 (2018). [CrossRef]
4. J. Lo, M. Heisler, V. Vanzan, et al., “Microvasculature segmentation and intercapillary area quantification of the deep vascular complex using transfer learning,” Trans. Vis. Sci. Tech. 9(2), 1–12 (2020). [CrossRef]
5. L. Mou, Y. Zhao, L. Chen, et al., “CS-Net: Channel and spatial attention network for curvilinear structure segmentation,” In MICCAI 2019, LNCS 11764, 721–730 (2019).
6. T. Pissas, E. Bloch, M. J. Cardoso, et al., “Deep iterative vessel segmentation in OCT angiography,” Biomed. Opt. Express 11(5), 2490–2510 (2020). [CrossRef]
7. M. Alam, D. Le, T. Son, et al., “AV-Net: Deep learning for fully automated artery-vein classification in optical coherence tomography angiography,” Biomed. Opt. Express 11(9), 5249–5257 (2020). [CrossRef]
8. M. Li, Y. Chen, Z. Ji, et al., “Image projection network: 3D to 2D image segmentation in OCTA images,” IEEE Trans. Med. Imaging 39(11), 3343–3354 (2020). [CrossRef]
9. Y. Liu, A. Carass, L. Zuo, et al., “Disentangled representation learning for OCTA vessel segmentation with limited training data,” IEEE Trans. Med. Imaging 41(12), 3686–3698 (2022). [CrossRef]
10. Y. Giarratano, E. Bianchi, C. Gray, et al., “Automated segmentation of optical coherence tomography angiography images: benchmark data and clinically relevant metrics,” Trans. Vis. Sci. Tech. 9(13), 1–10 (2020). [CrossRef]
11. Y. Ma, H. Hao, H. Fu, et al., “ROSE: A retinal OCT-Angiography vessel segmentation dataset and new model,” IEEE Trans. Med. Imaging 40(3), 928–939 (2020). [CrossRef]
12. M. Li, K. Huang, Q. Xu, et al., “OCTA-500: a retinal dataset for optical coherence tomography angiography study,” arXiv, arXiv:2012.07261 (2022). [CrossRef]
13. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” In MICCAI 2015, LNCS 9351, 234–241 (2015).
14. D. Stalling, M. Westerhoff, and H. Hege, “Amira: a highly interactive system for visual data analysis,” Visualization Handbook 27, 749–767 (2005). [CrossRef]
15. C. Lavia, S. Bonnin, M. Maule, et al., “Vessel density of superficial, intermediate, and deep capillary plexuses using optical coherence tomography angiography,” Retina 39(2), 247–258 (2019). [CrossRef]
16. N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern. 9(1), 62–66 (2007). [CrossRef]
17. R. F. Frangi, W. J. Niessen, K. L. Vincken, et al., “Multiscale vessel enhancement filtering,” Medical Image Computing and Computer Assisted Intervention MICCAI’98, Lecture Notes in Computer Science 1496, 130–137 (1998).
18. J. V. Soares, J. J. Leandro, R. M. Cesar, et al., “Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification,” IEEE Trans. Med. Imaging 25(9), 1214–1222 (2006). [CrossRef]
19. R. Annunziata and E. Trucco, “Accelerating Convolutional Sparse Coding for Curvilinear Structures Segmentation by Refining SCIRD-TS Filter Banks,” IEEE Trans. Med. Imaging 35(11), 2381–2392 (2016). [CrossRef]
20. M. E. Gegundez-Arias, A. Aquino, J. M. Bravo, et al., “A function for quality evaluation of retinal vessel segmentations,” IEEE Trans. Med. Imaging 31(2), 231–239 (2012). [CrossRef]
21. M. Alam, J. I. Lim, D. Toslak, et al., “Differential artery-vein analysis improves the performance of octa staging of sickle cell retinopathy,” Trans. Vis. Sci. Tech. 8(2), 3 (2019). [CrossRef]
22. M. Abtahi, D. Le, J. I. Lim, et al., “MF-AV-Net: an open-source deep learning network with multimodal fusion options for artery-vein segmentation in OCT angiography,” Biomed. Opt. Express 13(9), 4870–4888 (2022). [CrossRef]
23. M. Zhang, T. S. Hwang, J. P. Campbell, et al., “Projection-resolved optical coherence tomographic angiography,” Biomed. Opt. Express 7(3), 816–828 (2016). [CrossRef]
24. M. Li, Y. Chen, S. Yuan, et al., “OCTA-500,” IEEE (2023), https://ieee-dataport.org/open-access/octa-500.
