Nonlinear optical (NLO) microscopy based, e.g., on coherent anti-Stokes Raman scattering (CARS) or two-photon-excited fluorescence (TPEF) is a fast label-free imaging technique, with a great potential for biomedical applications. However, NLO microscopy as a diagnostic tool is still in its infancy; there is a lack of robust and durable nuclei segmentation methods capable of accurate image processing in cases of variable image contrast, nuclear density, and type of investigated tissue. Nonetheless, such algorithms specifically adapted to NLO microscopy present one prerequisite for the technology to be routinely used, e.g., in pathology or intraoperatively for surgical guidance. In this paper, we compare the applicability of different seeding and boundary detection methods to NLO microscopic images in order to develop an optimal seeding-based approach capable of accurate segmentation of both TPEF and CARS images. Among different methods, the Laplacian of Gaussian filter showed the best accuracy for the seeding of the image, while a modified seeded watershed segmentation was the most accurate in the task of boundary detection. The resulting combination of these methods followed by the verification of the detected nuclei performs high average sensitivity and specificity when applied to various types of NLO microscopy images.
Steve Bégin, Olivier Dupont-Therrien, Erik Bélanger, Amy Daradich, Sophie Laffray, Yves De Koninck, and Daniel C. Côté Biomed. Opt. Express 5(12) 4145-4161 (2014)
Piero Rangel-Fonseca, Armando Gómez-Vieyra, Daniel Malacara-Hernández, Mario C. Wilson, David R. Williams, and Ethan A. Rossi J. Opt. Soc. Am. A 30(12) 2595-2604 (2013)
Xun Chen, Yang Li, Nicole Wyman, Zheng Zhang, Hongming Fan, Michael Le, Steven Gannon, Chelsea Rose, Zhao Zhang, Jeremy Mercuri, Hai Yao, Bruce Gao, Shane Woolf, Thierry Pécot, and Tong Ye Biomed. Opt. Express 12(5) 2759-2772 (2021)
Zach Nadler, Bo Wang, Gadi Wollstein, Jessica E. Nevins, Hiroshi Ishikawa, Larry Kagemann, Ian A. Sigal, R. Daniel Ferguson, Daniel X. Hammer, Ireneusz Grulkowski, Jonathan J. Liu, Martin F. Kraus, Chen D. Lu, Joachim Hornegger, James G. Fujimoto, and Joel S. Schuman Biomed. Opt. Express 4(11) 2596-2608 (2013)
You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
Groups of Images Used for the Algorithm Evaluation
Group
Number of Images
Number of Nuclei (Manually Detected)
Image Resolution, Pix per μm
Image Size, μm ()
Microscopy Method
1
Brain tumors
18
640
1.609–6.803
47–355
TPEF
2
Larynx
27
626
2.276–4.551
74–228
TPEF
3
Skin tumors
27
533
2.276
158–329
TPEF, CARS
4
Keloidal skin
3
113
1.138
30–47
CARS
Total
75
1912
1.138–6.803
30–355
Table 2.
Average Accuracy (Sensitivity/Specificity) of the -Minima Seeding Using Different Values of the Parameter (%)
Group
1
Brain tumors
31.6 ()
62.2 ()
73.9 ()
70.8 ()
63.2 ()
52.4 ()
2
Larynx
37.0 ()
70.6 ()
78.7 ()
72.3 ()
58.1 ()
45.0 ()
3
Skin tumors
46.9 ()
78.3 ()
83.0 ()
76.3 ()
67.1 ()
54.9 ()
4
Keloidal skin
43.1 ()
59.2 ()
68.0 ()
73.6 ()
62.7 ()
54.0 ()
Average
39.5 ()
70.9 ()
78.7 ()
73.4 ()
62.7 ()
50.7 ()
Table 3.
Average Accuracy (Sensitivity/Specificity) of the CHT Using Different Types of Edges and Shape of the Increment Region (—Half-doughnut, —Quarter-doughnut), %
Group
Sobel,
Canny,
Sobel,
Canny,
1
Brain tumors
46.5 ()
45.0 ()
58.5 ()
55.5 ()
2
Larynx
45.2 ()
44.5 ()
57.1 ()
52.0 ()
3
Skin tumors
78.3 ()
72.3 ()
86.1 ()
81.1 ()
4
Keloidal skin
68.3 ()
59.2 ()
74.6 ()
67.6 ()
Average
58.4 ()
55.2 ()
68.6 ()
63.9 ()
Table 4.
Accuracy (Sensitivity/Specificity) of the LoG Filter after the Thresholding of the LoG Response at Different Threshold Values [thr] (%)
Group
max(R)
max(R)
max(R)
max(R)
1
Brain tumors
79.7 ()
85.1 ()
90.6 ()
89.8 ()
2
Larynx
87.9 ()
89.9 ()
91.1 ()
89.4 ()
3
Skin tumors
90.3 ()
92.4 ()
92.6 ()
87.9 ()
4
Keloidal skin
72.5 ()
78.0 ()
83.5 ()
84.7 ()
Average
86.2 ()
89.2 ()
91.2 ()
88.8 ()
Table 5.
Comparison of the Accuracy (Sensitivity/Specificity) of Seeding Methods (%)
Group
-Minima,
CHT, Sobel Edges, Doughnut
Scale LoG, max(R)
1
Brain tumors
73.9 ()
58.5 ()
90.6 ()
2
Larynx
78.7 ()
57.1 ()
91.1 ()
3
Skin tumors
83.0 ()
86.1 ()
92.6 ()
4
Keloidal skin
68.0 ()
74.6 ()
83.5 ()
Average
78.7 ()
68.6 ()
91.2 ()
Table 6.
Comparison of the Accuracy (Sensitivity/Specificity) of Boundary Detection Methods (%)
Group
Radial-Gradient-Based Detection
LoG Response
Active Contour
Seeded Watershed
1
Brain tumors
73.3 ()
76.9 ()
77.6 ()
82.9 ()
2
Larynx
70.2 ()
75.3 ()
74.8 ()
79.2 ()
3
Skin tumors
70.8 ()
71.6 ()
72.8 ()
80.4 ()
4
Keloidal skin
64.3 ()
77.8 ()
72.8 ()
75.3 ()
Average
71.0 ()
74.5 ()
74.7 ()
80.4 ()
Table 7.
Accuracy (Sensitivity/Specificity) of the Final Nuclei Segmentation Approach (%)
Group
Seeding Accuracy (%)
Boundary Accuracy (%)
1
Brain tumors
()
()
2
Larynx
()
()
3
Skin tumors
()
()
4
Keloidal skin
()
()
Average
()
()
Table 8.
Estimated Errors for the Calculated Quantitative Parameters (%)a
Quantitative Characteristics of SCC Metastasis in Human Brain
Characteristic
Healthy Brain Tissue
SCC Metastasis Region
Nuclear density per
115.5
4.3
Nuclear fraction (%)
27.5
0.9
Size ()
20.6 (14.5–31.3)
20.0 (10.8–28.0)
0.21 (0.14–0.27)
0.15 (0.12–0.19)
Texture
0.047
(0.044–0.050)
0.043
(0.041–0.046)
Eccentricity
2.14 (1.72–2.84)
1.88 (1.66–2.21)
Convexity
0.77 (0.68–0.86)
0.78 (0.74–0.86)
Table 10.
Quantitative Characteristics of Larynx Carcinoma Sample
Characteristic
Healthy Epithelium
Dysplastic Epithelium
Connective Tissue
Nuclear density per
67.6
84.6
108.9
Nuclear fraction (%)
15.3
27.6
21.4
Size ()
20.7
25.9
15.5
(14.7–28.5)
(16.3–43.1)
(11.4–23.5)
0.24
0.19
0.18
(0.17–0.30)
(0.14–0.24)
(0.13–0.24)
Texture
0.081
0.082
0.080
(0.078–0.084)
(0.078–0.085)
(0.076–0.083)
Eccentricity
2.38
2.10
2.04
(1.97–3.13)
(1.79–2.60)
(1.68–2.59)
Convexity
0.70
0.77
0.83
(0.61–0.78)
(0.68–0.86)
(0.74–0.88)
Tables (10)
Table 1.
Groups of Images Used for the Algorithm Evaluation
Group
Number of Images
Number of Nuclei (Manually Detected)
Image Resolution, Pix per μm
Image Size, μm ()
Microscopy Method
1
Brain tumors
18
640
1.609–6.803
47–355
TPEF
2
Larynx
27
626
2.276–4.551
74–228
TPEF
3
Skin tumors
27
533
2.276
158–329
TPEF, CARS
4
Keloidal skin
3
113
1.138
30–47
CARS
Total
75
1912
1.138–6.803
30–355
Table 2.
Average Accuracy (Sensitivity/Specificity) of the -Minima Seeding Using Different Values of the Parameter (%)
Group
1
Brain tumors
31.6 ()
62.2 ()
73.9 ()
70.8 ()
63.2 ()
52.4 ()
2
Larynx
37.0 ()
70.6 ()
78.7 ()
72.3 ()
58.1 ()
45.0 ()
3
Skin tumors
46.9 ()
78.3 ()
83.0 ()
76.3 ()
67.1 ()
54.9 ()
4
Keloidal skin
43.1 ()
59.2 ()
68.0 ()
73.6 ()
62.7 ()
54.0 ()
Average
39.5 ()
70.9 ()
78.7 ()
73.4 ()
62.7 ()
50.7 ()
Table 3.
Average Accuracy (Sensitivity/Specificity) of the CHT Using Different Types of Edges and Shape of the Increment Region (—Half-doughnut, —Quarter-doughnut), %
Group
Sobel,
Canny,
Sobel,
Canny,
1
Brain tumors
46.5 ()
45.0 ()
58.5 ()
55.5 ()
2
Larynx
45.2 ()
44.5 ()
57.1 ()
52.0 ()
3
Skin tumors
78.3 ()
72.3 ()
86.1 ()
81.1 ()
4
Keloidal skin
68.3 ()
59.2 ()
74.6 ()
67.6 ()
Average
58.4 ()
55.2 ()
68.6 ()
63.9 ()
Table 4.
Accuracy (Sensitivity/Specificity) of the LoG Filter after the Thresholding of the LoG Response at Different Threshold Values [thr] (%)
Group
max(R)
max(R)
max(R)
max(R)
1
Brain tumors
79.7 ()
85.1 ()
90.6 ()
89.8 ()
2
Larynx
87.9 ()
89.9 ()
91.1 ()
89.4 ()
3
Skin tumors
90.3 ()
92.4 ()
92.6 ()
87.9 ()
4
Keloidal skin
72.5 ()
78.0 ()
83.5 ()
84.7 ()
Average
86.2 ()
89.2 ()
91.2 ()
88.8 ()
Table 5.
Comparison of the Accuracy (Sensitivity/Specificity) of Seeding Methods (%)
Group
-Minima,
CHT, Sobel Edges, Doughnut
Scale LoG, max(R)
1
Brain tumors
73.9 ()
58.5 ()
90.6 ()
2
Larynx
78.7 ()
57.1 ()
91.1 ()
3
Skin tumors
83.0 ()
86.1 ()
92.6 ()
4
Keloidal skin
68.0 ()
74.6 ()
83.5 ()
Average
78.7 ()
68.6 ()
91.2 ()
Table 6.
Comparison of the Accuracy (Sensitivity/Specificity) of Boundary Detection Methods (%)
Group
Radial-Gradient-Based Detection
LoG Response
Active Contour
Seeded Watershed
1
Brain tumors
73.3 ()
76.9 ()
77.6 ()
82.9 ()
2
Larynx
70.2 ()
75.3 ()
74.8 ()
79.2 ()
3
Skin tumors
70.8 ()
71.6 ()
72.8 ()
80.4 ()
4
Keloidal skin
64.3 ()
77.8 ()
72.8 ()
75.3 ()
Average
71.0 ()
74.5 ()
74.7 ()
80.4 ()
Table 7.
Accuracy (Sensitivity/Specificity) of the Final Nuclei Segmentation Approach (%)
Group
Seeding Accuracy (%)
Boundary Accuracy (%)
1
Brain tumors
()
()
2
Larynx
()
()
3
Skin tumors
()
()
4
Keloidal skin
()
()
Average
()
()
Table 8.
Estimated Errors for the Calculated Quantitative Parameters (%)a