## Abstract

The advance in microscopy and artificial intelligence enables the application of digital pathology in various classification situations to help pathologists reduce the challenge of performing diagnosis purely based on their visualization experience. Human endometrium is receptive to the embryo only during a defined period in a menstrual cycle. The endometrial phase characterization is crucial for the formation of a healthy pregnancy. Polarization imaging is an emerging label-free and non-invasive technique that is good at characterizing the microstructures of biological tissues. In this study, polarization imaging was combined with digital pathology to characterize the microstructures of endometrium samples at the typical proliferative phase and typical secretory phase. The involved polarization parameters include Muller matrix polar decomposition (MMPD) derived parameters *δ*, *θ* and a set of rotation invariant parameters *P _{L}*,

*D*,

_{L}*q*,

_{L}*r*and their corresponding angular parameters

_{L}*α*,

_{P}*α*,

_{D}*α*and

_{q}*α*. The approaches for the digitalization of the polarization parameter images include the statistical mean analysis that does not involve image texture information, the Local Binary Pattern (LBP) analysis that involves partial image texture information, and the machine learning classifications that make full use of the polarization parameter image information. A class distance Score was defined to evaluate the performance of polarization parameters in the statistical mean and the image texture analysis. The statistical mean analysis indicates parameter

_{r}*D*that relate to the dichroism of the endometrial tissues shows the best class separation ability with the highest class distance Score. Image texture analysis indicates parameter

_{L}*D*still has the highest class distance Score. And compared with the statistical mean method, the class distance Score for

_{L}*D*increased after LBP process. The results of machine learning classification show parameter

_{L}*α*classified by Convolutional Neural Network (CNN) architecture 1 and parameter

_{D}*α*classified by CNN architecture 2 have the same highest accuracy of 87%. This study shows the potential of applying the digital pathology techniques on polarization parameter images to achieve endometrial phase characterization.

_{P}© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

## 1. Introduction

In recent years, the advance in microscopy and artificial intelligence enables the digitizing of whole-slide images of biological samples containing complex morphological features. In clinic, pathologists realize histological diagnosis through visualization and sometimes semi-quantification of the morphological patterns of the tissue samples. The accurate diagnosis based on the visualization of histological slides requires well trained pathologists with rich experience in pathology. Digital pathology based on artificial intelligence (AI) approaches has been employed to various image modalities and classification situations to assist pathologist in the process of diagnosis and prognosis. Bera *et al.* [1] summarizes AI approaches in pathology in two aspects that are hand-crafted feature-based approaches and deep neural network-based approaches. The hand-crafted feature-based approaches require pathologist, oncologist and AI expert to engineer the features to be analyzed by machine learning approaches [2,3]. The deep neural network-based approaches do not require feature engineering and can directly apply on primary data sets. Convolutional Neural Networks (CNNs) are widely used deep learning algorithms in various pathological image classification applications [4].

Mueller matrix imaging is an emerging label-free and non-invasive technique that good at characterizing the microstructures of biological tissues with anisotropy properties [5–8]. Recently, several researchers have been working on the Muller matrix imaging based digital pathology. Wang *et al.* [9] investigated on the potential of using Mueller matrix imaging for digital staining. Mueller matrix imaging based digital staining has several advantages such as provides quantitative information of tissue structures, reduces hands-on time of pathologists, reduces cost of chemical staining and help users avoid the toxic chemical stains. Lee *et al.* [10] reported an innovative polarization parameter extraction method based on the logarithmic Mueller matrix decomposition and explored the impact of tissue thickness variations on the results of digital histology. In this study, Mueller matrix imaging is combined with fast growing digital pathological techniques to quantitatively characterize the microstructures of endometrial samples at the typical proliferative phase and the typical secretory phase.

Endometrium undergoes cyclic morphological changes from proliferative phase to secretory phase and to menstrual phase in response to estrogen and progesterone secretion by the ovaries during a menstrual cycle for a woman in her reproductive years. Endometrial receptivity refers to the ability of the endometrium to attach and nourish the blastocyst during implantation, which plays an important role in the formation of a successful and healthy pregnancy. Endometrium is receptive to the embryo only during a defined window of implantation. The study and characterization of the phases of the endometrium is essential for pathologists to determine the endometrial receptivity and perform exogenetic regulation [11,12,13]. In the clinic, Hematoxylin and Eosin (H&E) staining and immunofluorescence staining are commonly used to observe the histological changes of the endometrium at different phases based on their visualization experience. Therefore, the label-free Mueller matrix imaging based digital pathology in this study can provide rich quantitative microstructural information of the endometrium samples, avoid the timing consuming staining procedures, reduce the costs of chimerical staining, and prevent the contact of toxic chemical stains.

The Mueller matrix imaging based digital pathology in this study include the statistical mean analysis, the image texture analysis through Local Binary Pattern (LBP) [14,15] technique, and the machine learning classifications done separately from the statistical mean and the image texture analysis. First of all, the statistical mean of various polarization parameters was calculated to study the endometrial morphological change induced polarization anisotropy differences. Polarization parameters investigated in this study including Mueller matrix polar decomposition (MMPD) [16] derived parameter *δ* [17] and *θ* and a set of rotation invariant parameter *P _{L}*,

*D*,

_{L}*q*,

_{L}*r*and their corresponding angular parameter

_{L}*α*,

_{P}*α*,

_{D}*α*and

_{q}*α*[18,19]. Parameter

_{r}*δ*represents the retardance and parameter

*θ*describes the orientation of retardance. Parameter

*P*and

_{L}*D*relate to the dichroism and parameter

_{L}*α*and

_{P}*α*are the corresponding angular parameters that describe the orientation of dichroism. Parameter

_{D}*q*and

_{L}*r*relate to the birefringence and parameter

_{L}*α*and

_{q}*α*are the corresponding angular parameters that describe the orientation of birefringence. Retardance parameter

_{r}*δ*is more sensitive to the changes of collagen and fibrous structures in biological samples. Parameter

*P*and

_{L}*D*are more sensitive to the cell nucleus changes in tissues. And parameter

_{L}*q*and

_{L}*r*are sensitive to the fibrous structures in tissues. Then, Local Binary Pattern (LBP) technique was employed to extract the image texture information from polarization parameter images. A class distance Score was defined based on the Fisher’s linear discriminant equation [20,21] to evaluate the performance of polarization parameters in the statistical mean and the image texture analysis.

_{L}Previous researches of using polarization parameters to characterize the microstructural features of biological samples such as in the study of breast ductal carcinoma [7], Crohn’s disease [22] and the cervical cancer [23] are pixel based analysis rather than image based analysis. In this study, to make full use of the polarization parameter image contained microstructural information of endometrial tissues, machine learning classifications were performed on the polarization parameter images. The machine learning classification algorithms used were Logistic Regression, Ridge [24], CNN architecture 1 that constructed based on AlexNet [25] and CNN architecture 2 that constructed based on Residual Neural Network (ResNet) [26]. Logistic Regression and Ridge are widely used linear classification models. Logistic Regression is based on a sigmoid function and can achieve relative high accuracy for linearly separable simple data sets. Ridge classification imposes a penalty on the size of the coefficients to reduce the variance of the estimates. Compared with simple linear regression, Ridge classification is a good technique to reduce model complexity and prevent over-fitting. Overall, the linear models have several advantages over CNNs including its simplicity and computational attractiveness. Linear models are relative easy to implement and interpret. And they can achieve an adequate accuracy with less training and validation time. CNNs are well known for their good performance in image pattern recognition [4]. In this study, we focus on make full use of the polarization parameter image information. Therefore, two CNN architectures were also constructed to distinguish the polarization parameter images at typical proliferative phase and typical secretory phase.

## 2. Methods and Materials

#### 2.1 Experimental setup

The Mueller matrix microscope is designed through the modification of a commercial transmission-light microscope (L2050, Guangzhou LISS Optical Instrumentation Co. LTD, China) based on the dual-rotating-retarder method [27]. Figure 1 illustrates the photograph and schematic of the Mueller matrix microscope. During the experiment, the incident beam from the LED passes through the polarization state generator (PSG), sample, objective lens, polarization state analyzer (PSA) and then the photons are recorded by a 12-bit CCD camera (QImaging 74-0107A, Canada). The two quarter-waves plates at 632 nm in PSG and PSA (R1 and R2) rotate with a rate of ω_{R2}=5ω_{R1} while the two polarizers (P1 and P2) are fixed in the horizontal direction [5,27]. The Mueller matrix of a sample is calculated from the 30 images with different polarization states corresponding to different angle combinations produced by PSG and PSA. The calibration of the Mueller matrix microscope is achieved through measuring the Mueller matrix of the air and other standard samples, and the maximum errors of the microscope are about 1%. Detailed information relative to the Mueller matrix calculation algorithm and the Mueller matrix imaging technique are demonstrated in Refs. [28,29].

#### 2.2 Mueller matrix polar decomposition parameters and rotation invariant parameters

Mueller matrix contains the microstructural information of the biological samples. It is challenging to link the individual Mueller matrix element directly to the microstructures of the samples. Therefore, Mueller matrix polar decomposition (MMPD) parameters [16,17] and various Mueller matrix transformation [18] and rotation invariant parameters [19] are derived to connect the physical meaning of polarization such as scattering and birefringence to the microstructural features of biological samples. The parameters selected for investigation in this study include MMPD retardance parameter *δ*, a set of rotation invariant parameter *P _{L}*,

*D*,

_{L}*q*,

_{L}*r*and their corresponding angular parameter

_{L}*θ*,

*α*,

_{P}*α*,

_{D}*α*and

_{q}*α*. The MMPD method is proposed by Lu and Chipman [16], which decomposes a Mueller matrix into diattenuation (

_{r}*D*), retardance (

*R*) and depolarization (

*Δ*). In this study, the MMPD method derived linear retardance (

*δ*) [17] and its orientation angle (

*θ*) were employed to study the endometrium tissues.

Besides MMPD parameters, a set of rotation invariant parameters recently proposed by Li *et al.* [19] shown in Eq. (1) were applied to quantitatively characterize the endometrial phases in a menstrual cycle. In Eq. (1), the Mueller matrix was normalized by m_{11} for the calculation of polarization parameters. The corresponding Mueller matrix elements were represented by m_{21}, m_{31}, m_{12}, m_{13}, m_{42}, m_{43}, m_{24} and m_{34}. Rotation invariant parameter *P _{L}* and

*D*relate to the dichroism of the tissues and their corresponding angular parameter

_{L}*α*and

_{P}*α*illustrate the orientation of the dichroism. Parameter

_{D}*q*and

_{L}*r*relate to the birefringence of the tissues and

_{L}*α*and

_{q}*α*show the orientation of the birefringence [19].

_{r}#### 2.3 Endometrium tissue samples

Endometrium has relatively simple tissue structures and is composed of tubular glands and vascular stroma [30]. In a period of 28 days for a normal menstrual cycle, endometrium undergoes cyclic regular growth and maturation from proliferative to secretory phase, and shedding occurs with the absence of pregnancy [11]. In typical proliferative phase as shown by the H&E staining in *Fig. 2*(a2), the glands are small and narrow and the glandular cells have uniform size of nuclei. The glandular epithelium forms a cubo-columnar structure and the stromal cells are loose aggregates of cells. From typical proliferative phase to typical secretory phase, the glandular cells gradually grow larger with a clear cytoplasm and the stromal cells increase in size and volume. At typical secretory phase as shown by the H&E staining in *Fig. 2*(b2), the glands are expanded to form a plumper structure. This study focus on using Mueller matrix imaging to characterize the microstructural features of 16-µm-thick non-stained endometrial tissues at typical proliferative phase and typical secretory phase.

In this study, the endometrial tissues were acquired through sampling method of dilation and curettage (D&C) without a hysteroscopy. There were 15 patients in typical proliferative phase and 15 patients in typical secretory phase. For each patient, one 4-µm-thick hematoxylin and eosin (H&E) stained pathological section and one adjacent 16-µm-thick non-stained and dewaxed pathological section were obtained. The H&E stained section was used by an experienced pathologist to label one region of interest (ROI) as shown in Fig. 2(a2) or (b2). And the corresponding ROI with a size of 1000 × 1000 pixels on the 16-µm-thick non-stained and dewaxed pathological section as shown in Fig. 2(a1) or (b1) was acquired for Mueller matrix imaging. In this study, for each patient, only one ROI was obtained for polarization imaging. And each ROI was treated as a sample. Therefore, there were 15 samples in typical proliferative phase and 15 samples in typical secretory phase. This study strictly followed the rules regulated by the Ethics Committee of the Shenzhen People’s Hospital.

#### 2.4 Quantitative characterization of polarization parameter images through statistical mean method and Local Binary Pattern (LBP)

In clinic, pathologists determine the phases of endometrium tissues by observing the H&E staining sections based on their visualization experience. This study aims at providing digitized quantitative assistance through analyzing the images of polarization parameters. The quantitative techniques include two parts: statistical mean analysis and Local Binary Pattern (LBP) analysis. First of all, for each sample, the statistical mean of each polarization parameter image with a ROI of 1000 × 1000 pixels was calculated. Then, a polarization classification Score was defined based on the Fisher’s linear discriminant equation [20,21] to assess the level of separation between two sets of endometrial data at typical proliferative phase and typical secretory phase. The equation for calculating the classification Score was shown in Eq. (2).

Where *PM _{0}* and

*STD*were the class mean and class standard deviation of a polarization parameter at typical proliferative phase. And

_{0}*PM*and

_{i}*STD*were the class mean and class standard deviation of a polarization parameter at the typical secretory phase. For a polarization parameter in each class, there were 15 mean values calculated from 15 samples. The class mean was the mean of these 15 mean values. And the class standard deviation was the standard deviation of these 15 mean values. For a polarization parameter, the Score calculated between two endometrial phases describes the separation ability of the polarization parameter. A polarization parameter with a larger Score has a better separation and classification ability.

_{i}The statistical mean analysis of a polarization parameter image does not involve much image texture information. Therefore, LBP as a powerful image texture extraction technique was used to obtain more endometrial microstructural information contained by the polarization parameter images. The implementation of LBP technique was achieved through MATLAB function ‘extractLBPFeatures’ [14,15]. The input of the LBP analysis was a polarization parameter image with a ROI of 1000 × 1000 pixels. The ‘CellSize’ of the MATLAB function equals 15. The output of the LBP analysis was a feature vector with an overall length of 257004 elements. To assess the classification ability of a polarization parameter after LBP analysis, the Score defined in Eq. (2) also can be used on the LBP feature vectors to calculate the class distance between two endometrial phases. For a polarization parameter, in each endometrial phase, there were 15 LBP feature vectors resulted from 15 samples. Then, the skewness and kurtosis were calculated for each feature vector. To calculate the class distance Score as shown in Eq. (2) based on the skewness of the feature vectors, the class mean *PM _{0}* or

*PM*was calculated by taking the mean of the 15 skewness values in typical proliferative phase or typical secretory phase. And the class standard deviation

_{i}*STD*or

_{0}*STD*was calculated by taking the standard deviation of the 15 skewness values in typical proliferative phase or typical secretory phase. In addition, the class distance Score of a polarization parameter based on the kurtosis of the feature vectors was also calculated using the same method.

_{i}#### 2.5 Endometrial phase characterization of polarization parameter images through machine learning classification

To make full use of the endometrial microstructural information contained by polarization parameter images, four widely used machine learning classification algorithms including Logistic Regression, Ridge [24], CNN architecture 1 that constructed based on AlexNet [25] and CNN architecture 2 that constructed based on ResNet [26] were applied directly on the polarization parameter images to distinguish the endometrial samples at typical proliferative phase and typical secretory phase.

In the machine learning classification, the Logistic Regression and Ridge classification were achieved through MATLAB linear classification learner template ‘templateLinear’ [24] with loss functions ${\ell _{Logistic}}$ and ${\ell _{Ridge}}$ as shown in Eq. (3) [31].

*b*is bias. The optimizer used to minimize the loss function for both Logistic Regression and Ridge was Limited-memory Broyden-Fletcher-Goldfarb-Shanno quasi-Newton (LBFGS) algorithm [32]. For Ridge classification, a statistical procedure was performed to tune the regulation strength λ and the selected λ to output classification results was 8.6${\times} $10

^{−5}.

The models CNN 1 and CNN 2 were achieved through MATLAB with detailed architectures shown in Fig. 3. The optimizer used for training network was Adaptive Moment Estimation (Adam) [33] for both CNN 1 and CNN 2. For CNN 1, the initial learning rate was 1${\times} $10^{−3} and the size of mini-batch was 32. For CNN 2, the initial learning rate was 2${\times} $10^{−3} and the size of mini-batch was 64. Based on the convergence of the model, the maximum number of epochs was 13 for both of the networks. In addition, for both CNN 1 and CNN 2, the training options not specifically mentioned were set to default of MATLAB function ‘trainingOptions’ [34].

In this study, there were 15 sample images for both typical proliferative phase and typical secretory phase. Before input into the classification algorithm, Random Image Cropping as a commonly used data augmentation method was applied to enrich the datasets and prevent overfitting [35]. Radom Image Cropping creates random subset image patches of an original image and prevent the model overfitting to specific features. In this study, for each polarization parameter of each endometrium sample, there was one polarization parameter image with a ROI of 1000 × 1000 pixels in size. The Random Image cropping method creates 400 subset image patches from the original polarization parameter image with a dimension of 64 × 64 pixels. In machine learning classification, polarization parameters were considered separately. For Logistic Regression and Ridge, the dimension of the input feature vector of an image patch was 4096. For CNN 1 and CNN 2, the input of classification was an image patch with a dimension of 64 × 64 pixels. In the classification accuracy calculation procedure, the 400 image patches generated from one original polarization parameter image of a single patient were treated as one sample.

In addition, Leave-one-out cross-validation (LOOCV) was performed to validate the performance of the selected machine learning algorithms [36]. In LOOCV, the distribution of the training and testing set was based on patients. The 30 endometrial samples from 30 patients can be separated into 30 folds. In each fold of cross-validation, one patient with 400 image patches was distributed into the testing set and the rest of 29 patients with 11600 image patches were distributed into the training set. The classification algorithm was applied once for each fold. In this study, to analyze the results of machine learning classification, the image patch accuracy of each sample during the LOOCV was calculated. The classification of the sample was considered right if its corresponding image patch accuracy was greater than 50%. The accuracy presented in the results section was calculated through divide the number of right samples by the number of total samples.

## 3. Results and discussion

#### 3.1 Polarization imaging results of endometrium tissue samples

2D images of Mueller matrix parameter *δ*, *P _{L}, D_{L}, q_{L}, r_{L}* and their corresponding angular parameter

*θ*,

*α*and

_{P}, α_{D}, α_{q}*α*for endometrium samples at typical proliferative phase and typical secretory phase were shown in Fig. 4. Parameter

_{r}*δ*represents retardance, parameters

*q*and

_{L}*r*relate to birefringence, and parameters

_{L}*P*and

_{L}*D*relate to the degree of dichroism of endometrium samples. Polarization imaging is sensitive to microstructures with anisotropy properties. The morphological changes of endometrial tissues at typical proliferative phase and typical secretory phase may be reflected by the changes of polarization parameters. In histological perspective, endometrium is composed of tubular glands and stroma [30]. As shown in Fig. 4, the images of the birefringence related parameters

_{L}*δ*,

*q*and

_{L}*r*show high contrast of endometrial glandular epithelium structures versus endometrial stromal structures. For dichroism related parameters

_{L}*P*and

_{L}*D*, there were signals of both glandular epithelium structures and stromal structures with a less obvious contrast. These results may indicate birefringence parameters are more sensitive to endometrial glandular epithelium structures. In addition, from typical proliferative phase to typical secretory phase, the endometrial glandular structures change from small and narrow shapes into plumper and tortuous structures. And the stromal cells change from loose groups to more cohesive groups. These endometrial morphological changes also can be reflected by the polarization parameter images as shown in Fig. 4. Previous researches [18] indicate parameter

_{L}*δ*,

*q*and

_{L}*r*are more sensitive to collagen and fibrous structures. Parameter

_{L}*P*and

_{L}*D*are more sensitive to cell nucleus structures. In Fig. 4, from typical proliferative phase to typical secretory phase, for birefringence related parameters

_{L}*δ*,

*q*and

_{L}*r*, the contrast of glandular epithelium structures versus stromal structures increased. And for dichroism related parameters

_{L}*P*and

_{L}*D*, there were increase of signals of both glandular epithelium structures and stromal structures with a less obvious contrast.

_{L}The angular polarization parameters shown in Fig. 4 provide additional orientation information that closely related to the endometrial microstructures at typical proliferative phase and typical secretory phase. Parameter *θ* illustrates the orientation of linear retardance. Parameter *α _{P}* and

*α*describe the orientation of dichroism and parameter

_{D}*α*and

_{q}*α*represent the orientation of birefringence. As shown in Fig. 4, images of parameter

_{r}*θ*,

*α*and

_{q}*α*show higher contrast than parameter

_{r}*α*and

_{P}*α*in characterizing the glandular epithelium structures. And in the images of parameter

_{D}*α*and

_{P}*α*, the contrast of glandular epithelium structures versus stromal structures was less obvious. Comparing the imaging results shown in Fig. 4 with the corresponding H&E staining shown in Fig. 2, we may conclude that polarization parameter images have the potential to distinguish the endometrial glandular and stromal morphological changes from typical proliferative phase to typical secretory phase.

_{D}#### 3.2 Quantitative characterization of polarization images of the endometrium samples at different phases

### 3.2.1 Statistical analysis of polarization parameter images

The statistical mean of polarization parameter images for each endometrial sample at typical proliferative phase and typical secretory phase were calculated as shown in Fig. 5. In Fig. 5, for parameter *δ*, *P _{L}*,

*D*,

_{L}*q*and

_{L}*r*, mean values increased from typical proliferative phase to typical secretory phase, which quantify the results visualized in previous polarization parameter images. The class distance Score for parameter

_{L}*D*,

_{L}*r*,

_{L}*δ*,

*P*and

_{L}*q*were 0.25, 0.21, 0.16, 0.15 and 0.12 in decreasing order, which indicate parameter

_{L}*D*shows the best separation ability. In biological samples, both dichroism and birefringence can cause anisotropy [23]. Previous researches [19,23,37,38] show

_{L}*q*and

_{L}*r*calculated by Mueller matrix elements m

_{L}_{42}, m

_{43}and m

_{24}, m

_{34}are relative to the birefringence of the sample. And

*P*and

_{L}*D*calculated by m

_{L}_{21}, m

_{31}and m

_{12}, m

_{13}are related to the dichroism of the sample. For endometrium tissues, from typical proliferative phase to typical secretory phase, the increase of

*P*and

_{L}*D*indicate the increase of dichroism induced anisotropy and the increase of

_{L}*q*and

_{L}*r*reveal the increase of birefringence structures in endometrium tissues. The results in Fig. 5(f) show both in typical proliferative phase and typical secretory phase, the sum of the mean values of parameter

_{L}*P*and

_{L}*D*was greater than the sum of the mean values of parameter

_{L}*q*and

_{L}*r*Compared with the ratio in typical proliferative phase, the ratio of the sum of parameter

_{L}.*P*and

_{L}*D*over the sum of parameter

_{L}*q*and

_{L}*r*increased in typical secretory phase, this indicates the increase contribution of dichroism induced anisotropy.

_{L}However, the class distance Score for all of the five Mueller matrix parameters were less than one show that there were mean value range overlaps between typical proliferative phase and typical secretory phase. In each endometrial phase of this study, the 15 ROIs used for experiments were come from 15 patients. The individual differences among the 15 patients and the quality of the 16-µm-thick non-stained pathological sections may all be the factors that affect the class distance Score. The mean value range overlaps between two phases bring challenge to distinguish the endometrial phases purely using statistical mean methods. In addition, the statistical mean method does not use much polarization parameter image information. Therefore, in the following section, LBP as a common image texture extraction method was used to involve more polarization parameter image texture information in the characterization of the endometrial microstructural features.

### 3.2.2 Image texture analysis through a local binary pattern (LBP)

Local Binary Pattern (LBP) technique was used to extract image texture information from polarization parameter images with a dimension of 1000 × 1000 pixels. The output of each polarization parameter image after LBP process was a feature vector. Ushenko *et al.* [39]. indicates skewness and kurtosis can be used to perform distribution analysis of Mueller matrix based histological slides. In order to quantitatively interpret the LBP feature vector, the skewness and kurtosis of each feature vector were calculated as shown in Fig. 6. The class distance Scores of a polarization parameter based on the skewness and kurtosis of the LBP feature vectors were also shown in Fig. 6. Figure 6(a1) – (a3) show the skewness of the LBP feature vectors of selected polarization parameter *δ*, *P _{L}* and

*D*with highest class distance Score. Figure 6(b1) – (b3) show the kurtosis of the LBP feature vectors of selected polarization parameter

_{L}*δ*,

*P*and

_{L}*D*with highest class distance Score.

_{L}For the convenience of comparison, Table 1 and Table 2 summarize the quantitative analysis results of the polarization parameters illustrated in Fig. 5 and Fig. 6. As shown in Table 1, the mean value of all five polarization parameters increased from typical proliferative phase to typical secretory phase. Table 2 indicates the class distance Scores of parameter *δ*, *P _{L}* and

*D*increased after adding the image texture information through LBP analysis. After LBP process, parameter

_{L}*δ*,

*P*and

_{L}*D*have higher class distance Scores than parameter

_{L}*q*and

_{L}*r*. For parameter

_{L}*δ*, class distance Score increases from 0.16 to 0.32 of LBP skewness analysis and to 0.28 of LBP kurtosis analysis. For parameter

*P*, class distance Score increases from 0.15 to 0.20 of LBP skewness analysis and to 0.17 of LBP kurtosis analysis. In the previous statistical mean analysis section, parameter

_{L}*D*has the highest class distance Score and shows the best separation ability. After LBP process, parameter

_{L}*D*still has the highest class distance Score and it increases from 0.25 to 0.66 of LBP skewness analysis and to 0.67 of LBP kurtosis analysis. Parameter

_{L}*δ*comes from the decomposition of an entire Mueller matrix and represents the retardance of the endometrial tissues. Parameter

*q*and

_{L}*r*are calculated by the lower right 3 × 3 elements of a Mueller matrix and relate to the birefringence structures of endometrial tissues. Parameter

_{L}*P*and

_{L}*D*represent the dichroism of the endometrial tissues. Generally, in biological perspective, parameter

_{L}*δ*,

*q*and

_{L}*r*are more sensitive to collagen and fibrous structures in endometrial tissues. Parameter

_{L}*P*and

_{L}*D*are more sensitive to cell nucleus structures in endometrial tissues. In quantitative perspective, the highest class distance Scores of parameter

_{L}*D*in the statistical mean analysis and the LBP analysis may indicate the characterization of cell nucleus structures in endometrial tissues plays an important role in distinguishing the typical proliferative phase and typical secretory phase. However, parameter

_{L}*δ*also has the third highest class distance Score in the statistical mean analysis and the second highest class distance Score in the LBP analysis. Therefore, characterization of the collagen and fibrous structures in endometrial tissues may also important for classification.

The classification ability of parameter *δ*, *P _{L}* and

*D*increased after adding the LBP related image texture information. However, there were still overlaps of the sample values in the two endometrial phases. Therefore, in the following section, to make full use of the polarization parameter image information, machine learning algorithms were applied directly on the polarization parameter images for endometrial phases classification.

_{L}#### 3.3 Machine learning algorithms for classification of the two endometrial phases

To fully use the endometrial microstructural information contained by polarization parameter images in endometrial phase characterization, four widely used machine learning algorithms including Logistic Regression, Ridge [24], CNN 1 and CNN 2 were applied directly on the polarization parameter images to classify the endometrial samples at typical proliferative phase and typical secretory phase. Leave-one-out cross-validation (LOOCV) was performed to validate the selected machine learning classification algorithms. Table 3 show the LOOCV validated accuracy, precision, recall and Area Under Curve (AUC) of the four classification algorithms for parameter *δ*, *P _{L}*,

*D*,

_{L}*q*,

_{L}*r*and their corresponding angular parameter

_{L}*θ*,

*α*,

_{P}*α*,

_{D}*α*and

_{q}*α*. Among the 10 polarization parameters, parameter

_{r}*α*classified by CNN 1 has the highest classification accuracy of 87%. And parameter

_{D}*α*classified by CNN 2 also has the highest classification accuracy of 87%. Parameter

_{P}*α*and

_{P}*α*are the corresponding angular parameters of

_{D}*P*and

_{L}*D*and they describe the orientation of the dichroism in endometrial tissues. Parameter

_{L}*α*and

_{P}*α*have higher classification accuracy than parameter

_{D}*P*and

_{L}*D*in all of the four classification algorithms. The outstanding performance of angular parameter

_{L}*α*and

_{P}*α*indicate the orientation of dichroism or the orientation of cell nucleus related structures is critical for endometrial phase characterization.

_{D}Logistic Regression and Ridge are linear classification algorithms and CNN 1 and CNN 2 are non-linear deep learning classification algorithms. In this study, machine learning classifications were carried out on one personal computer with Intel Core i9-9900 K and one GeForce GTX 2080Ti. The operating system was Ubuntu 18.04 with kernel 5.4.0. For a single polarization parameter, the training time per test sample (400 image patches) for Logistic Regression, Ridge, CNN 1 and CNN 2 were 11.89, 11.37, 70.46 and 85.71 seconds respectively. The predicting time per test sample were 0.0039, 0.0051, 0.0800 and 0.1291 seconds respectively. Therefore, the linear classification algorithms take less time for training and validation. The non-linear CNNs are more complex and time consuming models. However, CNNs are good at automatically and adaptively learning the spatial hierarchies of image features. The angular parameter *θ, α _{q}* and

*α*classified by Logistic Regression and Ridge have higher classification accuracy than their corresponding property parameter

_{r}*δ*,

*q*and

_{L}*r*. However, the angular parameter

_{L}*θ, α*and

_{q}*α*classified by CNN 2 have lower classification accuracy than parameter

_{r}*δ*,

*q*and

_{L}*r*. In addition, the angular parameter

_{L}*θ*and

*α*classified by CNN 1 also have lower classification accuracy than parameter

_{q}*δ*and

*q*. To sum up, generally, angular parameter

_{L}*θ, α*and

_{q}*α*perform better than the corresponding property parameter

_{r}*δ*,

*q*and

_{L}*r*when using linear classification algorithms for classification. And the polarization property parameter

_{L}*δ*,

*q*and

_{L}*r*perform better than their corresponding angular parameter

_{L}*θ, α*and

_{q}*α*when using non-linear deep learning classification algorithms for classification.

_{r}Among the five polarization parameter *δ*, *P _{L}*,

*D*,

_{L}*q*and

_{L}*r*, parameter

_{L}*δ*classified by CNN 1 and parameter

*q*classified by CNN 1 and CNN 2 have the same highest classification accuracy of 77%. Parameter

_{L}*D*classified by CNN 1 and CNN 2 have the same second highest classification score of 73%.

_{L}The precision, recall, and AUC were also shown in Table 3 to evaluate the selected best combinations of polarization parameters and classification algorithms. Among the 10 polarization parameters, parameter *α _{D}* classified by CNN 1 and parameter

*α*classified by CNN 2 overall have the same highest accuracy of 87%. The precision and recall of these two combinations were the same. However, the AUC of

_{P}*α*classified by CNN 2 was 0.90 and the AUC of

_{P}*α*classified by CNN 1 was 0.84. A larger AUC represents a better classification performance. Therefore, the best combination for machine learning classification in this study was

_{D}*α*classified by CNN 2.

_{P}## 4. Conclusions

In conclusion, clinically, pathologists use H&E staining to distinguish the endometrium phases in a menstrual cycle based on their visualization experience, which is time consuming and lacks quantitative assistance. The advance in imaging technique and AI stimulate the development of digital pathology. This work combines polarization imaging with digital pathology technique to quantitatively study the microstructural features of endometrium samples. The selected polarization parameters for investigation including *δ*, *P _{L}*,

*D*,

_{L}*q*,

_{L}*r*and their corresponding angular parameter

_{L}*θ*,

*α*,

_{P}*α*,

_{D}*α*and

_{q}*α*. The quantitative digital pathology approach in this study involves using statistical properties of various polarization parameters for optimal characterization of endometrial morphological changes. Then, the incorporation of image local texture information through LBP technique improves the characterization ability of polarization parameter images. To fully extract the endometrial microstructural information contained in polarization parameter images, machine learning classifiers were applied. Compared with our previous works which were pixel based polarization parameter analysis, polarization parameter images contain extra structural information of biological samples.

_{r}The statistical mean method indicates parameter *D _{L}* has the highest class distance Score to distinguish the endometrial phases at typical proliferative phase and typical secretory phase. After extracting the image texture information through LBP process, parameter

*D*still has the highest class distance Score. Machine learning algorithms make use of all the characteristic features encoded in the images of parameter

_{L}*δ*,

*P*,

_{L}*D*,

_{L}*q*,

_{L}*r*along with their corresponding angular parameter

_{L}*θ*,

*α*,

_{P}*α*,

_{D}*α*and

_{q}*α*. Among the 10 polarization parameters, parameter

_{r}*α*classified by CNN 1 and parameter

_{D}*α*classified by CNN 2 have the same highest accuracy of 87%. The AUC of parameter

_{P}*α*classified by CNN 2 was higher than the AUC of parameter

_{P}*α*classified by CNN 1. Therefore, parameter

_{D}*α*combined with CNN 2 performs best in endometrial phase classification. Among the five polarization parameter

_{P}*δ*,

*P*,

_{L}*D*,

_{L}*q*and

_{L}*r*, parameter

_{L}*δ*classified by CNN 1 and parameter

*q*classified by CNN 1 and CNN 2 have the same highest classification accuracy of 77%. Parameter

_{L}*D*classified by CNN 1 and CNN 2 have the same second highest classification score of 73%. To sum up, the statistical mean analysis and the LBP analysis selected parameter

_{L}*D*that relates to the dichroism of the endometrial samples and is more sensitive to cell nucleus structures. The machine learning classification selected parameter

_{L}*α*and

_{P}*α*that describe the orientation of the dichroism of the endometrial samples and are more sensitive to the orientation of the cell nucleus structures. These results indicate the difference of cell nucleus structures at different endometrial phases may play an important role in endometrial phase classification. In addition, machine learning classification also selected parameter

_{D}*δ*and parameter

*q*among the parameter

_{L}*δ*,

*P*,

_{L}*D*,

_{L}*q*,

_{L}*r*. The retardance parameter

_{L}*δ*and the birefringence parameter

*q*are more sensitive to the collagen and fibrous structures of endometrial samples. The characterization of the collagen and fibrous structures in endometrial tissues may also important for classification. This study demonstrates the feasibility of combining label-free and non-invasive polarization imaging with digital pathological techniques to characterize the microstructural features of endometrium samples at typical proliferative phase and typical secretory phase.

_{L}## Funding

National Natural Science Foundation of China (11974206, 61527826); Shenzhen Technical Project (JCYJ20170412170814624).

## Disclosures

The authors declare no conflicts of interest.

## References

**1. **K. Bera, K. A. Schalper, D. L. Rimm, V. Velcheti, and A. Madabhushi, “Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology,” Nat. Rev. Clin. Oncol. **16**(11), 703–715 (2019). [CrossRef]

**2. **G. Lee, R. W. Veltri, G. Zhu, S. Ali, J. I. Epstein, and A. Madabhushi, “Nuclear shape and architecture in benign fields predict biochemical recurrence in prostate cancer patients following radical prostatectomy: preliminary findings,” Eur. Urol. Focus **3**(4-5), 457–466 (2017). [CrossRef]

**3. **C. Lu, D. Romo-Bucheli, X. Wang, A. Janowczyk, S. Ganesan, H. Gilmore, D. Rimm, and A. Madabhushi, “Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers,” Lab. Invest. **98**(11), 1438–1448 (2018). [CrossRef]

**4. **T. Araújo, G. Aresta, E. Castro, J. Rouco, P. Aguiar, C. Eloy, A. Polónia, and A. Campilho, “Classification of breast cancer histology images using Convolutional Neural Networks,” PLoS One **12**(6), e0177544 (2017). [CrossRef]

**5. **Y. Wang, H. He, J. Chang, N. Zeng, S. Liu, M. Li, and H. Ma, “Differentiating characteristic microstructural features of cancerous tissues using Mueller matrix microscope,” Micron **79**, 8–15 (2015). [CrossRef]

**6. **Y. Wang, H. He, J. Chang, C. He, S. Liu, M. Li, N. Zeng, J. Wu, and H. Ma, “Mueller matrix microscope: a quantitative tool to facilitate detections and fibrosis scorings of liver cirrhosis and cancer tissues,” J. Biomed. Opt. **21**(7), 071112 (2016). [CrossRef]

**7. **Y. Dong, J. Qi, H. He, C. He, S. Liu, J. Wu, D. S. Elson, and H. Ma, “Quantitatively characterizing the microstructural features of breast ductal carcinoma tissues in different progression stages by Mueller matrix microscope,” Biomed. Opt. Express **8**(8), 3643–3655 (2017). [CrossRef]

**8. **Y. Dong, H. He, W. Sheng, J. Wu, and H. Ma, “A quantitative and non-contact technique to characterise microstructural variations of skin tissues during photo-damaging process based on Mueller matrix polarimetry,” Sci. Rep. **7**(1), 14702 (2017). [CrossRef]

**9. **W. Wang, L. G. Lim, S. Srivastava, J. Bok-Yan So, A. Shabbir, and Q. Liu, “Investigation on the potential of Mueller matrix imaging for digital staining,” J. Biophotonics **9**(4), 364–375 (2016). [CrossRef]

**10. **H. R. Lee, P. Li, T. S. H. Yoo, C. Lotz, F. K. Groeber-Becker, S. Dembski, E. Garcia-Caurel, R. Ossikovski, H. Ma, and T. Novikova, “Digital histology with Mueller microscopy: how to mitigate an impact of tissue cut thickness fluctuations,” J. Biomed. Opt. **24**(7), 1–9 (2019). [CrossRef]

**11. **M. Mihm, S. Gangooly, and S. Muttukrishna, “The normal menstrual cycle in women,” Anim. Reprod. Sci. **124**(3-4), 229–236 (2011). [CrossRef]

**12. **T. A. Snider, C. Sepoy, and G. R. Holyoak, “Equine endometrial biopsy reviewed: observation, interpretation, and application of histopathologic data,” Theriogenology **75**(9), 1567–1581 (2011). [CrossRef]

**13. **S. Bhagwat, D. Sc, R. Kakar, S. Davuluri, A. Bajpai, S. Nayak, S. Bhutada, K. Acharya, and G. Sachdeva, “Endometrial receptivity: a revisit to functional genomics studies on human endometrium and creation of HGEx-ERdb,” PLoS One **8**(3), e58419 (2013). [CrossRef]

**14. **T. Ojala, M. Pietikäinen, and T. Mäenpää, “Gray scale and rotation invariant texture classification with local binary patterns,” in * Computer Vision - ECCV 2000*, (Springer, 2000), pp. 404–420.

**15. **https://www.mathworks.com/help/vision/Ref./extractlbpfeatures.html?s_tid=srchtitle

**16. **S. Y. Lu and R. A. Chipman, “Interpretation of Mueller matrices based on polar decomposition,” J. Opt. Soc. Am. A **13**(5), 1106–1113 (1996). [CrossRef]

**17. **N. Ghosh, M. F. Wood, and I. A. Vitkin, “Mueller matrix decomposition for extraction of individual polarization parameters from complex turbid media exhibiting multiple scattering, optical activity, and linear birefringence,” J. Biomed. Opt. **13**(4), 044036 (2008). [CrossRef]

**18. **H. He, R. Liao, N. Zeng, P. Li, Z. Chen, X. Liu, and H. Ma, “Mueller matrix polarimetry—an emerging new tool for characterizing the microstructural feature of complex biological specimen,” J. Lightwave Technol. PP, 1 (2018).

**19. **P. Li, D. Lv, H. He, and H. Ma, “Separating azimuthal orientation dependence in polarization measurements of anisotropic media,” Opt. Express **26**(4), 3791–3800 (2018). [CrossRef]

**20. **R. A. Fisher, “The use of multiple measurements in taxonomic problems,” Annals of eugenics **7**(2), 179–188 (1936). [CrossRef]

**21. **W. S. Rayens, “Discriminant analysis and statistical pattern recognition,” Technometrics **35**(3), 324–326 (1993). [CrossRef]

**22. **T. Liu, M. Lu, B. Chen, Q. Zhong, J. Li, H. He, H. Mao, and H. Ma, “Distinguishing structural features between Crohn’s disease and gastrointestinal luminal tuberculosis using Mueller matrix derived parameters,” J. Biophotonics **12**(12), e201900151 (2019). [CrossRef]

**23. **M. Sun, H. He, N. Zeng, E. Du, Y. Guo, S. Liu, J. Wu, Y. He, and H. Ma, “Characterizing the microstructures of biological tissues using Mueller matrix and transformed polarization parameters,” Biomed. Opt. Express **5**(12), 4223–4234 (2014). [CrossRef]

**24. **https://www.mathworks.com/help/stats/templatelinear.html

**25. **A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, (Curran Associates Inc., Lake Tahoe, Nevada), 1097–1105 (2012)

**26. **K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778 (2016)

**27. **D. H. Goldstein, “Mueller matrix dual-rotating retarder polarimeter,” Appl. Opt. **31**(31), 6676–6683 (1992). [CrossRef]

**28. **R. M. A. Azzam, “Photopolarimetric measurement of the Mueller matrix by Fourier analysis of a single detected signal,” Opt. Lett. **2**(6), 148–150 (1978). [CrossRef]

**29. **D. Goldstein and R. Chipman, “Error analysis of a Mueller matrix polarimeter,” J. Opt. Soc. Am. A **7**(4), 693 (1990). [CrossRef]

**30. **M. Jiménez-Ayala and B. Jiménez-Ayala Portillo, “Cytology of the normal endometrium – cycling and postmenopausal,” Monogr. Clin. Cytol. **17**, 32–39 (2008). [CrossRef]

**31. **K. P. Murphy, * Machine learning: a probabilistic perspective* (MIT press, 2012).

**32. **J. Nocedal and S. Wright, * Numerical optimization* (Springer Science & Business Media, 2006).

**33. **D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).

**34. **https://www.mathworks.com/help/deeplearning/ref/trainingoptions.html

**35. **R. Takahashi, T. Matsubara, and K. Uehara, “Data Augmentation using Random Image Cropping and Patching for Deep CNNs,” IEEE Trans. Circuits Syst. Video Technol. **30**(9), 2917–2931 (2020). [CrossRef]

**36. **T.-T. Wong, “Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation,” Pattern Recognition **48**(9), 2839–2846 (2015). [CrossRef]

**37. **T. Sun, T. Liu, H. He, J. Wu, and H. Ma, “Distinguishing anisotropy orientations originated from scattering and birefringence of turbid media using Mueller matrix derived parameters,” Opt. Lett. **43**(17), 4092–4095 (2018). [CrossRef]

**38. **T. Liu, T. Sun, H. He, S. Liu, Y. Dong, J. Wu, and H. Ma, “Comparative study of the imaging contrasts of Mueller matrix derived parameters between transmission and backscattering polarimetry,” Biomed. Opt. Express **9**(9), 4413–4428 (2018). [CrossRef]

**39. **A. G. Ushenko, I. Z. Misevich, V. Istratiy, I. Bachyns’ka, A. P. Peresunko, O. K. Numan, and T. G. Moiysuk, “Evolution of statistic moments of 2D-distributions of biological liquid crystal net Mueller Matrix elements in the process of their birefringent structure changes,” Adv. Opt. Technol. **2010**, 1–8 (2010). [CrossRef]