We address an image segmentation method to detect concealed objects captured by passive millimeter wave (MMW) imaging. Passive MMW imaging can create interpretable imagery on the objects concealed under clothing, which gives the great advantage to the security system. In this paper, we propose the multi-level expectation maximization (EM) method to separate the concealed objects from the other area in the image. We apply the EM method to obtain a Gaussian mixture model (GMM) of the acquired image. In the experiments, we evaluate the performance by the average probability of error. We will show that the consecutive EM processes separates the object area more accurately than the conventional EM method.
©2010 Optical Society of America
Recently, the researches on passive millimeter wave (PMMW) imaging techniques have increased for the security and defense applications [1–3]. The PMMW system forms interpretable imagery in the low-visibility conditions such as haze, fog, clouds, smoke, or sandstorms. The millimeter wave provides the penetrability into specific materials such as paper, plastic, clothing and fabrics [4–6]. Thus, the MMW imaging can identify concealed objects under clothing. The passive MMW imaging system largely depends on the temperature distribution of the objects with their emissivity and reflectivity [7–9]. Since the received signal is usually a weak thermal signal, the sensitivity and the noise robustness of the system are crucial to acquire high quality images. However, the passive imaging system is free of speckle and glint, and has a simple and cost-effective hardware compared to the active imagery system .
Image segmentation is a process to separate the foreground object from the background area . The segmentation research has been extended to medical and microscopic imaging [12,13]. Various segmentation methods have been developed to detect the concealed objects on the body area in the MMW images [14–16]. One of the typical methods for the segmentation is a thresholding method [14,15]. The thresholding method is very effective when the image has the bimodal distributions and the distributions are located far enough from each other . The multi-level thresholding was addressed to detect the contours of the concealed objects from Terahertz (THz) images . The image segmentation using the Gaussian mixture model (GMM) has been researched on the metallic object detection .
In this paper, we propose the multi-level expectation maximization (EM) method to segment concealed objects from the body area. The histogram of the passive MMW image is modeled as a Gaussian mixture distribution. The EM method is an iterative solution for the maximization likelihood (ML) estimation of the GMM . It has been applied to the three-dimensional color object visualization and recognition . In the GMM, the probability density function (PDF) for each cluster is assumed to be Gaussian. The multi-level EM method finds two major clusters at each level. The EM algorithms are consecutively applied to one cluster, which is defined at the previous EM level. Thus, the first level EM separate the body area from background and the second level EM finds the concealed object in the body area. The performances of the conventional EM method and the proposed method are compared by the average probability of error [19–21]. In the experiments, it will be shown that the multi-level EM outperforms the conventional EM method.
The paper is organized as follows. In Section 2, we introduce the passive millimeter wave imaging system. The image segmentation using the multi-level EM algorithm is discussed in Section 3. In Section 4, we present the experimental results with the performance evaluation. The conclusions follow in Section 5.
2. A passive millimeter wave imaging system
The passive MMW imaging system is equipped with a Cassegrain dish antenna with a diameter of 0.5 m, as illustrated in Fig. 1 . Two feed horn antennas are located at the focal plane of the antenna receiving the regime of 8 mm wavelength with vertical and horizontal polarization. They are apart from each other by 2 cm in the horizontal direction. The receiver channel connected to the feed horn is composed of the wave guide, the Dicke receiver, three monolithic microwave integrated circuit (MMIC) amplifiers, and a shottky diode detector. The scanning mechanism rotates both the antenna and the receiver in vertical and horizontal directions with a constant angular step. The angular resolution is around 1.1° according to the Rayleigh criterion since λ is 8 mm and D is 0.5 m. The scanning ranges are 60° and 106° in vertical and horizontal directions, respectively. The angular step size and the integrating time can be modified from 0.1° to 1° and from 20 ms to 200 ms, respectively.
3. Image segmentation for concealed weapon detection
The overall process of the segmentation is composed of the Gaussian smoothing filtering (GSF), two-level image segmentations using the EM method, and decision processes as illustrated in Fig. 2 .
We adopt the GSF to remove the noise on the body contour for edge enhancement and the noise in the background for better visualization. The standard deviation and the window of the Gaussian smoothing filtering (GSF) are set at 0.5 and a 3 × 3 matrix, respectively. The first level EM extracts the body area from the background. The first Bayesian decision rule is followed by body area detection, which removes any segmented object inside the body area. During this process, we also perform the morphological erosion of the body area in order to estimate the accurate contour of the body area. The erosion process is also required since the body area has been dilated by the GSF. The second level EM is followed by the second Bayesian decision rule to detect the concealed objects in the body area.
At each level of the EM process, the histogram is modeled with two components of the Gaussian mixture distributions. The PDF of each pixel is assumed as follows17,18]
Since Eqs. (3)–(6) are highly interated non-linear functions, the EM algorithm is adopted to solve the ML estimation. The EM algorithm uses a set of training data to iteratively estimate the parameters until convergence. The disadvantage of the EM algorithm can be its slow convergence, and dependent of the initial values. However, it is noted that the proposed method does not require deciding the number of the component. Figure 3 shows the block diagram of the EM algorithm; i represents the number of the iteration, ε and are the termination criteria for the iteration, and is the likelihood . In the experiments, the size of the body is assumed to be comparable with the background and the concealed object is assumed to be smaller than the body area. Thus, we set the initial probabilities of the clusters for the first level EM as and , respectively, to extract the body area, and for the second level EM as and , respectively, to extract the concealed object. It is noted that the initial probabilities can defend on the normality of data. We have decided the initial probabilities heuristically when the best results are provided. The termination criteria is set ε at and and at 200 and 500 for the first and the second level EM, respectively.
In the experiment, we will show that the first EM separates the body area from the background area. The object is extracted from the body area by the second EM.Eqs. (9)–(12), the Bayesian probability error becomes
It is noted that, in the experiments, is the concealed object and is the body area except for the concealed object.
4. Experimental and simulation results
In this section, we present the segmentation results of the several PMMW images capturing the concealed objects. The several concealed objects are captured by the passive MMW system. Figures 4(a) and 4(b) are the images obtained by a visual camera. They represent a human subject and the object attached to the body, respectively. Figures 4(c) and 4(d) are the PMMW images with 8 mm horizontal and 8 mm vertical polarization, respectively. The bright part inside the body is the concealed axe.
Figures 5(a) and 5(b) are the fitting results by using the GMM to the histogram of the image after the first and second level EM segmentation, respectively. Figures 5(c) and 5(d) are the segmented body area and concealed object area after the 1st and 2nd segmentation processes. By fitting the Gaussian mixture distributions to the histograms, we can visualize how exactly the EM estimate the GMM models in Figs. 5(a) and 5(b).
Figure 6 shows other experimental results by the multi-level EM method. The first column images are obtained by a visual camera without the concealment. The second column images are the passive MMW images. The third column images are the segmented images using the conventional EM method. The fourth column images are the segmented images using the proposed method. As show in the segmentation results, the proposed method provides the better results than the conventional EM method.
Figure 7 shows the performance evaluation by the average probability of error. The multi-level EM has increased the performance by 36.7%, 39.4%, 68.4%, 90.0%, respectively, from the axe, gun, skin-aid, and water-pack. The average improvement is around 58.7%.
In this paper, we have presented the image segmentation techniques for concealed weapons detected with the multi-level EM method. The passive MMW images usually have the low contrast and resolution. Also, the images are often noisy due to the low-signal level. Therefore, a special segmentation method has been developed to detect the concealed objects. This method can be extended for multi-channel images in the future, taking advantage of the polarization effects and diversity. We also leave the comparison with other segmentation approaches with non-Gaussian mixture models or GMM with normality enhancement for the future research topic.
We are grateful to Mr. Hyoung Lee and Mr. Vladmir. P. Guschin for their experimental support in passive MMW imaging. This work was supported by the Korea Science and Engineering Foundation (KOSEF) grant funded by the Korea government (MEST) (No. R01-2008-000-2118-0).
References and links
1. H.-M. Chen, S. Lee, R. M. Rao, M.-A. Slamani, and P. K. Varshney, “Imaging for concealed weapon detection: a tutorial overview of development in imaging sensors and processing,” IEEE Signal Process. Mag. 22(2), 52–61 (2005). [CrossRef]
2. R. Appleby and R. N. Anderton, “Millimeter-wave and submillimeter-wave imaging for security and surveillance,” Proc. IEEE 95(8), 1683–1690 (2007). [CrossRef]
4. L. Yujiri, M. Shoucri, and P. Moffa, “Passive millimeter-wave imaging,” IEEE Microw. Mag. 4(3), 39–50 (2003). [CrossRef]
5. J. Son, V. P. Guschin, A. G. Denisov, S. Yeom, and D. Jung, “Comparisons of 3 dimensional temperature distributions of mm and IR images,” Proceedings of IEEE Conference on Microwaves, Radar and Remote Sensing Symposium, MRRS, Kiev, 132–135 (2008).
7. M. R. Fetterman, J. Grata, G. Jubic, W. L. Kiser Jr, and A. Visnansky, “Simulation, acquisition and analysis of passive millimeter-wave images in remote sensing applications,” Opt. Express 16(25), 20503–20515 (2008). [CrossRef] [PubMed]
8. K. B. Cooper, R. J. Dengler, N. Llombart, T. Bryllert, G. Chattopadhyay, I. Mehdi, and P. H. Siegel, “An approach for sub-second imaging of concealed objects using terahertz (THz) radar,” Int. J. Infrared Millim. Waves 30, 1297–1307 (2009).
9. A. Denisov, V. Gorishnyak, S. Kuzmin, V. Miklashevich, V. Obolonskv, V. Radzikhovsky, B. Shevchuk, B. Yshenko, V. Uliz'ko, and J. Son, “Some experiments concerning resolution of 32 sensors passive 8 mm wave imaging system,” Proceedings of the 20th International Symposium on Space Terahertz Technology, Charlottesville, 227–229 (2009).
10. M.-K. Jung, Y.-S. Chae, S.-K. Kim, M. Koji, and J.-K. Rhee, “Studies on the millimeter-wave passive imaging system,” J. Korean Inst. Electr. and Electro. Mater. Eng. 43, 182–188 (2006).
11. R. C. Gonzalez, Digital Image Processing 2/E (Prentice-Hall Inc., 2003).
13. T. Collier, P. Shen, B. de Pradier, K.-B. Sung, R. Richards-Kortum, M. Follen, and A. Malpica, “Near real time confocal microscopy of amelanotic tissue: dynamics of aceto-whitening enable nuclear segmentation,” Opt. Express 6(2), 40–48 (2000). [CrossRef] [PubMed]
14. D.-S. Lee, S. Yeom, J. Son, and S.-H. Kim, “Image segmentation of concealed objects detected by a PMMW imaging system,” presented at the 34th International Conference on Infrared, Millimeter, and Terahertz Waves, Paradise Hotel, Busan, Korea, 21–25 (2009).
15. X. Shen, C. R. Dietlein, E. Grossman, Z. Popovic, and F. G. Meyer, “Detection and segmentation of concealed objects in terahertz images,” IEEE Trans. Image Process. 17(12), 2465–2475 (2008). [CrossRef] [PubMed]
16. C. D. Haworth, Y. Saint-Pern, D. Clark, E. Trucco, and Y. R. Petillot, “Detection and tracking of multiple metallic objects in millimeter-wave images,” Int. J. Comput. Vis. 71(2), 183–196 (2007). [CrossRef]
17. A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc., B 39, 1–38 (1977).
18. S. Yeom, B. Javidi, P. Ferraro, D. Alfieri, S. Denicola, and A. Finizio, “Three-dimensional color object visualization and recognition using multi-wavelength computational holography,” Opt. Express 15(15), 9394–9402 (2007). [CrossRef] [PubMed]
19. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification (John Wiley & Sons, 2001).
20. K. Fukunaga, Introduction to Statistical Pattern Recognition 2nd (Academic Press, Boston, 1990).
21. W. A. Yasnoff, J. K. Mui, and J. W. Bacus, “Error measures for scene segmentation,” Pattern Recognit. 9(4), 217–231 (1977). [CrossRef]