The analysis of polarized filtered images has been proven useful in image dehazing. However, the current polarization-based dehazing algorithms are based on the assumption that the polarization is only associated with the airlight. This assumption does not hold up well in practice since both object radiance and airlight contribute to the polarization. In this study, a new polarization hazy imaging model is presented, which considers the joint polarization effects of the airlight and the object radiance in the imaging process. In addition, an effective method to synthesize the optimal polarized-difference (PD) image is introduced. Then, a decorrelation-based scheme is proposed to estimate the degree of polarization for the object from the polarized image input. After that, the haze-free image can be recovered based on the new polarization hazy imaging model. The qualitative and quantitative experimental results verify the effectiveness of this new dehazing scheme. As a by-product, this scheme also provides additional polarization properties of the objects in the image, which can be used in extended applications, such as scene segmentation and object recognition.
© 2014 Optical Society of America
The image quality of outdoor scenes can be severely limited by atmospheric aerosols which scatter and absorb the target signal out of the optical path and scatter unwanted light into the optical path from the surroundings. To eliminate such negative effects, several dehazing methods have been developed in recent years. The existing dehazing methods can be divided into two categories: single image dehazing [1–5] and polarization-based dehazing [6–10]. Because of their ill-posed nature, single image dehazing schemes rely on various assumptions to eliminate ambiguity, including dark channel prior , local consistency , neighboring smoothness and maximizing image contrast . Polarization-based dehazing methods use as few as two polarized images taken through a polarizer at different orientations. The representative schemes in polarization-based dehazing were proposed by Schechner et al. [6–8, 10], who assumed that only airlight is polarized (OAP assumption), as shown in Fig. 1(b). In particular, Namer and Schechner  pointed out that this assumption works fine in most cases with the exception of occasional specular dielectric objects in a scene. Furthermore, they performed a correction of the airlight map by detecting incorrect areas and re-estimating the polarization property of the airlight. The correction process sharply increased the computation cost while only obvious mistakes can be corrected, such as areas of water bodies and shiny construction materials.
In fact, polarizations of the airlight and the object radiance both commonly contribute to polarization of images, as shown in Fig. 1(c). The schemes based on this overly strong OAP assumption do not work well in certain situations, especially when polarization of the object radiance is much more than the polarization of the airlight. Moreover, the related studies have shown that polarization-based methods can also provide polarization information about the scene, which is beneficial to extended image processing applications such as object recognition [11–14], scene segmentation [15–18] and material classification [12, 19, 20].
Consequently, we propose a new dehazing method that jointly considers polarization effects of both airlight and object radiance. The remainder of this paper is organized as follows. First, the related optical models are introduced in Section 2, including both polarization imaging model and polarization hazy imaging model. Then, we prove that polarization of object radiance cannot be ignored, as shown by observation data presented in Section 3. Next, in Section 4, the new dehazing algorithm is described in detail. Subsequently, the dehazing results and their comparison with existing schemes are shown in Section 5. Finally, the conclusion and discussion are presented in Section 6.
2. Optical models
2.1 Stokes vector and polarization imaging
A common representation of a polarization state is the Stokes vector representation . In this representation, a 4 × 1 column vector is assembled over a scene with x-y spatial coordinates as
where represents the total intensity of the remitted and collected light; represents the difference in intensities between the horizontal and vertical linearly polarized components; represents the difference in intensities between linearly polarized components traveling at 45° and −45° with respect to the x-axis; and represents the difference in intensities between right and left circularly polarized light. In this study, we have chosen to ignore because circular polarization is relatively rare in airlight, and it has not appeared as a major component in natural scenes.
The Stokes vector for a partially polarized beam  can be considered as a superposition of a completely polarized Stokes vector and a non-polarized Stokes vector. The polarized portion of the beam represents a net polarization ellipse traced by the electric field vector as a function of time. From the Stokes vector, the degree of linear polarization (DoLP) and the orientation angle of polarization (AOP) ellipse are given by:
Suppose a perfect linear polarizer is placed in front of a camera. The observed intensity of image at pixel is a function of the angle that the polarization analyzer lies with respect to a reference direction adopted in prior studies [22, 23], which can be described as:
2.2 Polarization hazy imaging model
The following equation approximately represents the optical energy transferred through the homogeneous atmosphere medium by absorption and scattering processes [24, 25]. The total intensity of an image is proportional to its optical energy; therefore, image intensity is equal to the first element of the Stokes vector.Equation (8) describes the medium transmittance mainly depending on the distance d between the object and the observer. Thus the transmission map t can also be regarded as a scaled range map.
where denotes the maximum intensity at position (x, y) when rotating the polarizer, and the angle of polarization analyzer at this time is called . corresponds to the minimum intensity and the angle of polarization analyzer at this time is called . We define ΔI as the polarized-difference (PD) image and as the polarized-sum (PS) image [13, 23, 26]. Equation (9) is more suitable for the wideband signals.
Similarly, the DoLP of the direct transmission and the airlight can be defined as:
3. An influence of polarization of object radiance
To prove that polarization of object radiance cannot be ignored in most cases, we select 40,000 polarized images from the two-year observation data for a fixed scene. The observation scene included sky, mountain, buildings and trees, as shown in Fig. 2(a). Directly in front of the imaging device several features were positioned at the following linear distances from the camera: a lawn (0-130 m), a pond (130-280 m), a reservoir (280-1000 m), buildings and trees (1000-6000 m), and a mountain (6000 m). The underlying topography of the light path can be categorized as a complex surface type, as shown Fig. 2(b). Owing to the high humidity of the region, haze is quite common throughout the year. Thus, it is easy to capture hazy images.
Each collected image was then segmented into five regions, as shown in Fig. 2(c). The means of the DoLP for each region of each image group were computed. The means of the DoLP for the sky region is considered as the DoLP of airlight and that of the other regions are considered as the common DoLP of both the airlight and the object radiance. The statistical results are shown in Fig. 3, where the points represent the DoLPs of corresponding regions and the points on the same dotted line represent the DoLPs of a certain image group.
Schechner et al assumed that light emanating from scene objects is not polarized, so its energy is evenly distributed between the polarization components . This means and . According to Eqs. (9) and (11), it can be concluded that the mean of the DoLP for the sky region should be greater than or equal to that of the other regions. However, it is not the case from the results shown in Fig. 3, where mean of the DoLP of the sky region is often less than or close to that of the other regions. We believe this is due to the effect of the polarization of the object radiance. Therefore, the assumption that polarization is associated only with the airlight is not always true.
Moreover, we further quantitatively analyzed on the relationship between and. It can be derived that according to Eqs. (12) and (5). If , then . If is greater than , since ,then is much larger than . This means that . This shows that the polarization of the object radiance in fact contributes greatly to the image polarization, and consequently cannot be ignored.
In fact, neither nor , should be ignored in the analysis of polarization images. There are two main reasons for this. The first reason is that the polarization of airlight varies with meteorological situations. The second reason is that the polarization of object radiance occurs with respect to the incident angle and the properties of material surface. Because polarizations of airlight and object radiance are dynamic and change considerably with specific situations, we cannot decide which one is more important. Therefore, neither of them can be ignored.
According to the above observation and analysis, a well-founded conclusion can be drawn that both polarization of airlight and object radiance typically contributes to the polarization of images.
4. Improved dehazing algorithm
4.1 Image restoration model
Since the polarization of the object radiance cannot be ignored, we will use the new polarization hazy model presented in Section 2, which considers both the polarization effects of the airlight and the object radiance, to remove the undesired haze.Eqs. (5)–(7), and (13), we can get the expression of the intensity of the scenes radiance J (i.e. dehazed image) as follows:Eq. (14), we need to estimate the following parameters: the PD image ΔI, the infinity atmospheric intensity , the DoLP of the airlight and the DoLP of the object . The PD image ΔI can be obtained from the polarized images and . The method to synthesize the PD image will be described in Section 4.2. The infinity atmospheric intensity and the DoLP of the airlight are global parameters, and the estimation methods for these parameters will be introduced in Section 4.3. As for the DoLP of the direct transmission, i.e. the DoLP of the object , a decorrelation-based method will be proposed to deal with them and will be explained in detail in Section 4.4 and 4.5.
4.2 Synthesis of PD image ΔI
Tyo et al. [13, 26, 27] proposed a measuring method for a PD image by automatically selecting the optimal orthogonal directions of the polarizer. However, since polarization properties of the object radiance in the scene are different, the best PD image cannot be captured only relying on the optimal orthogonal direction. Here, we propose a synthesis method to obtain the optimal PD image.Eq. (15), the observed image intensity is maximum at and minimum at . Since the polarization properties of the observed objects are different (i.e. is different for various object in a scene), we cannot directly capture the images and by rotating the polarizer. Here, we can solve , , and when we have three polarized images as input with the linear polarizer setting at different orientations. However, we can synthesize images and, which can be expressed as:
4.3 Estimation of and
In this study, the atmosphere was assumed to be homogeneous. Therefore, and are constant for all of the pixels in the image. As in prior studies [6–8], the pixel values of the sky region in the image can be used to estimate the two parameters, since they correspond to the airlight radiance from an object at an infinite distance. Then, the two parameters are estimated as:28].
4.4 Estimation of and t
The transmission map t depends on the scene depth and the atmospheric attenuation coefficient, while the dehazed image J represents the inherent trait of the scene object. Therefore, it is reasonable to assume that they are not statistically correlated [4, 23] over a localized set of pixels sharing the same. It can be formulated as , . However, it does not always hold up in practice due to noise. Therefore, we attempt to estimate an optimal value for:Eqs. (13) and (14), we obtain:
The optimal solution of the above equation can be obtained by solving the differential equation . The method described here is similar to the Independent Component Analysis (ICA) method reported in prior work .
4.5 Refining and
It is inevitable that some artifacts, such as block effects and estimation errors, are introduced into the results obtained based on the proposed scheme, as shown in Figs. 6(d) and 6(e). In order to overcome these problems, we apply image guided filtering  to post-process the estimated transmission map t. This filter is an excellent edge-preserving smoothing operator, so it can be used not only to filter the errors by neighborhood pixels smoothing but also to preserve the edges and structures. In this section, as t is in vector form, we use the one-dimensional symbol i to denote the pixel index for simplicity. We denote the refined transmission map by and the original transmission map by t. Rewriting them in their vector form as and t, the filtering output at a point i is expressed as:29].
Moving the filter kernel through all positions, we can obtain the refined transmission map . Then, we substitute into Eq. (13) to obtain the refined DoLP map .
Notes that only the transmission map t of R-channel is refined in our experiments, for reducing computing cost. According to Eq. (8), we can get refined and by a simple calculation. The expressions are and , where and , and N is image size.
5. Experimental results
Figure 4 shows the flowchart of the proposed scheme consisting of three steps. The first step aims at synthesizing the polarized images and . In the second step, we estimate the key model parameter based on a decorrelation-based method. The other two parameters, and t, are also estimated in this step. Finally in the third step, in order to correct the estimations and eliminate the block effects, we apply an image guided filtering to the transmission map t. Then, the improved t is used to refine. After that, the recovered image J is obtained according to Eq. (14).
5.2 Experimental data
Aiming at developing schemes for continental aerosols and marine aerosols, we selected two typical hazy image sets to test our algorithm. First, we captured several sets of images at different level of atmospheric visibility in the city of Hefei, a typical inland city with air pollution in China. Therefore, image degradation is mostly due to continental aerosols. Typical examples are shown in Figs. 5(a)–5(d). Then, we captured sets of images on the island of Jinmen in the coastal East China Sea region, as shown in Figs. 5(e) and 5(f). The degradation of these images is mainly due to marine aerosols. These polarized images are captured by an automatic rotating-polarizer. The next step is to restore the degraded images using the proposed algorithm.
5.3 Dehazing experiments
In this section, the performance of the proposed scheme is tested on polarized hazy images. Taking Scene 1 as an example [as shown in Fig. 5(a)], the flow of the process and the interim results are shown in Fig. 6. Since objects have different polarization properties, as shown in Fig. 6(c), the PD map shows the outline the contour of the objects. Accordingly, the PD map can be of great use for image processing applications, such as recognition and scene segmentation. There are some isolated bright spots distributed in the sky and building regions in Fig. 6(e), which are obviously errors. These errors can be corrected in the refined transmission map, as shown in Fig. 6(f). Furthermore, the refined transmission map succeeds in capturing the sharp edge discontinuities. Comparing the results shown in Figs. 6(d) and 6(g), the DoLP map of the refined object is much smoother. Figure 6(i) shows the dehazing result with the rough , which has apparent errors in the regions outlined by the red rectangle and more noise in sky region than the final dehazing result Fig. 6(j). The contrast of Fig. 6(j) is 0.2501, and the contrast of hazy image is 0.0970. It can be seen that the proposed scheme can effectively improve image contrast and recover the details. Here the contrast is calculated according to the reference .
To evaluate the performance of proposed scheme on hazy images caused by marine aerosols, we also conducted experiments on Scenes 5 and 6. As can be seen in Fig. 7, the proposed approach can unveil the details and recover vivid color information even in some very dense haze regions. The experimental results show that the proposed scheme can also work well under the hazy weather of marine aerosols.
5.4 Dehazing comparison with and without considering
Different from the existing polarization-based dehazing algorithms, the proposed algorithm considers the polarization of both airlight and object radiance. As mentioned in prior studies [6–8], if only the polarization of airlight is considered, the real radiance of a scene, J, is expressed as follows:
Taking Scene 2 as example, we perform several experiments to compare the proposed scheme with those that only consider the polarization of airlight [6–8]. Comparing Fig. 8(a) (contrast: 0.1170)with Fig. 8(b) (contrast: 0.1932), it can be seen that there is a significant improvement in image contrast, especially at the region outlined by the red rectangle in the dehazing results. The enlarged red rectangles shown in Figs. 8(c) and 8(d) are more apparent.
To further evaluate the effectiveness of the proposed model, another two sets of experiments are carried out the results are shown in Fig. 9, whose input images are shown in Figs. 5(c) and 5(d). They are polarized images of the same scene at different atmospheric visibility. For Fig. 5(c), the DoLP of the sky region is similar to that of the object region. For Fig. 5(d), the DoLP of the sky region is less than that of the object region. Figures 9(a) and 9(c) (the contrasts: 0.1170 and 0.0344) are the dehazed images without considering the DoLP of object, which have lower contrast in comparison to the results shown in Figs. 9(b) and 9(d) (the contrasts: 0.1932 and 0.0572).
We calculate the noise levels of dehazing images according to the reference , which is shown in Table 1. From these experimental results, we can see that the noise level is higher in the dehazed images without considering . The value of in Eq. (22) is usually small. When is also very small, the image noise can be easily amplified in the process of image recovery. As for the model characterized by Eq. (14), the denominator is larger than that of Eq. (22) due to the contribution of the term . Therefore, the proposed scheme can recover the dehazed image with reduced noise level.
5.5 Dehazing comparison with the method of Schechner et al.’s
In prior work reported in [6–8], the input images are the best and the worst polarized images, because it is assumed that polarization is associated only with the airlight. Therefore, in order to compare our results with that of previous studies [6–8], we used the best and worst polarized images to approximate and images.
Comparing Figs. 10(b) and 10(c), it can be seen that the proposed scheme can achieve a better performance, especially in the region of specular reflection (outlined by the red circle in the dehazing results). This is because the scheme in  assumes that the light reflected from scene object has no significant polarization. This assumption fails to characterize the region of specular reflection. In order to deal with this situation, additional steps have to be applied to process the specular surface area individually, as in . In contrast, the proposed scheme simultaneously considers the light polarization from object radiance and airlight, and consequently achieves better results without additional post-processing. The quantitative comparison between Figs. 10(b) and 10(c) are also shown in Table 2.
5.6 The quantitative evaluation of dehazed images
Due to the lack of ground truth images, we adopt the no-reference natural image quality evaluator (NIQE)  to quantitatively assess and compare the dehazing results. The NIQE could evaluate total image quality including image structure, image noise level, image blur degree, edge sharpness, and so on. The NIQE model constructed a collection of quality awareness features and then fit them to a multivariate Gaussian (MVG) model. The quality awareness features were derived from a spatial natural scene statistic (NSS) model, which was the framework of locally normalized luminance coefficients. Mittal et al.  showed that the natural image normalized luminance coefficients closely follow a Gaussian-like distribution, but the degraded image did not hold well for this distribution. Thus, the quality of a given test image was simply expressed as the distance between an MVG fit of the NSS features extracted from the test image and an MVG model from the corpus of natural images. Mathematically, this is defined as:
We ran the NIQE code (http://live.ece.utexas.edu/research/Quality/index.htm) to evaluate the dehazed images. The results are shown in Table 2. Values in Table 2 represent the distance between the model statistics of clear natural images and those of the dehazed image. The smaller the value is, the closer the dehazed image is to the clear images. The quantitative evaluation shows that all of the dehazed images based on the proposed scheme have been improved over the images processed by the existing schemes that only considered the polarization of airlight.
In this paper, a new approach is presented for the recovery of haze-free images from polarized hazy images. The main difference between this scheme and the existing polarization dehazing algorithms is that this scheme considers the polarization of the airlight and the object radiance jointly. Three novel aspects are studied, including (1) a new polarization hazy imaging model including and ; (2) a decorrelation-based algorithm to obtain from polarized hazy images; (3) an effective method to synthesize the optimal PD image. After obtaining these key parameters, the haze-free image can be recovered according to polarization hazy imaging model. The qualitative and quantitative experimental results show that our dehazing scheme can considerably improve the image quality in contrast, detail and signal-to-noise ratio.
This scheme can also generate two valuable derivative results, the PD image and the DoLP of the object. Unlike the traditional approach which uses the selection of measuring two optimal orthogonal directions of the polarizer , the PD image is obtained by synthesizing the polarized images and . Since the polarization properties of the observed scene vary, the optimal PD image is actually unable to be directly measured in practice. However, the proposed scheme can obtain the optimal PD image, which can be applied into object perceptions, especially in foggy weather or underwater surroundings. In addition, this scheme can separate the DoLP of an object from the DoLP of an image. This scheme can obtain a much more stable and accurate DoLP of an object, comparing with the traditional approach which uses the DoLP of an image as a substitute. Since the DoLP of an object represents an essential attribute of the object, it will be useful in object recognition and scene segmentation.
Note that the proposed scheme requires that the input images contain some sky areas in order to estimate the parameters of airlight. However, the sky is sometimes cannot be seen within the field-of-view. Tarel  proposed an atmospheric veil and an algorithm of inferring atmospheric veil. In our future work, we will apply an atmospheric veil of polarized images to obtain the polarization of airlight.
The authors thank Prof. Rao Ruizhong and Dr. Wu Pengfei for fruitful discussions. This work was supported by the National Natural Science Foundation of China (No.61175033) and the Fundamental Research Funds for Central Universities (No.2010HGXJ0018, No.2012HGCX0001).
References and links
1. K. M. He, J. Sun, and X. O. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010). [PubMed]
3. J. P. Tarel and N. Hautiere, “Fast visibility restoration from a single color or gray level image,” in Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 2201–2208. [CrossRef]
4. R. Fattal, “Single image dehazing,” ACM Trans. Graph. 27(3), 988–992 (2008). [CrossRef]
5. R. T. Tan, “Visibility in bad weather from a single image,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8. [CrossRef]
6. Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Instant dehazing of images using polarization,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2001), pp. 325–332. [CrossRef]
8. E. Namer and Y. Y. Schechner, “Advanced visibility improvement based on polarization filtered images,” Proc. SPIE 5888, 36–45 (2005).
10. S. Shwartz, E. Namer, and Y. Y. Schechner, “Blind haze separation,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 1984–1991. [CrossRef]
11. M. Saito, Y. Sato, K. Ikeuchi, and H. Kashiwagi, “Measurement of surface orientations of transparent objects using polarization in highlight,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1999), pp. 381–386. [CrossRef]
12. H. Chen and L. B. Wolff, “Polarization phase-based method for material classification and object recognition in computer vision,” Proc. SPIE 2599, 54–63 (1996). [CrossRef]
15. L. B. Wolff, “Using polarization to separate reflection components,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1989), pp. 363–369. [CrossRef]
16. M. Ben-Ezra, “Segmentation with invisible keying signal,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2000), pp. 32–37. [CrossRef]
17. K.M. Yemelyanov, S.S. Lin, E.N. Pugh, Jr., and N. Engheta, “Polarization-based segmentation for enhancement of target detection in adaptive polarization-difference imaging,” in Frontiers in Optics, OSA Technical Digest Series (Optical Society of America, 2005), paper JWA51.
18. S. S. Lin, K. M. Yemelyanov, E. N. Pugh Jr, and N. Engheta, “Separation and contrast enhancement of overlapping cast shadow components using polarization,” Opt. Express 14(16), 7099–7108 (2006). [CrossRef] [PubMed]
19. L. B. Wolff, “Polarization-based material classification from specular reflection,” IEEE Trans. Pattern Anal. Mach. Intell. 12(11), 1059–1071 (1990). [CrossRef]
20. S. Tominaga and A. Kimachi, “Polarization imaging for material classification,” Opt. Eng. 47(12), 123201 (2008). [CrossRef]
21. M. Bass, Devices, Measurements, and Properties, Vol. 2 of Handbook of Optics (McGraw-Hill, 1995), Chap. 22.
22. M. W. Hyde, S. C. Cain, J. D. Schmidt, and M. J. Havrilla, “Material classification of an unknown object using turbulence-degraded polarimetric imagery,” in Proceedings of IEEE Transactions on Geoscience and Remote Sensing (IEEE, 2010), pp. 264–276. [CrossRef]
23. K. M. Yemelyanov, S. S. Lin, E. N. Pugh Jr, and N. Engheta, “Adaptive algorithms for 2-channel polarization sensing under various polarization statistics with nonuniform distributions,” Appl. Opt. 45(22), 5504–5520 (2006). [CrossRef] [PubMed]
25. S. G. Narasimhan and S. K. Nayar, “Vision and the atmosphere,” Int. J. Comput. Vis. 48(3), 233–254 (2002). [CrossRef]
27. J. S. Tyo, “Optimum linear combination strategy for an N-channel polarization sensitive vision or imaging system,” J. Opt. Soc. Am. A 15(2), 359–366 (1998). [CrossRef]
28. D. Hoiem, A. A. Efros, and M. Hebert, “Automatic photo pop-up,” ACM Trans. Graph. 24(3), 577–584 (2005). [CrossRef]
29. K. M. He, J. Sun, and X. Tang, “Guided image filtering,” in Proceedings of European Conference on Computer Vision, K. Daniilidis, P. Maragos, N. Paragios, eds. (Berlin, 2010), pp. 1–14.
30. R. M. Haralick, K. Shanmugam, and I. H. Dinstein, “Textural features for image classification,” IEEE Trans. Syst. Man Cybern. 3(6), 610–621 (1973). [CrossRef]
32. A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a completely blind image quality analyzer,” in Proceedings of IEEE Conference on Signal Processing Letters (IEEE, 2013), pp. 209–212.