Open Access
Add to BibSonomyAbstract
Many dehazing methods have proven to be effective in removing haze out of the hazy image, but few of them are adaptive in handling the dense haze. In this paper, based on the angle of polarization (AOP) distribution analysis we propose a kind of polarimetric dehazing method, which is verified to be capable of enhancing the contrast and the range of visibility of images taken in dense haze substantially. It is found that the estimating precision of the intensity of airlight is a key factor which determines the dehazing quality, and fortunately our method involves a high precision estimation inherently. In the experiments a good dehazing performance is demonstrated, especially for dense haze removal. We find that the visibility can be enhanced at least 74%. Besides, the method can be used not only in dense haze but also in severe sea fog.
© 2015 Optical Society of America
1. Introduction
The image quality in turbid media may be limited, because the light reflected from the natural scene is partially absorbed by haze particles, meanwhile, some unwanted scattering light, which is usually called as the airlight, is added in. Both of these two effects degrade the contrast of images, and some detailed information might be submerged into the noise. As a result, it is quite inconvenient for people to do further image processing by using these hazy images. Therefore, in past decades, many dehazing methods have been proposed to enhance the quality of hazy images, and some of them are truly impressive.
Image-dehazing methods can be roughly categorized into two kinds: the methods based on computer visions and those based on physical models. The advantage of computer-vision-based methods is that they can do the dehazing process by utilizing only one single image [1–5]. However, each of these dehazing methods may be just suitable for some specific hazy images, which is determined by their different key-parameter extracting algorithms. On the other hand, image-dehazing methods based on physical models, which can be divided into polarization-based dehazing methods [6–10] and multiresolution-fusion-based methods [11,12], are usually effective in all kinds of turbid media, including fog, haze and turbid water, etc. Besides, these methods have extra capability of retaining the detailed information. However, several raw images are needed to finally fuse into one dehazed image.
Although all these methods mentioned above can enhance the visibility of hazy images, they mainly focus on the haze which is not dense enough. When images are captured in dense haze, the light reflected from the scene is severely attenuated and at the meantime the airlight becomes stronger. Therefore, the light reflected from the scene is submerged into the airlight and the visibility of the images is extremely low. In this situation, it is difficult to distinguish these two kinds of irradiances in one single image, so it has to take much more time and complex algorithms for computer-vision-based methods to do the dehazing process. On the other hand, owing to the fact that infrared light can travel a longer path comparing with the visible light, multiresolution-fusion-based dehazing method, which utilizes the infrared information to do the dehazing process, has natural advantage in handling the dense haze. In [11], experimental results for dehazing the images with the dense haze demonstrate that the contrast is indeed enhanced; however, the noise is also amplified. This noise is mainly caused by the camera, and can be depressed by averaging several images with the same scene. Note that, in the polarization-based method, four images with the same scene need to be captured so the noise can be suppressed [13]. It implies that if polarization-based dehazing method can handle dense hazy images, the quality of dehazed images may be better. Besides, the cost of an infrared camera is quite high; while polarization-based dehazing methods are normally based on ordinary color camera, so it can be predicted that polarization-based dehazing method has broader potential application prospects. In this paper, aiming at handling the images captured in dense haze, we optimize our proposed polarimetric dehazing model [14] and propose a dehazing method based on the angle of polarization (AOP) distribution analysis of the airlight. The experimental results show that this method can effectively enhance the contrast of dense-hazy images, especially for enhancing the range of visibility in dense-hazy situtations.
2. Technique
The basic physical model of polarimetric dehazing method is proposed in [5]. In hazy weather, the irradiance received by the camera, I, is composed of the irradiance reflected from the scene through the haze (D) and the partial-polarized airlight irradiance (A), and it can be expressed as
where D and A can be respectively given by where L is the irradiance reflected from the scene without being attenuated by the haze, A∞ is the airlight corresponding to an imaginary object at an infinite distance, and t(z) is the transmittance of atmosphere [15]. In our case, the extinction coefficient β is assumed to be distance invariant, so t(z) can be easily expressed asCombining the above equations, one can find that the expression of L can be written asNote that the calculated L corresponds to the final dehazed image.From Eq. (5), we can see that A and A∞ are the two key parameters in the polarimetric dehazing method. The estimating algorithms of them have to be optimized in dense haze, because the behavior of images captured in dense haze is quite different from that in normal haze. The differences can mainly be summarized from the following aspects:
- a) The light reflected from the scene in long distance is severely attenuated. In this case, the range of visibility enhancement is much more important compared to the contrast enhancement.
- b) The quantization noise caused by the camera becomes a big problem in dense-hazy removal matter. Unlike normal haze, the signal in images captured in dense haze often possesses the same gray level as the noise. Therefore, when the signal is amplified enough for observation, the noise may inevitably experience a considerable amplification simultaneously.
- c) The polarization states of the light in long distance are quite different from that in short distance. As a result, at the cost of remarkably promote the range of visibility, some information in short distance may have to be lost.
Based on these facts mentioned above and our polarimetric dehazing model [14], we propose a new polarimetric dehazing method for dense haze removal. The scheme for precisely estimating the above two key parameters are described in the following.
First, put a linear polarizer in front of the camera. And four images with different angles of rotated polarizer need to be captured. In our case, we choose the polarizer’s angle to be 0°, 45°, 90° and 135°, and thus the intensities of four image are represented as I(0), I(45), I(90) and I(135), respectively. The Stokes parameters can be written as
where S0 represents the total intensity of the incident light, S1 and S2 represent the polarization states of the incident light. Note that the quantization noise is a kind of random noise and thus can be suppressed by averaging several images with the same scene. We firstly suppress the noise of S0 by adding the four polarized images, seeing Eq. (6).Similarly, S1 and S2 also suffer from severe noise. And from the expression of AOP θ shown in Eq. (7) we can see that these noises will lead to an imprecised estimation of AOP.
Figure 1 is one polarized image I(0) we took in the hazy weather, and its content is the sky region in dense haze. When θ is directly calculated by S1 and S2, the statistical graph of the AOP values is shown in Fig. 2(a). It is seen that the statistical graph is ruleless. The inset of Fig. 2(a) is the distribution of the AOP values in the image, from which the influence of noise is clearly indicated. Therefore, a preprocessing on AOP distribution analysis is needed to suppress this noise. Here, we find that the intensity of each pixel in raw hazy images can be modified by averaging the intensity itself and that of other pixels in a small surrounding window. The size of the window is chosen to 7 × 7 in our case and this size can provide adequate pixel values for averaging. This process can effectively smooth the raw images and reduce the noise. Note that one limitation needs to be appointed, where the difference between two adjacent pixel values should be less than 6. This limitation is helpful to distinguish whether these two pixels reflect the same object. Then, S1 and S2 with less noise can be obtained by the modified raw images. Finally, the AOP can be calculated by the optimized S1 and S2. After this process, one can obtain the statistical graph of the AOP values as shown in Fig. 2(b), it is seen that the distribution of the curve is more regular, and the inset of Fig. 2(b) contains less noise than that of Fig. 2(a). According to this figure, we set θ with the most probability as θA. In fact, in a hazy image the polarization state of the light reflected from the natural scene is usually very small, but that of the airlight is larger and distance invariant. Therefore, θA can be regarded as the AOP of the airlight. Also note from Fig. 2(b) that those pixels whose AOP values are θA almost concentrate in the center of the figure, this implies that such a horizontal sky area is more suitable for the estimation of θA. This can be explained that the incident lights in the upper and lower regions have different incident angles, which results in the calculation error. Moreover, it should be pointed out that the result shown in Fig. 2(b) has a good accordance with that in [6].
Fig. 2 (a) The AOP value of the incident light directly calculated by Eq. (7). (b) The AOP value of the incident light calculated by Eq. (7) after smoothing each pixel value of the four raw images.
Second, let us introduce how to determine the intensity of airlight A after obtaining θA. As shown in [14], A can be expressed by
where p is the degree of polarization (DOP) and determined by . Owing to the fact that S1, S0 and θA are all optimized, p can be much more precise. Note that in our algorithm, those pixels with θ values in the range of (1 ± 0.001)∙θA are chosen to be used for precisely estimating the DOP of the airlight, pA. Among them the largest DOP is regarded as pA.Note that, in principle, although A can be obtained by either of the last two terms of Eq. (8), the one with the bigger value between cos2θA and sin2θA is preferred, because small denominator may excessively amplify the noise. However, we need to point out that Eq. (8) is usually suitable for the situation of normal haze, not for the dense haze. As for the latter, the polarization states of the light reflected from the scene via a long transmission distance suffers from severely absorbing and scattering because of the haze particles. As a result, compared with the airlight, this part of light can be neglected in intensity and polarization state, which means the DOP of the total intensity can be regarded as pA in this part. Therefore, when the parameter p is substituted by pA in Eq. (8), the information of the long distant scene can be well restored and the noise introduced by p is eliminated. But the information of short distance scene might be somewhat lost.
Third, the method of estimating A∞ is detailedly described in our previous work [16], which is helpful to handle the hazy images without sky region.
In principle, after having obtained all these parameters, the dehazing process can be conducted according to Eq. (5). However, it is seen that the denominator in Eq. (5) might be close to zero if some pixels’ As are approximately close to A∞, which will bring in huge error to L. Therefore, for A or A∞, some modifications have to be done. Note that A reflects the airlight distribution of the whole image, while A∞ is only a scalar parameter, so we find that the modification of A is much more effective to the result of dense haze removal. However, two problems need to be further taken into consideration for modifying A. One is reducing the information of the scene in A. Due to our assumption of the DOP, some residual information of the scene has to be left in A, and this information should be reduced. Two parameters θA and pA directly affect the precision of estimating A, which can be seen from Eq. (8). Since θA has already optimized by the AOP distribution analysis as mentioned above, this problem can be further relieved by introducing a bias factor ε to pA, i.e., using ε∙pA substituting for pA in Eq. (8). The optimized expression of A can be written as:
Note that similar treatment of modulating pA has ever been adopted in some works of polarimetric dehazing [6, 8], but the treatment in our case is made in both numerator and denominator, which will lead to a more precise estimation of A. The other one is suppressing the noise in A. This process is very important in dehazing the dense haze. In our previous work, we found that our proposed polarimetric dehazing method based on the AOP of the airlight can suppress the noise better than theirs based on the difference of the two orthogonal image [16]. Here, we demonstrate that the noise can be further depressed by utilizing the smoothing method we used to estimate the θA.3. Dehazing process
In this paper, we focus on the condition of dense haze to find out whether our proposed polarimetric dehazing method can enhance the range of visibility, effectively retain the detailed information and meanwhile reduce the influence of the noise. The raw images are taken in hazy weather with the air quality index (AQI) of 264. This value reflects that the pollution level is heavily polluted [17] and usually the range of visibility is limited to only several hundred meters. Figure 3 shows the four polarized images taken in this haze, and 3(a-d) are I(0), I(45), I(90) and I(135), respectively.
Fig. 3 (a-d) are the polarized images taken with the specific angle of the polarizer 0°, 45°, 90° and 135°, respectively.
We transform Fig. 3 into gray images as an example to demonstrate the estimation of A. It should be noted that the value of ε is a key parameter in estimating A. Here, the appropriate and inappropriate values of ε are set to 1.61 and 3.61, respectively, and the detailed discussion on ε is presented in the following paragraph. Before the smoothing process, the original A of the image is given in Fig. 4(a) while the smoothed A of the image is shown in Fig. 4(b), with ε = 1.61. It is easily seen that the noise of Fig. 4(a) is severer than 4(b). Besides, we can see from Fig. 4(b) that some residual information of the scene in short distance is left in A, which will make this part of the dehazed image become a little darker. Figure 4(c) shows the A after smoothing with ε = 3.61. By comparing Fig. 4(b) with 4(c), the information of the scene in long distance is a little stronger in 4(c), which implies that this part of information will get lost during the dehazing process. Figure 5 (a-c) show the dehazing results with the A given in Fig. 4(a-c), respectively. It is clearly seen that the detail information of Fig. 5(a) is submerged into the severe noise, and that of Fig. 5(c) is also seriously lost. As a result, the buildings in distance can hardly be seen. In comparison, Fig. 5(b) is much better, the contrast of the image is increased, the detailed information is retained, and meanwhile the range of visibility is enhanced. The result in addition to verify our dehazing method for dense haze removal effective, it also indicates that the value of ε is pivotal to the final quality of the dehazed image.
Fig. 4 (a) and (b) are the estimated A of the image before and after smoothing with ε = 1.61; (c) is the estimated A of the image after smoothing with ε = 3.61.
Fig. 5 (a-c) are the results of the dehazing process using the A which shows in Fig. 4(a-c), respectively.
So, the next important thing is how to determine the value of ε automatically. Actually, image-evaluation functions normally cannot make a correct judgement without human subjective judgment [18]. Some of these functions regard high gradient to be a better figure, and as a result, regarding Fig. 5(a) better than (b). While the other may regard low gradient to be less noise, however, under this judgment, Fig. 5(c) seems to be preferable, where the sky area is so flattened because of some kind of ‘overamplification’ effect after the dehazing process. Here, we introduce the contrast expression to seek an optimal ε [19]:
where N is the number of pixels in the image, is the mean intensity of the image, and I(x, y) is the intensity of the pixel (x, y). By using Eq. (10), we can obtain the relationship between ε and C, as shown in Fig. 6. It can be seen that the behavior of the curve is indented. It is interesting that the quality of the dehazed image is better when the contrast is in the valley of the curve. Figure 7(a)-7(d) show the dehazing results of the sky region with ε = 1.61, 1.76, 1.77 and 1.90, respectively, where 1.61 and 1.77 correspond to the valleys; while 1.76 and 1.90 correspond to the peaks. From these four images, we can see that the noise in the sky region actually enhance the contrast of the whole image, but in this case, the quality of the dehazed image is not enhanced. Besides, we find from dozen groups of experiments that when the value of ε is in this range from 1.60 to 1.90, the ‘overamplification’ effect can be restrained. Therefore, we can automatically choose ε according to the valley of the contrast. Generally speaking, there may be several valleys in the value range, just like ε can be 1.61 or 1.77 in this case, and they are both optional.Fig. 7 (a-d) are the dehazing results of the sky region with ε = 1.61, 1.76, 1.77 and 1.90, respectively.
In color image dehazing process, one need to handle the red, green and blue channel independently, and then recombine them to one dehazed color image. The dehazing processes of the three channel are exactly the same, so here we just show the dehazing result of Fig. 3, as shown in Fig. 8(a). For comparison, we also show the dehazing result by using our previously proposed dehazing method [14] in Fig. 8(b). It can be seen that the contrast of Fig. 8(a) is better than (b), which means that this optimized polarimetric dehazing method has advantage in doing dehazing for the dense haze.
Fig. 8 (a) The dehazing result using our optimized polarimetric dehazing method; (b) the dehazing result using the polarimetric dehazing method in [14].
In order to quantitively estimate the capability of our dehazing method in enhancing the range of visibility, images containing longer distance objects in the same haze are captured. One raw polarized image and the dehazed image are shown in Fig. 9(a) and (b), respectively. From Fig. 9(a), the outline of the building in distance can be hardly seen, but in 9(b), these buildings can be easily found and the detailed information looks much better. Based on Fig. 3 and Fig. 9, we can roughly estimate the enhancing rate of the range of visibility. Normally, the contrast between the brightest and the darkest pixel values of an object of 5% is regarded as the longest range of visibility in an image. The definition of the contrast for determining the range of visibility V can be expressed as:
where I(x,y)max and I(x,y)min are the largest and the smallest intensities of the object pixels, respectively. Note that the same pixels should be chosen from hazy and dehazed images, which can provide objective calculated results. We estimated the range of visibility in hazy condition by using Fig. 3(a), and that in dehazed condition by using Fig. 9(b). The target buildings are marked by yellow rectangle, and the evaluating results are listed in Table 1. From the table, we can see that in this dense hazy condition, the distance of the building of Fig. 3(a) can be roughly regarded as the longest range of visibility, whose contrast is only 6.67%. This distance is 860m. While in the dehazed image, we regard the building in distance of Fig. 9(b) as the longest range of visibility. This distance is 1500m. It should be pointed out that the contrast 14.75% means the range of visibility is farther than 1500m actually. It can be easily estimated that our optimized dehazing method can enhance the range of visibility of 74% at least. Besides, the contrast of the dehazed images enhanced around 400% compared to the hazy images.Fig. 9 (a) The raw polarized image in four is taken in the same haze weather as Fig. 3; (b) the dehazed image is dehazed by using the optimized polarimetric dehazing method.
Table 1. The contrast of the target buildings in the same hazy condition.
Our dehazing method is also useful in removing the interruption of sea fog. We have captured a group of polarized raw images under the condition of extremely dense sea fog, and one of them is shown in Fig. 10(a). Using our proposed optimized polarimetric dehazing method, the dehazed image is obtained, as shown in Fig. 10(b). From the comparison of these two images, it can be seen that the visibility of 10(b) is much better than 10(a). The windows of the house can be clearly seen in 10(b), while it is totally invisible in 10(a). All these dehazing results in this paper prove that our polarimetric dehazing method is very effective in handling the dense turbid environment.
Fig. 10 (a) The raw polarized image in four is taken in extremely dense sea fog; (b) the dehazed image is dehazed by using the optimized polarimetric dehazing method.
4. Conclusion
In conclusion, we have proposed an optimized polarimetric dehazing method based on AOP distribution analysis, and it is proven to be effective in dehazing images with dense haze in this paper. Two important steps are added into the dehazing method. First, the method to estimate θA is given. Based on this optimized process, pA and A∞ can be estimated much precisely. Second, we analyze the influence of the estimated A for the dense haze removal. It is found that the bias factor ε determines the quality of the dehazing result. Due to the fact that the detailed information in dense haze is very weak, it is very important to find an appropriate ε. And the method of finding ε is given in this paper. According to our polarimetric dehazing method, we choose some images captured under dense haze to do the dehazing process. The dehazing results show that the range of visibility can be enhanced at least 74%. Besides, this dehazing method is proven to be not only adaptive to the dense haze but also to the severe sea fog.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under grants 61505246, 61535015 and 61275149.
References and links
1. K. Garg and S. K. Nayar, “Vision and rain,” Int. J. Comput. Vis. 75(1), 3–27 (2007). [CrossRef]
2. R. T. Tan, “Visibility in bad weather from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8. [CrossRef]
3. K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Anal. Mach. Intell. 33, 2341–2353 (2010). [PubMed]
4. K. Nishino, L. Kratz, and S. Lombardi, “Bayesian defogging,” Int. J. Comput. Vis. 98(3), 263–278 (2012). [CrossRef]
5. H. Lu, Y. Li, L. Zhang, and S. Serikawa, “Contrast enhancement for images in turbid water,” J. Opt. Soc. Am. A 32(5), 886–893 (2015). [CrossRef] [PubMed]
6. Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Polarization-based vision through haze,” Appl. Opt. 42(3), 511–525 (2003). [CrossRef] [PubMed]
7. T. Treibitz and Y. Y. Schechner, “Active polarization descattering,” IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 385–399 (2009). [CrossRef] [PubMed]
8. J. Mudge and M. Virgen, “Real time polarimetric dehazing,” Appl. Opt. 52(9), 1932–1938 (2013). [CrossRef] [PubMed]
9. S. Fang, X. S. Xia, X. Huo, and C. W. Chen, “Image dehazing using polarization effects of objects and airlight,” Opt. Express 22(16), 19523–19537 (2014). [CrossRef] [PubMed]
10. J. Fade, S. Panigrahi, A. Carré, L. Frein, C. Hamel, F. Bretenaker, H. Ramachandran, and M. Alouini, “Long-range polarimetric imaging through fog,” Appl. Opt. 53(18), 3854–3865 (2014). [CrossRef] [PubMed]
11. G. Woodell, D. J. Jobson, Z. Rahman, and G. Hines, “Enhancement of imagery in poor visibility conditions,” Sensors, and Command, Control, Communications, and Intelligence Technologies for Homeland Security and Homeland Defense 5778 (2005).
12. L. Schaul, C. Fredembach, and S. Süsstrunk, “Color image dehazing using the near-infrared,” in 16th IEEE International Conference on Image Processing (ICIP, 2009), pp. 1629–1632. [CrossRef]
13. T. Treibitz and Y. Y. Schechner, “Polarization: Beneficial for visibility enhancement?” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2009), 525–532 (2009).
14. J. Liang, L. Y. Ren, H. J. Ju, E. S. Qu, and Y. L. Wang, “Visibility enhancement of hazy images based on a universal polarimetric imaging method,” J. Appl. Phys. 116(17), 173107 (2014). [CrossRef]
15. C. F. Bohren and A. B. Fraser, “At what altitude does the horizon cease to be visible?” Am. J. Phys. 54(3), 222–227 (1986). [CrossRef]
16. J. Liang, L. Ren, E. Qu, B. Hu, and Y. Wang, “Method for enhancing visibility of hazy images based on polarimetric imaging,” Photonics Res. 2(1), 38–44 (2014). [CrossRef]
17. The AQI criterion can be read at http://en.wikipedia.org/wiki/Air_quality_index (2.2 Mainland China).
18. A. Santos, C. Ortiz de Solórzano, J. J. Vaquero, J. M. Peña, N. Malpica, and F. del Pozo, “Evaluation of autofocus functions in molecular cytogenetic analysis,” J. Microsc. 188(3), 264–272 (1997). [CrossRef] [PubMed]
19. Y. Y. Schechner and N. Karpel, “Recovery of underwater visibility and structure by polarization analysis,” IEEE J. Oceanic Eng. 30(3), 570–587 (2005). [CrossRef]
