Images/videos captured from optical devices are usually degraded by turbid media such as haze, smoke, fog, rain and snow. Haze is the most common problem in outdoor scenes because of the atmosphere conditions. This paper proposes a novel single image-based dehazing framework to remove haze artifacts from images, where we propose two novel image priors, called the pixel-based dark channel prior and the pixel-based bright channel prior. Based on the two priors with the haze optical model, we propose to estimate atmospheric light via haze density analysis. We can then estimate transmission map, followed by refining it via the bilateral filter. As a result, high-quality haze-free images can be recovered with lower computational complexity compared with the state-of-the-art approach based on patch-based dark channel prior.
© 2013 Optical Society of America
Different weather conditions such as haze, fog, smoke, rain, or snow will cause complex visual effects of spatial or temporal domains in images or videos [1–3]. Such artifacts may significantly degrade the performances of outdoor vision systems relying on image/video feature extraction  or visual attention modeling [5–7], such as event detection, object detection, tracking, and recognition, scene analysis and classification, image indexing and retrieval. For example, visual attention models  compute a saliency map topographically encoding for saliency at each location in the visual input that simulates which elements of a visual scene are likely to attract the attention of human observers. Nevertheless, the performances of the model for related applications may be degraded if haze directly interacts with the interested target in an image. A comprehensive survey of detection approaches for outdoor environmental factors to enhance the accuracy of video-based automatic incident detection systems can be found in . Moreover, hazing artifacts have been shown to significantly influence the performance of image/video coding in , where the dehazing effects in compression were investigated in terms of the coding artifacts and motion estimation in both cases of applying dehazing before and after compression. It has been concluded that both better dehazing performance and better coding efficiency can be achieved when the dehazing is applied before compression . More specifically, we have evaluated several vision-based object detection systems (e.g., [4,6,7]) with hazy and haze-removed images (obtained by the proposed method) as inputs to claim that detection accuracy for haze-removed images would be better, which will be shown in Sec. 4.1.
Removal of weather effects has recently received much attention , such as removals of haze [9–31], rain [32–35], and snow  from image/video. In this paper, we focus on haze removal, i.e., dehazing, from a single image. As illustrated in Fig. 1, haze is an example of the turbid medium (e.g., particles and water droplets) in the atmosphere, which will degrade outdoor images due to atmospheric absorption and scattering. The irradiance received by the camera from the scene point is attenuated along the line of sight. Furthermore, the incoming light is blended with the airlight, i.e., ambient light reflected into the line of sight by atmospheric particles. As a result, the degraded images will lose contrast and color fidelity. Based on the fact that the amount of scattering depends on the distance of the scene points from the camera, the degradation is spatially variant .
Based on the fact that haze is dependent on the unknown depth, dehazing is therefore a challenging problem. Furthermore, if the available input is only one single hazy image, the problem is under-constrained and more challenging. Hence, most traditional dehazing approaches [10–19] have been proposed based on using multiple hazy images as input or additional prior knowledge. Polarization-based methods [11–14] were proposed to remove the haze effects through two or more images taken with different degrees of polarization. In [15–17], more constraints obtained from multiple images of the same scene under different weather conditions were employed for haze/weather artifact removal. Moreover, in depth-based methods [18–21], it is required to provide some depth information from user inputs or known 3D models for dehazing or deweathering.
Nevertheless, taking multiple input images of the same scene is usually impractical in several real applications. Single image haze removal [22–30] has recently received much attention. The success of these approaches usually lies on using stronger priors or assumptions. In , it is observed that a haze-free image should have higher contrast compared with its hazy version. Hence, it was proposed to remove haze from a single image by maximizing the local contrast of the restored image. The results are usually visually compelling but may not be physically valid. Fattal  proposed to estimate the albedo of the scene and the medium transmission under the assumption that the transmission and the surface shading are locally uncorrelated for single image dehazing. This approach is physically sound and can usually produce impressive results. Nevertheless, it cannot well restore heavily hazy images and may fail in the cases where the assumption is invalid. Tarel and Hautière  proposed a fast visibility restoration algorithm for a single image based on median filter in order to preserve both edges and corners. This method has been shown to be not suitable to handle the region with discontinuous depth in . In addition, in , it was presented an algorithm for removing the effects of light scattering in a single underwater image. In , it was proposed to iteratively extract transmission under the assumption that large-scale chromaticity variations are due to transmission while small-scale luminance variations are due to scene albedo for single image dehazing. A filtering-based dehazing method for single image was proposed in , where the basic idea is to compute an accurate atmosphere veil that is not only smoother, but also respect with depth information of the underlying image. In , single image dehazing was extended to video dehazing with the maintenance of spatial and temporal coherence.
Recently, an effective image prior, called dark channel prior  was proposed to remove haze from a single image, where the key observation is that most local patches in outdoor haze-free images contain some pixels whose intensity is very low in at least one color channel. Based on this prior with the haze optical model, one can directly estimate the thickness of the haze and restore a high-quality haze-free image. Then, the dark channel prior was also employed in [29,30] for single image dehazing.
In this paper, inspired by the patch-based dark channel prior , we propose two novel image priors, called the pixel-based dark channel prior and the pixel-based bright channel prior. Based on the two novel priors with the haze optical model, we propose to estimate the atmospheric light via haze density analysis. We can then estimate the transmission map, followed by refining it via the bilateral filter . As a result, high-quality haze-free images can be recovered with lower computational complexity compared with . The main contribution of this paper is three-fold: (i) we propose the pixel-based dark channel prior for single image dehazing, which can significantly reduce the computational complexity of the original patch-based approach  while maintaining the dark channel property of an image; (ii) we propose the pixel-based bright channel prior, which can be integrated with the dark channel prior for haze density analysis and atmospheric light estimation; and (iii) in our method, the estimated atmospheric light for an image is pixel-based, which might be more accurate than using only a constant for an image employed in , resulting in more reliable estimation of transmission map. Moreover, we also investigate several real applications of single image dehazing, as shown in Sec. 4.1, which was rarely evaluated in the literature.
The rest of this paper is organized as follows. In Sec. 2, we briefly review the haze imaging model, the patch-based dark channel prior , and the dehazing method proposed in . Sec. 3 presents the proposed single-image-based dehazing framework. In Sec. 4, experimental results are demonstrated. Finally, Sec. 5 concludes this paper.
2. Related works
2.1 Haze optical model22–30].
2.2 Patch-based dark channel prior
The dark channel prior  is based on the observation in outdoor haze-free images that in most of the non-sky image patches, at least one color channel has some pixels whose intensity are very low and close to zero. That is, the minimum intensity in such a patch is close to zero. For an arbitrary image , its dark channel is given by :
Based on the concept of a dark channel, it has been shown that if is an outdoor haze-free image, except for the sky region, the intensity of ’s dark channel is low and tends to be zero, i.e., , which is called by dark channel prior. The low intensity in the dark channel is mainly derived from the three factors: (i) shadows; (ii) colorful objects or surfaces; and (iii) dark objects or surfaces. The patch-based dark channel prior has been recently used to single image dehazing [28–30], as described in Sec. 2.3.
2.3 Single image dehazing based on patch-based dark channel prior
In , a single image-based dehazing framework based on the patch-based dark channel prior was proposed. Based on the haze imaging model shown in Eq. (1), the authors  proposed to first estimate the atmospheric light and the transmission map , followed by refining via the soft matting process . For estimating , it is general to use the color of the most haze-opaque in as  or ’s initial guess . Based on the property that the dark channel of a hazy image can approximate the haze denseness, in , it was proposed to use the dark channel to detect the most haze-opaque region. They first picked the top 0.1% brightest pixels in the dark channel of , which are usually most haze-opaque. Among these pixels, the pixels with the highest intensity in are selected as the atmospheric light .28]), the definitions of , , and , respectively, are the same as those defined in Eq. (2), denotes the color channel of , and denotes the color channel of . Then, is refined via the soft matting process . Finally, the haze-free image can be recovered by:28]).
3. Proposed single image-based dehazing framework
In this section, we present the proposed single image-based dehazing framework, including our pixel-based dark/bright channel prior, haze density analysis, atmospheric light estimation, and transmission map estimation/refinement methods to improve the dehazing method via the patch-based dark channel prior proposed in . The main drawbacks of , includes: (i) the estimated atmospheric light of an image is a constant for all of the pixels, which is not accurate enough, and may induce lower accuracy of transmission map estimation; (ii) the computational complexity for calculating a patch-based dark channel prior for an image is too expensive, especially for large patch size (e.g., 15 × 15 is used in ); and (iii) the computational complexity of the transmission map refinement process via soft matting is too expensive. The issue of computational complexity will be discussed and evaluated in Sec. 4.3. The details of the proposed single image dehazing method are elaborated in the following subsections.
3.1 Pixel-based dark/bright channel prior
Based on our observation, most outdoor haze-free images contain many pixels whose intensity is very low in at least one color channel. We call this property as “pixel-based dark channel prior.” Even if the degree of darkness of the dark channel obtained by our pixel-based dark channel prior is less darker than that obtained by the patch-based approach , it is already sufficiently enough for the estimation of atmospheric light and transmission map, described in Secs. 3.2 and 3.3, respectively. More specifically, for an image , its pixel-based dark channel can be derived as:Figs. 2(b) and 2(c), we calculate the histogram (Fig. 2(c)) for the pixel-based dark channel (Fig. 2(b)) of the image shown in Fig. 2(a). It can be found that most pixels in the dark channel have very small values.
On the other hand, we also observe that most outdoor haze-free images contain many pixels whose intensity is very high in at least one color channel, which is called “pixel-based bright channel prior.” Hence, the pixel-based bright channel can be similarly derived as:Figs. 2(d) and 2(e), even if the degree of brightness of the obtained pixel-based bright channel prior is not really very “bright,” it is already sufficiently enough for the estimation of atmospheric light, described in Sec. 3.2.
3.2 Haze density analysis and estimation of atmospheric light
Based on the fact that the density of haze in different regions of an image should be different, the value of atmospheric light for each pixel should be different accordingly. In general, in a hazy image, the haze density is higher in the region of deeper depth, while the haze density is lower in the region of shallower depth. To estimate the haze density, we first convert an input hazy image to its color representation in the HSV (hue, saturation, and value) color space. HSV color space has been widely employed in several computer vision and multimedia applications with the main characteristic of better stability beneficial from separating the brightness and chromaticity information of an image [38,39]. We then define the HSV distance for each pixel as:
Then, we determine the pixel value of the color channel with the highest haze density in by selecting the pixel value corresponding to the top 0.1% brightest value in , while determining the pixel value of the color channel with the lowest haze density in by selecting the pixel value corresponding to the top 30% darkest value in . Based on the above-mentioned information about the haze density and the dark/bright channel prior for , we can estimate the color channel of the atmospheric light for , i.e., , as:Fig. 3. After estimating the atmospheric light , we can then estimate the transmission map , described in Sec. 3.3.
3.3 Estimation of transmission map and its refinement
Based on our atmospheric light estimation technique, the haze imaging model shown in Eq. (1) can be modified by replacing with . By normalizing the modified haze imaging equation with for the color channel , followed by calculating the pixel-based dark channel on both sides, we can obtain:Eq. (10). Moreover, similar to , we also keep a very small amount of haze for the distant objects by introducing a parameter for each pixel , . Different from using only a constant (set to 0.95) for all pixels of an image in , we propose to use the adaptive weighting function , adapted to the depth of , by setting . That is, is larger in the region of deeper depth, while is smaller in the region of shallower depth. Then, we can derive as:Fig. 4(b). We then refine via the bilateral filter  to reduce halo artifacts, as illustrated in Fig. 4(c), which is more efficient than the soft matting process with high complexity  employed in .The bilateral filter can efficiently smooth an image while preserving edges, by means of a nonlinear combination of nearby image values, which has been extensively applied and investigated recently for image processing, such as image denoising. We can then recover a hazy image based on Eq. (1), described in Sec. 3.4.
3.4 Recovery of Haze-free Image
After estimating and , we can derive based on the haze imaging model. Similar to , we also introduce a lower bound to preserve a small amount of haze in very dense haze regions ( is also set to 0.1 based on ). Finally, the haze-free image can be derived as:Fig. 4(d).
4. Experimental results
To demonstrate the practicability and performance of the proposed single image-based dehazing method, in Sec. 4.1, we evaluate the performance of the three existing vision-based object detection systems [4,6,7] with and without applying the proposed dehazing method as a preprocessing step. Then, in Sec. 4.2, we compare our method with the three state-of-the-art image-based dehazing methods [22–23,28].
4.1 Experiments on vision-based systems with/without image dehazing as preprocessing
As shown in Fig. 5, we applied the HOG (histogram of oriented gradients)-based pedestrian detector released from  to the hazy image shown in Fig. 5(a) and its haze-removed version (obtained by the proposed method presented in Sec. 3) shown in Fig. 5(b), respectively. It can be found that the detection accuracy for the haze-removed version is better. In addition, as shown in Fig. 6, we applied the visual attention model-based blob detector released from  to the hazy image shown in Fig. 6(a) and its haze-removed version (obtained by the proposed method) shown in Fig. 6(b), respectively. It can be found that the detection accuracy for the haze-removed version is better. Moreover, we also evaluate the graph-based approach released from  to calculate the visual saliency map for the hazy image shown in Fig. 7(a) and its haze-removed version (obtained by the proposed method) shown in Fig. 7(b), respectively. Similarly, it can also be found that the derived saliency map for the haze-removed version is better.
In Fig. 8, the proposed method can outperform both Fattal’s and He’s methods, especially in the recovery of the red wall region. Fattal’s method  is mainly based on local statistics and requires sufficient color information and variance. When the color information is faint induced by haze, the variance is not high enough for this method to estimate the transmission map. Moreover, the estimation of atmospheric light in He’s method  is somewhat rough, which may induce unreliable estimation of the transmission map, resulting in inaccurate color information recovery. The proposed method applies more reliable estimations of atmospheric light and transmission map, resulting in better image recovery. Similar observations can be also found in Figs. 9 and 10.
In addition, in Fig. 11, even if Fattal’s method  can remove the haze more completely than our method, the recovered image seems to be somewhat unnatural. For example, in Fig. 11(b), the sky region is too bright and the colors of some tree regions are too white. The proposed method employs more adaptive weighting function in estimation of the transmission map (Eq. (11)) to suitably keep a very small amount of haze, resulting in more natural image recovery. In Fig. 12, the proposed method can achieve better color information recovery than that of He’s method  based on better estimation of the transmission map.
Moreover, Figs. 13 and 14 show the comparison results of Tan’s  and the proposed methods, where it can be found that our method can significantly outperform Tan’s method. The main reason is that Tan’s method tends to maximize the contrast of a hazy image, which may result in overestimating the haze layer and over-saturating colors. However, the proposed method can recover the image structure without sacrificing the fidelity of the color information.
On the other hand, to subjectively evaluate the proposed method, we also invited 10 subjects to participate in the subjective test. All subjects were undergraduate or graduate students and none of them had knowledge about the evaluated algorithms, including Tan’s , Fattal’s , He’s , and the proposed algorithms. We selected 50 hazy images and applied the four evaluated methods to perform dehazing and obtained the respective 50 dehazed images for each method. Similar to the principle of the double stimulus continuous quality scale (DSCQS) quality assessment  for video quality measurement recommended by ITU-R BT.500, each subject was asked to subjectively give a score to each dehazed image. The score corresponding to the visual quality of an image is ranged from 5 to 1, denoting excellent (5), good (4), fair (3), poor (2), and bad (1), respectively. Table 1 lists the average score obtained by averaging the scores of the 50 dehazed images for each evaluated method. Based on Table 1, the proposed method outperforms the three methods [22,23,28] in terms of the subjective test.
4.3 Discussion on computational complexity
The proposed dehazing method was implemented in MATLAB® 64-bit on a personal computer equipped with Intel® Core i7-2600 processor and 4 GB memory. Table 2 lists the run-time (in seconds for processing an image) of the dehazing method based on patch-based dark channel prior (He’s method)  and the proposed method. It can be observed from Table 2 that the computational complexity of the proposed method is significantly lower than that of He’s method . The main reasons are: (i) the computational complexity for calculating a patch-based dark channel prior for an image is too expensive, especially for large patch size used in , while our method employs the proposed pixel-based dark channel prior, which can significantly reduce the complexity; and (ii) the computational complexity of the transmission map refinement process via soft matting  used in  is too expensive, while our method employs the bilateral filter , which is efficient and with lower complexity. As a result, higher-quality haze-free images can be recovered with lower computational complexity by our method compared with .
In this paper, we have proposed a novel single image-based dehazing framework, where we proposed two novel image priors, called the pixel-based dark channel prior and the pixel-based bright channel prior. Based on the two priors with the haze optical model, we can estimate the atmospheric light via haze density analysis. We can then estimate the transmission map, followed by refining it via the bilateral filter. Based on our experimental results, high-quality haze-free images can be recovered by our dehazing method which can outperform or be comparable with the three state-of-the-art methods [22,23,28] used for comparisons. For future work, we will extend our method to video dehazing while maintaining spatial and temporal consistency, which can be embedded into driving safety assistance device with the integration of our previous rain removal techniques [32,33].
This work was supported in part by the National Science Council, Taiwan, under Grants NSC99-2628-E-110-008-MY3 and NSC100-2218-E-224-017-MY3. Our thanks to Wen- Hung Xu for executing the program on the test data in this work; his timely assistance is greatly appreciated.
References and links
1. S. G. Narasimhan and S. K. Nayar, “Vision and the atmosphere,” Int. J. Comput. Vis. 48(3), 233–254 (2002). [CrossRef]
2. K. Garg and S. K. Nayar, “Vision and rain,” Int. J. Comput. Vis. 75(1), 3–27 (2007). [CrossRef]
4. S. Maji, A. C. Berg, and J. Malik, “Classification using intersection kernel support vector machines is efficient,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Anchorage, Alaska, USA, 2008), pp. 1–8. [CrossRef]
5. L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998). [CrossRef]
6. M. Jahangiri and M. Petrou, “An attention model for extracting components that merit identification,” in Proceedings of IEEE International Conference on Image Processing (Cairo, Egypt, 2009), pp. 965–968. [CrossRef]
7. J. Harel, C. Koch, and P. Perona, “Graph-based visual saliency,” in Proceedings of Advances in Neural Information Processing Systems (2007), pp. 545–552.
8. M. S. Shehata, J. Cai, W. M. Badawy, T. W. Burr, M. S. Pervez, R. J. Johannesson, and A. Radmanesh, “Video-based automatic incident detection for smart roads: the outdoor environmental challenges regarding false alarms,” IEEE Trans. Intell. Transp. Syst. 9(2), 349–360 (2008). [CrossRef]
10. S. G. Narasimhan and S. K. Nayar, “Removing weather effects from monochrome images,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Kauai, Hawaii, USA, 2001), pp. 186–193.
11. Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Instant dehazing of images using polarization,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Kauai, Hawaii, USA, 2001), pp. 325–332.
12. S. Shwartz, E. Namer, and Y. Y. Schechner, “Blind haze separation,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (New York, USA, 2006), pp. 1984 – 1991.
15. S. K. Nayar and S. G. Narasimhan, “Vision in bad weather,” in Proceedings of IEEE International Conference on Computer Vision (Kerkyra, Greece, 1999), pp. 820–827. [CrossRef]
16. S. G. Narasimhan and S. K. Nayar, “Chromatic framework for vision in bad weather,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Hilton Head Island, USA, 2000), pp. 598–605. [CrossRef]
17. S. G. Narasimhan and S. K. Nayar, “Contrast restoration of weather degraded images,” IEEE Trans. Pattern Anal. Mach. Intell. 25(6), 713–724 (2003). [CrossRef]
18. J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O. Deussen, M. Uyttendaele, and D. Lischinski, “Deep photo: model-based photograph enhancement and viewing,” ACM Trans. Graph. 27(5), 1–10 (2008). [CrossRef]
19. S. G. Narasimhan and S. K. Nayar, “Interactive deweathering of an image using physical models,” Proceedings of IEEE Workshop Color and Photometric Methods in Computer Vision (2003).
20. F. Cozman and E. Krotkov, “Depth from scattering,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (San Juan, USA, 1997), pp. 801–806. [CrossRef]
22. R. Tan, “Visibility in bad weather from a single image,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Anchorage, Alaska, USA, 2008), pp. 1–8. [CrossRef]
23. R. Fattal, “Single image dehazing,” ACM Trans. Graph. 27(3), 1–9 (2008). [CrossRef]
24. J. P. Tarel and N. Hautière, “Fast visibility restoration from a single color or gray level image,” in Proceedings of IEEE International Conference on Computer Vision (Kyoto, Japan, 2009), pp. 2201–2208. [CrossRef]
25. N. Carlevaris-Bianco, A. Mohan, and R. M. Eustice, “Initial results in underwater single image dehazing,” in Proceedings of MTS/IEEE OCEANS (Seattle, USA, 2010), pp. 1–8. [CrossRef]
26. J. Zhang, L. Li, G. Yang, Y. Zhang, and J. Sun, “Local albedo-insensitive single image dehazing,” Vis. Comput. 26(6–8), 761–768 (2010). [CrossRef]
27. C. Xiao and J. Gan, “Fast image dehazing using guided joint bilateral filter,” Vis. Comput. 28(6–8), 713–721 (2012). [CrossRef]
28. K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel Prior,” IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010). [PubMed]
29. C. T. Chu and M. S. Lee, “A content-adaptive method for single image dehazing,” Lect. Notes Comput. Sci. 6298, 350–361 (2010). [CrossRef]
30. J. Yu and Q. Liao, “Fast single image fog removal using edge-preserving smoothing,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, (Prague, Czech Republic, 2011), pp. 1245–1248. [CrossRef]
31. J. Zhang, L. Li, Y. Zhang, G. Yang, X. Cao, and J. Sun, “Video dehazing with spatial and temporal coherence,” Vis. Comput. 27(6–8), 749–757 (2011). [CrossRef]
33. L. W. Kang, C. W. Lin, C. T. Lin, and Y. C. Lin, “Self-learning-based rain streak removal for image/video,” in Proceedings of IEEE International Symposium on Circuits and Systems (Seoul, Korea, 2012), pp. 1871–1874. [CrossRef]
34. P. C. Barnum, S. Narasimhan, and T. Kanade, “Analysis of rain and snow in frequency space,” Int. J. Comput. Vis. 86(2–3), 256–274 (2010). [CrossRef]
35. J. Bossu, N. Hautière, and J. P. Tarel, “Rain or snow detection in image sequences through use of a histogram of orientation of streaks,” Int. J. Comput. Vis. 93(3), 348–367 (2011). [CrossRef]
36. C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proceedings of IEEE International Conference on Computer Vision (Bombay, 1998), pp. 839–846. [CrossRef]
37. A. Levin, D. Lischinski, and Y. Weiss, “A closed form solution to natural image matting,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (New York, USA, 2006), pp. 61–68. [CrossRef]
38. B. S. Manjunath, J. R. Ohm, V. V. Vasudevan, and A. Yamada, “Color and texture descriptors,” IEEE Trans. Circ. Syst. Video Tech. 11(6), 703–715 (2001). [CrossRef]
39. X. Wang and Z. Wang, “A novel method for image retrieval based on structure elements’ descriptor,” J. Vis. Commun. Image Represent. 24(1), 63–74 (2013). [CrossRef]
40. M. H. Pinson and S. Wolf, “Comparing subjective video quality testing methodologies,” in Proceedings of SPIE 5150, Visual Communications and Image Processing (2003).