This paper provides a novel Bayesian deringing method to reduce ringing artifacts caused by image interpolation and JPEG compression. To remove the ringing artifacts, the proposed method uses a Bayesian framework based on a SGLI (spatial-gradient-local-inhomogeneity) prior. The SGLI prior employs two complementary discontinuity measures: spatial gradient and local inhomogeniety. The spatial gradient measure effectively detects strong edge components in images. In addition, the local inhomogeniety measure successfully detects locations of the significant discontinuities by taking uniformity of small regions into consideration. The two complementary measures are elaborately combined to create prior probabilities of the Bayesian deringing framework. Thus, the proposed deringing method can effectively preserve the significant discontinuities such as textures of objects as well as the strong edge components in images while reducing the ringing artifacts. Experimental results show that the proposed deringing method achieves average PSNR gains of 0.09 dB in image interpolation artifact reduction and 0.21 dB in JPEG compression artifact reduction.
© 2010 OSA
The key issue of image interpolation is to get a high-resolution (HR) image from one or several original low-resolution (LR) images. Image interpolation techniques have been applied to a lot of image processing applications such as video surveillance systems, medical image diagnoses, resolution enhancement of digital cameras, etc. Single frame interpolation techniques, such as traditional bilinear, bicubic, and various B-spline interpolations, are commonly used as image magnification methods. Generally, these techniques make use of averaging effects on the neighborhoods of pixels [1–3]. However, in the case of the bicubic interpolation method and its related ones, the sharp transition or edge can cause ringing artifacts in the non-transition or non-edge regions at object boundaries. The main cause of ringing artifacts is due to a signal being bandlimited (specifically, not having high frequencies) or passed through a low-pass filter. That is to say, in terms of the spatial domain, the cause of the ringing artifacts is the ripples in the cubic and sync functions . The ringing artifacts are also observed at sharp transitions in JPEG compressed images, which is due to the abrupt truncation of the high frequency DCT or DWT coefficients [5,6]. For example, Fig. 1(b) shows the bicubic interpolated image corresponding to the down-sampled image of the original Cameraman image in Fig. 1(a). The down-sampling factor is 4 and the down-sampled image has been interpolated to the same size of the original image by bicubic interpolation. It can be observed that there are a lot of ringing artifacts around edge regions. Figure 1(c) shows the JPEG compressed image corresponding to the original image in Fig. 1(a). The compression rate is 56.6 Kbps and the ringing artifacts also occur around the edge regions.
Also, the lower the compression rate is, the more the ringing artifacts occur. Therefore, the ringing artifacts lead to bad image quality and can be annoying to viewers of the reconstructed images. Up to now, several methods have been proposed over the years to solve the ringing artifact problems [2,5–13], among which are Bayesian approaches. Bayesian approaches are frequently used due to their adaptability to various artifacts. A Bayesian framework works by interpreting observed images as an accumulation of original images and artifacts. In the framework, the original images, uncorrupted by artifacts, are reconstructed by maximum a posterior (MAP) estimator. By adding any prior knowledge of the original images in the framework, the Bayesian method treats image deringing as a probabilistic problem. In addition, the original images are estimated by minimizing an energy function using Markov random fields (MRF) [2,5,6,14–22].
As can be expected, it is very important that we assign an appropriate prior value to efficiently reduce the ringing artifacts and successfully reconstruct the original images. The performance of the Bayesian framework depends on the applied prior and its ability to extract artifacts. In this paper, we propose a novel Bayesian deringing method based on a SGLI prior, which achieves impressive performance with respect to artifact reduction. The idea of the SGLI prior comes from two discontinuity measures previously mentioned by the Chen and Park et al.’s works [23,24]: spatial gradient and local inhomogeniety. The spatial gradient measure which has been widely used as a discontinuity measure detects strong edge components effectively in images. In addition, the local inhomogeniety measure successfully detects locations of the significant discontinuities by taking uniformity of small regions into consideration. The two complementary measures are very effective not only for noise removal but also feature preservation. They are elaborately combined to be employed for creating prior probabilities of the Bayesian deringing framework . Therefore, the SGLI prior probability of the Bayesian framework is able to preserve feature components and remove ringing artifacts from corrupted images efficiently.
This paper is organized as follows. In Section 2, we describe the proposed Bayesian deringing method in detail. In Section 3, some experimental results and the corresponding analysis are provided. Finally, we make a conclusion in Section 4.
In the adaptive smoothing methods proposed by the Chen and Park et al.’s works [23,24], it has been proven that the spatial gradient and local inhomogeniety measures effectively preserves features while smoothing images. Thus, the two complementary measures are combined into a singular new prior to reduce the ringing artifacts in a Bayesian framework.
The spatial gradient of the observed image X(i, j) at a pixel (i, j) is defined as the first partial derivatives of its image intensity with respect to coordinates x and y:Eq. (1) is expressed as:
Therefore, we can design a Bayesian deringing framework using the SGLI prior energy U(X). Let Y be an N × M observed image corrupted by ringing artifacts from an unknown image X, the optimal solution X* is determined by the maximum a posteriori (MAP) estimation as follows .14,25]. For large λ, p(X) becomes flat, and for small λ, p(X) has sharp modes. Consequently, the probability function is converted into energy function by Eq. (10).
If we assume that the ringing artifacts are independent and identically distributed (i.i.d.) Gaussian , then we get:Eq. (9) can be expressed as:Eq. (13), and thus we can reconstruct the original image through energy optimization techniques. In our method, the termination of iteration is determined automatically based on the energy difference between iteration t and t-1 as follows:27]. UT ( t )(x,y) denotes the total energy of the image at iteration t. Figure 2 shows the evolution of UT ( t) for the Cameraman image as the iteration proceeds. As can be seen, energy function is monotonically decreasing as the number of iterations increases. In addition, the observed corrupted image gets smoothed as the iteration proceeds. We find that after a relatively small number of iterations, UT ( t) changes slightly in value from various images. Thus, we determine the optimal termination time of iterations from the energy difference Ψ(t). Our method is iterated until Ψ(t) is lower than 8% of the first energy UT (0).
To evaluate the efficiency of the proposed method, 6 typical HR images were used for the experiments. They are Lena, Cameraman, Man, Woman, Airfield, and House, whose sizes are 256x256 pixels as shown in Fig. 3 . The LR images were generated by low-pass filtering and down-sampling the HR images. The down-sampling factor was 4 and the down-sampled images were interpolated to the same size of the original HR image. The weight α of Eq. (12) was initially set as 0<α≤0.3. The optimal weight was selected through exhausting experiments. The optimal weight was 0.2 in the interpolation artifact reduction and 0.03 in the JPEG compression artifact reduction. The regularization parameters were chosen heuristically and the values of them were γ 1 = γ 2 = 1.0.
Evaluation of the estimated image was done using the following three measures:
3.1 Performance evaluation in the interpolation artifact reduction
Figure 4 shows the Bicubic interpolated results of the down-sampled images. It can be observed that a lot of ringing artifacts occur around sharp transition and edge regions in the interpolated results. This is because the cubic functions with the negative lobes produce some overshoot effects. As can be seen, the ringing artifacts are degrading the quality of picture seriously. Figure 5 shows the reduction results of the ringing artifacts obtained with the proposed method. We can see that the proposed method suppresses the ringing artifacts efficiently and improve the quality of picture, especially around edges where ringing is severe. Above all, in the case of the Cameraman image, the ringing artifacts in non-edge regions at object boundaries are removed effectively.
In order to provide more reliable performance evaluation of the results, the MSE, SNR, and PSNR values are measured over the 6 test images. Table 1 lists performance evaluation results of our proposed method compared with the bicubic interpolation, bilateral filtering , adaptive smoothing , image analogies , pointwise shape-adaptive DCT (SA-DCT) , and fields-of-experts (FoE) [20,21] methods. The image analogies method effectively handles the ringing artifacts in block-based DCT (BDCT) compressed cartoon images and we have obtained the corresponding demonstrative software for evaluation at http://www.cse.cuhk.edu.hk/~ttwong/demo/arti/arti.html. The pointwise SA-DCT method is effective in dealing with not only the image denoising problem but also the image deblocking and deringing ones from BDCT compression. The corresponding software is available at http://www.cs.tut.fi/~foi/SA-DCT, and we have used it for evaluation. Notice that the values of the parameters including smoothing factors and order-mixture parameters were not modified in the tests. In the FoE method, the FoE prior captures the statistics of natural scenes, and thus has been effectively employed for image denoising and inpainting. Moreover, it has been reported that the FOE prior is successfully applied to deblocking of BDCT compressed images . We have obtained the corresponding software for evaluation at http://www.gris.informatik.tu-darmstadt.de/~sroth/research/foe/index.html. In the experiments, the FoE filter size was 5x5 and the maximum number of iterations was 500.
In the table, the bold numbers represent the smallest MSE value of each image which means the best performance. As can be seen, the proposed method provides the best evaluation results in three cases: Cameraman, House, and Airfield. Although the FoE method produces the best evaluation results in the other three images, the proposed method performs better than any other methods in average performance. Our method achieves an average PSNR gain of 0.09 dB as compared to the bicubic interpolation method. The results show that our method reduces the interpolation ringing artifacts efficiently and improve picture quality successfully.
3.2 Performance evaluation in the JPEG compression artifact reduction
The ringing artifacts are also observed at sharp edges in JPEG compressed images. Figure 6 shows the JPEG compressed images when the compression rate is 56.6 Kbps. We can see that some ringing artifacts appear around sharp edges in the compressed images. The ringing artifacts in JPEG compressed images occur because of the abrupt truncation of the high frequency DCT or DWT coefficients. The reduction results of the ringing artifacts obtained with the proposed method are shown in Fig. 7 . It can be observed that the proposed deringing method reduces the ringing artifacts efficiently and reconstruct original images successfully. To provide more reliable performance evaluation of the results, the MSE, SNR, and PSNR values are also measured over the 6 test images. Table 2 lists performance evaluation results of the proposed method compared with JPEG compression, bilateral filtering , adaptive smoothing , image analogies , pointwise SA-DCT , and FoE [20,21] methods. In the table, the bold numbers represent the smallest MSE value of each image which means the best performance. As can be seen, the proposed method provides the best evaluation results in almost all cases. The conventional methods also remove the ringing artifacts successfully, but have a tendency to produce somewhat over-smoothed results in object areas. Thus, they are not able to efficiently preserve the significant discontinuities such as the textures of object areas. However, the proposed method effectively preserves the significant discontinuities including textures of objects as well as the strong edge components in images while reducing the ringing artifacts. This enables the proposed method to outperform the other methods for reducing the ringing artifacts. Also, the results show that the SGLI prior is effectively employed for reducing the ringing artifacts caused by JPEG compression. Consequently, the proposed method achieves an average PSNR gain of 0.21 dB as compared to the JPEG compressed images.
In this paper, we propose a novel Bayesian deringing method to reduce the ringing artifacts caused by image interpolation and JPEG compression. The ringing artifacts mainly appear around sharp edges in images because of loss of high frequency components. They can seriously degrade the quality of picture and be annoying to viewers of the reconstructed images. To remove the ringing artifacts, we have used a Bayesian framework based on a SGLI prior. The SGLI prior is very effective in preserving the strong edge components and significant discontinuities such as textures of objects while reducing the ringing artifacts from corrupted images. Experimental results show that the proposed method yields average PSNR improvements of 0.09 dB in the image interpolation artifact reduction and 0.21 dB in the JPEG compression artifact reduction.
Nowadays, displays of many different sizes have come into wide use. We believe that the proposed deringimg method can be effectively employed for enhancing image quality in the displays.
The authors would like to thank all the anonymous reviewers for their valuable comments and useful suggestions on this paper. This work was done during the study period of the Young Scientist Exchange Program between China and Korea, and the authors are also grateful to the China Postdoctoral Science Foundation and the National Research Foundation of Korea for their financial support. This work was supported by the National High Technology Research and Development Program (863 Program) of China (Nos. 2008AA01Z125 and 2009AA12Z210), the Key Scientific and Technological Innovation Special Projects of Shaanxi “13115” (No. 2007ZDKG-55), the National Natural Science Foundation of China (Nos. 60607010, 60201029, and 60971112), and the Program for Cheung Kong Scholars and Innovative Research Team in University (No. IRT0645).
References and links
1. J. D. Ouwerkerk, “Image super-resolution survey,” Image Vis. Comput. 24(10), 1039–1052 (2006). [CrossRef]
2. J. Sun, J. Sun, Z. Xu, and H. Y. Shum, “Image super-resolution using gradient prior,” in: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8.
3. F. Pan and L. Zhang, “New image super-resolution scheme based on residual error restoration by neural networks,” Opt. Eng. 42(10), 3038–3046 (2003). [CrossRef]
4. J. S. Chitode, Digital Signal Processing (Technical Publications, Pune-India, 2008).
5. S. Yang, Y. H. Hu, T. Q. Nguyen, and D. L. Tull, “Maximum-likelihood parameter estimation for image ringing-artifact removal,” IEEE Trans. Circ. Syst. Video Tech. 11(8), 963–973 (2001). [CrossRef]
6. Y. Wu, O. C. Au, E. Luo, D. Tu, and L. Yeung, “A novel deringing method based on MAP image restoration,” in: Proceedings of IEEE International Conference on Multimedia and Exposition (IEEE, 2009), pp. 217–220.
7. K. T. Block, M. Uecker, and J. Frahm, “Suppression of MRI truncation artifacts using total variation constrained data extrapolation,” Int. J. Biomed. Imaging 2008, 184123 (2008). [CrossRef] [PubMed]
9. V. B. S. Prasath, and A. Singh, “Ringing artifact reduction in blind image deblurring and denoising problems by regularization methods,” in: Proceedings of International Conference on Advances in Pattern Recognition (IEEE, 2009), pp. 333–336.
11. C. A. Segall, A. K. Katsaggelos, R. Molina, and J. Mateos, “Bayesian resolution enhancement of compressed video,” IEEE Trans. Image Process. 13(7), 898–911 (2004). [CrossRef]
12. G. Wang, T. T. Wong, and P. A. Heng, “Deringing cartoons by image analogies,” ACM Trans. Graph. 25(4), 1360–1379 (2006). [CrossRef]
13. A. Foi, V. Katkovnik, and K. Egiazarian, “Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images,” IEEE Trans. Image Process. 16(5), 1395–1411 (2007). [CrossRef] [PubMed]
14. S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(6), 721–741 (1984). [CrossRef]
16. T. A. Stephenson and T. Chen, “Adaptive Markov random fields for example-based super-resolution of faces,” EURASIP J. Appl. Signal Process. 2006, 1–12 (2006).
17. D. Rajan and S. Chaudhuri, “An MRF-based approach to generation of super-resolution images from blurred observations,” J. Math. Imaging Vis. 16(1), 5–15 (2002). [CrossRef]
18. H. S. Kim, C. Jung, S. Choi, S. Lee, and J. K. Kim, “A novel approach for Bayesian image Denoising using a SGLI Prior,” Lect. Notes Comput. Sci. 5879, 990–1011 (2009).
20. S. Roth, and M. J. Black, “Field of experts: a framework for learning image priors,” in: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (IEEE, 2005), pp. 860–867.
21. S. Roth and M. J. Black, “Fields of experts,” Int. J. Comput. Vis. 82(2), 205–229 (2009). [CrossRef]
24. Y. K. Park, S. L. Park, and J. K. Kim, “Retinex method based on adaptive smoothing for illumination invariant face recognition,” Signal Process. 88(8), 1929–1945 (2008). [CrossRef]
25. J. Besag, “On the statistical analysis of dirty pictures,” J. R. Stat. Soc. [Ser A] 48, 259–302 (1986).
26. A. B. Hamza and H. Krim, “A variational approach to maximum a posteriori estimation for image denoising,” Lect. Notes Comput. Sci. 2134, 19–34 (2001). [CrossRef]
27. Y. K. Park, K. Jung, Y. Oh, S. Lee, J. K. Kim, G. Lee, H. Lee, K. Yun, N. Hur, and J. Kim, “Depth-image-based rendering for 3DTV service over T-DMB,” Signal Process. Image Commun. 24(1-2), 122–136 (2009). [CrossRef]
28. C. Tomasi, and R. Manduch, “Bilateral filtering for gray and color images,” in: Proceedings of IEEE International Conference on Computer Vision (IEEE, 1998), pp. 839–846.