Fast Method for Noise Level Estimation and Integrated Noise Reduction

Angelo Bosco1, Arcangelo Bruna1, Giuseppe Messina1, Giuseppe Spampinato1

Abstract This paper describes a fast method for noise level estimation and denoising. Specifically, we address the problem of estimating the standard deviation of additive white Gaussian noise in digital images; the computed value is used to reduce Gaussian noise and eliminate defective pixels in a raw digital image. The method is particularly suitable for implementation in low power mobile devices with imaging capabilities such as camera phones, as well as Digital Still Cameras (DSC) and TVs. Index Terms Defect Correction, Noise Level Estimation, Noise Reduction.

I. INTRODUCTION Noise signal has a negative impact on the optimal functioning of the image processing pipeline as a whole and lowers the perceived quality of the acquired data. Hence a smart filter enabling successful noise reduction without affecting the tiny details of an image is of paramount importance in an imaging system. Camera phones and low-end digital still cameras are particularly subject to noise degradation, especially when images are acquired in low light; the usage of these devices in bad lighting conditions is very frequent especially because of their portability. In case of low light environments, such as indoor scenes, the image signal must be amplified so as to obtain an acceptable picture. The possible absence of a flashgun, typically in camera phones, further worsens the shooting conditions. Unfortunately, when boosting an already degraded signal, noise is also amplified. The noise reduction filter must be adaptive so as to change its strength according to the amount of noise present in the image. In order to implement adaptiveness, we need some measure to estimate the noise level in the image signal. Typically, noise level information is provided by the noise standard deviation V; many filters rely on V to adaptively change their smoothing effects [1][2][3][4]. In literature, various methods for the evaluation of V have been suggested; basically, noise level estimation techniques can be categorized in two main classes: block based and filter based methods. Briefly, a block based method partitions the image in a sequence of blocks; the estimation of sigma is carried out by properly calculating a weighted noise level obtained by
1 Angelo Bosco, Arcangelo Bruna, Giuseppe Messina and Giuseppe Spampinato are with STMicroelectronics, in the Imaging Group of the Advanced System Technology Catania Lab Italy. (e-mail:angelo.bosco@st.com; arcangelo.bruna@st.com; giuseppe.messina@st.com; giuseppe.spampinato@st.com)

averaging the noise levels of the most homogenous blocks. The filter based methods perform a pre-filtering operation in which the noisy image is blurred to suppress image structure; a difference image is then computed by subtracting the filtered image from the original one. The noise level is then estimated using the difference image, which is assumed to contain the noise signal only. An efficient technique based on the block based approach is described in [5]; a comparison between various methods for determining an approximation of the noise level in an image is given in [6] . In this paper we address the problem of implementing a fast method for noise level estimation in digital images degraded by Gaussian noise; the validity of the estimation is verified by integrating the algorithm in a noise reduction system. The paper is organized as follows: next section describes the assumed noise model and its effects on digital images degradation. Section III illustrates the proposed noise estimation method. Section IV shows noise estimation experimental results and how the estimated noise value is used to effectively filter a raw image.

II. NOISE MODEL Noise degrading the quality of digital images is usually modeled as stationary additive zero mean Gaussian noise (AWGN). The Gaussian function is defined by the following equation:
Gx 
 1 e V 2S
x

 P 2

2V 2

(1)

where V represents the noise standard deviation and P is the mean value of the distribution. The equation representing the image formation model is:
I N i, j  I NF i, j   N i, j 

(2)

Where IN and INF represent the noisy and the noise free images respectively, whilst N(i,j) is the AWGN additive noise contribution. From equation (1), it is clear that a key element to be investigated is V. The higher V the wider the Gaussian distribution bell. A further relevant characteristic is given by the laws that link the P and V values to the distribution samples:

- 68% of its samples fall in the interval [P-V, P+V] - 95% of its samples fall in the interval [P-2V, P+2V] - 99% of its samples fall in the interval [P-3V, P+3V]

P0

P1

P2

(3)
P3 Pc P4

Hence, pixels deviate from their correct value by some constant which is drawn from a Gaussian distribution; usually pixel fluctuations are small, but greater variations are also possible. Nonetheless, in 99% of the cases, the deviations do not exceed 3V in absolute value. Although to a less extent, large noise values generated by the distribution tails are possible; in this case the pixel of interest might appear as a spiky or dead element. The knowledge of a good V estimation allows filtering the image data properly, significantly reducing the unpleasant effects of Gaussian noise. Furthermore, V can also be a reference value for detecting outliers. III. PROPOSED NOISE ESTIMATION METHOD The classical method for estimating the noise standard deviation is based on the following general ideas [2]: 1) locate homogeneous areas in the image (because in flat areas pixel fluctuations are supposed to be due exclusively to random noise). 2) Compute the local variance in the detected flat areas. 3) Repeat steps 1) and 2) until the whole image has been processed. 4) Finally estimate the mean variance (or standard deviation) by averaging the computed local estimations. This method has a major drawback: it requires a high number of computations. The proposed approach requires only a few comparisons and sums; hence it is suitable for real-time applications which require low power consumption, such as camera phones. As stated in section II, the image signal is supposed to be affected by AWGN with a maximum deviation value Vmax. By exploiting property (3) of the Gaussian distribution, we can reasonably find a good estimation of the noise level in the image so as to obtain a good measure of the picture degradation. Without loosing generality and within a certain degree of confidence, it is possible to assume that an image cannot be affected by an arbitrary high noise level. Hence, a certain Vmax value can be set; i.e. we start from the hypothesis that a given pixel will not deviate, in absolute value, more than Vmax from its correct value. By fixing Vmax=3V we could straightforwardly obtain the V value by considering the 68% of the noise samples. Obviously, it is not that trivial; the noise samples and the value 3V are not available, so an approximation has to be found. Noise samples will be gathered by analyzing flat areas of the image; the Vmax threshold will be determined after a proper tuning phase of the algorithm. Processing the image on a pixel basis, the absolute differences d0, d1,, d7 between the central pixel and its neighborhood are computed.

P5

P6

P7

Fig. 1 Differences computation in flat areas (red case).

If Bayer CFA data [8] is considered, the differences are computed as in Fig. 1: di=abs(P-Pi), i=0,,7 (4)

Fig. 1 illustrates the case of a red pixel in CFA data. If all the 8 differences are very small, we can reasonably assume that a flat area has been detected; in this case, each absolute difference is plotted in a noise histogram that is used for the estimation of V. The noise samples di are counted and accumulated in the corresponding histogram bin.

IvrCvthhq6iyrIvrCvth
IvrThyr % IvrCv 6iIvrCv $

#

"

!



0 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 IvrWhyr 53 55 57

Fig. 2 Full Noise Histogram (shorter bars) and Absolute Noise Histogram (higher bars on the right hand side). Differences equal to 0 have been excluded.

After processing the whole image, the noise histogram will be half Gaussian-like distributed (higher bars in Fig.2); if simple differences were computed in place of the absolute differences, the histogram would appear as shown by the shorter bars in Fig. 2. The bin related to the differences equal to zero is discarded because it refers to noiseless samples. As soon as the population of the histogram is completed, it is possible to estimate V by counting the histogram elements until the 68% of the total samples has been reached. The index on the x-axis of the histogram allowing reaching the 68% of the samples represents the estimated noise level (Fig. 3).

@hyr)IvrGrry@vhvsurBrrThyr8y8uhry

Black Image (Maximum Noise Example) - Normalized Histogram

  

V H  O S P D 6

50 45 40 35 30 H(D(i)) RedN GreenN BlueN

                           IvrGrry       *UHHQ

25 20 1 3 5 7 15 9 11 13 10 15 17 19 5 21 23 25 0 27 29

D(i)

Fig. 3 The index on the x-axis allowing to reach the 68% of the total samples represents the current noise level estimation.

Fig. 6 Black Image Noise histogram.

Image structure interference is mostly avoided because the histogram is updated using only areas in which the absolute differences are all less than Vmax; this implicitly means that the pixel belongs to an almost flat area. The Vmax threshold must be selected with care because an excessive underestimation or overestimation can lead to a bad prediction.
Collected Noise Samples
300000

250000 200000 H(D(i)) 150000 100000 50000 0 1 3 5 7 9 11 13 15 Red Blue Green

If a Bayer CFA image sensor is considered, it contains twice as many green elements as red or blue; hence the corresponding bins contain a higher number of green samples: a full absolute histogram relative to an indoor acquired picture is shown in Fig. 4; the corresponding normalized absolute histogram is illustrated in Fig. 5 which shows that the blue channel contains higher noise if compared to the green and red channels. This is consistent with the fact that usually, the blue channel is the worst. Fig. 6 illustrates the noise histogram relative to an image acquired in complete darkness; hence it contains only noise. In this case the bins in the histogram contain approximately the same amount of noise for each color channel.

17

19

21

25

27

D(i)

23

29

(a) Fig. 4 Noise histogram of a CFA image.

8yyrprqIvrThyrIhyvrqCvth

50 45 40 35
 I   v  9  C

30 25 Rnorm% 20 15 10 5 0 Gnorm% Bnorm%

(b)

    

  

               'L1RUP      

Fig. 5 Normalized histogram of a CFA image.

(c) Fig. 7 (a) Original Noisy Color Image. (b) Output of the simple Vmax method. (c) Output of the flatness detector.

Fig. 3 was obtained by mixing the information from all color channels.

A visual representation of the detected homogeneous areas is shown in Fig. 7. Specifically, a color image was artificially bayerized and corrupted with zero mean AWGN having V=10. The pixels used to build the histogram using the Vmax threshold are shown in Fig. 8(b). The dark pixels are the ones selected to populate the noise histogram; the brighter ones are discarded as they belong to textured zones. The results obtained by using a texture analyzer [2] are shown in Fig.8(c); clearly, the results are quite similar and the variations do not affect the final estimation. Eventually, the method using the Vmax threshold generated the same noise estimation as the one using the flatness detector.

For the sake of clarity, we briefly summarize the old method proposed in [2][3][9]. The Gaussian filter averages only pixels that are similar to the central one in the filter mask. Two slightly different versions of the central pixel P are considered, i.e. the V-biased versions P-V and P+V. An interval whose wideness is directly proportional to V is used to select the pixels that can be safely averaged. In Fig. 9 the interval centered on P is chosen, because it maximizes the number of selected pixels.

IV. EXPERIMENTAL RESULTS The technique was first tested using a uniform reference grayscale image contaminated with various amounts of AWGN with V ranging from 1 to 18. The results of the test are shown in Fig. 8; the real estimation is plotted versus different estimations generated by choosing different Vmax possible values.
Ivr@vhv

P-V

P+V

Fig. 9 Only similar pixels are selected for filtering.

Pixels are successively averaged using different weights depending on their distance from the interval center. B. DEFECT CORRECTION
9hxQvry Hh Hv Pyvr 7vtu Qvry

FUHDO F F F

y r r G r v I q r h v @

                  

3V
F F F FUHDO
                                                                 

3V

Fig. 10 Central pixel outlier detection.

FUHDO  F F F

          

           
6WG'HY

Fig. 8 Experimental Results.

The trend of the curves shows the validity of the method. The creal bars represent the true noise standard deviations; c25, c30, c35 are the estimations curves obtained by using different values for Vmax. We are generally not interested in noise values higher than 15, hence for our practical purposes the method works reliably. The Vmax threshold can be obtained by fine tuning its value according to the image sensor in use, testing its behaviour under different illumination conditions. A. GAUSSIAN NOISE REDUCTION For a real case study, the proposed estimation technique has been inserted in a system for Gaussian noise reduction; its reliability basically depends on the V estimation. We have improved the method illustrated in [2][3] to obtain a more robust detection of the outliers. The updated technique uses the estimated V.

Unfortunately, if the central pixel P is itself an outlier (i.e. spike or dead element) then no pixels will be similar to it; in this case the V-biased versions of P provide no help, because the corresponding intervals may not include any pixel either. For example, if a spike is located in the central pixel of the filter mask, then, after sorting the pixels, the situation appears as the one depicted in Fig. 10. To overcome this issue, let us also consider the minimum (min) and the maximum (max) value contained in the neighborhood of the central pixel. These values, along with P-V and P+V are used to determine if the central element in the mask is either correct or affected by Gaussian noise or defective. Hence, if P-V is greater than the maximum value in the neighborhood, then it is likely to be an outlier (spike). If a pixel is classified as defective, it is substituted by a weighted average of the remaining pixels. C. OVERALL ESTIMATION AND FILTERING RESULTS We filter out noise in a pre-processing phase before color interpolation; this improves the efficiency of the pipeline [10]. Fig. 11a) shows noisy CFA data in false colors; the image is also corrupted by defective elements; in Fig. 11b) the filtered counterpart is shown.

Noise level is estimated as described in Section III. Defective elements are eliminated by using the aforementioned method (see section IV.B); finally Gaussian noise is reduced using the method described in section IV.A (see also [2][4] for further details).

V. CONCLUSIONS A fast method for noise level estimation has been presented. The technique is valid for estimating the noise level in images affected by Additive White Gaussian Noise. It requires low computational resources; hence it is useful for implementation in low power devices such as camera phones. The proposed estimation method has been validated by incorporating it into a system for noise reduction and defect correction of raw CFA digital images. REFERENCES
[1] J.Brailean, R.Kleihorst, S.Efstratiadis, A. Katsaggelos, R. Lagendijk, Noise Reduction filter for Dynamic Image Sequences: A Review, Proceedings of the IEEE, Vol.83, No.9, Sept. 1995 [2] S. Battiato, A.Bosco, M. Mancuso, G. Spampinato, Adaptive Temporal Filtering for CFA Video Sequences, In Proceedings of IEEE ACIVS02 Advanced Concepts for Intelligent Vision Systems 2002, pp. 19-24, Ghent University, Belgium, September 2002 [3] A. Bosco, K. Findlater, S. Battiato, A. Castorina " A Noise Reduction Filter for Full-Frame Imaging Devices" IEEE Transactions on Consumer Electronics Vol. 49, Issue 3, August 2003; [4] A. Bosco, K. Findlater, S. Battiato, A. Castorina, - "A Temporal Noise Reduction Filter Based on Full-Frame Data Image Sensors" - in Proceedings of IEEE - ICCE 2003 (International Conference on Consumer Electronics) Los Angeles, June 2003 [5] Aish Aishy Amer, Eric Dubois, Amar Mitiche, Reliable and Fast Structure-oriented Video Noise Estimation, IEEE ICIP 2002; [6] S.I.Olsen, Noise Variance Estimation in Images in Proceedings of the 8th Scandinavian Conference on Image Analysis, Troms , Norway 1993;

(a)

(b)

Fig. 11 (a) CFA data with detective elements. (b) Corrected CFA image plus Gaussian Filtering.

The effects of defects cancellation and Gaussian noise reduction on color interpolated images are shown in Fig. 12, which illustrates a cropped and magnified part of a noisy color interpolated image (a) and its filtered counterpart (b).

Fig. 12 (a) Cropped and Magnified part of a noisy color interpolated image (b) Denoised color interpolated counterpart (impulsive noise removal plus Gaussian noise reduction; noise level was estimated with the proposed method).

[7] G. Cortelazzo, G.A. Mian, R. Parolari, Statistical Characteristics of Granular Camera Noise , IEEE Transactions on Circuits and Systems for Video Technology, Vol. 4, No.6, December 1994 [8] B.E. Bayer, Color Imaging Array, U.S. Patent 3,971,065-1976; [9] A. M. Borneo, L. Salinari, D. Sirtori, An Innovative Adaptive Noise Reduction Filter for Moving Pictures Based on Modified Duncan Range Test, ST Journal of System Research, Vol.1, No.1, 2001; [10] O. Kalevo, H. Rantanen, Noise Reduction Techniques for Bayer-Matrix Images, Proceedings of SPIE 2002, Vol.4669.

The overall system architecture of the imaging pipeline is illustrated in Fig. 13:
QrQprvt 7ypx

6@ Tr 6A 6X7 IG@

AUTHORS
V
DBQhq DBQhq Ivr Srqpv Ivr Srqpv 8qrp 8qrp

8A6

8A6

8A6

Dt

Dt

EQ@BHQ@B

%XV
8A6 8A6 IG@)Ivr Grry @vh 6@)6@r

9S6H 9S6H

6X7)6Xuvr 7hyhpr 6A)6Ap DBQ)Dhtr BrrhvQvryvr

Fig. 13 Imaging Pipeline

Angelo Bosco was born in Catania, Italy, in 1972. He received the degree in computer Science in 1997 at the University of Catania with a thesis in the field of image processing. He joined STMicroelectronics in 1999 as a system engineer in the Imaging and Multimedia Group (AST Catania Lab - Italy). Since then, he has been working on noise reduction algorithms for cmos imagers, both for still pictures and video. His current research interests currently include digital image stabilization algorithms. He is author and co-author of various papers and patents in the image-processing field. Arcangelo Bruna received his Italian degree in Electronic Engineer in 1998 at the University of Palermo. First he worked in a telecommunication firm in Rome. After one year, he joined STMicroelectronics where he works in the Advanced System Technology (AST) Catania Lab - Italy. His current research interests are in the areas of digital image processing from the

The noise level estimation block (NLE) is inserted in the preprocessing block along with other algorithms. The estimated noise level is passed to the noise reduction block which actually filters the raw CFA data.

physical digital acquisition to the final image compression. He is the author of several papers and international patents on these activities. Giuseppe Messina received his degree in Computer Science at Catania University in 2000 doing a thesis about statistical methods for textures discrimination. Since March 2001 he has been working in STMicroelectronics in the AST Imaging & Multimedia Group as System Engineer (Catania Lab Italy). His current activities include research for resolution enhancement and SVG.

Giuseppe Spampinato received the Degree in Computer Science (summa cum laude) from the University of Catania (Italy) in 1996. From 1999 he works in AST (Advanced System Technology) Catania Lab of STMicroelectronics as system engineer. He is author of papers and patents in Image Processing field.