Ljubomir Jovanov Aleksandra PiˇzuricaStefan SchultePeter Schelkens Adrian Munteanu Etienne Kerre Wil fr ied Phi l ips

Combined Wavelet-Domain and Motion-Compensated Video Denoising Based on

Video Codec Motion Estimation MethodsIEEE TRANSACTIONS ON

CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 19, NO. 3, MARCH 2009

IntriductionMotion field refinement stepMotion compenstated temporal fi lterSpatial fi lterResult

motion refinement algorithm denoising results

OUTLINE

noise in the video sequences increases image entropy, thereby reducing the eff ective compression performance.

By reusing motion estimation resources from the video coding module for video denoising

motion fields produced by real-time video codecs cannot be directly employed in video denoising, since they, as opposed to noise filters, tolerate errors in the motion field

a novel motion-field filtering step that refines the accuracy of the estimated motion to a degree

a novel temporal filter is proposed that is robust against errors in the estimated motion field.

Introduction

Motion estimation, such as half-pixel motion field estimator defined in MPEG-4 standard used in our work, do not capture the realistic motion fields, because it does not use the neighboring motion vectors to impose a structure on a motion field.

we propose a motion field filtering technique that eliminates spurious motion vectors from the spatial areas in the video frames where no actual motion exists.

compare the MAD between the corresponding blocks with the average MAD, and based on that we decide if motion is present or not.

Motion Field Refinement Step

Frame k-1

Frame k

Block(i,j)

pixel(m,n)

We define a threshold for motion detection in the kth frame as follows:

we decide whether motion exists in each block simply by comparing the absolute block diff erence with the previously calculated threshold

Motion Field Refinement Step

γ : Scalar, 0.45 yield the best result for most of the test sequence

motion vector is set to zero Otherwise keep its original value

Denoising based on motion compensated fi ltering along the estimated trajectory is a very powerful approach.

But this approach can yield very disturbing artifacts at positions where the MVs are incorrect.

The main idea behind the proposed fi lter is to control switching between weaker and stronger temporal smoothing based on a motion detection variable .

Motion Compenstated Temporal Filter

=0 (no motion detected) apply standard recursive temporal filter =1 (moving positions ) filter along the trajectory and using different filter coefficients

Moreover, we take into account an estimate of the reliability of the estimated motion through prediction errors

expressing filtering unreliability through is to avoid wrong averaging of the diff erent pixel values along the estimated motion trajectory, and hence to avoid motion blur and ghosting artifacts

Proposed motion compensated fi lter :

Motion Compenstated Temporal Filter

ensure 0<ε<1

α,β are fixed parameters of the recursive filters in static and moving areasOptimal α = 0.45 ,β = 0.85

static

moving

K-1 k

Aiming at low complexity and a hardware-friendly solution ， we start from fuzzy fi lter of [11]

This filter applies to each wavelet coefficient a shrinkage factor, which is a function of two measurements: the coefficient magnitude and a local spatial activity indicator (LSAI) ,i.e., .

Spatial Filter

average of the wavelet coefficients in the (2K + 1) × (2K + 1) neighborhood around a given position (i, j).

Degree of activation of Fuzzy Rule 1

1 : signal of interest0 : not interestOtherwise : not sure

We propose the modification of the FuzzyShrink method to make it adaptive to spatially non-stationary noise by estimating σ locally, since the noise after temporal filtering has non-uniform variance.

We use 16*16 overlapping windows and shift these in steps of 8 pixels along each direction.

For each window we use Donoho’s wavelet domain median estimator

Spatial Filter

compare the mean squared error (MSE) in the motion field with and without the motion field refinement step

We observe that the MSE of the motion compensation decreases for the most of the test sequences, which proves the eff ectiveness of the motion field filtering step.

Result – motion refinement algorithm

Result – Motion Refinement Algorithm

The algorithm sets the motion vectors to zero in the smooth areas where no actual motion exists

Use four sequence with additive white Gaussian noise of σ = 10,15,20

Result – Denoising Results

SEQWT[7] : 0.5 - 1.4 dB ↑WST [6] : around 1 dB ↑• [6] slightly tends to degrade the

textures in the image, while preserving well static image edges

• “Miss America” contain less textures than other test sequence

Result – Denoising Results

Preserves texture!!

Less motion blur!!

Compare with [10]

It is important to notice that the method [10] is much more complex 1m40s per frame for the frame size 384*288 on a powerful 8*3 GHz processer

Moreover, the methods [10] requires up to 7 frames to be stored, while proposed method uses only current and previous frame.

Result – Denoising Results

input proposed [10]

Flower Gardan

28 dB 30.8 dB 31.3 dB

Salesman 28dB 34.61 dB 35.13 dB