An Improved 3DRS Algorithm for Video De-interlacing

Songnan Li, Jianguo Du, Debin Zhao, Qian Huang, Wen Gao

in IEEE Proc. Picture Coding Symposium (PCS), Beijing, China, Apr. 2006.

INTRODUCTION

Improved 3DRS Algorithm Bi-directional Block Matching Multi-resolution Searching

EXPERIMENTAL RESULTS

CONCLUSION

Introduction Traditional television systems adopted

interlaced scan As a tradeoff between frame rate and transmission

bandwidth requirements Some artifacts, such as

flicker, serration makes video coding

less efficient and more complex

de-interlacing as a pictureformat conversion frominterlaced to progressive pictures

De-interlacing methods(1/2) Line doubling ,

line interpolation frame rate identical to

the field rate used more frequently in

high end consumer electronics Weaving

forming a jagged edge

De-interlacing methods(2/2) Directional dependent filtering

Adaptive adjust Motion-Compensated (MC) interpolation

better results massive requirements on computational resources

and storage. 3D recursive searching (3DRS) algorithm has

superiority at both true motion estimation and hardware complexity forward block matching suffers from latent error

propagation fixed matching block size isn’t suitable for predicting

different kinds of motion

INTRODUCTION

Improved 3DRS Algorithm BI-DIRECTIONAL BLOCK MATCHING MULTI-RESOLUTION SEARCHING

EXPERIMENTAL RESULTS

CONCLUSION

Forward Block Matching for Original 3DRS For each block B(x) at block position x of fn,

the match error criterion is the Sum of Absolute Difference (SAD) which can be described as follow: d : motion vector, for x’ : y mod 2 = n

mod 2

leads to the interpolation errors propagating into subsequent output frames

),( yx ddd

BI-DIRECTIONAL BLOCK MATCHING assumption that the motion at forward and

backward fields is consistent For each block B(x) at position x of fn, the

match error criterion for bi-directional block matching is as follows:

d : motion vector, , the restriction : dy mod 2 = 0

),( yx ddd

MULTI-RESOLUTION SEARCHING Larger block size makes the motion estimation

less sensitive to noise reduces the aperture problem not suitable to small-area motion details

Adjust block size according to the video content. First, at the lowest resolution, matching block

sized 16x16 By split criterion, decide

whether to go down to a

higher resolution, four 8x8 smallest block size 4x4.

Split Criterion(1/2) Observation: A moving object from a long distance (at a low

resolution) It may contain many types of motion.

Pay attention to a part of it (at a high resolution) You will get a series of similar motions.

Split Criterion(2/2) formalized as follow:

Function f returns true or false depending on the similarity

of 9 MVs

get MV

split to four (n/2)x(n/2) block

higher resolution

At least 5MVs=MV5?

falsetrue

n x n block

3DRS motion estimation

Candidate Set at Different Resolution(1/2) Original 3DRS construct a candidate set

spatial candidates, temporal candidates and update candidates

Candidate Set at Different Resolution(2/2) Improved 3DRS algorithm adopts different

compositions

16x16

8x8

4x4

increase horizontalsearch range

INTRODUCTION

Improved 3DRS Algorithm Bi-directional Block Matching Multi-resolution Searching

EXPERIMENTAL RESULTS

CONCLUSION

Subjective Visual Quality(1/2) Step1: extract interlaced videos from

progressive sequences by dropping one field in each frame

Step2 : de-interlace the generated videos using original 3DRS algorithm improved 3DRS algorithm , respectively.

Video sequence:1. Paris2. Mobile & calendar

Subjective Visual Quality(2/2)1. Video sequence : paris

2. Video sequence : mobile & calender

a) Original 3DRS

b) improved 3DRS

a) Bi3DRS b) improved 3DRS

Objective PSNR Performance(1/2) 7 CIF progressive sequences and 2 HD

progressive sequences are selected.

test 3 de-interlacing algorithms: original 3DRS, Bi3DRS and improved 3DRS

algorithm. each of them is followed by the AR error

protection[11]

Objective PSNR Performance(2/2)

the parameter C1 of AR error protection determines the ratio of temporal and spatial information

If parameter C1 ↑ spatial information ↑ => the impact of error propagation

↓ benefits from accurate motion estimation ↓

CONCLUSION 3DRS algorithm is an efficient motion estimation

method for the video de-interlacing process

Improve 3DRS algorithm to overcome some deficiencies bi-directional block matching

to eliminate error propagation multi-resolution searching

to adjust matching block size according to the video content.

Experiments show that better subjective and objective results than the original 3DRS algorithm on most tested sequences.