Efficient multi-frame motion Efficient multi-frame motion estimation algorithms for estimation algorithms for MPEG-4 AVC/JVTH.264MPEG-4 AVC/JVTH.264

Mei-Juan Chen, Yi-Yen Chiang, Hung-Ju Li and Ming-Chieh ChiISCAS 2004

OutlineOutline

IntroductionIntroduction FDVSFDVS

– FDVS for multi-frameFDVS for multi-frame– FDVS for multi-frame in H.264FDVS for multi-frame in H.264

VADSVADS– VADVS in H.264

Refinement Experiment Conclusion

BackgroundsBackgrounds

H.264 can refer from previous 1~5 frames, that can get smaller prediction error and obtain better performance.

MPEG-4 AVC/JVT/H.264 motion estimation involves considerable complexity, which increases linearly with the number of reference frames.

IntroductionIntroduction

In multi-frame motion estimation, we reuse the motion information of each frame to its previous one frame for obtaining the MV.

IntroductionIntroduction

Forward Dominant Vector Selection (FDVS) for multi-frame motion estimation in h.264

Variable block size Activity Dominant Vector Selection (VADVS) in H.264

Refinement for the Composed MV

FDVS for multi-frameFDVS for multi-frame

FDVS ： For frame-rate conversion– new MVs for outgoing bit-stream need t

o be computed– generates the outgoing MV from the inc

oming MVs instead of performing the full-scale motion estimation

FDVSFDVS

MV(n-2)(estimated)=MV1+ MV2=I(n)

1+I(n-

1)2

FDVS for multi-frameFDVS for multi-frame

Dominant MV ： MV carried by a dominant macroblock

Dominant macroblock ： macroblock has the largest overlapping segment with the block pointed by the incoming motion vector

FDVS for multi-frameFDVS for multi-frameCurrentFrame

Ref.0Frame

FDVS for multi-frame motion estimation in h.264 H.264 standard can 16x16, 16x8,

8x16, 8x8, 8x4, 4x8, and 4x4 block size to perform the motion compensation.

Therefore, we must store the seven block size MV with for FDVS algorithm for the previous reference frame.

FDVS for multi-frame motion estimation in h.264 Variable block size in h.264 (16x16 8x8 4x4)Variable block size in h.264 (16x16 8x8 4x4)

MV5=I1(n)+I2(n-1)+I1(n-2)+I4(n-3)+I3(n-4)

FDVS for multi-frame motion estimation in h.264 Variable block Variable block

size in h.264size in h.264 16x8 and 8x416x8 and 8x4 8x16 and 4x88x16 and 4x8

Variable block size Activity Dominant Vector Selection (VADVS) FDVS selects the dominant block wit

h the largest overlapping area from the four neighboring blocks.

ADVS algorithm utilizes the activity information of the macroblock.– Counting the numbers of NZ (nonzero q

uantized DCT coefficients) of covered 8x8 residual blocks represents the activity information of a macroblock.

VADVS

8x8 dct residual

VADVS in h.264

In H.264 standard, residual coding uses 4X4 block size to perform the transformation.

H.264 standard uses integer transform instead of the DCT transform.

Dominant MV ： the MV carried by a dominant block.

dominant block ： block that has the maximum NZ

VADVS in h.264

Refinement for composed Refinement for composed MVMV

A composed MV is usually not optimal because each dominant motion vector is an approximated value of the optimal MV.

Horizontal and Vertical Search (HAVS) to refine the composed motion vector.

Refinement for composed Refinement for composed MVMV

ExperimentExperiment

bit-rate variation is less than 2%

ExperimentExperiment

PSNR difference is less than 0.1dB compared with the reference software JM61E_REF5.

ExperimentExperiment

ConclusionConclusion

In this paper, we propose the efficient algorithms to improve the coding speed by 4 times in average, and maintain the performance.