Vinh Quang Luu, Truong Quang Nguyen, Cuong The Pham, Kien ...
Ai-Mei Huang And Truong Nguyen Image processing, 2006 IEEE international conference on
-
Upload
shoshana-reynolds -
Category
Documents
-
view
65 -
download
0
description
Transcript of Ai-Mei Huang And Truong Nguyen Image processing, 2006 IEEE international conference on
A i - M e i H u a n g A n d Tr u o n g N g u y e nI m a g e p r o c e s s i n g , 2 0 0 6 I E E E i n t e r n a t i o n a l c o n f e r e n c e on
Motion vector processing based on residual energy information for
motion compensated frame interpolation
INTRODUCTION
• Low bandwidth requirements, i.e. video telephony• by skipping frames• low frame rate video usually
results in motion jerkiness
• Motion-compensated frame interpolation (MCFI)• Directly uses the received MVs• Suffer from annoying artifacts such as blockiness and ghost effect.• Motion re-estimation is not suited for mobile devices
Video in encoder
decoderReconstructed frame
Bit stream
Video coding/compression
Motion compensated frame interpolation
• In MCFI methods, a skipped frame is often bi-directionally interpolated • from its two neighboring reconstructed frames .• by using the received MVs of the second frame.
MV is not always reliable!
The correlation between motion vectorreliability and residual energy.
• Macroblock(MB) contain areas of
different motion.• Encoder favors
the MV that can represent most ofthe region
• The prediction residual will be generated and encoded.• Correct those unreliable MVs for frame interpolation• Avoid using those unreliable MVs to correct other MVs.
The proposed method
• Based on residual energy associated with each motion vector
• iterative process, stops when the MAP is no longer changed, or reaches a pre-defined maximum iteration, max_ite.
(1)motion vector classification based on residual energy
• Classify MVs to 3 Groups.• Calculate every 8x8 block’s
residual energy Em,n
• By taking the sum of the absolute value ofeach reconstructed prediction error for each pixel.• Em,n<threshold MV will be classified as
reliable and place into first group G1.• Em,n>=threshold MV will be classified as
unreliable and place into first group G3.• Unreliable MV’s neighboring MVs within the same MB will be
classified as possibly unreliable into the second group G2
(base unit)16*16 MB=>four 8*8 blocks
(2) motion vector correction
• Works on those unreliable MVs Correcting from neighbor reliable MV
• A residual-energy constrained median filter (RECMF) is used in this process and defined as
• S contains the neighboring MVs centered at v*m,n.• Select a new MV only from its neighboring reliable MVs.
• vm,n itself will be excluded from the candidates.• we prevent unreliable MVs to be used to correct other
unreliable MVs.
(3)motion vector similarity check
• Ensure v*m,n is not identical or similar to vm,n
• Remain unreliable
• Use angle variance by taking inner product of the two vectors.• < threshold => two MVs is similar ,fail in the check => bm,n still in G3• Pass in the check => bm,n take in G1
(4)motion vector re-sampling and smoothing
• In [7], each 8 x 8 block is further broken into four 4 x4 sub-blocks.• The MVs of these four sub-blocks can be obtained
simultaneously by minimizing a smoothness measure
, which is defined in the following.
• For example:
[7] G. Dane and T. Q. Nguyen, "Smooth motion vector resamplingfor standard compatible video post-processing," Proc. AsilomarConf: Signals, Systems and Computers, 2004.
SIMULATIONS
• two video sequences, FOREMAN and SILENT, • CIF frame resolution • 30 frame per second (fps).• both encoded using H.263 but even frames are skipped
by the encoder.• fix quantization parameter (QP) values.
• Avg. bitrate• FOREMAN is 240.3 Kbps • SILENT is184.59 Kbps