Interframe coding of video signalsee225b/fa12/lectures/11-Interframe... · Rate-distortion...

Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 1

Interframe coding of video signals

  Adaptive intra-interframe prediction   Conditional replenishment   Rate-distortion optimized mode selection   Motion-compensated prediction   Hybrid coding: combining interframe

prediction and intraframe transform coding


Interframe coding of video signals

. . . exploits similarity of successive pictures

Previous frame Current frame


“It has been customary in the past to transmit successive complete images of the transmitted picture.” [...] “In accordance with this invention, this difficulty is avoided by transmitting only the difference between successive images of the object.”


Interframe DPCM

Entropy coder Entropy decoder Quantizer

Frame store

e s + - s ^

s'

e' e'

s ^

s'

Coder Decoder

Frame store


Adaptive intra-interframe Prediction

Predictor is switched between two states: A: Intraframe prediction for moving or changed areas.

B: Interframe prediction (previous frame prediction) for still areas of the picture.

Sintra = a1S1 + a2S2 + a3S3 + a4S4 Sinter = S20


Intra-interframe DPCM: feedback adaptation

Entropy coder

Interframe predictor

Intraframe predictor

Entropy decoder Quantizer


Intraframe predictor Predictor

adaptation

e s + - s ^ s ^

inter

s ^ intra

s'

e' e'

s ^ s ^

inter

s ^ intra

s'

Predictor adaptation

Coder Decoder


Intra-interframe DPCM: feedforward adaptation



Entropy decoder Entropy coder Quantizer



Predictor adaptation

e s + - s ^ s ^

inter

s ^ intra

s'

e' e'

s ^ s ^

inter

s ^ intra

s'

intra-/interframe switching information

Coder Decoder


Conditional replenishment

  Still areas: repeat from frame store   Moving areas: encode and transmit address and waveform

CODER DECODER

SIGNAL INPUT

SEGMENTER (MOVEMENT DETECTOR)

CODING, ADDRESSING, BUFFERING

BUFFERING, DECODING, ADDRESSING

FRAME DELAY

TRANSMISSION CHANNEL

SIGNAL OUTPUT

Change detector

Frame store Frame store


Change detection   Example of a pixel-wise change detector

  Example of a block-wise change detector

ABS Average of 3x3 window

Eliminate isolated points or pairs of points

+

-

Current frame

Previous frame

Decision changed/ unchanged

Threshold

ABS Accumulate over NxN blocks

+

-

Current frame

Previous frame

Decision changed/ unchanged

Threshold


Example: pixel-wise change detection

Two successive video frames Change detection

mask

[Xinqiao Liu, EE368B class project, 2000]


Crawford noise reduction filter


“Dirty Window” effect

  Conditional replenishment scheme with change detection threshold set too high leads to the subjective impression of looking through a dirty window.

Moving area picked up by change detector

Moving areas missed by change detector


Rate-distortion optimized mode selection

  How to choose the decision threshold, if distortion D shall be minimized for a given rate R?

  Assumptions   Blockwise mode selection, block index i   Additive overall distortion and rate

  Lagrangian cost function

  Strategy: minimize Ji for each block i separately, using a common Lagrange multiplier λ


Rate-distortion optimized mode selection (cont.)

D

R

D1(R1)

D2(R2)

•  Consider 2 blocks with D(R) = D1(R1) +D2(R2) •  Intra-coding only


Rate-distortion optimized mode selection (cont.)

D

R

D1(R1)

D2(R2)

Interframe replenishment

Intraframe coding


Motion-compensated prediction

x

y

stationary background

moving object

shifted object

time t

previous frame

current frame

Prediction for the luminance signal S(x,y,t) within the moving object:

S(x, y,t) = S(x − dx , y − dy ,t − Δt)

„Displacement vector“


Combining transform coding and prediction

- -

Transform domain prediction Space domain prediction


Intra-frame Decoder

Motion- Compensated

Predictor

Control Data

DCT Coefficients

Motion Data

0

Intra/Inter

Coder Control

Decoder

Motion Estimator

Intra-frame DCT Coder -

Motion-compensated hybrid coder


Intra-frame Decoder

Motion- Compensated

Predictor

Control Data

DCT Coefficients

Motion Data

0

Intra/Inter

Decoder

Motion-compensated hybrid decoder


Reading

  B. Girod, H. Almer, L. Bengtsson, B. Christensson, P. Weiss, “A Subjective Evaluation of Noise-Shaping Quantization for Adaptive Intra-/Interframe DPCM Coding of Color Television Signals,” IEEE Trans. Communications, vol. 36, no. 3, pp. 332-346, March 1988.

  B. Haskell, P. Gordon, R. Schmidt, J. Scattaglia, “Interframe Coding of 525-Line, Monochrome Television at 1.5 Mbits/s,” IEEE Trans. Communications, vol. 25, no. 11, pp. 1339-1348, Nov. 1977.

  S. Ericsson, “Fixed and Adaptive Predictors for Hybrid Predictive/Transform Coding,” IEEE Trans. Communications, vol. 33, no. 12, pp. 1291-1302, Dec. 1985.

Interframe coding of video signalsee225b/fa12/lectures/11-Interframe... · Rate-distortion...

Documents

Transcript of Interframe coding of video signalsee225b/fa12/lectures/11-Interframe... · Rate-distortion...