Wyner-Ziv Residual Coding of Video Anne Aaron, David Varodayan and Bernd Girod Information Systems...

Wyner-Ziv Residual CodingWyner-Ziv Residual Coding of Videoof Video

Wyner-Ziv Residual CodingWyner-Ziv Residual Coding of Videoof Video

Anne Aaron, DavAnne Aaron, David Varodayan and id Varodayan and Bernd GirodBernd Girod

Information Systems LaboratoryInformation Systems LaboratoryStanford UniversityStanford University

A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of Video April 26, 2006 2

OutlineOutline

• Wyner-Ziv video coding• Pixel-domain Wyner-Ziv residual video codec• Simulations Results– Side information from motion-compensated

interpolation

– Side information from hash-based motion estimation at the receiver


“Motion JPEG”

Decoder

“Motion JPEG”

Decoder

“Motion JPEG”

Encoder

“Motion JPEG”

Encoder

Wyner-ZivInterframe Decoder

Wyner-ZivInterframe Decoder

Wyner-ZivIntraframe Encoder

Wyner-ZivIntraframe Encoder

Side Information

YX X

Video Coding with Low ComplexityVideo Coding with Low Complexity

[Witsenhausen, Wyner, 1980] [Puri, Ramchandran, Allerton 2002][Aaron, Zhang, Girod, Asilomar 2002]


Xn

Wyner-Ziv Residual CodingWyner-Ziv Residual Coding

Wyner-ZivDecoder

Wyner-ZivDecoder

Wyner-Ziv Encoder

Wyner-Ziv Encoder

Side Information

X’nY

1ˆ

nX



Wyner-ZivDecoder

Wyner-ZivDecoder

Wyner-Ziv Encoder

Wyner-Ziv Encoder

Motion-Compensated Side Information

X’n

Previous Frame Side Information-

Xn

Frame difference

Y

1ˆ

nX1

ˆnX



Wyner-ZivDecoder

Wyner-ZivDecoder

Wyner-Ziv Encoder

Wyner-Ziv Encoder

X’n

-

Xn

Frame difference-

Y

Side Info

1ˆ

nX1ˆ

nX


Wyner-Ziv Decoder

Scalar Quantizer

LDPC Encoder

Buffer

WZ frames

X

Wyner-Ziv Encoder

LDPC Decoder

Request bits

Slepian-Wolf Codec

Interpolation/ Extrapolation

Reconstruction

Y

Key frames

I Conventional Intraframe coding

Conventional Intraframe decoding

X’

I’

Side information

Pixel-Domain Wyner-Ziv Pixel-Domain Wyner-Ziv Residual Video CodecResidual Video Codec

Xer

-Xer-Q-1

Frame Memory

[Varodayan, Aaron, Girod, 2005]


Y

Set-up 1: Side Information from Set-up 1: Side Information from Motion-Compensated InterpolationMotion-Compensated Interpolation

WZ frame

W W’

Decoded WZ frames

Side information

Wyner-Ziv Residual Encoder


WZ parity bits

InterpolationInterpolationDecoded frames

Wyner-Ziv Residual Decoder


I II

Previous key frame as encoder reference


Rate-Distortion Performance – SalesmanRate-Distortion Performance – Salesman

Alternating key frames and WZ frames

Side information from motion-compensated interpolation

Salesman QCIF sequence 15fps, 96 frames

100 200 300 400 500 600 700 800 90026

28

30

32

34

36

38

40

42

44

Rate (kbps)

PS

NR

(dB

)

H.263+ I-B-I-B

WZ DCT

WZ Pixel Residual

Motion-JPEG+


Rate-Distortion Performance – ForemanRate-Distortion Performance – Foreman

Alternating key frames and WZ frames

Side information from motion-compensated interpolation

Foreman QCIF sequence 15fps, 96 frames

100 200 300 400 500 600 700 800 90026

28

30

32

34

36

38

40

42

44

Rate (kbps)

PS

NR

(dB

)

H.263+ I-B-I-B

WZ DCT

WZ Pixel Residual

Motion-JPEG+


Previous frame as encoder reference Y

Set-up 2: Hash-Based Side InformationSet-up 2: Hash-Based Side Information

WZ frame

W W’

Decoded WZ frames

Side information



WZ parity bits

Motion-compensated Extrapolation

Motion-compensated Extrapolation

Hash Generator(with previous hash

store)

Hash Generator(with previous hash

store)

Hash bits Decoded previous

frame



“I-P-P-P structure”

I



100 200 300 400 500 60026

28

30

32

34

36

38

40

42

44

Rate (kbps)

PS

NR

(dB

)

H.263+ I-P-P-P

WZ DCT with DCT Hash

WZ Pixel Residual with Pixel Residual Hash

Motion-JPEG+

Every 8th frame is a key frame

Side information from hash-based motion estimation at the decoder



100 200 300 400 500 60026

28

30

32

34

36

38

40

42

44

Rate (kbps)

PS

NR

(d

B)

H.263+ I-P-P-P


Motion-JPEG+

WZ Pixel with Pixel Hash







100 200 300 400 500 600 700 800 90026

28

30

32

34

36

38

40

42

44

Rate (kbps)

PS

NR

(dB

)

H.263+ I-P-P-P

WZ DCT with DCT Hash


Motion-JPEG+Every 8th frame is a key frame





100 200 300 400 500 600 700 800 90026

28

30

32

34

36

38

40

42

44

Rate (kbps)

PS

NR

(dB

)

H.263+ I-P-P-P


Motion-JPEG+

WZ Pixel with Pixel Hash




Motion-JPEG+ RateY=194 kbps

PSNRY=29.9 dB

Wyner-Ziv Pixel Residual Coding RateY=192 kbps

PSNRY=36.6 dB

Salesman at 15 fpsSalesman at 15 fps


SummarySummary

• Wyner-Ziv coding on residual video frames• Encoder can use simple reference frame while

decoder applies more sophisticated motion compensation

• Good rate-distortion performance for pixel-domain system– Up to 8 dB improvement over independently encoding

the frames– Comparable performance to transform-domain Wyner-

Ziv coding scheme

Wyner-Ziv Residual Coding of Video Anne Aaron, David Varodayan and Bernd Girod Information Systems...

Documents

Transcript of Wyner-Ziv Residual Coding of Video Anne Aaron, David Varodayan and Bernd Girod Information Systems...