Block Loss Recovery Techniques for Image and Video Communications

Block Loss Recovery Techniques for Image and Video Communications

Jiho Park

The Computational Intelligence Applications (CIA) Lab.

Department of Electrical Engineering

University of Washington

May 21, 2002

2

Outline

Introduction

Background for Block Recovery

Spatial Block Loss Recovery Technique

Temporal Block Loss Recovery Technique

Interpolation based Coding and Data Priority

Setting

Conclusion & Future Work

3

Image and Video Coding Standards

Adopt block based coding (e.g. JPEG, MPE

G, H.261/3+)

Lossy coding techniques

Images are segmented into blocks

Predictive Coding, Discrete Cosine Transform

and Quantization are adopted

4

Background –Block Coding & Transmission Error

Block based coding technique

N x N block

Discrete Cosine

TransformQ RLC Packet

Encoding Packetizing Network Depacketizing DecodingImage/ Video

Source

image/video transmission system.

5

Transmission & Error

During transmission, some packets are lost due to bit error, congestion of network, noise burst, or other reasons.

Packet

Packet error during transmission causes all data loss in a corresponding block.

6

Background – Error Correction Automatic Retransmission Request(ARQ) – Decoder sen

ds retransmission requests to encoder when error happens. pro : error-free. cons : additional delays, unacceptable for real-time. example : TCP/IP – unreal-time internet file transfer protocol

Forward Error Correction(FEC) – Encoder inserts additional codes so that decoder can correct errors. pro : error resilient. con : additional bits

example : 18-bit FEC code in H.261(video-conferencing standard)

Error Concealment/Block Loss Recovery – Post-processing to restore damaged blocks by decoder

pro : no additional bits. con : complex decoder

7

Spatial EC : Exploiting spatially neighboring image data of a missing block.

Temporal EC : Exploiting temporally adjacent frame of video.

Background –Error Concealment / Block Loss Recovery

tt-1

8

Spatial Block Loss Recovery

Spatial block loss recovery is suitable for

image coding data, (e,g, JPEG, Image

based coding frame of MPEG/H.261/263+)

9

Averaging the values of pixels in the same position of surrounding blocks.

Previous Research (Spatial EC) -Edge-based Average Interpolation (Ancis ’99)

1x 2x 3x

5x4x

6x 7x 8x

8

1

8

1

,, /i

k

i

jikk

jim wxwx

where wk = 0, 1, according to edges in surrounding blocks

10

Previous Research (Spatial EC) - Interpolation Exploiting Interblock Correlation (Hemami ’95)

Tx

RxLx

Bx

BBRRLLTTM xwxwxwxwx

To get weight, W, the values of boundary pixels in a missing block are set to the value of adjacent known pixels.

Weight, W, are computed by solving the equations using Linear Least Squares Problem.

SXWXM

S''' XWX M

(1~64)

(1~30)

11

Set Y2 to zero =>

Previous Research (Spatial EC) – DCT-based interpolation (Ziad, 2000)

,2211 XTXT

XTY

unknown.: known,: 21 XX

2

1

2221

1211

2

1

X

X

TT

TT

Y

Y

222121 XTXT0 121222

1 XTTX

Kernel DCT:T

12

Previous Research – Other Algorithms

Maximally Smooth Image Recovery (Wang, ’93)

- minimizing errors between a missing block and surrounding pixels

DCT Coefficients Block Recovery (Park, ’97)

- minimizing errors between boundary pixels of a missing block and surrounding pixels

Block Recovery using POCS (Sun, ’95)

13

Conventional Algorithms -

1) Not adaptive to local image structure (Ziad, Hemami,…)

2) ill-utilizing surrounding image structure (hemami,…)

3) no ability of recover certain frequency bands (Sun, Ziad, Wan

g, Park…)

Previous Research – Problem Formulation

14

Block Loss Recovery – Research Goal

To design a spatial block loss recovery Technin

que More Reliable

Adaptive to Image Structure

No Retransmission

No Additional Bits for error correction

15

Projections based Block Recovery – Motivation

Conventional Algorithms use information of all surrounding area. Using only highly correlated area

16

Alternating Projections is projecting between two or more convex sets iteratively.

Projections onto Convex Sets –Alternating Projections

Converging to a common point

17

Signal Restoration using Alternating Projections – Papoulis-Gerchberg Algorithm

Band Limiting.Convex Set, C1

IFT

Identical Middle.Convex Set, C2

FT

FT

Original Signal

Corrupted Signal

C2

18

Projections based Block Recovery – Algorithm

2 Steps Pre Process : 1) Edge orientation detection

2) Surrounding vector extraction

3) Recovery vector extraction

Projections : 1) Projection operator P1

2) Projection operator P2

3) Projection operator P3

19

Edge orientation in the surrounding area(S) of a missing block(M). In order to extend the geometric structure to the missing block.

Simple line masks at every i, j coordinate in surrounding area(S) of the missing block(M) for edge detection.

Pre Process 1 –Edge Orientation Detection

121

121

121

vL

111

222

111

hL

Horizontal Line Mask Vertical Line Mask

20

Pre Process 1 – Edge Orientation Detection

Responses of the line masks at window W :

Total magnitude of responses :

Th > Tv ; Horizontal line dominating area

Th < Tv ; Vertical line dominating area

987

654

321

www

www

www

W987654321 w-w-w-w2w2w2w-w--w hR

987654321 w-w2w-w-w2w-w-w2-w vR

,||T S

hh R S

vv R ||T

21

Pre Process 2 – Surrounding Vectors

Surrounding Vectors, sk, are extracted from surrounding area of a missing block by N x N window.

Each vector has its own spatial and spectral characteristic. The number of surrounding vectors, sk, is 8N.

}W),(),,(:{ jijixxks

22

Pre Process 3 – Recovery Vector Recovery vectors are extracted to restore missing pixels. Two positions of recovery vectors are possible according to the

edge orientation.

Recovery vectors consist of known pixels(white color) and missing pixels(gray color).

The number of recovery vectors, rk, is 2.

}W),(),,(:{ jijixxkr

Vertical line dominating area Horizontal line dominating area

23

Projections based Block Recovery –Projection operator P1

Recovery vectors, ri, for i = 1, 2

Surrounding vectors, sj , for j = 1 ~ 8N

Surrounding vectors, S, form a convex hull in N2-dimensional space

Recovery vectors, R, are orthogonally projected onto the line defined by the closest surrounding vector, si, j : Projection Operator P1.

24


Projection operator P1 :

Convex hull (formed by surrounding vectors, containing information of local image structure)

25


Surrounding vectors, sj , for j = 1 ~ 8N Recovery vectors, ri, for i = 1, 2

The closest vertex, sdi , from a recovery vector, ri.

or equivalently in DCT domain,

P1 :

Njiford jij

i 81,21||||minarg sr

Njiford jij

i 81,21||||minarg SR

21,||||

,)(

2

ii

i

d di

idiiP S

R

RSRS

26

Convex set C2 acts as an “identical middle”.



otherwise

nforFFC

o

n

ff

ff

:

L: maxmin2

otherwise

nFforF

nFforF

P

n

n

n

n

f

f

f

f L

L

max,max

min,min

2

27

Convex set C3 acts as a convex constraint between missing pixels and adjacent known pixels, (fN-1 fN).

where, and

is a N x N recovery vector in column vector form.

Projections based Block Recovery – Projection operator P3

fN-1 fN

}||:{3 EC n gg

)}(....,),{( ,,10,0,1 NNNNNN ffffg }....,,,{ 21 Nffff

28

Projections based Block Recovery – Projection operator P3


otherwise

nEforE

nEforE

P

mn

nmn

nmn

mn

,

,1

,1

,3 L,

L,

f

gf

gf

f

29

Projections based Block Recovery –Iterative Algorithm

Missing pixels in recovery vectors are restored by iterative algorithm of alternating projections :

N x N windows moving :

ii fPPPf 3211

Vertical line dominating area Horizontal line dominating area

30

Projections based Block Recovery - Summary

Edge Orientation Detection

Surrounding Vector Extraction

Recovery Vector Extraction

Projection Operator P1



Iteration=I?

All pixels?

31

Simulation Results – Test Data and Error

512 x 512 “Lena”, “Masquerade”, “Peppers”, “Boat”, “Elaine”, “Couple”

176 x 144 “Foreman” 352 x 240 “Flower Garden”

8 x 8 pixel block loss 16 x 16 pixel block loss 8 x 8 consecutive block losses

Peak Signal – Noise – Ratio

)|),(ˆ),(|

255log(10

1 1

2

2

N

i

M

j

jixjix

MNPSNR

32

Simulation Results –Lena, 8 x 8 block loss

Original Image Test Image

33


Ancis, PSNR = 28.68 dB Hemami, PSNR = 31.86 dB

34


Ziad, PSNR = 31.57 dB Proposed, PSNR = 34.65 dB

35


Ancis

PSNR = 28.68 dB

Hemami

PSNR = 31.86 dB

Ziad

PSNR = 31.57 dB

Proposed

PSNR = 34.65 dB

36

Simulation Results – Each StepLena 8 x 8 block loss

(a)

(b)

(c)

37

Simulation Results –Peppers, 8 x 8 block loss


38

Simulation Results – Peppers, 8 x 8 block loss

Ancis, PSNR = 27.92 dB Hemami, PSNR = 31.83 dB

39

Simulation Results – Peppers, 8 x 8 block loss


40

Simulation Results – PSNR (8 x 8)

Lena Masqrd Peppers Boat Elaine Couple

Ancis 28.68 25.47 27.92 26.33 29.84 28.24

Sun 29.99 27.25 29.97 27.36 30.95 28.45

Park 31.26 27.91 31.71 28.77 32.96 30.04

Hemami 31.86 27.65 31.83 29.36 32.07 30.31

Ziad 31.57 27.94 32.76 30.11 31.92 30.99

Proposed 34.65 29.87 34.20 30.78 34.63 31.49

41

Simulation Results –Lena, 8 x one row block loss


42

Simulation Results –Lena, 8 x one row block loss

Hemami, PSNR = 26.86 dB Proposed, PSNR = 30.18 dB

43

Simulation Results –Masquerade, 8 x one row block loss


44

Simulation Results –Masquerade, 8 x one row block loss

Hemami, PSNR = 23.10 dB Proposed, PSNR = 25.09 dB

45



46



47

Simulation Results –Foreman, 16 x 16 block loss



48

Simulation Results –Flower Garden, 16 x 16 block loss



49

Simulation Results – PSNR (Row, 16 x 16)

(16 x 16) Lena Foreman Garden

Ziad 28.75 25.65 20.40

Proposed 32.70 30.34 22.62

(8 x Row) Lena Maskrd Peppers Boat Elaine Couple

Hemami 26.86 23.10 25.41 24.54 26.87 24.30

Proposed 30.18 25.09 28.31 26.06 30.11 26.12

50

Simulation Results –Mean & Variance of Error (8 x 8)

,|ˆ|mean Br

xx 22mean|ˆ|variance

Br

xx

Lena Maskrd Peppers Boat Elaine Couple

m v m v m v M v m v m V

Ancis 11.5 242.2 18.6 437.9 12.4 290.9 15.9 392.2 11.5 154.2 12.2 264.2

Sun 12.6 117.2 15.6 276.2 12.3 126.6 16.3 241.1 11.8 81.6 14.4 186.7

Park 8.2 138.7 13.4 267.5 7.9 123.1 11.4 236.4 7.8 76.9 10.0 172.4

Hemami 7.2 127.4 12.7 313.3 7.7 121.2 10.6 208.9 8.7 94.8 8.4 185.7

Ziad 7.3 137.8 12.3 290.7 7.4 91.3 10.0 170.4 9.4 88.5 7.3 165.1

Proposed 5.4 64.8 10.0 184.3 5.9 69.3 9.1 147.4 6.8 47.1 7.2 143.6

51

Projections based Spatial Block Recovery – Discussion

New projection based block recovery algorithm was

presented.

8 images and 3 error cases were tested.

5 existing block recovery algorithms were used for

comparison.

The proposed algorithm outperforms other techniques

in all error cases of all images.

52

Temporal Block Loss Recovery

In video coding (e, g, MPEG, H.261/3+),

temporal recovery is more effective.

tt-1

53

Background – MPEG, H.261/3+

Standards for video coding Block and predictive coding techniques are

adopted.

I B B BP B P B B

Group of pictures

P B B

54

Predictive Coding : Eliminating the inter-pixel redundancies

Background – Predictive Coding

- =Previous FrameCurrent Frame

MV(x, y) = (x0 – x1, y0 – y1)

(x0 , y0) (x1 , y1)

MV(x, y) Redundancy

MPEG/H26x bitstream

55

Errors in video streams during transmissions

Background – Error in MPEG

MV(x, y) Redundancy

MV(x, y) Redundancy

MV(x, y) Redundancy

56

Motion Vector Recovery Techniques

Motion Flow Estimation Method

Spatial Correlation Exploit Method

Temporal Correlation Exploit Method

Previous Research – Motion Vector Recovery

57

Zero Motion Vector (ZM)

Average of Surrounding Motion Vectors (AV)

Previous Research - Motion Flow Estimation

MV(x, y) = (0, 0)

Simplest but poor performance

Simple but poor performance

Smn

mn yxMVyxMV,

6/),(),(

58

Motion Flow Interpolation (MFI) (IEEE 1999)

Previous Research - Motion Flow Estimation

Pro : Simple

Con : different Motion Vectors in the same block

N

vyvyNvxvxNyxd

BnTnRnLn

nn

2

)()(),(

59

Boundary Matching Algorithm (BMA) (IEEE 1993)

Previous Research –Spatial Correlation Exploit Method

),1,(),,( 00

10

0

nyxfnyxfC RC

x

xx

A

N

),,1(),,( 000

10

0

nyxfnyxfC RC

y

yy

L

N

),,(),1,( 000

10

0

nNyxfnNyxfC RC

x

xx

B

N

)(minarg,,

LBAxx DDDmvmvyx

A

L

B

60

Decoder Motion Vector Estimation (DMVE) (IEEE 2000)

Previous Research –Temporal Correlation Exploit Method

),,(),,(11 0

0

0

0

nyxfnyxfC RC

y

wyy

x

xx

A

N

)(minarg,,

LBAxx DDDmvmvyx

),,(),,(11 0

0

0

0

nyxfnyxfC RC

wy

yy

x

xx

B

N

N

N

),,(),,(11 0

0

0

0

nyxfnyxfC RC

wy

wyy

x

wxx

L

N

A

L

B

61

Forward-Backward Block Matching (FB BM) (IEEE 2000)

Previous Research –Temporal Correlation Exploit Method

)),,()1,,((minarg,11 0

0

0

0

,njifnjyixfmvmv

y

yj

x

xiyx

AyAx

N

N

)),,()1,,((minarg,121 0

0

0

0

,njifnjyixfmvmv

N

N

N y

yj

x

xiyx

ByBx

),,()1,,((minarg,11 0

0

0

0

,njifnjyixfmvmv

y

yj

x

xiyx

CyCx

N

N

)),,()1,,(121 0

0

0

0

njifnjyixfN

N

N y

yj

x

xi

}}{,,,,,arg{min, , yxCyCxByBxAyAxyx Dmvmvmvmvmvmvmvmv

62

Advantage - Good Performance

Disadvantage -

1) Not Adaptive to Local Image Structure

2) Not Resilient to Propagation Error

Previous Research – Problem Formulation

Temporal/Spatial Correlation Exploit Methods

63

Temporal Block Loss Recovery – Research Goal

To design a Temporal Restoration Technique More reliable

Adaptive to local image structure

Adaptive/Resilient to propagation error

Utilizing Projections

No retransmission

No additional bits for error correction

64

3 Steps

Pre Process – Edge extension using Projections P1, P2

Motion vector recovery - 1) Local image structure estimation

2) Propagation error

estimation

Post process - Spatial Compensation using Projections P3

Adaptive Temporal Block Recovery – Algorithm

65

Vertical edge detection using vertical line mask

on SN and SS

Temporal Block Loss recovery –Edge extension with projections P1, P2

,||T NS

vnv R ,||T sS

vsv R Total Magnitude :

121

121

121

vL

Vertical Line Mask

54321 w2w-w-w2-w vR9876 w-w2w-w-

66

Adaptive Motion Vector Recovery 1 –Edge Extension with projections P1, P2

If , extend edge by projecting recovery pixels to above

and below reference blocks (we assume there is strong edge in thr

ee blocks.)

csvnv TT ,

otherwise

nforFFC

o

n

ff

ff

:

L: maxmin2

otherwise

nFforF

nFforF

P

n

n

n

n

f

f

f

f L

L

max,max

min,min

2

21,||||

,)(

2

ii

i

d di

iiiP S

SS R

R

RRRP1:

67

Adaptive Motion Vector Recovery –Distortion Metric

},),,(:{ 0000 NyjyxiKxjixxfn

},),,(:{ 0000 NyjyVNxiNxjixxfs

},),,(:{ 0000 yjLyNxixjixxfw

},),,(:{ 0000 NyjyVxixjixxfmt

},),,(:{ 0000 NyjyNxiVNxjixxfmb

68

),,(),,( 0000,,,

1 1

Rjijiji NjyyixxfNjyixfD n

K

i

N

j

nn

),,(),,( 0000,,,

1 1

Rjijiji NjyyiNxxfNjyiNxfD s

K

i

N

j

ss

),,(),,( 0000,,,

1 1


V

i

N

j

nmt

),,(),,( 0000,,,

1 1

Rjijiji NjyyiNxxfNjyiNxfD n

V

i

N

j

nmb

),,(),,( 0000,,,

1 1


N

i

L

j

nw

)(minarg,,

wmbmtsnxx DDDDDmvmvyx

Adaptive Motion Vector Recovery –Distortion Metric (Temporal Correlation)

69

: Weight for Local Image Structure

: Weight for Local Image Correlation

Adaptive Motion Vector Recovery –Weights in the Distortion Metric

ji ,ji , : Weight for Propagation Error

Three weights enable the proposed algorithm to be adaptive to local image structure and propagation error

ji ,

70

Adaptive Motion Vector Recovery 1 –Weight i,j for Local Image Structure

v

vjisjin

Tround ,,,, ,

2

)(1

,,,,,,

jisjinjiw

1,0 ,,,, jisjin

2

)( ,,,, jisjin

jiw ,,1

1

,

,||T NS

vnv R ,||T sS

vsv R

71

Adaptive Motion Vector Recovery 1 –Weights i,j for Local Image Correlation

/)1(,

dji ec

ji,

d

72

Adaptive Motion Vector Recovery 2 – Propagation Error Estimation for Weights i,j

Error Propagation and Error States

73

Adaptive Motion Vector Recovery 2 –Weights i,j for Propagation Error

Error States of Pixels

Sc(i,j) = 0, when correctly received

Sres(i,j) = Sref(i,j)+1, when restored

Weight, ji ,

10,1 ,,, jijiji S

1

ji ,

1

S (i,j)

74

Adaptive Motion Vector Recovery – Weights and Motion Vector

)(minarg,,

wmbmtsnxx DDDDDmvmvyx

)( ),(),(),(),(),( mvxmvxwmvxmvxmbmvxmvxmtmvxmvxsmvxmvxn DDDDDe

/)1(,

dji ec

v

vjisjin

Tround ,,,, ,

2

)(1

,,,,,,

jisjinjiw

10,1 ,,, jijiji S

75

if , apply spatial constraint

Temporal Block Loss Recovery –Spatial Compensation using Projection P3

eTe

otherwise

nEforE

nEforE

P

mn

nmn

nmn

mn

,

,1

,1

,3 L,

L,

f

gf

gf

f

where, andis a N x N recovery vector in column vector form.

)}(....,),{( ,,10,0,1 NNNNNN ffffg }....,,,{ 21 Nffff

76

Adaptive Temporal Block Loss Recovery – Summary

Local Image Structure Estimation

Propagation Error Estimation

Motion Vector Recovery

Spatial Constraints, P3

e > Te

End

Edge Extension, P1, P2

77

352 x 240 “Flower Garden”, “Table Tennis”, “Mobile”, “Football”

176 x 144 “Foreman”

Simulation Results – Test Data and Parameters

K = 3, L = 1, V = 1. (size of surrounding area)

=v= 3500 ()

= 1.1 if d=1, = 1.0 else = 0.1 ()

Tc=7000 (edge extension)

Te=1800 (spatial compensation)

78

Simulation Results –Errors

I B P P P

- Error in every B & P frame

- 11 / 12 frame error (91.7 % frame error)

. . .

. .

.

B B BB B B B

I B P P PB B BB B B B

79

Simulation Results – Flower Garden

Original Sequence Test Sequence

80


ZM, PSNR = 16.15 dB AV, PSNR = 18.64 dB

81


MFI, PSNR = 19.29 dB BMA, PSNR = 19.83 dB

82


FB BM, PSNR = 19.21 dB Proposed, PSNR = 20.71 dB

83


84

Simulation Results – Foreman

Original Sequence Test Sequence

ZM PSNR = 24.71 dB AV PSNR = 26.22 dB

85


MFI PSNR = 27.09 dB BMA PSNR = 28.76 dB

FB BM PSNR = 27.46 dB Proposed PSNR = 29.82 dB

86


87

Simulation Results – Average PSNR

Garden Tennis Football Mobile Foreman

MV 16.15 22.40 18.06 17.49 24.71

AV 18.64 21.98 18.72 19.03 26.22

BMA 19.83 23.55 19.41 19.75 28.76

DMVE 19.88 24.04 19.64 20.02 28.77

MFI 19.29 22.77 19.29 19.60 27.09

F-B BM 19.21 22.49 19.05 19.59 27.46

Proposed 20.71 24.52 20.32 20.66 29.82

88

Adaptive Temporal Block Loss Recovery – Discussion

New adaptive temporal block loss recovery algorithm

was presented.

5 sequences were tested.

6 existing block recovery algorithms were used for

comparison.

The proposed algorithm outperforms other techniques

in all sequences.

89

Interpolation based Coding and Data Priority Setting Technique

Block loss recovery technique only by decode part

does not provide a good error resilient performance

Inter-active method between encoder and decoder is

more useful for image/video communication.

Data priority setting problem using the proposed

spatial block loss recovery.

90

Interpolation based Coding and Data Priority Setting – Problem Formulation

Many encoders have a function that sets priorities to image/video data for transmission.

When network congestions happen, the network discards low priority data and preserves high priority data.

Coding standards set high priority to low frequency data and low priority to high frequency data.

It does not consider block loss recovery ability of decoders during priority setting.

Packet Packet. . . . . .PacketL H H

91

Interpolation based Coding and Priority Setting Method – Research Goal

To make coding system be able to give priorities to

image data by considering decoder’s block loss

recovery ability

To outperform compression rates of JPEG

JPEG compatible

Compatible to the proposed spatial block recovery

algorithm

92

Interpolation based Coding - Motivation

If the block loss recovery algorithm in encoder restores missing blocks faithfully, then those blocks can be set as low priority blocks and discarded during network congestions, or not need be transmitted by sender.

Original Point

Compressed Point by JPEG

Interpolated Point by projections

distance(A-B) < distance(A-C) Low priority distance(A-B) > distance(A-C) High priority

93

Interpolation based Coding – Algorithm 1

Compute PSNRj & Bit Rate(R

b) of every block

compressed image/Video

Restore every block by the proposed algorithm. and Compute PSNRr of every restored b

lock

Sort blocks by distortion & bit rate.

((PSNRr - PSNRj) * Rb)

Set high priority to or Store/Transmit only the block of low distortion & low bit rate

94

Interpolation based Coding – Result 1

JPEG Coding

PSNR = 32.27 dB

Size = 0.30 BPP = 9,902 Byte

w/ Removed Blocks

Blocks : 447 / 4096 = 11%

Size = 0.29 BPP

I-based Coding

PSNR = 32.35 dB

Size = 0.29 BPP = 9,634 Byte

95

Interpolation based Coding – Algorithm 2

We applied algorithm1 and setting priorities to blocks

compressed image/Video

Investigate which surrounding blocks are used for the

recovery of low priority blocks

Set highest priority to the used surrounding blocks

Apply algorithm 1 to unused surrounding blocks

Set second lowest priority to some blocks with the results

96


JPEG Coding

PSNR = 32.27 dB

Size = 0.30 BPP = 9,902 Byte

w/ Removed Blocks

Blocks : 557 / 4096 = 14%

Size = 0.27 BPP

I-based Coding

PSNR = 32.37 dB

Size = 0.27 BPP = 9,570 Byte

97


JPEG Coding

PSNR = 32.27 dB

Size = 0.30 BPP = 9,902 Byte

I-based Coding

PSNR = 32.37 dB

Size = 0.27 BPP = 9,570 Byte

20% Packet Drop

PSNR = 29.70 dB

Size = 0.30 BPP = 9,902 Byte

98

Interpolation based Coding - Result

PSNR vs. BPP in Lena Image

99

Interpolation based Coding - Result

PSNR vs. BPP in Peppers Image

100

Interpolation based Coding – Discussion

New interpolation based image coding algorithm was presented.

Priority data setting by encoder using decoder’s block loss recovery ability

JPEG baseline coding technique was used for comparison

The proposed coding algorithm outperforms JPEG coding on different bit rates.

New priority setting using proposed spatial block loss recovery algorithm was presented

101

Conclusion Spatial block recovery

New method of exploiting and utilizing highly correlated surrounding area to a missing block is developed.

New alternating projections based algorithm is proposed. Proposed algorithm outperforms existing image block

recovery techniques.

Temporal block recovery Adaptive motion vector recovery is proposed. Edge extension for MV recovery is introduced. Adaptive spatial compensation is developed. Proposed algorithm outperforms existing video recovery

techniques.

102

Conclusion Interpolation-based low-bit-rate coding

Decoder based data priority setting is presented.

Active error resilient method is presented.

Presented coding technique outperforms JPEG standard.

103

Future Work

Fast search method for the proposed algorithms will

reduce search time and can be implemented in the

system requiring fast computation.

FEC technique for the proposed algorithms will

improve image/video communication reliability.

Block Loss Recovery Techniques for Image and Video Communications

Documents

Transcript of Block Loss Recovery Techniques for Image and Video Communications