Post on 09-Jan-2016
description
Block Loss Recovery Techniques for Image and Video Communications
Jiho Park
The Computational Intelligence Applications (CIA) Lab.
Department of Electrical Engineering
University of Washington
May 21, 2002
2
Outline
Introduction
Background for Block Recovery
Spatial Block Loss Recovery Technique
Temporal Block Loss Recovery Technique
Interpolation based Coding and Data Priority
Setting
Conclusion & Future Work
3
Image and Video Coding Standards
Adopt block based coding (e.g. JPEG, MPE
G, H.261/3+)
Lossy coding techniques
Images are segmented into blocks
Predictive Coding, Discrete Cosine Transform
and Quantization are adopted
4
Background –Block Coding & Transmission Error
Block based coding technique
N x N block
Discrete Cosine
TransformQ RLC Packet
Encoding Packetizing Network Depacketizing DecodingImage/ Video
Source
image/video transmission system.
5
Transmission & Error
During transmission, some packets are lost due to bit error, congestion of network, noise burst, or other reasons.
Packet
Packet error during transmission causes all data loss in a corresponding block.
6
Background – Error Correction Automatic Retransmission Request(ARQ) – Decoder sen
ds retransmission requests to encoder when error happens. pro : error-free. cons : additional delays, unacceptable for real-time. example : TCP/IP – unreal-time internet file transfer protocol
Forward Error Correction(FEC) – Encoder inserts additional codes so that decoder can correct errors. pro : error resilient. con : additional bits
example : 18-bit FEC code in H.261(video-conferencing standard)
Error Concealment/Block Loss Recovery – Post-processing to restore damaged blocks by decoder
pro : no additional bits. con : complex decoder
7
Spatial EC : Exploiting spatially neighboring image data of a missing block.
Temporal EC : Exploiting temporally adjacent frame of video.
Background –Error Concealment / Block Loss Recovery
tt-1
8
Spatial Block Loss Recovery
Spatial block loss recovery is suitable for
image coding data, (e,g, JPEG, Image
based coding frame of MPEG/H.261/263+)
9
Averaging the values of pixels in the same position of surrounding blocks.
Previous Research (Spatial EC) -Edge-based Average Interpolation (Ancis ’99)
1x 2x 3x
5x4x
6x 7x 8x
8
1
8
1
,, /i
k
i
jikk
jim wxwx
where wk = 0, 1, according to edges in surrounding blocks
10
Previous Research (Spatial EC) - Interpolation Exploiting Interblock Correlation (Hemami ’95)
Tx
RxLx
Bx
BBRRLLTTM xwxwxwxwx
To get weight, W, the values of boundary pixels in a missing block are set to the value of adjacent known pixels.
Weight, W, are computed by solving the equations using Linear Least Squares Problem.
SXWXM
S''' XWX M
(1~64)
(1~30)
11
Set Y2 to zero =>
Previous Research (Spatial EC) – DCT-based interpolation (Ziad, 2000)
,2211 XTXT
XTY
unknown.: known,: 21 XX
2
1
2221
1211
2
1
X
X
TT
TT
Y
Y
222121 XTXT0 121222
1 XTTX
Kernel DCT:T
12
Previous Research – Other Algorithms
Maximally Smooth Image Recovery (Wang, ’93)
- minimizing errors between a missing block and surrounding pixels
DCT Coefficients Block Recovery (Park, ’97)
- minimizing errors between boundary pixels of a missing block and surrounding pixels
Block Recovery using POCS (Sun, ’95)
13
Conventional Algorithms -
1) Not adaptive to local image structure (Ziad, Hemami,…)
2) ill-utilizing surrounding image structure (hemami,…)
3) no ability of recover certain frequency bands (Sun, Ziad, Wan
g, Park…)
Previous Research – Problem Formulation
14
Block Loss Recovery – Research Goal
To design a spatial block loss recovery Technin
que More Reliable
Adaptive to Image Structure
No Retransmission
No Additional Bits for error correction
15
Projections based Block Recovery – Motivation
Conventional Algorithms use information of all surrounding area. Using only highly correlated area
16
Alternating Projections is projecting between two or more convex sets iteratively.
Projections onto Convex Sets –Alternating Projections
Converging to a common point
17
Signal Restoration using Alternating Projections – Papoulis-Gerchberg Algorithm
Band Limiting.Convex Set, C1
IFT
Identical Middle.Convex Set, C2
FT
FT
Original Signal
Corrupted Signal
C2
18
Projections based Block Recovery – Algorithm
2 Steps Pre Process : 1) Edge orientation detection
2) Surrounding vector extraction
3) Recovery vector extraction
Projections : 1) Projection operator P1
2) Projection operator P2
3) Projection operator P3
19
Edge orientation in the surrounding area(S) of a missing block(M). In order to extend the geometric structure to the missing block.
Simple line masks at every i, j coordinate in surrounding area(S) of the missing block(M) for edge detection.
Pre Process 1 –Edge Orientation Detection
121
121
121
vL
111
222
111
hL
Horizontal Line Mask Vertical Line Mask
20
Pre Process 1 – Edge Orientation Detection
Responses of the line masks at window W :
Total magnitude of responses :
Th > Tv ; Horizontal line dominating area
Th < Tv ; Vertical line dominating area
987
654
321
www
www
www
W987654321 w-w-w-w2w2w2w-w--w hR
987654321 w-w2w-w-w2w-w-w2-w vR
,||T S
hh R S
vv R ||T
21
Pre Process 2 – Surrounding Vectors
Surrounding Vectors, sk, are extracted from surrounding area of a missing block by N x N window.
Each vector has its own spatial and spectral characteristic. The number of surrounding vectors, sk, is 8N.
}W),(),,(:{ jijixxks
22
Pre Process 3 – Recovery Vector Recovery vectors are extracted to restore missing pixels. Two positions of recovery vectors are possible according to the
edge orientation.
Recovery vectors consist of known pixels(white color) and missing pixels(gray color).
The number of recovery vectors, rk, is 2.
}W),(),,(:{ jijixxkr
Vertical line dominating area Horizontal line dominating area
23
Projections based Block Recovery –Projection operator P1
Recovery vectors, ri, for i = 1, 2
Surrounding vectors, sj , for j = 1 ~ 8N
Surrounding vectors, S, form a convex hull in N2-dimensional space
Recovery vectors, R, are orthogonally projected onto the line defined by the closest surrounding vector, si, j : Projection Operator P1.
24
Projections based Block Recovery –Projection operator P1
Projection operator P1 :
Convex hull (formed by surrounding vectors, containing information of local image structure)
25
Projections based Block Recovery –Projection operator P1
Surrounding vectors, sj , for j = 1 ~ 8N Recovery vectors, ri, for i = 1, 2
The closest vertex, sdi , from a recovery vector, ri.
or equivalently in DCT domain,
P1 :
Njiford jij
i 81,21||||minarg sr
Njiford jij
i 81,21||||minarg SR
21,||||
,)(
2
ii
i
d di
idiiP S
R
RSRS
26
Convex set C2 acts as an “identical middle”.
Projection operator P2 :
Projections based Block Recovery –Projection operator P2
otherwise
nforFFC
o
n
ff
ff
:
L: maxmin2
otherwise
nFforF
nFforF
P
n
n
n
n
f
f
f
f L
L
max,max
min,min
2
27
Convex set C3 acts as a convex constraint between missing pixels and adjacent known pixels, (fN-1 fN).
where, and
is a N x N recovery vector in column vector form.
Projections based Block Recovery – Projection operator P3
fN-1 fN
}||:{3 EC n gg
)}(....,),{( ,,10,0,1 NNNNNN ffffg }....,,,{ 21 Nffff
28
Projections based Block Recovery – Projection operator P3
Projection operator P3 :
otherwise
nEforE
nEforE
P
mn
nmn
nmn
mn
,
,1
,1
,3 L,
L,
f
gf
gf
f
29
Projections based Block Recovery –Iterative Algorithm
Missing pixels in recovery vectors are restored by iterative algorithm of alternating projections :
N x N windows moving :
ii fPPPf 3211
Vertical line dominating area Horizontal line dominating area
30
Projections based Block Recovery - Summary
Edge Orientation Detection
Surrounding Vector Extraction
Recovery Vector Extraction
Projection Operator P1
Projection Operator P2
Projection Operator P3
Iteration=I?
All pixels?
31
Simulation Results – Test Data and Error
512 x 512 “Lena”, “Masquerade”, “Peppers”, “Boat”, “Elaine”, “Couple”
176 x 144 “Foreman” 352 x 240 “Flower Garden”
8 x 8 pixel block loss 16 x 16 pixel block loss 8 x 8 consecutive block losses
Peak Signal – Noise – Ratio
)|),(ˆ),(|
255log(10
1 1
2
2
N
i
M
j
jixjix
MNPSNR
32
Simulation Results –Lena, 8 x 8 block loss
Original Image Test Image
33
Simulation Results –Lena, 8 x 8 block loss
Ancis, PSNR = 28.68 dB Hemami, PSNR = 31.86 dB
34
Simulation Results –Lena, 8 x 8 block loss
Ziad, PSNR = 31.57 dB Proposed, PSNR = 34.65 dB
35
Simulation Results –Lena, 8 x 8 block loss
Ancis
PSNR = 28.68 dB
Hemami
PSNR = 31.86 dB
Ziad
PSNR = 31.57 dB
Proposed
PSNR = 34.65 dB
36
Simulation Results – Each StepLena 8 x 8 block loss
(a)
(b)
(c)
37
Simulation Results –Peppers, 8 x 8 block loss
Original Image Test Image
38
Simulation Results – Peppers, 8 x 8 block loss
Ancis, PSNR = 27.92 dB Hemami, PSNR = 31.83 dB
39
Simulation Results – Peppers, 8 x 8 block loss
Ziad, PSNR = 32.76 dB Proposed, PSNR = 34.20 dB
40
Simulation Results – PSNR (8 x 8)
Lena Masqrd Peppers Boat Elaine Couple
Ancis 28.68 25.47 27.92 26.33 29.84 28.24
Sun 29.99 27.25 29.97 27.36 30.95 28.45
Park 31.26 27.91 31.71 28.77 32.96 30.04
Hemami 31.86 27.65 31.83 29.36 32.07 30.31
Ziad 31.57 27.94 32.76 30.11 31.92 30.99
Proposed 34.65 29.87 34.20 30.78 34.63 31.49
41
Simulation Results –Lena, 8 x one row block loss
Original Image Test Image
42
Simulation Results –Lena, 8 x one row block loss
Hemami, PSNR = 26.86 dB Proposed, PSNR = 30.18 dB
43
Simulation Results –Masquerade, 8 x one row block loss
Original Image Test Image
44
Simulation Results –Masquerade, 8 x one row block loss
Hemami, PSNR = 23.10 dB Proposed, PSNR = 25.09 dB
45
Simulation Results –Lena, 16 x 16 block loss
Original Image Test Image
46
Simulation Results –Lena, 16 x 16 block loss
Ziad, PSNR = 28.75 dB Proposed, PSNR = 32.70 dB
47
Simulation Results –Foreman, 16 x 16 block loss
Original Image Test Image
Ziad, PSNR = 25.65 dB Proposed, PSNR = 30.34 dB
48
Simulation Results –Flower Garden, 16 x 16 block loss
Original Image Test Image
Ziad, PSNR = 20.40 dB Proposed, PSNR = 22.62 dB
49
Simulation Results – PSNR (Row, 16 x 16)
(16 x 16) Lena Foreman Garden
Ziad 28.75 25.65 20.40
Proposed 32.70 30.34 22.62
(8 x Row) Lena Maskrd Peppers Boat Elaine Couple
Hemami 26.86 23.10 25.41 24.54 26.87 24.30
Proposed 30.18 25.09 28.31 26.06 30.11 26.12
50
Simulation Results –Mean & Variance of Error (8 x 8)
,|ˆ|mean Br
xx 22mean|ˆ|variance
Br
xx
Lena Maskrd Peppers Boat Elaine Couple
m v m v m v M v m v m V
Ancis 11.5 242.2 18.6 437.9 12.4 290.9 15.9 392.2 11.5 154.2 12.2 264.2
Sun 12.6 117.2 15.6 276.2 12.3 126.6 16.3 241.1 11.8 81.6 14.4 186.7
Park 8.2 138.7 13.4 267.5 7.9 123.1 11.4 236.4 7.8 76.9 10.0 172.4
Hemami 7.2 127.4 12.7 313.3 7.7 121.2 10.6 208.9 8.7 94.8 8.4 185.7
Ziad 7.3 137.8 12.3 290.7 7.4 91.3 10.0 170.4 9.4 88.5 7.3 165.1
Proposed 5.4 64.8 10.0 184.3 5.9 69.3 9.1 147.4 6.8 47.1 7.2 143.6
51
Projections based Spatial Block Recovery – Discussion
New projection based block recovery algorithm was
presented.
8 images and 3 error cases were tested.
5 existing block recovery algorithms were used for
comparison.
The proposed algorithm outperforms other techniques
in all error cases of all images.
52
Temporal Block Loss Recovery
In video coding (e, g, MPEG, H.261/3+),
temporal recovery is more effective.
tt-1
53
Background – MPEG, H.261/3+
Standards for video coding Block and predictive coding techniques are
adopted.
I B B BP B P B B
Group of pictures
P B B
54
Predictive Coding : Eliminating the inter-pixel redundancies
Background – Predictive Coding
- =Previous FrameCurrent Frame
MV(x, y) = (x0 – x1, y0 – y1)
(x0 , y0) (x1 , y1)
MV(x, y) Redundancy
MPEG/H26x bitstream
55
Errors in video streams during transmissions
Background – Error in MPEG
MV(x, y) Redundancy
MV(x, y) Redundancy
MV(x, y) Redundancy
56
Motion Vector Recovery Techniques
Motion Flow Estimation Method
Spatial Correlation Exploit Method
Temporal Correlation Exploit Method
Previous Research – Motion Vector Recovery
57
Zero Motion Vector (ZM)
Average of Surrounding Motion Vectors (AV)
Previous Research - Motion Flow Estimation
MV(x, y) = (0, 0)
Simplest but poor performance
Simple but poor performance
Smn
mn yxMVyxMV,
6/),(),(
58
Motion Flow Interpolation (MFI) (IEEE 1999)
Previous Research - Motion Flow Estimation
Pro : Simple
Con : different Motion Vectors in the same block
N
vyvyNvxvxNyxd
BnTnRnLn
nn
2
)()(),(
59
Boundary Matching Algorithm (BMA) (IEEE 1993)
Previous Research –Spatial Correlation Exploit Method
),1,(),,( 00
10
0
nyxfnyxfC RC
x
xx
A
N
),,1(),,( 000
10
0
nyxfnyxfC RC
y
yy
L
N
),,(),1,( 000
10
0
nNyxfnNyxfC RC
x
xx
B
N
)(minarg,,
LBAxx DDDmvmvyx
A
L
B
60
Decoder Motion Vector Estimation (DMVE) (IEEE 2000)
Previous Research –Temporal Correlation Exploit Method
),,(),,(11 0
0
0
0
nyxfnyxfC RC
y
wyy
x
xx
A
N
)(minarg,,
LBAxx DDDmvmvyx
),,(),,(11 0
0
0
0
nyxfnyxfC RC
wy
yy
x
xx
B
N
N
N
),,(),,(11 0
0
0
0
nyxfnyxfC RC
wy
wyy
x
wxx
L
N
A
L
B
61
Forward-Backward Block Matching (FB BM) (IEEE 2000)
Previous Research –Temporal Correlation Exploit Method
)),,()1,,((minarg,11 0
0
0
0
,njifnjyixfmvmv
y
yj
x
xiyx
AyAx
N
N
)),,()1,,((minarg,121 0
0
0
0
,njifnjyixfmvmv
N
N
N y
yj
x
xiyx
ByBx
),,()1,,((minarg,11 0
0
0
0
,njifnjyixfmvmv
y
yj
x
xiyx
CyCx
N
N
)),,()1,,(121 0
0
0
0
njifnjyixfN
N
N y
yj
x
xi
}}{,,,,,arg{min, , yxCyCxByBxAyAxyx Dmvmvmvmvmvmvmvmv
62
Advantage - Good Performance
Disadvantage -
1) Not Adaptive to Local Image Structure
2) Not Resilient to Propagation Error
Previous Research – Problem Formulation
Temporal/Spatial Correlation Exploit Methods
63
Temporal Block Loss Recovery – Research Goal
To design a Temporal Restoration Technique More reliable
Adaptive to local image structure
Adaptive/Resilient to propagation error
Utilizing Projections
No retransmission
No additional bits for error correction
64
3 Steps
Pre Process – Edge extension using Projections P1, P2
Motion vector recovery - 1) Local image structure estimation
2) Propagation error
estimation
Post process - Spatial Compensation using Projections P3
Adaptive Temporal Block Recovery – Algorithm
65
Vertical edge detection using vertical line mask
on SN and SS
Temporal Block Loss recovery –Edge extension with projections P1, P2
,||T NS
vnv R ,||T sS
vsv R Total Magnitude :
121
121
121
vL
Vertical Line Mask
54321 w2w-w-w2-w vR9876 w-w2w-w-
66
Adaptive Motion Vector Recovery 1 –Edge Extension with projections P1, P2
If , extend edge by projecting recovery pixels to above
and below reference blocks (we assume there is strong edge in thr
ee blocks.)
csvnv TT ,
otherwise
nforFFC
o
n
ff
ff
:
L: maxmin2
otherwise
nFforF
nFforF
P
n
n
n
n
f
f
f
f L
L
max,max
min,min
2
21,||||
,)(
2
ii
i
d di
iiiP S
SS R
R
RRRP1:
67
Adaptive Motion Vector Recovery –Distortion Metric
},),,(:{ 0000 NyjyxiKxjixxfn
},),,(:{ 0000 NyjyVNxiNxjixxfs
},),,(:{ 0000 yjLyNxixjixxfw
},),,(:{ 0000 NyjyVxixjixxfmt
},),,(:{ 0000 NyjyNxiVNxjixxfmb
68
),,(),,( 0000,,,
1 1
Rjijiji NjyyixxfNjyixfD n
K
i
N
j
nn
),,(),,( 0000,,,
1 1
Rjijiji NjyyiNxxfNjyiNxfD s
K
i
N
j
ss
),,(),,( 0000,,,
1 1
Rjijiji NjyyixxfNjyixfD n
V
i
N
j
nmt
),,(),,( 0000,,,
1 1
Rjijiji NjyyiNxxfNjyiNxfD n
V
i
N
j
nmb
),,(),,( 0000,,,
1 1
Rjijiji NjyyixxfNjyixfD n
N
i
L
j
nw
)(minarg,,
wmbmtsnxx DDDDDmvmvyx
Adaptive Motion Vector Recovery –Distortion Metric (Temporal Correlation)
69
: Weight for Local Image Structure
: Weight for Local Image Correlation
Adaptive Motion Vector Recovery –Weights in the Distortion Metric
ji ,ji , : Weight for Propagation Error
Three weights enable the proposed algorithm to be adaptive to local image structure and propagation error
ji ,
70
Adaptive Motion Vector Recovery 1 –Weight i,j for Local Image Structure
v
vjisjin
Tround ,,,, ,
2
)(1
,,,,,,
jisjinjiw
1,0 ,,,, jisjin
2
)( ,,,, jisjin
jiw ,,1
1
,
,||T NS
vnv R ,||T sS
vsv R
71
Adaptive Motion Vector Recovery 1 –Weights i,j for Local Image Correlation
/)1(,
dji ec
ji,
d
72
Adaptive Motion Vector Recovery 2 – Propagation Error Estimation for Weights i,j
Error Propagation and Error States
73
Adaptive Motion Vector Recovery 2 –Weights i,j for Propagation Error
Error States of Pixels
Sc(i,j) = 0, when correctly received
Sres(i,j) = Sref(i,j)+1, when restored
Weight, ji ,
10,1 ,,, jijiji S
1
ji ,
1
S (i,j)
74
Adaptive Motion Vector Recovery – Weights and Motion Vector
)(minarg,,
wmbmtsnxx DDDDDmvmvyx
)( ),(),(),(),(),( mvxmvxwmvxmvxmbmvxmvxmtmvxmvxsmvxmvxn DDDDDe
/)1(,
dji ec
v
vjisjin
Tround ,,,, ,
2
)(1
,,,,,,
jisjinjiw
10,1 ,,, jijiji S
75
if , apply spatial constraint
Temporal Block Loss Recovery –Spatial Compensation using Projection P3
eTe
otherwise
nEforE
nEforE
P
mn
nmn
nmn
mn
,
,1
,1
,3 L,
L,
f
gf
gf
f
where, andis a N x N recovery vector in column vector form.
)}(....,),{( ,,10,0,1 NNNNNN ffffg }....,,,{ 21 Nffff
76
Adaptive Temporal Block Loss Recovery – Summary
Local Image Structure Estimation
Propagation Error Estimation
Motion Vector Recovery
Spatial Constraints, P3
e > Te
End
Edge Extension, P1, P2
77
352 x 240 “Flower Garden”, “Table Tennis”, “Mobile”, “Football”
176 x 144 “Foreman”
Simulation Results – Test Data and Parameters
K = 3, L = 1, V = 1. (size of surrounding area)
=v= 3500 ()
= 1.1 if d=1, = 1.0 else = 0.1 ()
Tc=7000 (edge extension)
Te=1800 (spatial compensation)
78
Simulation Results –Errors
I B P P P
- Error in every B & P frame
- 11 / 12 frame error (91.7 % frame error)
. . .
. .
.
B B BB B B B
I B P P PB B BB B B B
79
Simulation Results – Flower Garden
Original Sequence Test Sequence
80
Simulation Results – Flower Garden
ZM, PSNR = 16.15 dB AV, PSNR = 18.64 dB
81
Simulation Results – Flower Garden
MFI, PSNR = 19.29 dB BMA, PSNR = 19.83 dB
82
Simulation Results – Flower Garden
FB BM, PSNR = 19.21 dB Proposed, PSNR = 20.71 dB
83
Simulation Results – Flower Garden
84
Simulation Results – Foreman
Original Sequence Test Sequence
ZM PSNR = 24.71 dB AV PSNR = 26.22 dB
85
Simulation Results – Foreman
MFI PSNR = 27.09 dB BMA PSNR = 28.76 dB
FB BM PSNR = 27.46 dB Proposed PSNR = 29.82 dB
86
Simulation Results – Foreman
87
Simulation Results – Average PSNR
Garden Tennis Football Mobile Foreman
MV 16.15 22.40 18.06 17.49 24.71
AV 18.64 21.98 18.72 19.03 26.22
BMA 19.83 23.55 19.41 19.75 28.76
DMVE 19.88 24.04 19.64 20.02 28.77
MFI 19.29 22.77 19.29 19.60 27.09
F-B BM 19.21 22.49 19.05 19.59 27.46
Proposed 20.71 24.52 20.32 20.66 29.82
88
Adaptive Temporal Block Loss Recovery – Discussion
New adaptive temporal block loss recovery algorithm
was presented.
5 sequences were tested.
6 existing block recovery algorithms were used for
comparison.
The proposed algorithm outperforms other techniques
in all sequences.
89
Interpolation based Coding and Data Priority Setting Technique
Block loss recovery technique only by decode part
does not provide a good error resilient performance
Inter-active method between encoder and decoder is
more useful for image/video communication.
Data priority setting problem using the proposed
spatial block loss recovery.
90
Interpolation based Coding and Data Priority Setting – Problem Formulation
Many encoders have a function that sets priorities to image/video data for transmission.
When network congestions happen, the network discards low priority data and preserves high priority data.
Coding standards set high priority to low frequency data and low priority to high frequency data.
It does not consider block loss recovery ability of decoders during priority setting.
Packet Packet. . . . . .PacketL H H
91
Interpolation based Coding and Priority Setting Method – Research Goal
To make coding system be able to give priorities to
image data by considering decoder’s block loss
recovery ability
To outperform compression rates of JPEG
JPEG compatible
Compatible to the proposed spatial block recovery
algorithm
92
Interpolation based Coding - Motivation
If the block loss recovery algorithm in encoder restores missing blocks faithfully, then those blocks can be set as low priority blocks and discarded during network congestions, or not need be transmitted by sender.
Original Point
Compressed Point by JPEG
Interpolated Point by projections
distance(A-B) < distance(A-C) Low priority distance(A-B) > distance(A-C) High priority
93
Interpolation based Coding – Algorithm 1
Compute PSNRj & Bit Rate(R
b) of every block
compressed image/Video
Restore every block by the proposed algorithm. and Compute PSNRr of every restored b
lock
Sort blocks by distortion & bit rate.
((PSNRr - PSNRj) * Rb)
Set high priority to or Store/Transmit only the block of low distortion & low bit rate
94
Interpolation based Coding – Result 1
JPEG Coding
PSNR = 32.27 dB
Size = 0.30 BPP = 9,902 Byte
w/ Removed Blocks
Blocks : 447 / 4096 = 11%
Size = 0.29 BPP
I-based Coding
PSNR = 32.35 dB
Size = 0.29 BPP = 9,634 Byte
95
Interpolation based Coding – Algorithm 2
We applied algorithm1 and setting priorities to blocks
compressed image/Video
Investigate which surrounding blocks are used for the
recovery of low priority blocks
Set highest priority to the used surrounding blocks
Apply algorithm 1 to unused surrounding blocks
Set second lowest priority to some blocks with the results
96
Interpolation based Coding – Result 2
JPEG Coding
PSNR = 32.27 dB
Size = 0.30 BPP = 9,902 Byte
w/ Removed Blocks
Blocks : 557 / 4096 = 14%
Size = 0.27 BPP
I-based Coding
PSNR = 32.37 dB
Size = 0.27 BPP = 9,570 Byte
97
Interpolation based Coding – Result 2
JPEG Coding
PSNR = 32.27 dB
Size = 0.30 BPP = 9,902 Byte
I-based Coding
PSNR = 32.37 dB
Size = 0.27 BPP = 9,570 Byte
20% Packet Drop
PSNR = 29.70 dB
Size = 0.30 BPP = 9,902 Byte
98
Interpolation based Coding - Result
PSNR vs. BPP in Lena Image
99
Interpolation based Coding - Result
PSNR vs. BPP in Peppers Image
100
Interpolation based Coding – Discussion
New interpolation based image coding algorithm was presented.
Priority data setting by encoder using decoder’s block loss recovery ability
JPEG baseline coding technique was used for comparison
The proposed coding algorithm outperforms JPEG coding on different bit rates.
New priority setting using proposed spatial block loss recovery algorithm was presented
101
Conclusion Spatial block recovery
New method of exploiting and utilizing highly correlated surrounding area to a missing block is developed.
New alternating projections based algorithm is proposed. Proposed algorithm outperforms existing image block
recovery techniques.
Temporal block recovery Adaptive motion vector recovery is proposed. Edge extension for MV recovery is introduced. Adaptive spatial compensation is developed. Proposed algorithm outperforms existing video recovery
techniques.
102
Conclusion Interpolation-based low-bit-rate coding
Decoder based data priority setting is presented.
Active error resilient method is presented.
Presented coding technique outperforms JPEG standard.
103
Future Work
Fast search method for the proposed algorithms will
reduce search time and can be implemented in the
system requiring fast computation.
FEC technique for the proposed algorithms will
improve image/video communication reliability.