8/18/2019 Fundamentals of Vector Quantization
1/87
8/18/2019 Fundamentals of Vector Quantization
2/87
8/18/2019 Fundamentals of Vector Quantization
3/87
8/18/2019 Fundamentals of Vector Quantization
4/87
8/18/2019 Fundamentals of Vector Quantization
5/87
8/18/2019 Fundamentals of Vector Quantization
6/87
8/18/2019 Fundamentals of Vector Quantization
7/87
8/18/2019 Fundamentals of Vector Quantization
8/87
8/18/2019 Fundamentals of Vector Quantization
9/87
8/18/2019 Fundamentals of Vector Quantization
10/87
8/18/2019 Fundamentals of Vector Quantization
11/87
8/18/2019 Fundamentals of Vector Quantization
12/87
8/18/2019 Fundamentals of Vector Quantization
13/87
8/18/2019 Fundamentals of Vector Quantization
14/87
8/18/2019 Fundamentals of Vector Quantization
15/87
8/18/2019 Fundamentals of Vector Quantization
16/87
8/18/2019 Fundamentals of Vector Quantization
17/87
8/18/2019 Fundamentals of Vector Quantization
18/87
8/18/2019 Fundamentals of Vector Quantization
19/87
8/18/2019 Fundamentals of Vector Quantization
20/87
8/18/2019 Fundamentals of Vector Quantization
21/87
8/18/2019 Fundamentals of Vector Quantization
22/87
8/18/2019 Fundamentals of Vector Quantization
23/87
8/18/2019 Fundamentals of Vector Quantization
24/87
8/18/2019 Fundamentals of Vector Quantization
25/87
8/18/2019 Fundamentals of Vector Quantization
26/87
8/18/2019 Fundamentals of Vector Quantization
27/87
8/18/2019 Fundamentals of Vector Quantization
28/87
8/18/2019 Fundamentals of Vector Quantization
29/87
8/18/2019 Fundamentals of Vector Quantization
30/87
8/18/2019 Fundamentals of Vector Quantization
31/87
8/18/2019 Fundamentals of Vector Quantization
32/87
8/18/2019 Fundamentals of Vector Quantization
33/87
8/18/2019 Fundamentals of Vector Quantization
34/87
8/18/2019 Fundamentals of Vector Quantization
35/87
8/18/2019 Fundamentals of Vector Quantization
36/87
8/18/2019 Fundamentals of Vector Quantization
37/87
8/18/2019 Fundamentals of Vector Quantization
38/87
8/18/2019 Fundamentals of Vector Quantization
39/87
8/18/2019 Fundamentals of Vector Quantization
40/87
8/18/2019 Fundamentals of Vector Quantization
41/87
8/18/2019 Fundamentals of Vector Quantization
42/87
8/18/2019 Fundamentals of Vector Quantization
43/87
8/18/2019 Fundamentals of Vector Quantization
44/87
8/18/2019 Fundamentals of Vector Quantization
45/87
8/18/2019 Fundamentals of Vector Quantization
46/87
8/18/2019 Fundamentals of Vector Quantization
47/87
8/18/2019 Fundamentals of Vector Quantization
48/87
8/18/2019 Fundamentals of Vector Quantization
49/87
8/18/2019 Fundamentals of Vector Quantization
50/87
8/18/2019 Fundamentals of Vector Quantization
51/87
8/18/2019 Fundamentals of Vector Quantization
52/87
8/18/2019 Fundamentals of Vector Quantization
53/87
8/18/2019 Fundamentals of Vector Quantization
54/87
8/18/2019 Fundamentals of Vector Quantization
55/87
8/18/2019 Fundamentals of Vector Quantization
56/87
8/18/2019 Fundamentals of Vector Quantization
57/87
8/18/2019 Fundamentals of Vector Quantization
58/87
8/18/2019 Fundamentals of Vector Quantization
59/87
8/18/2019 Fundamentals of Vector Quantization
60/87
8/18/2019 Fundamentals of Vector Quantization
61/87
8/18/2019 Fundamentals of Vector Quantization
62/87
8/18/2019 Fundamentals of Vector Quantization
63/87
8/18/2019 Fundamentals of Vector Quantization
64/87
8/18/2019 Fundamentals of Vector Quantization
65/87
8/18/2019 Fundamentals of Vector Quantization
66/87
8/18/2019 Fundamentals of Vector Quantization
67/87
8/18/2019 Fundamentals of Vector Quantization
68/87
8/18/2019 Fundamentals of Vector Quantization
69/87
8/18/2019 Fundamentals of Vector Quantization
70/87
8/18/2019 Fundamentals of Vector Quantization
71/87
8/18/2019 Fundamentals of Vector Quantization
72/87
8/18/2019 Fundamentals of Vector Quantization
73/87
8/18/2019 Fundamentals of Vector Quantization
74/87
8/18/2019 Fundamentals of Vector Quantization
75/87
8/18/2019 Fundamentals of Vector Quantization
76/87
8/18/2019 Fundamentals of Vector Quantization
77/87
8/18/2019 Fundamentals of Vector Quantization
78/87
8/18/2019 Fundamentals of Vector Quantization
79/87
If N very large, then lattice codes chosen so thatVoronoi cell has minimum normalized moment of
inertia are nearly optimal in the ECVQ problem.Tradeoff: Lattice codes mean simple NN selectionand nearly optimal variable rate performance,but need to entropy code to attain performance.ECVQ design can do better, and sometimesmuch better if rate not asymptotically big.
Note:
•For the xed rate case, cells should haveroughly equal partial distortion.•For the variable rate case, cells should haveroughly equal volume.
In neither case should you try to make cells haveequal probability (maximum entropy).
79
8/18/2019 Fundamentals of Vector Quantization
80/87
Summary
D(Q) ≈E [ m(X )λ(X )2/k ]N −
2/k
D̂k,f (R) ≈C k||f X ||k/ (k+2) 2−2R
D̂k,v(R)
≈C k2
2h(X )k 2−2R
Can use high rate quantization theory toquantify the gains of vector quantization overscalar quantization as a function of dimension k,and to separate out the gains as those duememory, to space lling (moment of inertia), andto shape (of density function).
80
8/18/2019 Fundamentals of Vector Quantization
81/87
Gish & Pierce
Asymptotic theory implies that for iid sources,high rates (low distortion)
D̂1,v(R)D̂k,v(R) ≤
C 1C ∞≈
1.533dB
Or, equivalently, for low distortionR̂1,v(D) − R̂k,v(D) ≤0.254 bits
famous “quarter bit” result.
Suggests at high rates there may be little to begained by using vector quantization (but stillneed to use vectors to do entropy coding!)
Ziv (1985) showed that for all distortions
R̂1,v
(D)
− R̂
k,v(D)
≤0.754 bits
using a dithering argument: X̂ = Q(X + Z ) −Z with Z uniform and independent of X .(subtractive dither)
81
8/18/2019 Fundamentals of Vector Quantization
82/87
Comment: If x R and let k → ∞, thenD̂
k,v(R) →
k → ∞D(R)
D̂k,f (R) →k → ∞D(R)
the Shannon DRF.
Bennett theory recently extended toinput-weighted squared error measures.
82
8/18/2019 Fundamentals of Vector Quantization
83/87
Recent Extensions
Can generalize to input-dependent squared errorand to non-difference distortion measures thatbehave locally in this way (have a well-behavedTaylor series expansion)
d(X, ˆX ) = (X −
ˆX )∗BX (X −
ˆX ),Fixed rate:
D ≥DL(Qopt) = kk + 2
C −2k
k N −2k×
[f (x)(det(B (x)))1k]
11+2/k dx
}k+2
k .Optimal point density
λ(x)∝( p(x)(det(B (x)))1k)
kk+2 .
Variable rate:
D ≥ DL(Qopt)=
kC −2k
kk + 2 ·e−
2k(H Q−h( p)−12 log(det(B(x)))f (x)dx)
83
8/18/2019 Fundamentals of Vector Quantization
84/87
λopt
(x) = (det(B (x)))
12
x∈G(det(B (x)))12dx
. (4)
84
8/18/2019 Fundamentals of Vector Quantization
85/87
Final Comments
•The Shannon compression model:codes, rate, and distortion Mathematical model of compression system.Successfully applied to asymptoticperformance bounds and design algorithms.
•Optimalility properties and quantizer design Lloyd optimal codes. For high rate, latticecodes + lossless codes nearly optimal (variablerate)Rate-distortion ideas can be used to improvestandards-compliant and wavelet codes. (e.g.,Ramchandran, Vetterli, Orchard)Multiple distortion measures (include othersignal processing)MPEG 4 style image structure?
85
8/18/2019 Fundamentals of Vector Quantization
86/87
Compression on networks, joint source andchannel coding
•Structured vector quantizers Transform/pyramid/subband/wavelet arecurrently best (rate, distortion, complexity)for image compression, especially embeddedzerotrees
Many variations and alternative structurescome and go, some stay.
•Optimal achievable performance:quantization and source coding theory Very general Shannon theorems exist. Recenturry of work on universal and adaptive codes.Multiple distortion measures, non-difference(perceptual) distortion measures.Current work on convergence and rate of convergence (Vapnic-Cernovenkis theory), usein classication and regression (Devroye,Gyor, Gabor, Nobel, Olshen)
86
8/18/2019 Fundamentals of Vector Quantization
87/87
Open problems: Gersho’s conjecture, correctcombination of wavelet theory + Bennett
(Goldberg did for traditional bit allocationapproach, not yet done for embedded zerotreecodes and related), other improvedcombinations of wavelet and compressiontheory and design (Mallat, Orchard).
Another issue: Perceived quality of compressedaudio and images.Different applications have different fundamentalquality requirements:Entertainment, browsing, screening, diagnostic,legal, scienticQuantitative as predictors for subjective anddiagnostic.
Top Related