Lattices for Distributed Source Coding -

Reconstruction of a Linear function of Jointly

Gaussian Sources

Dinesh Krithivasan and S. Sandeep Pradhan

University of Michigan, Ann Arbor

1

Presentation Overview

• Problem Formulation

• A straightforward coding scheme using random coding

• A new coding scheme using lattice coding

– Motivation for the coding scheme

– An overview of lattices and lattice coding

– A lattice based coding scheme

• Results and Extensions

• Conclusion

2

Problem Formulation

• Source (X1, X2) is a bivariate Gaussian.

• X1, X2 ∼ N (0, 1), E(X1X2) = ρ > 0.

• Encoders observe X1, X2 separately and quantize them at rates

R1 and R2.

• Decoder interested in a linear function Z , X1 − cX2.

• Lossy reconstruction to within a mean squared error distortion

D.

• Objective: Achievable rates (R1, R2) at distortion D.

3

Berger-Tung based Coding Scheme

• Encoders: Quantize X1 to W1 and X2 to W2. Transmit W1

and W2.

• Decoder: Reconstruct Z , E(Z | W1, W2).

• Can use Gaussian test channels for P (W1 | X1) and

P (W2 | X2) to derive achievable rates and distortion.

• Known to be optimal for c < 0 with Gaussian test channels.

4

Motivation for our Coding Scheme

• Korner and Marton considered lossless coding of Z = X1 ⊕ X2

• Coding Scheme if the encoding is centralized?

5

Motivation for our Coding Scheme

• Korner and Marton considered lossless coding of Z = X1 ⊕ X2

• Coding Scheme if the encoding is centralized?

– Compute Z = X1 ⊕ X2. Compress it to f(Z) and transmit.

– f(·) is any good source coding scheme.

5-a

Motivation for our Coding Scheme

• Korner and Marton considered lossless coding of Z = X1 ⊕ X2

• Coding Scheme if the encoding is centralized?

– Compute Z = X1 ⊕ X2. Compress it to f(Z) and transmit.

– f(·) is any good source coding scheme.

• Suppose f(·) distributes over ⊕?

5-b

Motivation for our Coding Scheme

• Korner and Marton considered lossless coding of Z = X1 ⊕ X2

• Coding Scheme if the encoding is centralized?

– Compute Z = X1 ⊕ X2. Compress it to f(Z) and transmit.

– f(·) is any good source coding scheme.

• Suppose f(·) distributes over ⊕?

– Compress X1 and X2 as f(X1) and f(X2).

– Decoder computes f(X1) ⊕ f(X2) = f(X1 ⊕ X2) = f(Z)

5-c

Motivation for our Coding Scheme

• Korner and Marton considered lossless coding of Z = X1 ⊕ X2

• Coding Scheme if the encoding is centralized?

– Compute Z = X1 ⊕ X2. Compress it to f(Z) and transmit.

– f(·) is any good source coding scheme.

• Suppose f(·) distributes over ⊕?

– Compress X1 and X2 as f(X1) and f(X2).

– Decoder computes f(X1) ⊕ f(X2) = f(X1 ⊕ X2) = f(Z)

• No difference between centralized and distributed coding.

• Choosing f(·) as a linear code will work.

5-d

An overview of Lattices

• An n dimensional lattice Λ - collection of integer combinations

of columns of a n × n generator matrix G.

• Nearest neighbor quantizer

QΛ(x) , {λ ∈ Λ: ‖ x − λ ‖≤‖ x − λ′

‖ ∀λ′

∈ Λ}.

• Quantization error : x mod Λ , x − QΛ(x).

• Voronoi region V0(Λ) , {x ∈ Rn : QΛ(x) = 0n}.

• The second moment σ2(Λ) of lattice Λ - expected value per

dimension of a random vector uniformly distributed in V0(Λ).

• Normalized second moment G(Λ) = σ2(Λ)V 2/n(Λ)

. V =∫

V0

dx.

6

Introduction to Lattice Codes

• Lattice code - a subset of the lattice points are used as

codewords.

• Have been used for both source and channel coding problems.

• Various notions of goodness:

– A good source D-code if log(2πeG(Λ)) ≤ ǫ and σ2(Λ) = D.

– A good channel σ2z -code if it achieves the Poltyrev exponent

on the unconstrained AWGN channel of variance σ2z .

7

Introduction to Lattice Codes

• Lattice code - a subset of the lattice points are used as

codewords.

• Have been used for both source and channel coding problems.

• Various notions of goodness:

– A good source D-code if log(2πeG(Λ)) ≤ ǫ and σ2(Λ) = D.

– A good channel σ2z -code if it achieves the Poltyrev exponent

on the unconstrained AWGN channel of variance σ2z .

• Why lattice codes?

7-a

Introduction to Lattice Codes

• Lattice code - a subset of the lattice points are used as

codewords.

• Have been used for both source and channel coding problems.

• Various notions of goodness:

– A good source D-code if log(2πeG(Λ)) ≤ ǫ and σ2(Λ) = D.

– A good channel σ2z -code if it achieves the Poltyrev exponent

on the unconstrained AWGN channel of variance σ2z .

• Why lattice codes?

– Lattices achieve the Gaussian rate distortion bound

– Lattice encoding distributes over the function Z = X1 − cX2

7-b

Nested Lattice Codes

• (Λ1, Λ2) is a nested lattice if Λ1 ⊂ Λ2.

• Nested lattice codes widely used in multiterminal

communications.

• Need nested lattice codes such that both Λ1 and Λ2 are good

source and channel codes.

• Such nested lattices termed “good nested lattice codes”.

• Known that such nested lattices do exist in sufficiently high

dimension.

8

The Coding Scheme

• A good nested lattice code (Λ11, Λ12, Λ2) with Λ2 ⊂ Λ11, Λ12.

• Dithers Ui ∼ Unif V0(Λ1i), i = 1, 2.

• σ2(Λ11) = q1, σ2(Λ12) =

Dσ2

Z

σ2

Z−D− q1, σ

2(Λ2) =σ4

Z

σ2

Z−D

• Note: Second encoder scales the input before encoding.

9

The Coding Scheme contd.

• Rate of a nested lattice (Λ1, Λ2) is R = 12 log σ2(Λ2)

σ2(Λ1).

• Thus, Encoder rates are

R1 =1

2log

σ4Z

q1(σ2Z − D)

, R2 =1

2log

σ4Z

Dσ2Z − q1(σ2

Z − D).

• Sum rate R1 + R2 = log2σ2

Z

Dachievable at distortion D.

10

The Coding Scheme contd.

• Rate of a nested lattice (Λ1, Λ2) is R = 12 log σ2(Λ2)

σ2(Λ1).

• Thus, Encoder rates are

R1 =1

2log

σ4Z

q1(σ2Z − D)

, R2 =1

2log

σ4Z

Dσ2Z − q1(σ2

Z − D).

• Sum rate R1 + R2 = log2σ2

Z

Dachievable at distortion D.

Theorem 1 The set of all rate-distortion tuples (R1, R2, D) that

satisfy

2−2R1 + 2−2R2 ≤

(

σ2Z

D

)−1

are achievable.

10-a

Outline of the Proof

• Key Idea: Distributive property of lattice mod operation

((x mod Λ) + y) mod Λ = (x + y) mod Λ ∀x, y

• Using this, one of the mod-Λ2 operation can be removed from

the signal path.

• Simplified but equivalent coding scheme is

• eq1and eq2

are subtractive dither quantization noises

independent of the sources.

11

Proof Outline contd.

• Effective dither is eq = eq1− eq2

with varianceDσ2

Z

σ2

Z−D.

• Decoder operation can be described as

Z =

(

σ2Z − D

σ2Z

)

((Z + eq) mod Λ2).

• But Λ2 is a goodσ4

Z

σ2

Z−Dchannel code.

• Implies that ((Z + eq) mod Λ2) = (Z + eq) with high

probability.

• Decoder reconstruction Z =(

σ2

Z−D

σ2

Z

)

(Z + eq).

• Can be checked that E(Z − Z)2 = D.

12

Comments about the Lattice Coding Scheme

• Larger rate region than the random coding scheme for some

source statistics and some range of D.

• Coding scheme here is lattice vector quantization followed by

“correlated” lattice-structured binning.

• Reconstructing a function of more than two sources

Z =∑K

i=1 ciXi

– Some sources coded together using the lattice coding

scheme. Others coded separately using Berger-Tung coding.

– Possible to present a unified rate region combining all such

strategies.

13

Comparison of the Rate Regions

• Compare sum rates for ρ = 0.8 and c = 0.8.

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.40

1

2

3

4

5

6

7

Distortion D

Sum

Rat

e

Comparison between Berger−Tung and Lattice based Coding Schemes

rho = 0.8 c = 0.8

Berger−Tung Sum rate Lattice Sum rate

• Lattice based scheme - lower sum rate for small distortions.

• Time sharing between the two schemes - Better rate region

than either scheme alone.

14

Range of Values for Lower Sum Rate

• Shows where (ρ, c) should lie for lattice sum rate to be lower

than Berger-Tung sum rate for some distortion D.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

rho

c

Region where lattice scheme outperforms Berger−Tung scheme

0

0.1

0.3

0.8

1.5

• Contour marked R - Lattice sum rate lower by R units.

• Improved performance only for c > 0.

15

Conclusion

• Considered lossy reconstruction of a function of the sources in

a distributed setting.

• Presented a coding scheme of vector quantization followed by

“correlated” binning using lattices.

• Improves upon the rate region of the natural random coding

scheme.

• Currently working on extending the scheme to discrete sources

and arbitrary functions.

16