Coding Theory: Packing, Covering, and 2-Player Games

Robert Ellis

Menger Day 2008: Recent Applied Mathematics

Research Advances

April 14, 2008

2

Overview

A problem from integrated circuit design

Coding Theory– Error-correcting codes and packings– Error-correcting codes as a 2-player liar game– Covering codes– Covering codes as a football pool

Coding with Feedback– A liar game and an adaptive football pool– Near-perfect radius 1 adaptive codes

Results and Research Questions in Liar Games

3

A VLSI Layout Problem

Silicon substrate

Wires & components

Inert metal fill

Fill Library

26 patterns 23 patterns,Compression ratio: 50%

4

An Asymmetric Covering Code

Fill library (6,2)-asymmetric binary code

Size bound (2n/nR) (Cooper,Ellis,Kahng `02)

Application to VLSI Layout(Ellis,Kahng,Zheng `03)

Improved fixed-parameter codes: Applegate,Rains,Sloane `03; Exoo `04; Östergård,Seuranen `04

Improved size bound (Krivelevich,Sudakov,Vu `03)

000000 000010 010100 111000

000001 000011 101100 010111

011101

Codeword: 010100

5

Therefore K+(4,2) = 6 (length=4, radius=2).

Smallest (4,1)-Asymmetric Covering Code

01011100 0011011010011010

1011 011111011110

1111

0000

0100 0010 00011000

0000

0100 0010 00011000

1100 10011010

1000

01011100 0110

0100

0011

1011 0111

1100

11011110

1111

6

00…00

11…11

Select each word to be inthe code with probability p(n)

Any uncovered word isadded as a codeword

This plus hypercube structureyields codes of size (2n/nR)

Best possible up to a constant,since middle ball volumes are(nR)

Good (n,R)-Asymmetric Covering Codes

7

Coding Theory Overview

Coding theory concerns the properties of sets of codewords, or fixed-length strings from a finite alphabet.

Primary uses: Error-correction for transmission in the presence of noiseCompression of data with or without loss

Many viewpoints afforded:Packings and coverings of Hamming balls in the n-cube2-player perfect information games

8

Noisy communication:add redundancy to counteract noise

Noiseless communication:compress data using redundancy

The binary symmetricchannel for noise0 ≤ p < 1/2

Information Theory (Shannon Model)

sender receiverencoder decoder

Noise 1…n

x1…xn (x1+1)…(xn+ n)m m

Claude Shannon0 0

1 1

p

p

1-p

1-p

9

Transmit blocks of length n

Noise changes≤ e bits per block(||||1 ≤ e)

Repetition code 111, 000– length: n = 3

– e = 1

– information rate: 1/3

Coding Theory: (n,e)-Codes

x1…xn

(x1+1)…(xn+ n)

110 010 000

000

101

000 111111

Received:

Decoded:

blockwise majority vote

Richard Hamming

10

Block Codes from now on

Restricting to block codes still includes

Convolutional codes (cell phones, Bluetooth)

Reed-Solomon codes (CDs, DSL, WiMAX)

Turbo codes (Mars Reconnaissance Orbiter)

(assumptions on noise for these codes will vary)

11

0010011

3 errors: incorrect decoding

Coding Theory – A Hamming (7,1)-Code

1 0 0 0 1 1 1 0 1 1 0 1 1 0

0 1 0 0 0 1 1 0 1 0 1 1 0 1

0 0 1 0 1 0 1 0 0 1 1 0 1 1

0 0 0 1 1 1 0 1 1 1 0 0 0 1

0 0 0 0 0 0 0 1 1 0 1 0 1 0

1 1 0 0 1 0 0 1 0 1 1 1 0 0

1 0 1 0 0 1 0 0 1 1 1 0 0 0

1 0 0 1 0 0 1 1 1 1 1 1 1 1

Length n=7, corrects e=1 error

1001011

received

decoded

1001001

1 error: correct decoding

12

(3,1)-code: 111, 000

Pairwise distance = 3 1 error can be corrected

The M codewords of an(n,e)-code correspond toa packing of Hamming ballsof radius e in the n-cube

A Repetition Code as a Packing

110 011101

111

000

010 001100

000

010 001100

110 011101

111

A packing of 2 radius 1 Hamming balls

in the 3-cube

13

A (5,2)-Code as a Packing

0100101100

01110 01101

00100

11100

01000

11110 11101 01111

00000

0101011000 10100 00110 00101

10110

10011

10001 100101101100011 10111

00001 00010 1000011111 101010011101011 11001 11010

•(5,2)-code: 01100, 10011

•(disjoint) packing in 5-cube

Volume:Sphere Bound: for an (n,e)-code with M codewords,

14

(5,1)-code: 11111, 10100, 01010, 00001

A (5,1)-Code as a 2-Player Game

0What is the 5th bit?

1What is the 4th bit?

0What is the 3rd bit?

0What is the 2nd bit?

0What is the 1st bit?

CarolePaul 11111

00001

10100

01010

0 1 >1

# errors

11111 0000110100 01010

01111 00100 00010 0001100100

01010

000100001000010

00001000010000111111 10100 01010 00001

15

Covering is the companion problem to packing

Packing: (n,e)-code

Covering: (n,R)-code

Covering Codes

lengthpacking radius

covering radius

110 011101

111

000

010 001100

000

010 001100

110 011101

111

(3,1)-packing code and(3,1)-covering code

“perfect code”11111

00001

10100

01010

11111

11000

01111

10111 00001

00100

00010

(5,1)-packing code (5,1)-covering code

16

Optimal Length 5 Packing & Covering Codes

0100101100

01110 01101

00100

11100

01000

11110 11101 01111

00000

0101011000 10100 00110 00101

10110 10011

1000110010

11011

00011

10111

000010001010000

11111

10101 00111010111100111010

01110 01101

0100101100

00100

11100

01000

11110 11101 01111

00000

0101011000 10100 00110 00101

10110 10011

1000110010

11011

00011

10111

000010001010000

11111

10101 00111010111100111010

(5,1)-packing code

(5,1)-covering code

17

A (5,1)-Covering Code as a Football Pool

WLLLLBet 7

LWLLLBet 6

LLWLLBet 5

LLLWWBet 4

WWWLWBet 3

WWWWLBet 2

WWWWWBet 1

Round 5Round 4Round 3Round 2Round 1

Payoff: a bet with ≤ 1 bad predictionQuestion. Min # bets to guarantee a payoff? Ans.=7

00100

01111

11000

10111

00001

00010

11111

18

Codes with Feedback (Adaptive Codes)

sender receiver

Noise

Noiseless FeedbackElwyn Berlekamp

FeedbackNoiseless, delay-less report of actual received bits

Improves the number of decodable messagesE.g., from 20 to 28 messages for an (8,1)-code

1, 0, 1, 1, 0 1, 1, 1, 1, 0

1, 1, 1, 1, 0

19

A (5,1)-Adaptive Code as a 2-Player Liar Game

A

D

B

C

0 1 >1# liesYIs the message C?

NIs the message D?

NIs the message B?

NIs the message A or C?

YIs the message C or D?

CarolePaul

00101

Message

Originalencoding

Adaptedencoding

A B C D

01110 0101011000 10011

1**** 1****11*** 10*** 10*** 1000*111** 100**1000* 1000010001

Y 1, N 0

20

A (5,1)-Adaptive Covering Code as a Football Pool

LWLLWCarole

LBet 6

LBet 5

LBet 4

WBet 3 W

L

L

WWBet 2

L

W

W

W

W

W

L

L

WWBet 1

Round 5Round 4Round 3Round 2Round 1

Payoff: a bet with ≤ 1 bad predictionQuestion. Min # bets to guarantee a payoff?

Ans.=6

Bet 3

Bet 6

Bet 4

Bet 5

0 1 >1# bad

predictions(# lies)

Bet 2

Bet 1

21

Form of an adaptive Hamming ball (radius 1)

Example: n = 5, e = R = 1

Feedback and Adaptive Hamming Balls

Adapted encoding after

1

1

1

1

0

1

1

1

1

0

*

1

0

0

1

*

*

0

1

0

*

*

*

1

Error in 5th bit1Child 5

Error in 4th bit*Child 4

Error in 3rd bit*Child 3

Error in 2nd bit*Child 2

Error in 1st bit*Child 1

Original encoding0Root

0101011010

1000101111 11100 11001 11011

22

Classification of Coding Problems

Packing

No feedbackError-correcting

codes P(n,e)

FeedbackAdaptive error-

correcting codes P’(n,e)

Covering

FeedbackAdaptive

covering codes K’(n,R)

No feedback Covering codes K(n,R)

Sphere Bound

≤≤

≤≤

23

Near-Perfect Radius 1 Adaptive Codes

Theorem (E.`05+). For all n ≥ 2 and e = R = 1, there exists an adaptive packing contained in an adaptive covering with sizes given by

where (The sphere bound is )

P’(n,1) K’(n,1)

24

Proof Idea: Near-Perfect Radius 1 Adaptive Codes

01

11

00

10

packing covering

0

1

00

01

10

11

steal 00

01

10

11

Q1 Q2

duplicate

0Q1 1Q1

duplicate

00

01

10

11

Q2

000

001

010

011

100

101

110

111 steal

0Q2 1Q2

000

001

010

011

100

101

110

111

Q3

000

001

010

011

100

101

110

111

Q3

01

11

00

10Q2 packing Q2 covering

25

Adaptive Coding as an (M,n,e)-Liar Game

M = # chips n = # rounds e = max # liesCarole picks a distinguished x 2 {1,…,M}

1

M

2

0 1 >e# lies

…

e…

(1) Paul bipartitions {1,…,M} = A0 [ A1 and asks “Is x 2 A1?” (2) Carole responds “Yes” or “No”, and may lie up to e times.

Each

Round

9

0 1 >32 3

321

4 65 8

7A0 A1

“Yes” “No”

9

0 1 >32 3

421

3

6

5 87

9

0 1 >32 3

321

4 65

87

26

Lose

Original and Pathological Liar Games

Two variants– Original liar game (Berlekamp, Rényi, Ulam)

Paul wins iff at most 1 chip survives after n rounds

– Pathological liar game (Ellis&Yan)Paul wins iff at least 1 chip survives after n rounds

…

0 1 >32 3

…

0 1 >32 3

…

0 1 >32 3Lose Win Win

…

0 1 >32 3

…

0 1 >32 3

…

0 1 >32 3WinWin

27

Adaptive error-correcting codes

Liar game

Classification of Coding Problems

Covering codes

Adaptivecovering codes

Error-correctingcodes

K(n,R)No feedback

K’(n,R)Feedback

Covering

P’(n,e)Feedback

P(n,e)No feedback

Packing

Sphere Bound

≤≤

≤≤

Pathologicalliar game

28

3 Chip Original Liar Game

Given M=3 chips, in how many rounds can Paul guarantee winning the game with e lies?

Label each chip with its distance to being eliminated

Introduce weight function f(x1,x2,x3)=x1+x2+x3-1

f(6,4,3) = 12 Each round:

– Paul can force f to reduce by 1– Carole can prevent f from reducing by more than 1

0 1 >ee…

6

0 1 >55

34

2 3 4A0 A1

Paul wins iffn ≥ 3e+2

29

4-8 Chip Original Liar Game

Order chip labels so that x1 ≥ x2 ≥ … ≥ xM.

M=4 chips: f4(x1,x2,x3,x4)=x1+x2+x3-1 = 12

M=5 chips: f5(x1,x2,x3,x4,x5)=x1+x2+x3+(x1=x5)-1

Exercise: find/verify the weight function for M=4,…,8 (Ellis&Łuczak) Research Problem: find the weight function for M>8

6

0 1 >55

34

2 3 4A0 A1

1

6

0 1 >552 3 4

6666

f5=18+1-1=18

5

0 1 >552 3 4

5666

f5=18+0-1=17(f3=18-1=17)

30

2-4 Chip Pathological Liar Game

M=2 chips g2(x1,x2)=x1+x2-1 = 9

M=3 chips g3(x1,x2,x3)=x1+x2-1 = 9

M=4 chips g4(x1,x2,x3,x4)=x1+x2+(x1=x4)-1 = 12

M=2,3,4 (Ellis&Stanford) M>4: Research Problem

6

0 1 >55

4

2 3 4

A0 A1

6

0 1 >55

34

2 3 4

6

0 1 >552 3 4

666

g4=12+1-1=12

5

0 1 >552 3 4

566

g4=12+0-1=11(g2=12-1=11)

31

Perfect Splits and the Pathological Liar Game

0 1 >eek-1 k

2k

… … …

0 1 >eek-1 k

2k-1

… … …2k-1

0 1 >eek-1 k

1 … … …1

1 round

k-1 rounds

0 1 >eek-1 k

0 … … …00 0

Removechips2k’

0 1 >eek-1 k

0 … … …00 00

0

1

Repeat until1 chip leftat position e

… …

32

Upper bound on M=2k: e/n is the overall fraction of lies after k rounds the chips at position (e/n)k determine whether Paul wins

Lower bound on M=2k: Each chip survives in only out of 2n possible outcomes of the game; i.e.,

Perfect Splits and the Pathological Liar Game

0 1 >eek-1 k

0 … … …00 02k’ 0

error fraction

33

Many Open Questions at Every Level!

Research problems appropriate forUndergraduates, Graduate students, Dissertations, and beyond!

Fixed parameter games Games with constrained lies Non-binary alphabets Restricted feedback List decoding (win with L

chips instead of 1)

Applying feedback coding to real-world problems