Adaptive Coding from a Diffusion Process on the Integer Line Robert Ellis October 26, 2009 Joint...
-
date post
21-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Adaptive Coding from a Diffusion Process on the Integer Line Robert Ellis October 26, 2009 Joint...
Adaptive Coding from a Diffusion
Process on the Integer Line
Robert Ellis
October 26, 2009
Joint work with Joshua Cooper,
University of South Carolina
Outline of Talk
Coding theory overview– Packing (error-correcting) & covering codes– Coding as a 2-player game– Liar game and pathological liar game
Diffusion processes on Z– Simple random walk (linear machine)– Liar machine– Pathological liar game, alternating question strategy
Improved pathological liar game bound– Reduction to liar machine– Discrepancy analysis of liar machine versus linear machine
Concluding remarks
2
Coding Theory Overview
Codewords:fixed-length strings from a finite alphabet
Primary uses: Error-correction for transmission in the presence of noiseCompression of data with or without loss
Viewpoints:Packings and coverings of Hamming balls in the hypercube2-player perfect information games
Applications:Cell phones, compact disks, Mars Reconnaissance Orbiter
3
Transmit blocks of length n
Noise changes≤ e bits per block(||||1 ≤ e)
Repetition code 111, 000– length: n = 3 – e = 1– information rate: 1/3
Coding Theory: (n,e)-Codes
x1…xn
(x1+1)…(xn+ n)
110 010 000
000
101
000 111111
Received:
Decoded:
blockwise majority vote
Richard Hamming
4
0010011
3 errors: incorrect decoding
Coding Theory – A Hamming (7,1)-Code
1 0 0 0 1 1 1 0 1 1 0 1 1 0
0 1 0 0 0 1 1 0 1 0 1 1 0 1
0 0 1 0 1 0 1 0 0 1 1 0 1 1
0 0 0 1 1 1 0 1 1 1 0 0 0 1
0 0 0 0 0 0 0 1 1 0 1 0 1 0
1 1 0 0 1 0 0 1 0 1 1 1 0 0
1 0 1 0 0 1 0 0 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 1 1 1 1 1
Length n=7, corrects e=1 error
1001011
received
decoded
1001001
1 error: correct decoding
5
A Repetition Code as a Packing
(3,1)-code: 111, 000
Pairwise distance = 3 1 error can be corrected
The M codewords of an(n,e)-code correspond toa packing of Hamming ballsof radius e in the n-cube
110 011101
111
000
010 001100
000
010 001100
110 011101
111
A packing of 2 radius-1 Hamming balls
in the 3-cube
6
A (5,1)-Packing Code as a 2-Player Game
(5,1)-code: 11111, 10100, 01010, 00001
0What is the 5th bit?
1What is the 4th bit?
0What is the 3rd bit?
0What is the 2nd bit?
0What is the 1st bit?
CarolePaul 11111
00001
10100
01010
0 1 >1
# errors
11111 0000110100 01010
01111 00100 00010 0001100100
01010
000100001000010
00001000010000111111 10100 01010 00001
7
Covering Codes
Covering is the companion problem to packing
Packing: (n,e)-code
Covering: (n,R)-code
lengthpacking radius
covering radius
110 011101
111
000
010 001100
000
010 001100
110 011101
111
(3,1)-packing code and(3,1)-covering code
“perfect code”11111
00001
10100
01010
11111
11000
01111
10111 00001
00100
00010
(5,1)-packing code (5,1)-covering code
8
Optimal Length 5 Packing & Covering Codes
0100101100
01110 01101
00100
11100
01000
11110 11101 01111
00000
0101011000 10100 00110 00101
10110 10011
1000110010
11011
00011
10111
000010001010000
11111
10101 00111010111100111010
01110 01101
0100101100
00100
11100
01000
11110 11101 01111
00000
0101011000 10100 00110 00101
10110 10011
1000110010
11011
00011
10111
000010001010000
11111
10101 00111010111100111010
(5,1)-packing code
(5,1)-covering code
9
Sphere bound:
A (5,1)-Covering Code as a Football Pool
WLLLLBet 7
LWLLLBet 6
LLWLLBet 5
LLLWWBet 4
WWWLWBet 3
WWWWLBet 2
WWWWWBet 1
Round 5Round 4Round 3Round 2Round 1
Payoff: a bet with ≤ 1 bad predictionQuestion. Min # bets to guarantee a payoff? Ans.=7
00100
01111
11000
10111
00001
00010
11111
10
Codes with Feedback (Adaptive Codes)
FeedbackNoiseless, delay-less report of actual received bits
Improves the number of decodable messagesE.g., from 20 to 28 messages for an (8,1)-code
sender receiver
Noise
Noiseless FeedbackElwyn Berlekamp
1, 0, 1, 1, 0 1, 1, 1, 1, 0
1, 1, 1, 1, 0
11
A (5,1)-Adaptive Packing Code as a 2-Player Liar Game
A
D
B
C
0 1 >1# liesYIs the message C?
NIs the message D?
NIs the message B?
NIs the message A or C?
YIs the message C or D?
CarolePaul
00101
Message
Originalencoding
Adaptedencoding
A B C D
01110 0101011000 10011
1**** 1****11*** 10*** 10*** 1000*101** 100**1000* 1000010001
Y $ 1, N $ 0
12
A (5,1)-Adaptive Covering Code as a Football Pool
LWLLWCarole
LBet 6
LBet 5
LBet 4
WBet 3 W
L
L
WWBet 2
L
W
W
W
W
W
L
L
WWBet 1
Round 5Round 4Round 3Round 2Round 1
Payoff: a bet with ≤ 1 bad predictionQuestion. Min # bets to guarantee a payoff?
Ans.=6
Bet 3
Bet 6
Bet 4
Bet 5
0 1 >1# bad
predictions(# lies)
Bet 2
Bet 1
13
Optimal (5,1)-Codes14
Code type Optimal size
(5,1)-code 4
(5,1)-adaptive code 4
Sphere bound 5 1/3
(5,1)-adaptive covering code 6
(5,1)-covering code 7
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
15
11
Linear Machine on Z
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Linear Machine on Z16
5.5 5.5
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Linear Machine on Z17
2.75 5.5 2.75
Time-evolution is proportional to rows of Pascal’s triangle
Liar Machine on Z18
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Liar machine time-stepNumber chips left-to-right 1,2,3,…Move odd chips right, even chips left(Reassign numbers every time-step)
11 chips
t=0
Liar Machine on Z19
Liar machine time-stepNumber chips left-to-right 1,2,3,…Move odd chips right, even chips left(Reassign numbers every time-step)
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=1
Liar Machine on Z20
Liar machine time-stepNumber chips left-to-right 1,2,3,…Move odd chips right, even chips left(Reassign numbers every time-step)
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=2
Liar Machine on Z21
Liar machine time-stepNumber chips left-to-right 1,2,3,…Move odd chips right, even chips left(Reassign numbers every time-step)
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=3
Liar Machine on Z22
Liar machine time-stepNumber chips left-to-right 1,2,3,…Move odd chips right, even chips left(Reassign numbers every time-step)
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=4
Liar Machine on Z23
Liar machine time-stepNumber chips left-to-right 1,2,3,…Move odd chips right, even chips left(Reassign numbers every time-step)
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=5
Liar Machine on Z24
Liar machine time-stepNumber chips left-to-right 1,2,3,…Move odd chips right, even chips left(Reassign numbers every time-step)
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=6
Liar Machine on Z25
Liar machine time-stepNumber chips left-to-right 1,2,3,…Move odd chips right, even chips left(Reassign numbers every time-step)
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Height of linear machine at t=7
t=7
(6,1)-Liar Game26
Liar game time stepPaul bipartitions chips: green, purpleCarole moves one color to right
Paul’s goal: disqualify all but ≤1 chip after t=6 time steps
0 1 2
t=0
disqualified
Paul bipartitionsCarole moves purple9 chips
(6,1)-Liar Game27
Liar game time stepPaul bipartitions chips: green, purpleCarole moves one color to right
Paul’s goal: disqualify all but ≤1 chip after t=6 time steps
0 1 2
t=1
disqualified
Paul bipartitionsCarole moves green
(6,1)-Liar Game28
Liar game time stepPaul bipartitions chips: green, purpleCarole moves one color to right
Paul’s goal: disqualify all but ≤1 chip after t=6 time steps
0 1 2
t=2
disqualified
Paul bipartitionsCarole moves green
(6,1)-Liar Game29
Liar game time stepPaul bipartitions chips: green, purpleCarole moves one color to right
Paul’s goal: disqualify all but ≤1 chip after t=6 time steps
0 1 2
t=3
disqualified
Paul bipartitionsCarole moves purple
t=4
(6,1)-Liar Game30
Liar game time stepPaul bipartitions chips: green, purpleCarole moves one color to right
Paul’s goal: disqualify all but ≤1 chip after t=6 time steps
0 1 2
disqualified
Paul bipartitionsCarole moves purple
t=5
(6,1)-Liar Game31
Liar game time stepPaul bipartitions chips: green, purpleCarole moves one color to right
Paul’s goal: disqualify all but ≤1 chip after t=6 time steps
0 1 2
disqualified
Paul bipartitionsCarole moves green
t=6
(6,1)-Liar Game32
Liar game time stepPaul bipartitions chips: green, purpleCarole moves one color to right
Paul’s goal: disqualify all but ≤1 chip after t=6 time steps
0 1 2
disqualified
Two chips survive: Paul loses
A Liar Game Strategy for Carole
Weight function for n rounds left; xi = #chips with i lies:
Lemma (Berlekamp)
Refined sphere boundLiar game. Carole keeps half of weight every step.Initial weight > 2n ) Final weight >1 ) Carole wins.
Pathological variant. Carole reduces half of weight every step.Initial weight < 2n ) Final weight <1 ) Carole wins.
33
(6,1)-Pathological Liar Game34
Paul’s goal: preserve ¸ 1 chip after t=6 time steps
0 1 2
t=0
disqualified
Paul bipartitionsCarole moves green9 chips
wt6-t(x)=wt6(x)=26-1
Carole moves green
(6,1)-Pathological Liar Game35
0 1 2
t=1
disqualified
wt5(x)=25-3
Paul bipartitions
Paul’s goal: preserve ¸ 1 chip after t=6 time steps
Paul bipartitions
(6,1)-Pathological Liar Game36
0 1 2
t=2
disqualified
Carole moves green
wt4(x)=24-2
Paul’s goal: preserve ¸ 1 chip after t=6 time steps
Paul bipartitionsCarole moves purple
(6,1)-Pathological Liar Game37
0 1 2
t=3
disqualified
wt3(x)=23-1
Paul’s goal: preserve ¸ 1 chip after t=6 time steps
Paul bipartitions t=4
(6,1)-Pathological Liar Game38
0 1 2
disqualified
wt2(x)=22-1
Carole moves purple
Paul’s goal: preserve ¸ 1 chip after t=6 time steps
Carole moves greenPaul bipartitions t=5
(6,1)-Pathological Liar Game39
0 1 2
disqualified
wt1(x)=21-1
Paul’s goal: preserve ¸ 1 chip after t=6 time steps
t=6
(6,1)-Pathological Liar Game40
0 1 2
disqualified
No chips survive: Paul loses
wt0(x)=20-1<1
Paul’s goal: preserve ¸ 1 chip after t=6 time steps
Optimal (6,1)-Codes41
Code type Optimal #chips
(6,1)-code 8
(6,1)-adaptive code(Liar game)
8
Sphere bound 9 1/7
(6,1)-adaptive covering code(Pathological liar game)
10
(6,1)-covering code 12
New Approach to the Pathological Liar Game
Spencer and Winkler (`92) reduced the liar game to the liar machine, a discrete diffusion process on the integer line.
Ellis and Yan (`04) introduced the pathological liar game.
Cooper and Spencer (`06) use discrepancy analysis to compare the Propp-machine to simple random walk on Zd.
Here: (1) We reduce the pathological liar game to the liar machine, (2) Use discrepancy analysis to compare the liar machine to simple random walk on Z, and thereby (3) Improve the best known pathological liar game strategy when the number of lies is a constant fraction of the number of rounds.
42
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Liar Machine vs. Pathological Liar Game43
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
9 chips
9 chips
t=0
disqualified
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Liar Machine vs. Pathological Liar Game44
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=1
disqualified
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Liar Machine vs. Pathological Liar Game45
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=2
disqualified
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Liar Machine vs. Pathological Liar Game46
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=3
disqualified
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Liar Machine vs. Pathological Liar Game47
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=4
disqualified
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Liar Machine vs. Pathological Liar Game48
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=5
disqualified
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Liar Machine vs. Pathological Liar Game49
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
t=6
disqualified
No chips survive: Paul loses
(6,1)-Pathological Liar Game, Liar Machine50
Code type Optimal #chips
Sphere bound 9 1/7
(6,1)-adaptive covering code(Pathological liar game)
10
(6,1)-liar machine 12
(6,1)-liar machine optimum: Minimum number of initial chips for ¸ 1 chip to be at position · -4 when t=6
9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8
(6,1)-Liar machine started with 12 chips after 6 rounds
disqualified
Reduction to Liar Machine51
Reduction to Liar Machine52
Liar Machine Versus Linear Machine53
Saving One Chip in the Liar Machine54
Pathological Liar Game Theorem55
Further Exploration
Tighten the discrepancy analysis for the special case of initial chip configuration f0(z)=M 0(z).
Generalize from binary questions to q-ary questions, q ¸ 2.
Improve analysis of the original liar game from Spencer and Winkler `92.
Prove general pointwise and interval discrepancy theorems for various discretizations of random walks.
56
Reading List
This paper: Linearly bounded liars, adaptive covering codes, and deterministic random walks, preprint (see homepage).
The liar machine– Joel Spencer and Peter Winkler. Three thresholds for a liar. Combin.
Probab. Comput.,1(1):81-93, 1992. The pathological liar game
– Robert Ellis, Vadim Ponomarenko, and Catherine Yan. The Renyi-Ulam pathological liar game with a fixed number of lies. J. Combin. Theory Ser. A, 112(2):328-336, 2005.
Discrepancy of deterministic random walks– Joshua Cooper and Joel Spencer, Simulating a Random Walk with
Constant Error, Combinatorics, Probability, and Computing, 15 (2006), no. 06, 815-822.
– Joshua Cooper, Benjamin Doerr, Joel Spencer, and Gabor Tardos. Deterministic random walks on the integers. European J. Combin., 28(8):2072-2090, 2007.
57