The 1’st annual (?) The 1’st annual (?)

workshopworkshop

2

Communication Communication under under

Channel Uncertainty:Channel Uncertainty:Oblivious channelsOblivious channels

Michael Langberg

California Institute of Technology

3

Coding theory Coding theory

X Ym {0,1}k Noisex = C(m) {0,1}n

y

C: {0,1}k {0,1}n

Error correcting codes m

decode

4

Communication channelsCommunication channelsDesign of C depends on properties of channel.

•Channel W: W(e|x) = probability that error e is imposed by channel when x=C(m) is transmitted.In this case y=xe is received.

BSCp: Binary Symmetric Channel. Each bit flipped with probability p.

W(e|x)=p|e|(1-p)n-|e|

X Yx e y=xe

5

Success criteriaSuccess criteria•LetLet D: D: {0,1}{0,1}nn {0,1} {0,1}kk be a decoder. be a decoder.

•CC is said to allow the communication is said to allow the communication ofof mm over over WW (with (with DD) ) if if PrPree[D(C(m)[D(C(m)e)=m] ~ 1e)=m] ~ 1..•Probability over Probability over W(e|C(m))W(e|C(m))..

C: {0,1}k {0,1}n

X Yx=C(m) y=xee

•CC is said to allow the communication is said to allow the communication ofof {0,1}{0,1}kk over over WW (with (with DD) if ) if PrPrm,em,e[D(C(m)[D(C(m)e)=m] ~ 1e)=m] ~ 1..•Probability uniform over Probability uniform over {0,1}{0,1}k k and over and over W(e|C(m))W(e|C(m))..

•RateRate of of CC is is k/n.k/n.

BSCp [Shannon]: exist codes with rate ~ 1-H(p) (optimal).

6

Channel uncertaintyChannel uncertainty•What if properties of the channel are not known?What if properties of the channel are not known?

•Channel can be any channel in family Channel can be any channel in family WW = = {W}{W}. .

•ObjectiveObjective: design a code that will allow communication : design a code that will allow communication not matter which not matter which WW is chosen in is chosen in WW..

•CC is said to allow the communication of is said to allow the communication of {0,1}{0,1}kk over over channel family channel family WW if there exists a decoder if there exists a decoder D D s.t. for s.t. for each each WWWW : C,D : C,D allow communication of allow communication of {0,1}{0,1}kk over over WW. .

X Y?

7

•A channel A channel WW is a is a pp-channel if it can only -channel if it can only change a change a pp-fraction of the bits transmitted: -fraction of the bits transmitted: W(e|x)=0W(e|x)=0 if if |e|>pn|e|>pn..

• WWp p = = family of all family of all pp-channels.-channels.

•Communicating overCommunicating over WWpp: design a code that : design a code that enables communication no matter which enables communication no matter which pp-fraction -fraction of bits are flipped. of bits are flipped.

The family The family WWpp

Power constrain on W

X Y

Adversarial model in which the channel W is chosen maliciously by an adversarial jammer within limits of WWpp..

8

* *

* * * * *

* *

Communicating over Communicating over WWpp•Communicating overCommunicating over WWpp: design a code : design a code CC that enables that enables

communication no matter which communication no matter which pp-fraction of bits are -fraction of bits are flipped. flipped.

•““Equivalently”Equivalently”: minimum distance of : minimum distance of CC is is 2pn2pn..

C

{0,1}n

Min. distance

X YWWpp

•What is the maximum achievable rate over WWpp??•Major open problem.Major open problem.

•Known: Known: 1-H(2p) ≤ R < 1-H(p)1-H(2p) ≤ R < 1-H(p)

9

This talkThis talk•Communication over Communication over WWp p not fully understood.not fully understood.

• WWp p does not allow communication w/ rate does not allow communication w/ rate 1-H(p)1-H(p)..

•BSCBSCpp allows communication at rate allows communication at rate 1-H(p)1-H(p)..

•In “essence” In “essence” BSCBSCppWWp p (power constraint). (power constraint).

•Close gap by considering restriction of Close gap by considering restriction of WWpp..

•Oblivious channelsOblivious channels•Communication over Communication over WWp p with thewith the assumption that assumption that

the channel has a limited view of the transmitted the channel has a limited view of the transmitted codeword.codeword.

X Y

11

Oblivious channelsOblivious channels• Communicating overCommunicating over WWpp: only : only pp-fraction of bits can be flipped.-fraction of bits can be flipped.

•Think of channel as adversarial jammer. Think of channel as adversarial jammer.

•Jammer acts maliciously according to codeword sent.Jammer acts maliciously according to codeword sent.

•Additional constraintAdditional constraint: Would like to limit the amount of : Would like to limit the amount of information the adversary has on codeword information the adversary has on codeword xx sent. sent.

•For example:For example:•Channel with a “window” view.Channel with a “window” view.•In general: correlation between codeword In general: correlation between codeword xx and and

error error ee imposed by imposed by WW is limited. is limited.

X Y

12

Oblivious channels: Oblivious channels: modelmodel•A channel A channel WW is is obliviousoblivious if if W(e|x)W(e|x) is independent of is independent of xx..

•BSCBSCpp is an oblivious channel. is an oblivious channel.

•A channel A channel WW is is partially-obliviouspartially-oblivious if the dependence of if the dependence of W(e|x)W(e|x) on on x x is limited:is limited:•Intuitively Intuitively I(e,x)I(e,x) is small. is small.

• Partially obliviousPartially oblivious - definition: - definition: •For each For each xx: : W(e|x)=WW(e|x)=Wxx(e)(e) is a distribution over is a distribution over {0,1}{0,1}nn..•Limit the size of the family Limit the size of the family {W{Wxx|x}|x}. .

X Y

Let W0 and W1 be two distributions over errors.Define W as follows:W(e|x) = W0(e) if the first bit of x is 0.W(e|x) = W1(e) if the first bit of x is 1.W is almost completely oblivious.

13

Families of oblivious Families of oblivious channelschannels•A family of channelsA family of channels WW** W Wp p is (partially) oblivious if is (partially) oblivious if

each each WWWW** is (partially) oblivious.is (partially) oblivious.

•Study the rate achievable when comm. over Study the rate achievable when comm. over WW**..

•Jammer Jammer WW** is limited in is limited in powerpower and and knowledgeknowledge..

•BSCBSCpp is an oblivious channel “in” is an oblivious channel “in” WWpp..

•Rate on Rate on BSCBSCpp ~ 1-H(p) ~ 1-H(p)..

•Natural questionNatural question: Can this be extended to any : Can this be extended to any family of oblivious channels?family of oblivious channels?

14

Our resultsOur results

•Study both oblivious and partially oblivious families.Study both oblivious and partially oblivious families.

•For oblivious families For oblivious families WW**one can achieve rate ~ one can achieve rate ~ 1-H(p)1-H(p)..

•For families For families WW** of partially oblivious channels in which of partially oblivious channels in which

WWWW* * : : {W{Wxx|x}|x} of size at most of size at most 22nn..

Achievable rate ~ Achievable rate ~ 1-H(p)-1-H(p)- (if (if < (1-H(p))/3 < (1-H(p))/3).).

•Sketch proof for oblivious Sketch proof for oblivious WW**..

X Y

15

Previous workPrevious work•Oblivious channels in Oblivious channels in WW** have been addressed by have been addressed by

[CsiszarNarayan] [CsiszarNarayan] as a special case of as a special case of Arbirtrarily Arbirtrarily Varying ChannelsVarying Channels with state constraints. with state constraints.

• [CsiszarNarayan] [CsiszarNarayan] show that rate ~ show that rate ~ 1-H(p)1-H(p) for oblivious for oblivious channels in channels in WW** using the using the “method of types”“method of types”..

•Partially oblivious channels not defined previously.Partially oblivious channels not defined previously.

•For partially oblivious channels For partially oblivious channels [CsiszarNarayan] [CsiszarNarayan]

implicitly show implicitly show 1-H(p)-301-H(p)-30 (compare with(compare with 1-H(p)- 1-H(p)-).).

•Our proof technique are Our proof technique are substantially differentsubstantially different..

17

Proof technique: Random Proof technique: Random codescodes•Let Let CC be a code (of rate be a code (of rate 1-H(p)1-H(p)) in which each ) in which each

codeword is picked at random.codeword is picked at random.

•ShowShow: with high probability : with high probability CC allows comm. over allows comm. over any oblivious channel in any oblivious channel in WW* * (any channel (any channel WW which which always imposes the same distribution over errors).always imposes the same distribution over errors).

•ImpliesImplies: Exists a code : Exists a code C C that allows comm. over that allows comm. over WW**

with rate with rate 1-H(p)1-H(p)..

18

Proof sketchProof sketch

• ShowShow: with high probability : with high probability CC allows comm. over any oblivious allows comm. over any oblivious channel in channel in WW**..

• Step 1Step 1: show that : show that CC allows comm. over allows comm. over WW* * iff iff CC allows comm. allows comm. over channels over channels WW that always impose a that always impose a singlesingle error error ee (|e| ≤ pn(|e| ≤ pn).).

• Step 2Step 2: Let : Let WWee be the channel that always imposes error be the channel that always imposes error ee..

Show that w.h.p. Show that w.h.p. CC allows comm. over allows comm. over WWee..

• Step 3Step 3: As there are only ~ : As there are only ~ 22H(p)nH(p)n channels channels WWee: use union bound. : use union bound.

X Yx e y=xe

19

Proof of Step 2Proof of Step 2• Step 2Step 2: Let : Let WWee be the channel that always imposes error be the channel that always imposes error ee..

Show that w.h.p. Show that w.h.p. CC allows comm. over allows comm. over WWee..

• Let Let DD be the be the Nearest NeighborNearest Neighbor decoder. decoder.

• By definitionBy definition: : CC allows comm. over allows comm. over WWee iff iff

for most codewords for most codewords x=C(m)x=C(m):: D(x D(xe)=me)=m..

• Codeword Codeword x=C(m)x=C(m) is is disturbeddisturbed if if D(xD(xe)e)mm..

• RandomRandom CC: : expectedexpected number of disturbed number of disturbed codewords is small (i.e. in expectation codewords is small (i.e. in expectation CC allows communication). allows communication).

•Need to prove that number of disturbed codewords is small w.h.p.Need to prove that number of disturbed codewords is small w.h.p.

X Yx e y=xe

* * * * * * * *

* *C

e

20

ConcentrationConcentration

• Expected number of Expected number of disturbeddisturbed codewords is small. codewords is small.

•Need to prove that number of disturbed Need to prove that number of disturbed codewords is small w.h.p.codewords is small w.h.p.

• Standard toolStandard tool - Concentration inequalities: - Concentration inequalities:Azuma, Talagrand, Chernoff.Azuma, Talagrand, Chernoff.

•Work well when random variable has smallWork well when random variable has smallLipschitz coefficientLipschitz coefficient..

• Study Lipschitz coefficient of our process.Study Lipschitz coefficient of our process.

X Yx e y=xe

* * * * * * * *

* *C

e

21

Lipschitz coefficientLipschitz coefficient

Lipschitz coefficient in our setting:Lipschitz coefficient in our setting:• Let Let CC and and C’C’ be two codes that differ be two codes that differ

in a single codeword.in a single codeword.

• Lipschitz coefficient = difference between Lipschitz coefficient = difference between number of number of disturbeddisturbed codewords in codewords in CC and and C’ C’ w.r.t.w.r.t. W Wee..

• Can show that L. coefficient is very large.Can show that L. coefficient is very large.

• Cannot apply standard concentration techniques.Cannot apply standard concentration techniques.

•What next?What next?

X Yx e y=xe

* * * * * * * *

* *C

e

22

Lipschitz coefficientLipschitz coefficient

Lipschitz coefficient in our setting is large.Lipschitz coefficient in our setting is large.•However one may show that However one may show that “on average”“on average”

Lipschitz coefficient is small.Lipschitz coefficient is small.

• This is done by studying the This is done by studying the list decodinglist decodingproperties of random properties of random CC..

•Once we establish that “average” Lipschitz Once we establish that “average” Lipschitz coef. is small one may use recent concentration coef. is small one may use recent concentration result of Vu to obtain proof.result of Vu to obtain proof.

• Establishing “average” Lipschitz coef. is technically involved.Establishing “average” Lipschitz coef. is technically involved.

X Yx e y=xe

* * * * * * * *

* *C

e

[Vu]: Random process in which Lipschitz coefficient has small “expectation” and “variance” will have exponential concentration:probability of deviation from expectation is exponential in deviation.

[KimVu]: concentration of low degree multivariate polynomials (extends Chernoff).

23

Conclusions and future Conclusions and future researchresearch

•Theme:Theme: Communication over Communication over WWpp not fully understood. not fully understood. Gain understanding of certain relaxations of Gain understanding of certain relaxations of WWpp..

•SeenSeen::•Oblivious channels Oblivious channels WW** WWpp..•Allows rate Allows rate 1-H(p)1-H(p)..

•Other relaxationsOther relaxations::•““Online adversaries”.Online adversaries”.•Adversaries restricted to changing certain Adversaries restricted to changing certain

locations (unknown to X and Y).locations (unknown to X and Y).