Post on 15-Jan-2016
description
Fighting Byzantine Adversaries in Networks: Network Error-Correcting Codes
Michelle Effros
Michael Langberg
Tracey Ho
Sachin Katti
Muriel Médard
Dina Katabi
Sidharth Jaggi
Obligatory Example/Historys
t1 t2
b1 b2
b2
b2
b1
b1 b1
b1 b1
b1 (b1,b2)
b1+b2
b1+b2b1+b2
(b1,b2)
[ACLY00] [ACLY00] Characterization Non-constructive
[LYC03], [KM02] Constructive (linear) Exp-time design
[JCJ03], [SET03] Poly-time design Centralized design
[HKMKE03], [JCJ03] Decentralized design
EVER
BETTER
.
.
.
C=2
[This work] All the above, plus security
Tons of work
[SET03] Gap provably exists
Multicast
Wired
Wireless
Simplifying assumptions• All links unit capacity
•(1 packet/transmission)• Acyclic network
Network = Hypergraph
ALL of Alice’sinformationdecodableEXACTLYbyEACH Bob
Network Model
[GDPHE04],[LME04] – No intereference
Multicast Networks
Webcasting
P2P networks
Sensor networks
Multicast Network Model
ALL of Alice’sinformationdecodableEXACTLYbyEACH Bob
3
2
2
Upper bound for multicast capacity C,
C ≤ min{Ci}
[ACLY00] With mixing, C = min{Ci} achievable!
[LCY02],[KM01],[JCJ03],[HKMKE03] Simple (linear) distributed codes suffice!
Mixing
)2(1,0)...( 21mm
m Fxbbb
2x
kx
b1b2 bmx
1x
kk xxx ...2211
β1
β2
βk
F(2m)-linear network[KM01]
Source:- Group together m bits,
Every node:- Perform linear combinations over finite field F(2m)
Generalization: The X arelength n vectors over F(2m)
X1
X2
Xk
kk XXX ...2211
Problem!
Eavesdropped links
Attacked links
Corrupted links
Setup
1. Scheme A B C2. Network
C3. Message A C4. Code C5. Bad links C6. Coin A7. Transmit B C8. Decode B
Eureka
Eavesdropped links ZI
Attacked links ZO
Who knows what
Stage
Privacy
Result(s)First codes Optimal rates (C-2ZO,C-ZO) Poly-time Distributed Unknown topology End-to-end Rateless Information theoretically secure Information theoretically private Wired/wireless
[HLKMEK04],[JLHE05],[CY06],[CJL06],[GP06]
Error Correcting Codes
Y=TX+E
Generator matrix
Low-weightvector
YX
(Reed-Solomon Code)
1
0
0
0
0
c
T
E
Error Correcting Codes
X
TY
TZ
Z
Y=TX+E=TX+TZZ
Networktransform matrices
Low-weightvector
Unknown
When stuck…“ε-rate secret uncorrupted channels”
•Useful abstraction/ building block
•Existing model ([GP06],[CJL06])
•We improve!
Example
1X
2X
3X
Z
ZX 111
ZX 222
ZX 333 C=3
ZO=1
ZβXαY
ZβXαY
ZβXαY
33 33
22 22
11 11
n-length vectors
3n known 4n unknown
scalars
4n+6 unknownX3=X1+X2
non-linear
R = C - Zo
2 3 1
6 secret hashes of X
4n+6 known4n known
)1()1(0)1(
)1()1(0)1(
)1()1(0)1(
333
222
111
yzx
yzx
yzx
)2()2(22)2(
)2()2(1)2(
)2()2(1)2(
3333
2222
1111
yzx
yzx
yzx
3
2
1
)1(
z
'
'
'
)2(
2 3
2
1
3
2
1
z
'
'
'
3
2
1
)3()3(33)3(
)3()3(22)3(
)3()3(1)3(
3333
2222
1111
yzx
yzx
yzx
'
'
'
)3(
3
2
3
2
1
3
2
1
zZ''βXαY
Z''βXαY
Z''βXαY
33 33
22 22
11 11
Redundancy addedat source 'β,'β,'βααα 3213,2,1,Solve for
Example
1X
2X
3X
Z
ZX 111
ZX 222
ZX 333 C=3
ZO=1
X3=X1+X2
6 secret hashes of X
4n+6 known4n+6 unknown
3
2
1
2
1
333
22
11
Y
Y
Y
Z'
X
X
'βαα
'βα0
'β0α
Z''βXαY
Z''βXαY
Z''βXαY
33 33
22 22
11 11
Invertible with high probability
3
2
1
3
2
1
)1(
'
'
'
z
Z=(0 z(2) z(3)… z(n))
3
2
1
3
2
1
0
'
'
'
3
2
1
2
1
33
2
1
Y
Y
Y
Z'
X
X
0αα
0α0
00α
Thm 1,ProofTheorem 1: Rate C-ZO-ε achievable with ZI={E},ε-rate secret uncorrupted channelImproves on [GP06/Avalanche] (Decentralized) and [CJL06] (optimal)
R = C - Zo
01...0000)()()1(
0...1...00)()()1(
0...10000)()()1( 111
nxjxx
nxjxx
nxjxx
X
RRR
iii
CxC identitymatrix
n>>C
[HKMKE03] IXX 1
T
packets
TTXTY 11
Thm 1,ProofTheorem 1: Rate C-ZO-ε achievable with ZI={E},ε-rate secret uncorrupted channel
TTXTY 11
LZ1
T
TZ
LTTZTTXTY ZZ 111 '
LTTT
ZTTXY
Z
Z
'
111
LTTT
LXTLXTZTTXY
Z
ZZZ
'
11111
LTTT
LXZTXTY
Z
Z
'
)(' 1111
Crrr ...21
nnnC
C
C
rrr
rrr
rrr
P
21
21
21
222
PXH 1CxC matrix
HTPYS '1 )('))('( 1111 PXTPLXZTXT Z
PLXZTZ )( 11
Q
XSTY
11 '
Invertible w.h.p.
Thm 2Theorem 2: Rate C-2ZO-ε achievable with ZI={E}
Example revisited
1X
2X
3X
Z
ZX 111
ZX 222
ZX 333
ZβXαY
ZβXαY
ZβXαY
33 33
22 22
11 11
X3=X1+X2
n more constraints added on X
3
2
1
3
2
1
)1(
'
'
'
z
Z=(0 z(2) z(3)… z(n))
3
2
1
3
2
1
0
'
'
'
DX=0
Z=(0 0 0… 0)
R = C – Zo - redundancyR = C – Zo
2 3 11 3 1 1
R = C – 2Zo
Tight (ECC, [CY06])
nZO
nZO
Thm 2,“Proof”Theorem 2: Rate C-2ZO-ε achievable with ZI={E}
R = C - 2Zo
01...0000)()()1(
0...1...00)()()1(
0...10000)()()1( 111
nxjxx
nxjxx
nxjxx
X
ZoCZoCZoC
iii
01 DX
nZO extra constraints
D chosen uniformly at random,known to Alice, Bob and Calvin
)(' 1111 LXZTXTY Z
Theorem 2: Rate C-2ZO-ε achievable with ZI={E}
Disjoint
?
T’’
''''' 11 ZTXTY
non-linearlinear
0DXInvertible
Basis changeMay not be
0'
''
XD
ITIZ
I
D of appropriate dimensions crucial
Thm 2,“Proof”
Thm 3,ProofTheorem 3: Rate C-ZO-ε achievable, with ZI+2ZO<C
ZI<C-2ZO
Using algorithm 2 for small header, can transmit secret, correct information…
… which can be used foralgorithm 1 decoding!
Algorithm 2 rate
Eavesdropping rate
ZI<R Information-theoretic Privacy
Theorem 4, etc:
SummaryRate Conditions
Thm 1 C-ZO Secret
Thm 2 C-2ZO Omniscient
Thm 3 C-ZO Limited
Optimal rates Poly-timeDistributedUnknown topologyEnd-to-endRatelessInformation theoretically secure/privateWired/wireless
Backup slides
Network Coding “Justification”
R. Ahlswede, N. Cai, S.-Y. R. Li and R. W. Yeung,"Network information flow," IEEE Trans. on Information
Theory, vol. 46, pp. 1204-1216, 2000.
http://tesla.csl.uiuc.edu/~koetter/NWC/Bibliography.html ≈ 200 papers in 3 years
NetCod Workshops, DIMACS working group, ISIT 2005 - 4+ sessions, tutorials, …
Several patents, theses…
“The core notion of network coding is to allow and encourage mixing of data at intermediate network nodes.”
(Network Coding homepage)
But what IS Network Coding?
Point-to-point flows
)(maxmin)(
cutsizeCflowtscut
C
1P
2P
CP
Min-cut Max-flow (Menger’s) Theorem [M27]
Ford-Fulkerson Algorithm [FF62]
s
t
Multicasting
Webcasting
P2P networks
Sensor networks
s1
t1
t2
t|T|
Network
s|S|
Justifications revisited - I
s
t1 t2
b1 b2
b2
b2
b1
b1 ?b1
b1 b1
b1 (b1,b2)
b1+b2
b1+b2b1+b2
(b1,b2)[ACLY00]
Throughput
Gap Without Coding
. . .
. . .
h2
hh2
Coding capacity = h Routing capacity≤2
[JSCEEJT05]
s
Multicasting
Upper bound for multicast capacity C,
C ≤ min{Ci}
s
t1
t2
t|T|
C|T|
C1
C2
Network
[ACLY00] - achievable!
[LYC02] - linear codes suffice!!
[KM01] - “finite field” linear codes suffice!!!
Multicasting
)2(1,0)...( 21mm
m Fbbb
2
k
b1b2 bm
1
kk ...2211
β1
β2
βk
F(2m)-linear network[KM01]
Source:- Group together `m’ bits,
Every node:- Perform linear combinations over finite field F(2m)
Multicasting
Upper bound for multicast capacity C,
C ≤ min{Ci}
s
t1
t2
t|T|
C|T|
C1
C2
Network
[ACLY00] - achievable!
[LYC02] - linear codes suffice!!
[KM01] - “finite field” linear codes suffice!!!
[JCJ03],[SET03] - polynomial time code design!!!!
Thms: Deterministic Codes
For m ≥ log(|T|), exists an F(2m)-linear network which can be designed in O(|E||T|C(C+|T|)) time.
[JCJ03],[SET03]
Exist networks for which minimum m≈0.5(log(|T|))
[JCJ03],[LL03]
Justifications revisited - II
s
t1 t2
One link breaks
Robustness/Distributeddesign
Justifications revisited - II
s
t1 t2
b1 b2
b2
b2
b1
b1
(b1,b2)
b1+b2
Robustness/Distributeddesign
(b1,b2)
b1+2b2
(Finite field arithmetic)b1+b2 b1+b2
b1+2b2
Thm: Random Robust Codes
s
t1
t2
t|T|
C|T|
C1
C2
Original Network
C = min{Ci}
Thm: Random Robust Codes
s
t1
t2
t|T|
C|T|'
C1'
C2'
Faulty Network
C' = min{Ci'}
If value of C' known to s,same code can achieve C' rate!
(interior nodes oblivious)
Thm: Random Robust Codesm sufficiently large, rate R<C
Choose random [ß] at each node
Probability over [ß] thatcode works
>1-|E||T|2-m(C-R)+|V|
[JCJ03] [HKMKE03]
(different notions of linearity)
Decentralized design
b1b2 bm
b’1b’2 b’m
b’’1b’’2 b’’m
’
’’
Much “sparser” linear operations
(O(m) instead of O(m2)) [JCE06]
Vs. prob of error - necessary evil?
Zero-error Decentralized CodesNo a priori network topological
information available - informationcan only be percolated down links
Desired - zero-error code design
One additional resource - eachnode vi has a unique ID number i(GPS coordinates/IP address/…)
Need to use yet other types of linear codes[JHE06?]
Inter-relationships between notions of linearity
C
B
M
M Multicast G General
Global Local I/O ≠ Local I/O =
a Acyclic
A AlgebraicB BlockC Convolutional
Does not exist
Є epsilon rate loss
G
a
GЄ
A Ma
Ma
Ma
G?
M
G
a
G
Ma G
G
[JEHM04]
Justifications revisited - III
s
t1 t2
Security
Evil adversary hiding in networkeavesdropping,
injecting false information[JLHE05],[JLHKM06?]
Greater throughputRobust against random errors...
Aha!Network Coding!!!
??
?
Xavier
Yvonne1
Zorba
???
Yvonne|T|
???
.
.
.
Setup
1. Scheme X Y Z2. Network Z3. Message X Z4. Code Z5. Bad links Z6. Coin X7. Transmit Y Z8. Decode Y
Eureka
WiredWireless (packet losses, fading)
Eavesdropped links ZI
Attacked links ZO
Who knows what
Stage
Xavier
Yvonne1
?
Zorba
??
Zorba sees MI links ZI, controls MO links ZO pI=MI/C, pO=MO/C
Xavier and Yvonnes share no resources (private key, randomness)
Zorba computationally unbounded; Xavier and Yvonnes -- “simple” computations
Setup
Zorba knows protocols and already knows almost all of Xavier’s message (except Xavier’s private coin tosses)
Goal: Transmit at “high” rate and w.h.p. decode correctly
Zorba (hidden) knows network; Xavier and Yvonnes don’t
C
MO
Yvonne|T|
??
?
Distributed design (interior nodes oblivious/overlay to network coding)
Background
Noisy channel models (Shannon,…)Binary Symmetric Channel
p (“Noise parameter”)0
1
1
C
(C
apac
ity)
0 1
H(p)
0.5
Background
Noisy channel models (Shannon,…) Binary Symmetric Channel Binary Erasure Channel
p (“Noise parameter”)0
1
1
C
(C
apac
ity)
0 E
1-p
0.5
Background
Adversarial channel models “Limited-flip” adversary, pI=1 (Hamming,Gilbert-Varshanov,McEliece et al…)
Large alphabets (Fq instead of F2)
Shared randomness, cryptographic assumptions…
pO (“Noise parameter”)0
1
1
C
(C
apac
ity)
0 1
0.5
pO (“Noise parameter”)
0
1
1
C
(C
apac
ity)
Upper bounds
0.5
0.5
1-pO
pO (“Noise parameter”)
0
1
1
C
(C
apac
ity)
Upper bounds
0.5
0.5
??
?
0
pI=pO (“Noise parameter” = “Knowledge parameter”)
0
1
1
C
(C
apac
ity)
Unicast – Results [JLHE05]
0.5
0.5
pO (“Noise parameter”)
0
1
1
C
(C
apac
ity)
Full knowledge [Folklore]
0.5
(“Knowledge parameter” pI=1)
t1
t|T|
S
Multicast Networks [HKMKE03]
ys(j)=Txs(j)
x
y1
β1
βi
βh
y|T|
xb(i)
01...0000),(),()1,(
0...1...00),(),()1,(
0...10000),1(),1()1,1(
nhxjhxhx
nixjixix
nxjxx
xb(i)
xs(j)
xb(1)
xb(h)
Rate h=C-MO
Block
Slice
hxh identitymatrix
x’b(i)
h<<n
T
xs(j)=T-1ys(j)
pO
0
1
1
C
(N
orm
aliz
ed b
y h)
0.5
0.5
Multicast Networks
R1
R|T|
S
S’|Z|
S’2
S’1
Observation 1: Can treatadversaries as new sources
Multicast Networks
)(']T' T[)('
)( 1 jyjx
jxs
s
s
01...0000),(),()1,(
0...1...00),(),()1,(
0...10000),1(),1()1,1(
nhxjhxhx
nixjixix
nxjxx
y’s(j)=Txs(j)+T’x’s(j)
SS
Supersource
Observation 2: w.h.p. over network code design, {TxS(j)} and {T’x’S(j)} do not intersect (robust codes…).
Corrupted Unknown
Multicast Networksy’s(j)=Txs(j)+T’x’s(j)
ε redundancy
xs(2)+xs(5)-xs(3)=0
ys(2)+ys(5)-ys(3)=vector in {T’x’s(j)}
{T’x’s(j)}{Txs(j)}
xs(3)+2xs(9)-5xs(1)=0
ys(3)+2ys(9)-5ys(1)=another vector in {T’x’s(j)}
Multicast Networksy’s(j)=Txs(j)+T’x’s(j)
ε redundancy
{T’x’s(j)}{Txs(j)}
Repeat MO timesDiscover {T’x’s(j)}“Zero out” {T’x’s(j)}
when you have eliminated the impossible, whatever remains, however improbable, must be the truth
Estimate T (redundant xs(j) known)
Linear algebra Decode
Multicast Networksy’s(j)=Txs(j)+T’x’s(j)
xs(2)+xs(5)-xs(3)=0
ys(2)+ys(5)-ys(3)=vector in {T’x’s(j)}
x’s(2)+x’s(5)-x’s(3)=0
ys(2)+ys(5)-ys(3)=0
Scheme 1(a)“ε-rate secret uncorrupted channels”
Useful abstraction
Scheme 1(b)“sub-header based scheme”
Works… kind of…
… for “many” networks
Scheme 2“distributed network error-correcting code”
(Knowledge parameter pI=1)
[CY06] – bounds, high complexity construction
[JHLMK06?] – tight, poly-time construction
pO (“Noise parameter”)0
1
1
C
(C
apac
ity)
0.5
Scheme 2“distributed network error-correcting code”
pO
pO
y’s(j)=Txs(j)+T’x’s(j)error vector
1-2pO
Scheme 2“distributed network error-correcting code”
y’s(j)=Txs(j)+T’x’s(j)
01...0000),(),()1,(
0...1...00),(),()1,(
0...10000),1(),1()1,1(
nhxjhxhx
nixjixix
nxjxx
Scheme 2“distributed network error-correcting code”
y’s(j)=T’’xs(j)+T’x’s(j)
01...0000),(),()1,(
0...1...00),(),()1,(
0...10000),1(),1()1,1(
nhxjhxhx
nixjixix
nxjxx
e
e
e’
Scheme 2“distributed network error-correcting code”
y’s(j)=T’’xs(j)+T’x’s(j)
e
e
e’
Linear algebra
Scheme 3“non-omniscient adversary”
y’s(j)=T’’xs(j)+T’x’s(j)
MI+2MO<C
MI<C-2MO Scheme 2 rate
Zorba’s observations
Using Scheme 2 as small header, can transmit secret, correct information…
… which can be used forScheme 1(a) decoding!
Variations - FeedbackC
p
0
1
1
Variations – Know thy enemyC
p
0
1
1C
p
0
1
1
Variations – Random NoiseC
p
0
CN
1
SEPARATION