Distributed Inference: High Dimensional Consensus
-
Upload
griffith-quinn -
Category
Documents
-
view
27 -
download
0
description
Transcript of Distributed Inference: High Dimensional Consensus
Carnegie Mellon
Distributed Inference: High Dimensional Consensus
José M. F. MouraWork with: Usman A. Khan (Upenn), Soummya Kar
(Princeton)
The Australian National UniversityRSISE Systems and Control Series Seminar
Canberra, Australia, July 30, 2010
Acknowledgements: AFOSR grant # FA95501010291, NSF grant # CCF1011903, ONR MURI N000140710747
Carnegie Mellon
2
Outline
Motivation for networked systems and distributed algorithms
Identify main characteristics of networked systems and distributed algorithms
Consensus algorithms and emerging behavior Example: Localization Conclusions
Carnegie Mellon
Motivation Networked systems: agents, sensors
Applications: inference (detection, estimation, filtering, …) Distributed algorithms:
Consensus:
More general algorithms – High dimensional consensus
Realistic conditions: Randomness: infrastructure (link failures), random protocols (gossip),
communication noise Quantization effects Measurement updates
Issues: convergence – design topology to speed convergence; prove Applications
Localization
Carnegie Mellon
4
Example
1 2
3
W12 =W21 =0
W =
2
4W11 0 W13
0 W22 W23
W31 W32 W33
3
5
• If link not available,• W is symmetric, sparse• W reflects the topology of network• Neighborhoods:
Wi j =Wj i =0
3 = f1;2g; 1 = f3g; 2 = f3g
xn(i +1) =Wnnxn(i) +P
l2 nWnlxl(i)
x(i) = [x1(i) ¢¢¢xN (i)]T
x(i +1) =Wx(i)
In matrix form, consensus is:
Consensus is linear & iterative – issues: convergence and rate of convergence
Networked Systems: Consensus
Carnegie Mellon
Consensus: Optimization
Consensus
Convergence
Limit
Spectral condition
1 2
3
Kar & Moura, Transactions Signal Processing, vol. 56, no. 6, June 2008
Carnegie Mellon
Topology Design
Speed convergence by making small
Choose where nonzero entries of W are and the actual values of the nonzero entries of W
W =
2
4W11 0 W13
0 W22 W23
W31 W32 W33
3
51 2
3
1 2
3
Kar & Moura, Transactions Signal Processing, vol. 56, no. 6, June 2008
Carnegie Mellon
Topology Design
Design Laplacean to minimize
Equal weights : weight α (Xiao and Boyd, CDC, Dec 2003)
Graph design:
subject to constraints, e.g., number of edges M, structure of graph
Kar & Moura, Transactions Signal Processing, vol. 56, no. 6, June 2008
Carnegie Mellon
Topology Design
Nonrandom topology: (topology static or fixed) Class 1: Noiseless communication
Class 2: Noisy communication Random topology: Class 3 links may fail intermittently
Random topology with communication costs and budget constraint: Class 4 Communication in link (i,j) has cost Link (i,j) fails with probability Average comm. network budget constraint per iteration
x(i +1) =Wx(i)
x(i +1) =Wx(i) +n(i)
random
Kar & Moura, Transactions Signal Processing, vol. 56, no. 6, June 2008
Carnegie Mellon
9
Topology Design – Class 1: Ramanujan (LPS)
Fig.1. A non-bipartite LPS Ramanujan graph with degree k = 6, and number of vertices N = 62 (Figure constructed using software Pajek)
Lubotzky, Phillips, Sarnak (LPS) (1988) and Margulis (1988)
(22/12/1887 – 26/4/1920)
Kar & Moura, Transactions Signal Processing, vol. 56, no. 6, June 2008
Carnegie Mellon
10
Comparison Studies
Convergencespeed, Sc, of P ne (i) ! Pe
Performance Metric
LPS Ramanujan(We use a non-bipartite Ramanujan graph construction from LPS and call it LPS-II.)
Regular Ring Lattice (RRL)Highly-structured regular graphswith nearest-neighbor connectivity
Erdos-Renyi (ER)Random networks
Watts-Strogatz (WS-I)Small-world networks usingWatts-Strogatz construction
vs vs vs
Carnegie Mellon
LPS Ramanujan vs Regular Ring Lattice (RRL)
m1 = ¡ m0 = ¹ 1; K =¾2I ; SNR = 10log³¹ 2
¾2
´= ¡ 25 db
1000 1200 1400 1600 1800 2000 2200200
300
400
500
600
700
800
900
N
Sc(L
PS¡II)
Sc(R
ingLatt
ice)
Fig.2. Ratio of convergence speed of LPS-II graphs to RRL graphs for different values of N and k = 18.
Regular graph
Ratio
spe
ed c
onve
rgen
ce
Ramanujan
Kar & Moura, Transactions Signal Processing, vol. 56, no. 6, June 2008
Carnegie Mellon
12
LPS-II vs Erdös-Renýi (ER)
The top blue line corresponds to the LPS-II graphs. The LPS-II graphs perform much better than the best ER graphs.
The relative performance of the LPS-II graphs over the ER graphs increases steadily with increasing N.
m1 = ¡ m0 = ¹ 1; K =¾2I ; SNR = 10log³¹ 2
¾2
´= ¡ 25 db
1000 1500 2000 2500 3000 35000.1
0.12
0.14
0.16
0.18
0.2
0.22
N
Sc
LPS-IIER(max)ER(avg)ER(min)
Fig.3. Comparison of convergence speed between LPS-II graphs and ER graphs for varying N and kavg = 18.
Kar & Moura, Transactions Signal Processing, vol. 56, no. 6, June 2008
Carnegie Mellon
13
LPS-II vs Watts-Strogatz (WS-I)
0.2 0.4 0.6 0.8 10.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
pw
Sc
LPS-IIWS-I(max)WS-I(avg)WS-I(min)
SmallWorld
Random
Fig. 4. Study of convergence speed Sc of WS-I graphs with N = 6038 and kavg = 18. The top blue line denotes the LPS-II graph with N = 6038 and k = 18.
Kar & Moura, Transactions Signal Processing, vol. 56, no. 6, June 2008
Carnegie Mellon
Topology Design–Class 4: Communication Costs Communication in link (i,j) has cost Link (i,j) fails with probability Average comm. network budget constraint per iteration
Convex optimization (SDP):
Kar & Moura, Transactions Signal Processing, vol. 56:7, July 2008
Carnegie Mellon
15
Random Topology w/ Comm. Cost
Fig. 4. Per step convergence gain Sg: N = 80 and |E| = 9N=720
Kar & Moura, Transactions Signal Processing, vol. 56:7, July 2008
Carnegie Mellon
High Dimensional Consensus
LOCAL INTERACTIONS: The local updates are given as
GLOBAL BEHAVIOR: Under whatconditions does HDC converge:
for some appropriate function, wl
limt! 1
xl(t+1) =wl (x(0);u(0)) ; (1)
16Khan, Kar, Moura, ICASSP ‘09, ‘10, ASILOMAR ‘09, IEEE TSP ‘10.
uk(t +1) = uk(t); k 2 · ;xl ;j (t +1) = f l
¡xK (l);j (t)
¢+gl
¡uK · (l);j (t)
¢; l 2 ;
(1)xl(t +1) =X
j 2K (l)[ f lg
pl j xj (t) +X
k2K · (l)
blkuk(t): (1)
Carnegie Mellon
Distributed Localization
m = 2-D plane
Localize M sensors with unknown locations in m-dimensional Euclidean space [1]
Minimal number, n=m+1 , of anchors with known locations
Sensors only communicate in a neighborhood
Only local distances in the neighborhood are known to the sensor
There is no central fusion center[1] Khan, Kar, Moura, “Distributed Sensor Localization in Random Environments using Minimal Number of Anchor Nodes,” IEEE Tr. on Sign. Pr., 57(5), pp. 2000-2016, May 2009.
Carnegie Mellon
Distributed Sensor Localization
Assumptions Sensors lie in convex hull of anchors Anchors not on a hyper-plane Sensors find m+1 neighbors so they lie in their
convex hull Only local distances available
Distributed localization (DILOC) algorithm Sensor updates position estimate as convex l.c. of n=m+1 neighbors Weights of l.c. are barycentric coordinates Barycentric coordinates: ratio of generalized volumes Barycentric coordinates: Cayley-Menger determinants (local distances)
xl(t +1) = pl lxl(t) +X
j 2K (l)
pl j xj (t) +X
k2K · (l)
blkuk(0):
(TRIANGULATION)
Carnegie Mellon
Barycentric Coord. & Cayley-Menger Det.
pl3 =A1l2
A123
1
32
l
Barycentric coordinates: Example 2D:
Cayley-Menger determinants:
plk =Af lg[ £ l nf kg
A£ l
A2· =
1sm+1
¯¯¯¯
0 1Tm+11m+1 ¡
¯¯¯¯;
where ¡ = fd2lj g; l; j 2 · ; is the matrix of squared distances, dl j , among them+1 points in · and
sm =2m(m!)2
(¡ 1)m+1 ; m= f0;1;2;: : :g:
Its ¯rst few terms are ¡ 1;2;¡ 16;288;¡ 9216;460800;: : : :
Carnegie Mellon
Set-up phase: Triangulation
Test to find a triangulation set, Convex hull inclusion test: based on the following observation.
The test becomes
A l12+A l13+A l23 =A123 A l12+A l13+A l23 >A123
32 32
1
l
1
l
l 2 C(£ l ); ifX
k2£ l
A£ l [ f lgnf kg = A£ l ;
l =2 C(£ l); ifX
k2£ l
A · [ f lgnf kg >A£ l
Carnegie Mellon
Distributed Localization Distributed localization algorithm (DILOC)
K anchors and M sensors (K+M=N) in m dimensions:
Matrix form:
xl(t +1) = pl lxl(t) +X
j 2K (l)
pl j xj (t) +X
k2K · (l)
blkuk(0):
U =£u1(t) ¢¢¢um(t)
¤; U : K £ m
X (t) =£x1(t) ¢¢¢xm(t)
¤; X (t) : M £ m
·U
X (t+1)
¸=
·I 0B P
¸ ·U
X (t)
¸
limt! 1
X (t) = [I ¡ P ]¡ 1BU
Carnegie Mellon
Distributed Localization: DILOC
DILOC:
Assume:
·U
X (t+1)
¸=
·I 0B P
¸ ·U
X (t)
¸
{𝚼§ k2 l plk +§ k2· l blk = 1
(Triangulation)
(Barycentric Coordinates)
limt! 1
X (t) = [I ¡ P ]¡ 1BU
Theorem [Convergence]: Under above assumptions:1. The underlying Markov chain with transition probability is
absorbing.2. DILOC converges to the exact sensor coordinates:
𝚼
Carnegie Mellon
23
Distributed Localization: Simulations N=7 node network in 2-d plane
M= 4 sensors, K = m+1 = 3 anchors
M = 497 sensors
Carnegie Mellon
Convergence of DILOC
Theorem [Convergence]:
Connected on average
Random network
Persistence cond.
Noisy communication vj (t)
Distributed distance localization algorithm converges
Errors in intersensor distances P¡¹dt
¢and B
¡¹dt¢
1t t t d d d,
Khan, Kar, and Moura, “DILAND: An Algorithm for Distributed Sensor Localiz. with Noisy Distance Meas.,” IEEE Tr. Signal Pr., 58:3, 1940-1947, March 2010
Carnegie Mellon
Proof of Theorem Proof: Cannot use standard stochastic approx. techniques because
function of past measurements, non Markovian
Study path behavior of error process wrt idealized update
Define error process wrt idealized update:
Dynamics of error process:
Error goes to zero:
Carnegie Mellon
Conclusion
High Dimensional Consensus Optimization: Topology design Distributed localization (DILOC):
Linear iterative Local communications Barycentric coordinates Cayley-Menger determinants
Convergence: Deterministic networks (protocols): standard Markov chain
arguments Random networks: structural (link) failures, noisy comm, quantized
data − standard stochastic approximation algorithms not sufficient to prove convergence
Carnegie Mellon
Bibliography Soummya Kar, Saeed Aldosari, and José M. F. Moura, “
Topology for Distributed Inference on Graphs,” IEEE Transactions on Signal Processing, volume 56 number 6, pp. 2609-2613, June 2008.
Soummya Kar and José M. F. Moura, “Sensor Networks with Random Links: Topology Design for Distributed Consensus,” IEEE Transactions on Signal Processing, 56:7, pp. 3315-3326, July 2008.
U. A. Khan, S. Kar, and J. M. F. Moura, “Distributed Sensor Localization in Random Environments using Minimal Number of Anchor Nodes,” IEEE Transactions on Signal Processing, 57: 5, pp. 2000-2016, May 2009; DOI:10.1109/TSP.2009.2014812.
U. A. Khan, S. Kar, and J. M. F. Moura, “DILAND: An Algorithm for Distributed Sensor Localization with Noisy Distance Measurements,” IEEE Transactions on Signal Processing, Vol. 58:3, pp.:1940-1947, March 2010.
U. A. Khan, S. Kar, and J. M. F. Moura, “Higher dimensional consensus: Learning in large-scale Networks,” IEEE Transactions on Signal Processing, Vol. 58:5, pp. 2836 -2849, May 2010.
Carnegie Mellon
The ENDThanks