Distributed Nuclear Norm Minimization for Matrix Completion
description
Transcript of Distributed Nuclear Norm Minimization for Matrix Completion
![Page 1: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/1.jpg)
1
Distributed Nuclear Norm Minimization for Matrix Completion
Morteza Mardani, Gonzalo Mateos and Georgios Giannakis
ECE Department, University of Minnesota
Acknowledgments: MURI (AFOSR FA9550-10-1-0567) grant
Cesme, TurkeyJune 19, 2012
![Page 2: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/2.jpg)
2
Learning from “Big Data” `Data are widely available, what is scarce is the ability to extract wisdom from them’
Hal Varian, Google’s chief economist
BIG Fast
Productive
Revealing
Ubiquitous
Smart
K. Cukier, ``Harnessing the data deluge,'' Nov. 2011.
Messy
![Page 3: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/3.jpg)
33
Context Imputation of network data
Goal: Given few incomplete rows per agent, impute missing entries in a distributed fashion by leveraging low-rank of the data matrix.
Preference modeling
Network cartography
Smart metering
![Page 4: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/4.jpg)
(as) has low rank Goal: denoise observed entries, impute missing ones
44
Low-rank matrix completion Consider matrix , set
Given incomplete (noisy) data
Nuclear-norm minimization [Fazel’02],[Candes-Recht’09]
Sampling operator
Noisy Noise-free
s.t.
?
?
??
??
? ??
?
?
?
![Page 5: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/5.jpg)
55
Problem statement
Goal: Given per node and single-hop exchanges, find
n
Network: undirected, connected graph
(P1)
?
?
??
?
??
?
?
?
Challenges Nuclear norm is not separable Global optimization variable
![Page 6: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/6.jpg)
66
Separable regularization Key result [Recht et al’11]
New formulation equivalent to (P1)
(P2)
Proposition 1. If stationary pt. of (P2) and , then is a global optimum of (P1).
Nonconvex; reduces complexity:
Lxρ≥rank[X]
![Page 7: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/7.jpg)
77
Distributed estimator
Network connectivity (P2) (P3)
(P3)
Consensus with neighboring nodes
Alternating-directions method of multipliers (ADMM) solver Method [Glowinski-Marrocco’75], [Gabay-Mercier’76] Learning over networks [Schizas et al’07]
Primal variables per agent :
Message passing:n
![Page 8: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/8.jpg)
88
Distributed iterations
![Page 9: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/9.jpg)
99
Highly parallelizable with simple recursions Unconstrained QPs per agent No SVD per iteration
Low overhead for message exchanges is and is small Comm. cost independent of network size
Recap:(P1) (P2) (P3)
CentralizedConvex
Sep. regul.Nonconvex
ConsensusNonconvex
Stationary (P3) Stationary (P2) Global (P1)
Attractive features
![Page 10: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/10.jpg)
1010
Optimality
Proposition 2. If converges to and , then: i) ii) is the global optimum of (P1).
ADMM can converge even for non-convex problems [Boyd et al’11]
Simple distributed algorithm for optimal matrix imputation Centralized performance guarantees e.g., [Candes-Recht’09] carry over
![Page 11: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/11.jpg)
1111
Synthetic data Random network topology
N=20, L=66, T=66
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
Data , ,
![Page 12: Distributed Nuclear Norm Minimization for Matrix Completion](https://reader035.fdocuments.in/reader035/viewer/2022062501/56816102550346895dd04439/html5/thumbnails/12.jpg)
1212
Real data
Abilene network data (Aug 18-22,2011) End-to-end latency matrix N=9, L=T=N 80% missing data
Network distance prediction [Liau et al’12]
Data: http://internet2.edu/observatory/archive/data-collections.html
Relative error: 10%