UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. ·...
Transcript of UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. ·...
![Page 1: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/1.jpg)
Iterative Projection Methods
for noisy and corrupted systems of linear equations
Deanna Needell
February 1, 2018
Mathematics
UCLA
joint with Jamie Haddock and Jesus De Loera
https://arxiv.org/abs/1605.01418 and forthcoming articles
1
![Page 2: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/2.jpg)
Setup
We are interested in solving highly overdetermined systems of equations,
Ax = b, where A ∈ Rm×n, b ∈ Rm and m >> n. Rows are denoted aTi .
2
![Page 3: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/3.jpg)
Projection Methods
If {x ∈ Rn : Ax = b} is nonempty, these methods construct an
approximation to an element:
1. Randomized Kaczmarz Method
2. Motzkin’s Method(s)
3. Sampling Kaczmarz-Motzkin Methods (SKM)
3
![Page 4: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/4.jpg)
Randomized Kaczmarz Method
Given x0 ∈ Rn:
1. Choose ik ∈ [m] with probability‖aik ‖
2
‖A‖2F
.
2. Define xk := xk−1 +bik−a
Tikxk−1
||aik ||2aik .
3. Repeat.
4
![Page 5: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/5.jpg)
Kaczmarz Method
x0
5
![Page 6: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/6.jpg)
Kaczmarz Method
x0
x1
5
![Page 7: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/7.jpg)
Kaczmarz Method
x0
x1
x2
5
![Page 8: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/8.jpg)
Kaczmarz Method
x0
x1
x2
x3
5
![Page 9: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/9.jpg)
Convergence Rate
Theorem (Strohmer - Vershynin 2009)
Let x be the solution to the consistent system of linear equations Ax = b.
Then the Random Kaczmarz method converges to x linearly in
expectation:
E||xk − x||22 ≤(
1− 1
||A||2F ||A−1||22
)k
||x0 − x||22.
6
![Page 10: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/10.jpg)
Motzkin’s Relaxation Method(s)
Given x0 ∈ Rn:
1. If xk is feasible, stop.
2. Choose ik ∈ [m] as ik := argmaxi∈[m]
|aTi xk−1 − bi |.
3. Define xk := xk−1 +bik−a
Tikxk−1
||aik ||2aik .
4. Repeat.
7
![Page 11: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/11.jpg)
Motzkin’s Method
x0
8
![Page 12: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/12.jpg)
Motzkin’s Method
x0
x1
8
![Page 13: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/13.jpg)
Motzkin’s Method
x0
x1x2
8
![Page 14: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/14.jpg)
Convergence Rate
Theorem (Agmon 1954)
For a consistent, normalized system, ‖ai‖ = 1 for all i = 1, ...,m,
Motzkin’s method converges linearly to the solution x:
‖xk − x‖2 ≤(
1− 1
m‖A−1‖2
)k
‖x0 − x‖2.
9
![Page 15: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/15.jpg)
Our Hybrid Method (SKM)
Given x0 ∈ Rn:
1. Choose τk ⊂ [m] to be a sample of size β constraints chosen
uniformly at random from among the rows of A.
2. From among these β rows, choose ik := argmaxi∈τk
|aTi xk−1 − bi |.
3. Define xk := xk−1 +bik−a
Tikxk−1
||aik ||2aik .
4. Repeat.
10
![Page 16: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/16.jpg)
SKM
x0
11
![Page 17: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/17.jpg)
SKM
x0
x1
11
![Page 18: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/18.jpg)
SKM
x0
x1
x2
11
![Page 19: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/19.jpg)
SKM Method Convergence Rate
Theorem (De Loera - Haddock - N. 2017)
For a consistent, normalized system the SKM method with samples of
size β converges to the solution x at least linearly in expectation: If sk−1
is the number of constraints satisfied by xk−1 and
Vk−1 = max{m − sk−1,m − β + 1} then
E‖xk − x‖2 ≤(
1− 1
Vk−1‖A−1‖2
)‖x0 − x‖2
≤(
1− 1
m‖A−1‖2
)k
‖x0 − x‖2.
12
![Page 20: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/20.jpg)
Convergence
13
![Page 21: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/21.jpg)
Convergence Rates
. RK: E||xk − x||22 ≤(
1− 1||A||2F ||A−1||22
)k
||x0 − x||22.
. MM: ‖xk − x‖2 ≤(
1− 1m‖A−1‖2
)k
‖x0 − x‖2.
. SKM: E‖xk − x‖2 ≤(
1− 1m‖A−1‖2
)k
‖x0 − x‖2.
. Why are these all the same?
14
![Page 22: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/22.jpg)
Convergence Rates
. RK: E||xk − x||22 ≤(
1− 1||A||2F ||A−1||22
)k
||x0 − x||22.
. MM: ‖xk − x‖2 ≤(
1− 1m‖A−1‖2
)k
‖x0 − x‖2.
. SKM: E‖xk − x‖2 ≤(
1− 1m‖A−1‖2
)k
‖x0 − x‖2.
. Why are these all the same?
14
![Page 23: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/23.jpg)
Convergence Rates
. RK: E||xk − x||22 ≤(
1− 1||A||2F ||A−1||22
)k
||x0 − x||22.
. MM: ‖xk − x‖2 ≤(
1− 1m‖A−1‖2
)k
‖x0 − x‖2.
. SKM: E‖xk − x‖2 ≤(
1− 1m‖A−1‖2
)k
‖x0 − x‖2.
. Why are these all the same?
14
![Page 24: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/24.jpg)
Convergence Rates
. RK: E||xk − x||22 ≤(
1− 1||A||2F ||A−1||22
)k
||x0 − x||22.
. MM: ‖xk − x‖2 ≤(
1− 1m‖A−1‖2
)k
‖x0 − x‖2.
. SKM: E‖xk − x‖2 ≤(
1− 1m‖A−1‖2
)k
‖x0 − x‖2.
. Why are these all the same?
14
![Page 25: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/25.jpg)
An Accelerated Convergence Rate
Theorem (Haddock - N. 2018+)
Let x denote the solution of the consistent, normalized system Ax = b.
Motzkin’s method exhibits the (possibly highly accelerated) convergence
rate:
‖xT − x‖2 ≤T−1∏k=0
(1− 1
4γk‖A−1‖2
)· ‖x0 − x‖2
Here γk bounds the dynamic range of the kth residual, γk := ‖Axk−Ax‖2
‖Axk−Ax‖2∞
.
. improvement over previous result when 4γk < m
15
![Page 26: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/26.jpg)
γk : Gaussian systems
γk .m
logm
16
![Page 27: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/27.jpg)
γk : Gaussian systems
γk .m
logm
16
![Page 28: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/28.jpg)
Gaussian Convergence
17
![Page 29: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/29.jpg)
Is this the right problem?
xLS
. noisy
. corrupted
x∗
xLS
18
![Page 30: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/30.jpg)
Is this the right problem?
xLS
. noisy
. corrupted
x∗
xLS
18
![Page 31: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/31.jpg)
Noisy Convergence Results
Theorem (N. 2010)
Let A have full column rank, denote the desired solution to the system
Ax = b by x, and define the error term e = Ax− b. Then RK iterates
satisfy
E‖xk − x‖2 ≤(
1− 1
‖A‖2F‖A−1‖2
)k
‖x0 − x‖2 + ‖A‖2F‖A−1‖2‖e‖2
∞
Theorem (Haddock - N. 2018+)
Let x denote the desired solution of the system Ax = b and define the
error term e = b−Ax. If Motzkin’s method is run with stopping criterion
‖Axk − b‖∞ ≤ 4‖e‖∞, then the iterates satisfy
‖xT − x‖2 ≤T−1∏k=0
(1− 1
4γk‖A−1‖2
)· ‖x0 − x‖2 + 2m‖A−1‖2‖e‖2
∞
19
![Page 32: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/32.jpg)
Noisy Convergence Results
Theorem (N. 2010)
Let A have full column rank, denote the desired solution to the system
Ax = b by x, and define the error term e = Ax− b. Then RK iterates
satisfy
E‖xk − x‖2 ≤(
1− 1
‖A‖2F‖A−1‖2
)k
‖x0 − x‖2 + ‖A‖2F‖A−1‖2‖e‖2
∞
Theorem (Haddock - N. 2018+)
Let x denote the desired solution of the system Ax = b and define the
error term e = b−Ax. If Motzkin’s method is run with stopping criterion
‖Axk − b‖∞ ≤ 4‖e‖∞, then the iterates satisfy
‖xT − x‖2 ≤T−1∏k=0
(1− 1
4γk‖A−1‖2
)· ‖x0 − x‖2 + 2m‖A−1‖2‖e‖2
∞
19
![Page 33: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/33.jpg)
Noisy Convergence
20
![Page 34: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/34.jpg)
What about corruption?
x0
xM1
xM2
xM3
xRK1
xRK2
xRK3
21
![Page 35: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/35.jpg)
Problem
Problem: Ax = b + e
(Corrupted) Error (e): sparse, arbitrarily large entries
Solution (x∗): x∗ ∈ {x : Ax = b}
Applications: logic programming, error correction in telecommunications
Problem: Ax = b + e
(Noisy) Error (e): small, evenly distributed entries
Solution (xLS): xLS ∈ argmin‖Ax− b− e‖2
22
![Page 36: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/36.jpg)
Problem
Problem: Ax = b + e
(Corrupted) Error (e): sparse, arbitrarily large entries
Solution (x∗): x∗ ∈ {x : Ax = b}
Applications: logic programming, error correction in telecommunications
Problem: Ax = b + e
(Noisy) Error (e): small, evenly distributed entries
Solution (xLS): xLS ∈ argmin‖Ax− b− e‖2
22
![Page 37: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/37.jpg)
Problem
Problem: Ax = b + e
(Corrupted) Error (e): sparse, arbitrarily large entries
Solution (x∗): x∗ ∈ {x : Ax = b}
Applications: logic programming, error correction in telecommunications
Problem: Ax = b + e
(Noisy) Error (e): small, evenly distributed entries
Solution (xLS): xLS ∈ argmin‖Ax− b− e‖2
22
![Page 38: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/38.jpg)
Why not least-squares?
x∗
xLS
23
![Page 39: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/39.jpg)
MAX-FS
MAX-FS: Given Ax = b, determine the largest feasible subsystem.
. MAX-FS is NP-hard even when restricted to homogenous systems
with coefficients in {−1, 0, 1} (Amaldi - Kann 1995)
. no PTAS unless P = NP
24
![Page 40: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/40.jpg)
MAX-FS
MAX-FS: Given Ax = b, determine the largest feasible subsystem.
. MAX-FS is NP-hard even when restricted to homogenous systems
with coefficients in {−1, 0, 1} (Amaldi - Kann 1995)
. no PTAS unless P = NP
24
![Page 41: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/41.jpg)
MAX-FS
MAX-FS: Given Ax = b, determine the largest feasible subsystem.
. MAX-FS is NP-hard even when restricted to homogenous systems
with coefficients in {−1, 0, 1} (Amaldi - Kann 1995)
. no PTAS unless P = NP
24
![Page 42: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/42.jpg)
Proposed Method
Goal: Use RK to detect the corrupted equations with high probability.
We call ε∗/2 the detection horizon.
25
![Page 43: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/43.jpg)
Proposed Method
Goal: Use RK to detect the corrupted equations with high probability.
Lemma (Haddock - N. 2018+)
Let ε∗ = mini∈[m] |Ax∗ − b|i = |ei | and suppose |supp(e)| = s. If
||ai || = 1 for i ∈ [m] and ||x− x∗|| < 12ε∗ we have that the d ≤ s indices
of largest magnitude residual entries are contained in supp(e). That is,
we have D ⊂ supp(e), where
D = argmaxD⊂[A],|D|=d
∑i∈D
|Ax− b|i .
We call ε∗/2 the detection horizon.
25
![Page 44: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/44.jpg)
Proposed Method
Goal: Use RK to detect the corrupted equations with high probability.
x∗
xk
We call ε∗/2 the detection horizon.
25
![Page 45: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/45.jpg)
Proposed Method
Goal: Use RK to detect the corrupted equations with high probability.
x∗
xk
We call ε∗/2 the detection horizon.
25
![Page 46: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/46.jpg)
Proposed Method
Method 1 Windowed Kaczmarz
1: procedure WK(A,b, k ,W , d)
2: S = ∅3: for i = 1, 2, ...W do
4: xik = kth iterate produced by RK with x0 = 0, A, b.
5: D = d indices of the largest entries of the residual, |Axik − b|.6: S = S ∪ D
7: return x, where ASC x = bSC
26
![Page 47: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/47.jpg)
Example
WK(A,b,k = 2,W = 3,d = 1): j = 1, i = 1, S = ∅
x∗
x10
H1
H2
H3
H4
H5
H6 H727
![Page 48: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/48.jpg)
Example
WK(A,b,k = 2,W = 3,d = 1): j = 1, i = 1, S = ∅
x∗
x10
x11
H1
H2
H3
H4
H5
H6 H727
![Page 49: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/49.jpg)
Example
WK(A,b,k = 2,W = 3,d = 1): j = 2, i = 1, S = {7}
x∗
x10
x11
x12
H1
H2
H3
H4
H5
H6 H727
![Page 50: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/50.jpg)
Example
WK(A,b,k = 2,W = 3,d = 1): j = 1, i = 2, S = {7}
x∗
x20
H1
H2
H3
H4
H5
H6 H727
![Page 51: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/51.jpg)
Example
WK(A,b,k = 2,W = 3,d = 1): j = 1, i = 2, S = {7}
x∗
x20
H1
H2
H3
H4
H5
H6 H7
x21
27
![Page 52: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/52.jpg)
Example
WK(A,b,k = 2,W = 3,d = 1): j = 2, i = 2, S = {7, 5}
x∗
x20
H1
H2
H3
H4
H5
H6 H7
x21
x22
27
![Page 53: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/53.jpg)
Example
WK(A,b,k = 2,W = 3,d = 1): j = 1, i = 3, S = {7, 5}
x∗
x30
H1
H2
H3
H4
H5
H6 H727
![Page 54: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/54.jpg)
Example
WK(A,b,k = 2,W = 3,d = 1): j = 1, i = 3, S = {7, 5}
x∗
x30
H1
H2
H3
H4
H5
H6 H7
x31
27
![Page 55: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/55.jpg)
Example
WK(A,b,k = 2,W = 3,d = 1): j = 2, i = 3, S = {7, 5, 6}
x∗
x30
H1
H2
H3
H4
H5
H6 H7
x31
x32
27
![Page 56: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/56.jpg)
Example
Solve ASC x = bSC .
x∗
H1
H2
H3
H4
27
![Page 57: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/57.jpg)
Theoretical Guarantees
Theorem (Haddock - N. 2018+)
Assume that ‖ai‖ = 1 for all i ∈ [m] and let 0 < δ < 1. Suppose
d ≥ s = |supp(e)|, W ≤ bm−nd c and k∗ is as given in the detection
horizon lemma. Then the Windowed Kaczmarz method on A,b will
detect the corrupted equations (supp(e) ⊂ S) and the remaining
equations given by A[m]−S ,b[m]−S will have solution x∗ with probability
at least
pW := 1−[
1− (1− δ)(m − s
m
)k∗]W.
28
![Page 58: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/58.jpg)
Theoretical Guarantee Values (Gaussian A ∈ R50000×100)
pW := 1−[
1− (1− δ)(
m−sm
)k∗]W
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
pW
s = 1
s = 10
s = 50
s = 100
s = 200
s = 300
s = 400
29
![Page 59: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/59.jpg)
Experimental Values (Gaussian A ∈ R50000×100)
0 0.2 0.4 0.6 0.8 10.4
0.5
0.6
0.7
0.8
0.9
1S
uccess r
atio
s = 100
s = 200
s = 500
s = 750
s = 1000
30
![Page 60: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/60.jpg)
Experimental Values (Gaussian A ∈ R50000×100)
0 500 1000 1500 2000
k
0
0.2
0.4
0.6
0.8
1S
uccess r
atio
s = 100
s = 200
s = 500
s = 750
s = 1000
31
![Page 61: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/61.jpg)
Experimental Values (Gaussian A ∈ R50000×100)
32
![Page 62: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/62.jpg)
Experimental Values (Gaussian A ∈ R50000×100)
33
![Page 63: UCLA Mathematics - Iterative Projection Methodsdeanna/IterativeProjection... · 2018. 3. 21. · jja ijj= 1 for i 2[m] and jjx x jj](https://reader036.fdocuments.in/reader036/viewer/2022062417/6147b09dafbe1968d37a36a6/html5/thumbnails/63.jpg)
Conclusions and Future Work
• Motzkin’s method is accelerated even in the presence of noise
• RK methods may be used to detect corruption
• identify useful bounds on γk for other useful systems
• reduce dependence on artificial parameters in corruption detection
bounds
34