Rank Minimization over Finite Fields - ECE@NUS · Rank Minimization over Finite Fields Vincent Y....

Rank Minimization over Finite Fields

Vincent Y. F. Tan

Laura Balzano, Stark C. Draper

Department of Electrical and Computer Engineering,University of Wisconsin-Madison

ISIT 2011

Vincent Tan (UW-Madison) Rank Minimization over Finite Fields ISIT 2011 1 / 20

Connection between Two Areas of Study

Area of Matrix Completion Rank-Metric CodesStudy Rank Minimization

Applications Collaborative Filtering Crisscross Error CorrectionMinimal Realization Network Coding

Field Real R, Complex C Finite Field Fq

Techniques Functional Analysis Algebraic coding

Decoding Convex Optimization Berlekamp-MasseyNuclear Norm Error Trapping

Can we draw analogies between the two areas of study?

Rank Minimization over finite fields

Rank Minimization over finite fieldsVincent Tan (UW-Madison) Rank Minimization over Finite Fields ISIT 2011 2 / 20

Minimum rank distance decoding

Transmit some codeword C ∈ C

A rank-metric code C is a non-empty subset of Fm×nq endowed with

the rank distance

Receive some received matrix R ∈ Fm×nq

Decoding problem (under mild conditions) is

C = arg minC∈C

rank(R− C)

Minimum distance decoding since rank induces a metric

X ≡ R− C is known as the error matrix – assumed low-rank

Assume linear code. Rank minimization over finite field:

X = arg minX∈ coset

rank(X)

the rank distance

C = arg minC∈C

rank(R− C)

rank(X)

the rank distance

C = arg minC∈C

rank(R− C)

rank(X)

the rank distance

C = arg minC∈C

rank(R− C)

rank(X)

the rank distance

C = arg minC∈C

rank(R− C)

rank(X)

When do low-rank errors occur?

Probabilistic crisscross error correction

Roth, IT Trans 1997

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 43, NO. 5, SEPTEMBER 1997 1425

Probabilistic Crisscross Error CorrectionRon M. Roth, Member, IEEE

Abstract—The crisscross error model in data arrays is consid-ered, where the corrupted symbols are confined to a prescribednumber of rows or columns (or both). Under the additionalassumption that the corrupted entries are uniformly distributedover the channel alphabet, and by allowing a small decoding errorprobability, a coding scheme is presented where the redundancycan get close to one half the redundancy required in minimum-distance decoding of crisscross errors.

Index Terms—Crisscross errors, probabilistic coding, rankmetric.

I. INTRODUCTION

CONSIDER an application where information symbols(such as bits or bytes) are stored in arrays, with the

possibility of some of the symbols recorded erroneously. Theerror patterns are such that all corrupted symbols are confinedto a prescribed number of rows or columns (or both). Werefer to such an error model ascrisscross errors. A crisscrosserror pattern that is confined to two rows and three columnsis shown in Fig. 1.

Crisscross errors can be found in various data storageapplications; see, for instance, [3], [5], [6], [11], [16]–[19].Such errors may occur in memory chip arrays, where rowor column failures occur due to the malfunctioning of rowdrivers or column amplifiers. Crisscross errors can also befound in helical tapes, where the tracks are recorded in adirection which is (conceptually) perpendicular to the directionof the movement of the tape; misalignment of the reading headcauses burst errors to occur along the track (and across thetape), whereas scratches on the tape usually occur along thetape (and across the tracks). Crisscross error-correcting codescan also be applied in linear magnetic tapes, where the tracksare written along the direction of the movement of the tapeand, therefore, scratches cause bursts to occur along the tracks;still, the information and check symbols are usually recordedacross the tracks. Computation of check symbols is equivalentto decoding of erasures at the check locations, and in this casethese erasures are perpendicular to the erroneous tracks.

Crisscross errors can be analyzed through the followingcover metric. Acover of an array over a fieldis a set of rows or columns that contain all the nonzero entriesin . The cover weightof , denotedw , is the sizeof the smallest cover of . The cover distancebetween two

Manuscript received October 23, 1995; revised February 18, 1997. Part ofthis work was performed at Hewlett-Packard Israel Science Center, Haifa,Israel. The material in this paper was presented in part at the 34th AnnualAllerton Conference on Communication, Control, and Computing, Urbana-Champaign, IL, October 2–4, 1996.

The author is with Hewlett-Packard Laboratories, Palo Alto, CA 94304USA, on leave from the Computer Science Department, Technion–IsraelInstitute of Technology, Haifa 32000, Israel,

Publisher Item Identifier S 0018-9448(97)05407-2.

Fig. 1. Typical crisscross error pattern.

arrays over is the cover weight of their difference. Anarray code over is a -dimensional linear

subspace of the vector space of all matrices oversuch that is the smallest cover distance between any

two distinct elements of or, equivalently, the smallest coverweight of any nonzero element of. The parameter isreferred to as theminimum cover distanceof and the term

stands for theredundancyof .The Singleton bound on the minimum cover distance states

that the minimum cover distance and the redundancy of anyarray code over a field satisfy the relation

where we assume that (see [9] and [19]).Let be the “transmitted” array and be the

“received” array, where is the error array. The number ofcrisscross errors is bounded from below byw . Sincecover distance is a metric, then by using anarray code, we can recover any pattern of up tocrisscross errors. On the other hand, if we wish to be ableto recoverany pattern of up to crisscross errors, then wemustuse an array code with minimum cover distance whichis at least . The Singleton bound on the minimum coverdistance implies that the number of redundancy symbols mustbe at least , namely, at least twice the maximum numberof erroneous symbols that need to be corrected.

A simple constructive technique to combat crisscross errorsis through theskewing methodwhich we explain next (see[9], [19], [20]). Let be a conventional linear

0018–9448/97$10.00 1997 IEEE

Data storage applications: Data stored in arrays

Error patterns confined to two rows and three columns above

Low-rank errors

Roth, IT Trans 1997

I. INTRODUCTION

0018–9448/97$10.00 1997 IEEE

Low-rank errors

Roth, IT Trans 1997

I. INTRODUCTION

0018–9448/97$10.00 1997 IEEE

Low-rank errors

Roth, IT Trans 1997

I. INTRODUCTION

0018–9448/97$10.00 1997 IEEE

Low-rank errors

Problem Setup: Rank minimization over finite fields

Square matrix X ∈ Fn×nq has rank ≤ r ≤ n

!!!"Assume that the sensing matrices H1, . . . ,Hk ∈ Fn×n

q are random.

Arithmetic is performed in the field Fq ⇒ ya ∈ Fq

Given (yk,Hk), find necessary and sufficient conditions on k andsensing model such that recovery is reliable, i.e., P(En)→ 0

Assume that the sensing matrices H1, . . . ,Hk ∈ Fn×nq are random.

q are random.

Main Results

k: Num. of linear measurementsn: Dim. of matrix Xr: Max. rank of matrix X

γ = rn : Rank-dimension ratio

Critical Quantity

2γ (1− γ/2) n2

= 2rn− r2

Result Statement Consequence

Converse k < (2− ε)γ(1− γ/2)n2 P(En) 9 0P(En)→ 0

Achievability (Uniform) k > (2 + ε)γ(1− γ/2)n2 P(En) ≈ q−n2E(R)

Achievability (Sparse) k > (2 + ε)γ(1− γ/2)n2 P(En)→ 0

Achievability (Noisy) k ' (3 + ε)(γ + σ)n2 P(En)→ 0(q assumed large)

Main Results

k: Num. of linear measurementsn: Dim. of matrix Xr: Max. rank of matrix Xγ = r

n : Rank-dimension ratio

Critical Quantity

2γ (1− γ/2) n2

= 2rn− r2

Main Results

Critical Quantity

2γ (1− γ/2) n2

= 2rn− r2

Main Results

Critical Quantity

2γ (1− γ/2) n2

= 2rn− r2

Main Results

Critical Quantity

2γ (1− γ/2) n2

= 2rn− r2

Converse k < (2− ε)γ(1− γ/2)n2 P(En) 9 0

P(En)→ 0Achievability (Uniform) k > (2 + ε)γ(1− γ/2)n2 P(En) ≈ q−n2E(R)

Main Results

Critical Quantity

2γ (1− γ/2) n2

= 2rn− r2

Main Results

Critical Quantity

2γ (1− γ/2) n2

= 2rn− r2

Main Results

Critical Quantity

2γ (1− γ/2) n2

= 2rn− r2

A necessary condition on number of measurements

Given k measurements ya ∈ Fq and sensing matrices Ha ∈ Fn×nq , we

want a necessary condition for reliable recovery of X.

Proposition (Converse)Assume

X drawn uniformly at random from all matrices in Fn×nq of rank ≤ r

Sensing matrices Ha, a = 1, . . . , k jointly independent of X

r/n→ γ (constant)

If the number of measurements satisfies

k < (2− ε)γ(

1− γ

then P(X 6= X) ≥ ε/2 for all n sufficiently large.

A necessary condition on number of measurements

Given k measurements ya ∈ Fq and sensing matrices Ha ∈ Fn×nq , we

want a necessary condition for reliable recovery of X.

Proposition (Converse)Assume

X drawn uniformly at random from all matrices in Fn×nq of rank ≤ r

Sensing matrices Ha, a = 1, . . . , k jointly independent of X

k < (2− ε)γ(

1− γ

then P(X 6= X) ≥ ε/2 for all n sufficiently large.

A sensing model: Uniform model

Now we assume that X is non-random

rank(X) ≤ r = γn

Each entry of each sensing matrix Ha is i.i.d. and has a uniformdistribution in Fq:

P([Ha]i,j = h) =1q, ∀ h ∈ Fq

rank(X) ≤ r = γn

P([Ha]i,j = h) =1q, ∀ h ∈ Fq

rank(X) ≤ r = γn

P([Ha]i,j = h) =1q, ∀ h ∈ Fq

rank(X) ≤ r = γn

P([Ha]i,j = h) =1q, ∀ h ∈ Fq

The min-rank decoder

We employ the min-rank decoder

minimize rank(X)

subject to 〈Ha, X〉 = ya, a = 1, . . . , k

NP-hard, combinatorial

Denote the set of optimizers as S

Define the error event:

En := |S| > 1 ∪ (|S| = 1 ∩ X∗ 6= X)

We want the solution to be unique and correct

minimize rank(X)

En := |S| > 1 ∪ (|S| = 1 ∩ X∗ 6= X)

minimize rank(X)

En := |S| > 1 ∪ (|S| = 1 ∩ X∗ 6= X)

minimize rank(X)

En := |S| > 1 ∪ (|S| = 1 ∩ X∗ 6= X)

minimize rank(X)

En := |S| > 1 ∪ (|S| = 1 ∩ X∗ 6= X)

Achievability under uniform model

Proposition (Achievability under uniform model)Assume

Sensing matrices Ha drawn uniformly

Min-rank decoder is used

k > (2 + ε)γ(

1− γ

then P(En)→ 0.

Proof Sketch

En =⋃

Z6=X:rank(Z)≤rank(X)

〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k

By the union bound, the error probability can be bounded as

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

At the same time, by uniformity and independence,

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

That the number of n× n matrices over Fq of rank ≤ r is boundedabove by

4q2γ(1−γ/2)n2

Remark: Can be extended to noisy case

Proof Sketch

En =⋃

〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

4q2γ(1−γ/2)n2

Proof Sketch

En =⋃

〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

4q2γ(1−γ/2)n2

Proof Sketch

En =⋃

〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

4q2γ(1−γ/2)n2

Proof Sketch

En =⋃

〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

4q2γ(1−γ/2)n2

The Reliability Function

DefinitionThe rate of a sequence of linear measurement models is defined as

R := limn→∞

1− kn2 , k = # measurements

Analogy to coding: The rate of the code

C = C : 〈C,Ha〉 = 0, a = 1, . . . , k

is lower bounded by 1− k/n2.

DefinitionThe reliability function of the min-rank decoder is defined as

E(R) := limn→∞

− 1n2 logq P(En)

R := limn→∞

C = C : 〈C,Ha〉 = 0, a = 1, . . . , k

E(R) := limn→∞

− 1n2 logq P(En)

R := limn→∞

C = C : 〈C,Ha〉 = 0, a = 1, . . . , k

E(R) := limn→∞

− 1n2 logq P(En)

Our achievability proof gives a lower bound on E(R)

Upper bound on E(R) is more tricky. But...

Proposition (Reliability Function of Min-Rank Decoder)Assume

Then,E(R) =

∣∣∣(1− R)− 2γ(

1− γ

)∣∣∣+Note |x|+ := maxx, 0.

Then,E(R) =

∣∣∣(1− R)− 2γ(

1− γ

)∣∣∣+Note |x|+ := maxx, 0.

Then,E(R) =

∣∣∣(1− R)− 2γ(

1− γ

)∣∣∣+Note |x|+ := maxx, 0.

Interpretation

E(R) ≈∣∣∣∣ kn2 − 2γ

(1− γ

)∣∣∣∣+

The more the ratio kn2 exceeds 2γ

(1− γ

)The larger E(R)

The faster P(En) decays

Linear relationship for the min-rank decoder

de Caen’s lower bound: Let B1, . . . ,BM be events:

M∑m=1

P(Bm)2∑Mm′=1 P(Bm ∩ Bm′)

Exploit pairwise independence to make statements about errorexponents (linear codes achieve capacity in symmetric DMCs)

Interpretation

E(R) ≈∣∣∣∣ kn2 − 2γ

(1− γ

)∣∣∣∣+The more the ratio k

n2 exceeds 2γ(1− γ

The larger E(R)

M∑m=1

Interpretation

E(R) ≈∣∣∣∣ kn2 − 2γ

(1− γ

)The larger E(R)

M∑m=1

Interpretation

E(R) ≈∣∣∣∣ kn2 − 2γ

(1− γ

)The larger E(R)

M∑m=1

Interpretation

E(R) ≈∣∣∣∣ kn2 − 2γ

(1− γ

)The larger E(R)

M∑m=1

Interpretation

E(R) ≈∣∣∣∣ kn2 − 2γ

(1− γ

)The larger E(R)

M∑m=1

Interpretation

E(R) ≈∣∣∣∣ kn2 − 2γ

(1− γ

)The larger E(R)

M∑m=1

Sparse sensing model

Assume as usual that X is non-random

rank(X) ≤ r = γn

Sensing matrices are sparse

Arithmetic still performed in Fq

rank(X) ≤ r = γn

Each entry of each sensing matrix Ha is i.i.d. and has a δ-sparsedistribution in Fq:

1−δ

δ/(q−1) δ/(q−1) δ/(q−1) δ/(q−1)

P([Ha]ij = h) =

1− δ h = 0δ

q−1 h 6= 0

Fewer adds and multiplies since Ha sparse⇒ Encoding cheaper

May help in decoding via message-passing algorithms

How fast can δ, the sparsity factor, decay with n for reliablerecovery?

1−δ

δ/(q−1) δ/(q−1) δ/(q−1) δ/(q−1)

P([Ha]ij = h) =

1− δ h = 0δ

q−1 h 6= 0

1−δ

δ/(q−1) δ/(q−1) δ/(q−1) δ/(q−1)

P([Ha]ij = h) =

1− δ h = 0δ

q−1 h 6= 0

1−δ

δ/(q−1) δ/(q−1) δ/(q−1) δ/(q−1)

P([Ha]ij = h) =

1− δ h = 0δ

q−1 h 6= 0

Problem becomes more challenging

X is not sensed “as much”

Measurements yk do not contain as much information about X

The equality

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

no longer holds

Nevertheless....

The equality

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

no longer holds

Nevertheless....

The equality

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

no longer holds

Nevertheless....

The equality

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

no longer holds

Nevertheless....

The equality

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k) = q−k

no longer holds

Nevertheless....

Achievability under sparse model

Theorem (Achievability under sparse model)Assume

Sensing matrices Ha drawn according to δ-sparse distribution

If the sparsity factor δ = δn satisfies

δ ∈ Ω

(log n

)∩ o(1)

and the number of measurements satisfies

k > (2 + ε)γ(

1− γ

then P(En)→ 0.

δ ∈ Ω

(log n

)∩ o(1)

k > (2 + ε)γ(

1− γ

then P(En)→ 0.

δ ∈ Ω

(log n

)∩ o(1)

k > (2 + ε)γ(

1− γ

then P(En)→ 0.

Proof Strategy

Approximate with uniform model

P(En) ≤∑

Z6=X:rank(Z)≤rank(X)‖Z−X‖0≤βn2

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

Z6=X:rank(Z)≤rank(X)‖Z−X‖0>βn2

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k)

First termFew termsLow Hamming weight – can afford loose bound on probability

Second termMany termsTight bound on probability (circular convolution of δ-sparse pmf)

Set β = Θ( δlog n) to be the optimal “split”

Proof Strategy

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k)

Proof Strategy

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

P(〈Z,Ha〉 = 〈X,Ha〉,∀a = 1, . . . , k)

Proof Strategy

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

Proof Strategy

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

Proof Strategy

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

Proof Strategy

P(En) ≤∑

P(〈Z,Ha〉 = 〈X,Ha〉, ∀a = 1, . . . , k)

Concluding remarks

Derived necessary and sufficient conditions for rank minimizationover finite fields

Analyzed the sparse sensing model

Minimum rank distance properties of rank-metric codes[analogous to Barg and Forney (2002)] were derived

Drew analogies between number of measurements and minimumdistance properties

Open Problem: Can the sparsity level of

(log n

)be improved (reduced) further?

http://arxiv.org/abs/1104.4302

Concluding remarks

(log n

Concluding remarks

(log n

Concluding remarks

(log n

Concluding remarks

(log n

Concluding remarks

(log n

http://arxiv.org/abs/1104.4302Vincent Tan (UW-Madison) Rank Minimization over Finite Fields ISIT 2011 20 / 20

Rank Minimization over Finite Fields - ECE@NUS · Rank Minimization over Finite Fields Vincent Y....

Documents

Transcript of Rank Minimization over Finite Fields - ECE@NUS · Rank Minimization over Finite Fields Vincent Y....

A Penalty Method for Rank Minimization Problems in ...

FINITE ELEMENT SIMULATION OF DURELLFS …vuir.vu.edu.au/17904/1/NGUYEN_1999compressed.pdf · FINITE ELEMENT SIMULATION OF DURELLFS PHOTOELASTIC STRESS MINIMIZATION METHOD FOR SHAPE

Rank aggregation via nuclear norm minimization · rank approximation of Y^ that models the elements, and ideally one that is rank-2. Phrased in this way, it is a natural candidate

An approximation theory of matrix rank minimization and its

Simpler & More General Minimization for Weighted Finite-State Automata

VISUALIZING THE MINIMIZATION OF A DETERMINISTIC FINITE ... · VISUALIZING THE MINIMIZATION OF A DETERMINISTIC FINITE STATE AUTOMATON by Kshatriya Jagannath Rajini Singh …

SUBGROUPS OF MAXIMAL RANK IN FINITE EXCEPTIONAL GROUPS …baumeist/sommerschule/maxexceptional.pdf · SUBGROUPS OF MAXIMAL RANK IN FINITE EXCEPTIONAL GROUPS OF LIE TYPE MARTIN W.

A Rank Minimization Approach to Learning Dynamical Systems ...

Rank aggregation via nuclear norm minimization

Combinatorics on wordsTheory of Automata Binary alphabet Finite Sturmian words K-ary alphabet Finite episturmian words Minimization of DMA Minimization.

Robust Simultaneous 3D Registration via Rank Minimization

Rank Minimization with Structured Data Patterns · 2014. 7. 1. · Rank Minimization with Structured Data Patterns Viktor Larsson 1, Carl Olsson , Erik Bylow , and Fredrik Kahl;2

Iterative Reweighted Algorithms for Matrix Rank Minimization

Visualizing the Minimization of a Deterministic Finite State - CiteSeer

Rank Regularized Estimation of Approximate Factor …sn2294/papers/baing17_slides.pdfApproximate Factor ModelsRank Minimization: NP HardApproximate-Rank MinimizationRank Regularized

Efﬁcient Structured Matrix Rank Minimizationweiyu/Adams_Wei_Yu_Homepage_files/...Efﬁcient Structured Matrix Rank Minimization Adams Wei Yu y, Wanli Ma , Yaoliang Yu , Jaime G.

Rank/sparsity minimization and latent variable graphical ...parrilo//pubs/talkfiles/IPAM-October2010.pdfRank/sparsity minimization and latent variable graphical model selection Pablo

1 FINITE STATE MACHINES - II STATE MINIMIZATION PARTITIONING MINIMIZATION PROCEDURE VENDING MACHINE EXAMPLE ANALYSIS OF SYNCHRONOUS SEQUENTIAL CIRCUITS.

Interference Alignment as a Rank Constrained Rank Minimization Dimitris S. Papailiopoulos and Alexandros G. Dimakis USC Globecom 2010.

Synthesis For Finite State Machines. FSM (Finite State Machine) Optimization State tables State minimization State assignment Combinational logic optimization.