Quick Tour of Linear Algebra and Graph Theory
Quick Tour of Linear Algebra and GraphTheory
CS224w: Social and Information Network AnalysisFall 2012 Yu ”Wayne” Wu
Based on Borja Pelato’s version in Fall 2011
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Matrices and Vectors
Matrix: A rectangular array of numbers, e.g., A ∈ Rm×n:
A =
a11 a12 . . . a1na21 a22 . . . a2n
......
...am1 am2 . . . amn
Vector: A matrix consisting of only one column (default) orone row, e.g., x ∈ Rn
x =
x1x2...
xn
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Matrix Multiplication
If A ∈ Rm×n, B ∈ Rn×p, C = AB, then C ∈ Rm×p:
Cij =n∑
k=1
AikBkj
Special cases: Matrix-vector product, inner product of twovectors. e.g., with x , y ∈ Rn:
xT y =n∑
i=1
xiyi ∈ R
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Matrix Multiplication
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Properties of Matrix Multiplication
Associative: (AB)C = A(BC)
Distributive: A(B + C) = AB + ACNon-commutative: AB 6= BA
Proof of associativity:Let L = (AB)C and R = A(BC), then we can showLij =
∑pk = 1
∑nl = 1 (ail ◦ blk ) ◦ ckj =∑n
l = 1∑p
k = 1 ail ◦(blk ◦ ckj
)= Rij .
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Operators and properties
Transpose: A ∈ Rm×n, then AT ∈ Rn×m: (AT )ij = Aji
Properties:(AT )T = A(AB)T = BT AT
(A + B)T = AT + BT
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Identity Matrix
Identity matrix: I = In ∈ Rn×n:
Iij =
{1 i=j,0 otherwise.
∀A ∈ Rm×n: AIn = ImA = A
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Diagonal Matrix
Diagonal matrix: D = diag(d1,d2, . . . ,dn):
Dij =
{di j=i,0 otherwise.
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Other Special Mtrices
Symmetric matrices: A ∈ Rn×n is symmetric if A = AT .Orthogonal matrices: U ∈ Rn×n is orthogonal ifUUT = I = UT U
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Linear Independence and Rank
A set of vectors {x1, . . . , xn} is linearly independent if@{α1, . . . , αn}:
∑ni=1 αixi = 0
Rank: A ∈ Rm×n, then rank(A) is the maximum number oflinearly independent columns (or equivalently, rows)Properties:
rank(A) ≤ min{m,n}rank(A) = rank(AT )rank(AB) ≤ min{rank(A), rank(B)}rank(A + B) ≤ rank(A) + rank(B)
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Rank from row-echelon forms
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Matrix Inversion
If A ∈ Rn×n, rank(A) = n, then the inverse of A, denotedA−1 is the matrix that: AA−1 = A−1A = IProperties:
(A−1)−1 = A(AB)−1 = B−1A−1
(A−1)T = (AT )−1
The inverse of an orthogonal matrix is its transpose
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Eigenvalues and Eigenvectors
A ∈ Rn×n, λ ∈ C is an eigenvalue of A with thecorresponding eigenvector x ∈ Cn (x 6= 0) if:
Ax = λx
eigenvalues: the n possibly complex roots of thepolynomial equation det(A− λI) = 0, and denoted asλ1, . . . , λn
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Eigenvalues and Eigenvectors Properties
Usually eigenvectors are normalized to unit length.If A is symmetric, then all the eigenvalues are real and theeigenvectors are orthogonal to each other.tr(A) =
∑ni=1 λi
det(A) =∏n
i=1 λi
rank(A) = |{1 ≤ i ≤ n|λi 6= 0}|
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Matrix Eigendecomposition
A ∈ Rn×n, λ1, . . . , λn the eigenvalues, and x1, . . . , xn theeigenvectors. P = [x1|x2| . . . |xn], D = diag(λ1, . . . , λn), then:
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Matrix Eigendecomposition
Therefore, A = PDP−1.In addition:
A2 = (PDP−1)(PDP−1) = PD(P−1P)DP−1 = PD2P−1
By induction, An = PDnP−1.A special case of Singular Value Decomposition
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Convex Optimization
A set of points S is convex if, for any x , y ∈ S and for any0 ≤ θ ≤ 1,
θx + (1− θ)y ∈ S
A function f : S → R is convex if its domain S is a convexset and
f (θx + (1− θ)y) ≤ θf (x) + (1− θ)f (y)
for all x , y ∈ S, 0 ≤ θ ≤ 1.
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Logistic Regression
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Gradient Descent
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Submodularity
A function f : S → R is submodular if for any subset A ⊆ B,
f (A ∪ {x})− f (A) ≥ f (B ∪ {x})− f (B)
Submodular functions allow approximate discreteoptimization.
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Greedy Set Cover
If the optimal solution contains m sets, greedy algorithm finds aset cover with at most m loge n sets.
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Proofs
Induction:1 Show result on base case, associated with n = k0
2 Assume result true for n ≤ i . Prove result for n = i + 13 Conclude result true for all n ≥ k0
Example:For all natural number n, 1 + 2 + 3 + ...+ n = n∗(n+1)
2Base case: when n = 1, 1 = 1.Assume statement holds for n = k , then1 + 2 + 3 + ...+ k = k∗(k+1)
2 .We see 1+2+3+ ...+(k +1) = k∗(k+1)
2 +(k +1) = (k+1)(k+2)2 .
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Proofs
Contradiction (reductio ad absurdum):1 Assume result is false2 Follow implications in a deductive manner, until a
contradiction is reached3 Conclude initial assumption was wrong, hence result true
Example:Let’s try to prove there is no greatest even integer.First suppose there is one, name N, and for any even integer n,we have N ≥ n.Now let define M as N + 2. Then we see M is even, but alsogreater than N. Thus by contradiction we prove the statement.
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Graph theory
Definitions: vertex/node, edge/link, loop/cycle, degree,path, neighbor, tree, clique,. . .Random graph (Erdos-Renyi): Each possible edge ispresent with some probability p(Strongly) connected component: subset of nodes that canall reach each otherDiameter: longest minimum distance between two nodesBridge: edge connecting two otherwise disjoint connectedcomponents
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Basic algorithms
BFS: explore by “layers”DFS: go as far as possible, then backtrackGreedy: maximize goal at each stepBinary search: on ordered set, discard half of the elementsat each step
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
BFS
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
DFS
Quick Tour of Linear Algebra and Graph Theory
Basic Linear Algebra
Complexity
Number of operations as a function of the problemparameters.Examples
1 Find shortest path between two nodes:DFS: very bad idea, could end up with the whole graph as asingle pathBFS from origin: good ideaBFS from origin and destination: even better!
2 Given a node, find its connected componentLoop over nodes: bad idea, needs N path searchesBFS or DFS: good idea
Top Related