Topics in Algorithms and Data Science

Random Graphs (2nd part)

Omid Etesami

Phase transitions for CNF-SAT

Phase transitions for other random structures

• We already saw phase transitions for random graphs

• Other random structures, like Boolean formula in conjunctive normal form (CNF), also have phase transitions

Random k-CNF formula

• n variables

• m clauses

• k literals per clause (k constant)

• literal = variable or negation

• each clause independently chosen from possible clauses.

• Unsatisfiability is an increasing property, so it has phase transition.

Satisfiability conjecture

• Conjecture. There is a constant rk such that m = rkn is a sharp threshold for satisfiability.

The conjecture was recently proved for large k by Ding, Sly, Sun!

Upper bound on rk

• Let m = cn.

• Each truth assignment satisfies the CNF with probability (1 – 2-k)cn.

• The probability that the CNF is satisfiable is at most 2n(1 – 2-k)cn.

• Thus rk ≤ 2k ln 2.

3-SAT solution space (height represents # of unsatisfied constraints)!

Lower bound on rk

• Lower bound more difficult. 2nd moment method doesn’t work.

• We focus on k = 3.

• Smallest Clause (SC) heuristic finds a satisfying solution almost surely when m = cn and constant c < 2/3. Thus r3 ≥ 2/3.

Smallest Clause (SC) heuristic

While not all clauses satisfied

assign true to a random literal in a random smallest-length clause

delete satisfied clauses; delete unsatisfied literals.

If a 0-length clause is ever found, we have failed.

Queue of 1-literal and 2-literal clauses

• While queue is not empty, a member of the queue is satisfied.

• Setting a literal to true, may add other clauses to the queue.

• We will show that while the queue is non-empty, the arrival rate is less than the departure rate.

Principle of deferred decisions

• We pretend that we do not know the literals appearing in each clause.

• During the algorithm, we only know the size of each clause.

Queue arrival rate

• When the t’th literal is assigned value, each 3-literal clause is added to the queue with probability 3/(2(n-t+1)).

• (With the same probability, the clause is satisfied.)

• Therefore, the average # of clauses added to the queue at each step is at most 3(cn – t + 1)/(2(n-t+1)) = 1 – Ω(1).

The waiting time is O(lg n)

Thm. The # steps any clause remains in the queue is Ω(lg n) with probability at most 1/n3.

The probability that the queue is empty at step t and remains non-empty in steps t, t + 1, …, t + s - 1 is at most exp(-Ω(s)) by multiplicative Chernoff bound: the # arrivals should be at least s while mean # arrivals is s(1 – Ω(1)).

(We upper-bound # arrivals with sum of independent Bernoullies.)

There are only n choices for t. Therefore for suitable choice of s0 = Ө(lg n), any non-empty episode is of length at most s0 with probability 1 – 1/n3.

The probability that setting a literal in the i’th clause makes the j’th clause false is o(1/n2)

If this trouble happens, then

• either of i’th or j’th clause is added to the queue at some step t,

• j’th clause consists of 1 literal when trouble happens,

• by SC rule i’th clause also consists of 1 literal when its literals is assigned,

• with probability 1 – 1/n3 the waiting time for both clauses is O(lg n).

If a1, a2, … is the sequence of literals that would be set to true (if clauses i and j didn’t exist), then 4 of the literals in these two clauses are the negation of the literals in at, at+1, …, at’ for t’ = t + O(lg n).

This happens with probability O((ln 4 n)/n4) times # choices for t.

Since there are O(n2) pairs of clauses, the algorithm fails with probability o(1) by union bound.

Nonuniform models of random graphs

Nonuniform models

• Fix a degree distribution: there is f(d) vertices of degree d

• Choose a random graph among all graphs with this degree distribution

• Edges are no longer independent

Degree distribution: vertex perspective vs edge perspective • Consider a graph where half of vertices have degree 1, half have degree 2

• A random vertex is equally likely of degree 1 or 2

• A random vertex of a random edge is twice more

likely to be of degree 2

• In many algorithms, we traverse a random edge

to reach an endpoint: the probability of reaching

a vertex of degree i is then proportional to i λi ,

where λi is the fraction of vertices of degree i

Giant component in random graphs with given degree distribution

[Molloy, Reed] There will be a giant component iff • Intuition: Consider BFS (branching process) from a fixed vertex.

• After the first level, a vertex of degree i has exactly i – 1 children.

• The branching process has probability of extinction < 1 iff the expected # children E[i – 1] ≥ 1, or in other words E[i – 2] >= 0.

• In calculating the expectation, the probability of degree i is from the edge perspective (and not the vertex perspective). Thus it is proportional to i λi.

Example: G(n, p=1/n)

Poisson degree distribution

If vertices have Poisson degree distribution with mean d,

then

random endpoint of a random edge has degree distribution

1 + Poisson(d).

Growth model without preferential attachment

Growing graphs

• Vertices and edges are added over time.

• Preferential attachment = selecting endpoints for a new edge with probability proportional to degrees

• Without preferential attachment = selecting endpoints for a new edge uniformly at random from the set of existing vertices

With preferential attachment

Basic growth model without preferential attachment • Start with zero vertices and zero edges

• At each time t, add a new vertex

• With probability δ, join two random vertices by an edge

The resulting graph may become a multigraph.

But since there are t2 pairs of vertices and O(t) existing edges, a multiple edge or self-loop happens at each step with small probability, and we ignore these cases.

new vertex

new edge

# vertices of each degree

Let dk(t) be expected # vertices of degree k at time t.

new vertex

new edge

degree distribution

Let dk(t) = pkt in the limit as t tends to infinity.

Geometric distribution which like the Poisson

Erdos-Renyi distribution falls off exponentially fast,

unlike preferential attachment power-law.

# components of each finite size

Let nk(t) be expected # components of size k at time t

• A randomly picked component is of size k with probability proportional to nk(t)

• A randomly picked vertex is in a component of size k with probability equal to k nk(t)

Components of size 4 and 2

Recurrence relation for nk(t)

• We use expectations instead of actual # of components of each size?!

• We ignore edges falling inside components since we are interested in small component sizes.

j vertices

k – j vertices

Recurrence relation for ak=nk(t) / t

j vertices

k – j vertices

Phase transition for non-finite components

Size of non-finite components below critical threshold

Summary of phase transition

Comparison with static random graph having degree distribution

• Could you explain why giant components appear for smaller δ in the grown model?

Why is δ = 1/4 the threshold for static model?

Growth model with preferential attachment

Description of the model

• Begin with empty graph

• At each time, add a new vertex

and with probability δ, attach the new vertex

to a vertex selected at random

with probability proportional to its degree

Obviously the graph has no cycles.

Degree of vertex i at time t

Let di(t) be the degree of vertex i at time t

Thus di(t) = a t1/2.

Since di(i) = δ,

we have di(t) = δ (t/i)1/2.

Power-law degree distribution

Vertex number tδ2/d2 has degree d.

Therefore, # of vertices of degree d is

In other words, probability of degree d is 2δ2/d3.

Small world graphs

Milgram’s experiment

• Ask one in Nebraska to send a letter to one in

Massachusetts with given address and occupation

• At each step, send to someone you know on a

“first name” basis who is closer

• In successful experiments, it took 5 or 6 steps

• Called “six degrees of separation”

The Kleinberg model for random graphs

• n × n grid with local and global edges

• From each vertex u, there is a long-distance edge

to a vertex v

• Vertex v is chosen with probability proportional to

d(u,v)-r where distance is Manhattan distance.

Normalization factor

• Let .

• # nodes of distance k from u is at most 4k.

• # nodes of distance k from u is at least k for k ≤ n/2.

• We have

• cr(u) = Θ(1) when r > 2.

• cr(u) = Θ(lg n) when r = 2.

• cr(u) = Ω(n2-r) when r < 2.

No short (polylogarithmic) paths exist when r > 2. • Expected # of edges connecting vertices of distance ≥ d* is

• Thus, with high probability there is no edge connecting vertices at distance at least d* for some d* = n1-Ω(1).

• Since many pairs of vertices are at distance Ω(n) from each other, the shortest path between these pairs is at least nΩ(1).

A pair of vertices with distance Ω(n)

Local algorithm when r = 2

The algorithm is local and greedy:

At each step follow the edge that takes us closest

to the target.

Target w

Analysis of the algorithm

Claim: with high probability for any pair of vertices u, t,

within O(ln2 n) steps the distance from u to t decreases by half:

Proof: If distance between u and t is k, there are Θ(k2) vertices at distance ≤ k/2 from t.

All these vertices are at distance Θ(k) from u.

Thus, with probability Θ(k2 k-r/cr(u))=Θ(1/ln n), there is an edge to a vertex half distant to t.

Repeating this process Θ(ln2 n) steps, by independence of edges, we succeed with probability o(1/n4).

Since there are n4 pairs, by union bound, we succeed for every pair (u, t).

Local algorithm when r = 2 takes polylogarithmic steps

Since the distance is at most 2n in the beginning

and halves every O(ln2 n) steps,

we reach the target within O(ln3 n) steps.

Target w

No local algorithm finds polylogarithmic paths when r < 2

Take u and t at distance ≥ nδ.

We show any local algorithm with high probability takes ≥ nδ steps to go from u to t for small constant δ > 0.

Otherwise, the algorithm should use an edge that takes us to a point at distance < nδ.

At each step, this happens with probability ≤ O(n2δ/cr) = O(n-2+r+2δ), since we cannot plan on the outgoing edges of vertices we haven’t yet visited in a local algorithm.

Since we should find such an edge in the first nδ steps, we can find it only with probability O(n-2+r+3δ), which is o(1) for small δ.

local algorithm for finding short path does not exist, despite existence of short paths

Proof that logarithmic paths exist when r = 0

• We show the diameter is O(lg n) in a way similar to the proof for Erdos-Renyi.

• Partition the grid into 3×3 squares: Now there are 9 non-local edges going out of each square.

• There are Ω(lg n) squares at distance Ө(lg1/2 n)

of any square.

• W.h.p. the non-local neighbors of these squares

are at least twice these squares,

since 9 > 2 and by Chernoff bound.

Proof that logarithmic paths exist when r = 0 (continued) • Similarly, one can show that while half of squares have not been

visited, the neighbors visited at each level is at least twice the number in the previous level

(since # outgoing edges × fraction of remaining squares ≥ 9 × 1/2 > 2)

• Therefore more than half the squares are can be reached with O(lg n) edges from any vertex.

• Any two sets consisting of more than half the squares have nonempty intersection. Q.E.D.