Transparency No. 5-1 Discrete Mathematics Chapter 5 Advanced Counting Techniques.

Transparency No. 5-1

Discrete Mathematics

Chapter 5

Advanced Counting Techniques

Discrete Mathematics advanced counting techniques


Contents

Recurrence RelationsSolving Recurrence RelationsDivide-and-conquer RelationsThe Inclusion-Exclusion principleApplications of the Inclusion-Exclusion principle



Recurrence relations#bacteria doubles every hour.

Initially (t=0), there are 5 bacteria

==> #bacteria after n hours = ?

sol: Let an = #bacteria after n hours.

=> a0 = 5 --- (1) initial condition

an = 2 an-1 for n > 0. --- (2) recurrence relation

(1) and (2) uniquely determine an for all n N.

-- called a recurrence definition of the sequence a0,a1,..={an}nN

Goal: find explicit formula for an satisfying the recurrence relation (and initial conditions)



Definition of recurrence relations

A recurrence relation for the sequence {an} is a formula that express an in terms of one or more of the previous terms of the sequence (i.e., some aj's with j < n).

A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation.

Ex1: If {an} satisfies

(1) an = an-1 - an-2 for n = 2,3,...

(2) a0=3, a1 = 5.

=> a2 = ? and a3 = ?

Many problems can be modeled(or expressed) more naturally by recurrence relations.



recurrence relation exampleEx3: deposit $10,000 in an account with 11% compound interest annually.==> How much will be in the account after 30 years ?

Sol: let Pn = amount of the account after n years

==> 1. p0 = 10,000

2. pn = pn-1 + pn-1 x 0.11.

==> Pn = 1.11 Pn-1 = 1.112 Pn-2=...= 1.11n P0 = 1.11n x 10000.

==> P30 = 1.1130 x 10000 = 22,892,297.Ex4: Rabbits and Fibonacci number.A pair of rabbits produce one pair per month after 2 month old.Initially there is only one pair.=> How many pairs of rabbits are there after n months?

sol: fn = #pairs after n months.

=>1. f0 = f1 = 1 --initial case 2. fn = fn-1 + fn-2 for n > 1 , where fn-1: #old-rabbits and f n-2 is #new born rabbits.



More recurrence examples

(The tower of Hanoi) 3 pegs: A,B,C. Initially n disks placed on peg A. Disks can be moved from peg to peg only if the disk is on top of the original disk and smaller than that on top of the target peg.The goal is to have all disks moved to the 2nd pegs.

Let Hn = #moves required to solve the problems with n disks

=> Hn = ?

Sol: 1. H1 = 1.

2. Hn = H n-1 + 1 + H n-1 = 2Hn-1 + 1.

A--n-->B A--n-1-->C A--1-->B C--n-1-->B.

Hence Hn = 2 Hn-1 + 1 = 22 Hn-2 + 2 + 1 = ...

= 2n-1 H1 + 2n-2 + ... + 1 = 2 n -1.

How big is H64 ?

: assume one move takes 1 sec => 264 - 1 = 1.8 x 10 19 = 500 billion years!!



more recurrence examples

Ex6: #bit strings of length n not containing two consecutive 0s.

sol: an = #bit strings of such kind of length n.

==> 1. a0 = 1, a1 = 2.

2. an = #bits string ending with 1 + #bit-strings-ending with 10

= an-1 + an-2.

Ex7: an = # n-digit-strings containing an even number of 0.

Sol: valid strings must be in one of the forms:

1. x1 x2 ... xn-1 xn with xn = 1..9 or

2. x1 x2 ... xn-1 0 with x1 x2 ... xn-1 containing odd number of 0.

==> 1. a0 = 1, a1 = 9.

2. an = an-1 x 9 + (10n-1 - an-1) for n > 1.



5.2 Solving recurrence relations

Linear homogeneous recurrence relation of degree k with constant coefficients: any recurrence of the form:

an = c1 an-1 + c2an-2+...+ckan-k,

where k > 0 and every cj is a constant.

Theorem: The recurrence relation: an = c1 an-1 + c2an-2+...+ckan-k,

and initial conditions: a0 = t0, ..., ak-1 = tk-1 uniquely determine a sequence (satisfying both conditions)

Pf: 1. Existence: trivial. (by Math Ind)

2. Uniqueness: If {n} and {n} satisfy the relation =>

{n} = {n} (i.e., n =n for all n.)

simple math. ind. Left as an exercise.



Solving linear homogeneous recurrence relations with constant coefficients

an = c1 an-1 + c2an-2+...+ckan-k ------ (1) : a recurrence relation => an = rn is a solution of (1) iff

rk = c1 rk-1 + c2 rk-2+...+ck rk-k -------(2) (2) is called the characteristic equation of (1)

an = c1 an-1 + c2an-2 ------ (3)

Theorem 1: If r2 = c1 r + c2 has two distinct roots r1, r2 then sequence {an} is a solu of (3) iff

an = d1 r1n + d2 r2

n for n = 0,1,2,.. (*) where d1 and d2 are constants.pf: <= : simple substitution.

=>: Let b0,b1,... be any solution of (3).

==> For {bn} to satisfy (*) ==> b0 = d1 + d2; b1 = d1r1 + d2 r2

==> (by ind.) for any k > 1:

bk = c1bk-1 + c2bk-2 = (by ind.hyp.) c1(d1r1k-1+d2r2

k-1) + c2 (d1r1k-2 +d2r2

k-

2) = d1(c1r1k-1+c2r1

k-1) +d2(c1r2k-2+c2r2

k-2)

= (by *) d1 r1k + d2 r2

k. QED



Examples:

Ex3: Find solu of an = an-1 + 2 an-2 for n > 1 and a0 = 2 and a1 = 7.

sol: char equ: r2 = r +2 has roots 2, -1.

Hence an = d1 2n + d2 (-1)n for all n for some d1, d2.

==> a0 = d1 + d2

a1 = 2 d1 - d2 => d1 = 3; d2 = -1

==> an = 3 x 2n - (-1)n for n 0.

Ex4: find solu of the Fibonacci sequence: f1=f0=1 and fn = fn-1 + fn-2 for n > 1.

Sol: The char equ: r2 = r + 1 has roots: =(1+rt(5)) /2, =(1-rt(5))/2. note: 1<<2 and -1 < <0.

Hence fn = d1 n + d2 n with

f0 = 1 = d1 + d2 and

f1 = 1 = d1 + d2 => d1 = rt(5)/5 and d2 = -rt(5)/5.

=> fn = d1n + d2n = O(n) grows exponentially.



Solving recurrence relation

Theorem 2: If r2 = c1 r + c2 has only one r0 then any sequence {an} is a solu of (3) iff an = d1 r0

n + d2 n r0n for n = 0,1,2,.. (*) where d1 and d2

are any constants.

Pf: => : Similar to Theorem 1.

<= : a n-1 = d1 r0n-1 + d2 (n-1) r0

n-1

a n-2 = d1 r0n-2 + d2 (n-2) r0

n-2

==> c1 an-1 + c2 an-2 = d1 (c1 r0

n-1 + c2 r0n-2) +

d2 (c1(n-1) r0n-1 + c2(n-2) r0

n-2)

= d1 r0n-2(c1 ro + c2) + d2 r0

n-2((n-1)(c1ro+c2) -c2)

= d1 r0n + d2 r0

n-2((n-1) r02 + r0

2) -- since r2- c1r -c2 = (r-r0)2.

Ex5: find solu. of an = 6an-1 - 9 an-2 with a0 = 1 and a1 = 6.

sol: the char equ has one root 3.

=> an = d1 3n + d2 n 3 n. => d1 = 1 and 3d1 + 3 d2 = 6

=> an = 3n + n 3n. for n >= 0.



Generalization

Theorem 3': If the equ. rk = c1 rk-1 + c2rk-2+...+ ck has solutions r1m1, r

2m2,..,rs

ms (with m1+...+ms = k) where mi is the multiplicity of the root ri.

(i.e. rk - c1 rk-1 +...-ck = i=1..s (x-ri)mi ) then

{an} is a solution of the recurrence relation:

an = c1 an-1 +...+ck an-k iff

an = i=1,s (j=0,mi-1 dij nj ri

n ), where dij's are constants.

Ex: The recurrence relation an = 5 an-1 + 9 an-2 -7 an-3 + 2 an-4

has char. equ. r4 = 5r3 -9r2 -7r + 2, which is equ. to

(r-2)(r-1)3 = 0. Hence r has roots: 2, 13.

Then the relation has general solu: an = d02n + e01n + e1 n1n + e2 n2

1n. where d0, e0,e1 and e2 are constants determined by initial conditions.



simultaneous recurrence relations

Ex24: Solving the simultaneous recurrence relations:

1. an = 3 an-1 + 2 bn-1

2. bn = an-1 + 2 bn-1, with a0 = 1 and b0 = 2.sol: (1,2) can be represented in matrix form:

Let Y be the eigenvector of A: (I.e., AY =Y for some .)=> (A-I)Y = 0 => det(A-I) = 0 => (3-)(2-) - 2 = 0 => =1,4.

=> Y1 = (1,-1)T and Y2 = (2,1)T. Now assume X0 = d1Y1 + d2 Y2

=> Xn = A Xn-1 = ... = An Xo = An-1 (AX0) = An-1 (d1AY1 + d2AY2) = An-1 (d1 1Y1 + d22Y2) = ... = d1 1

n Y1 + d22n Y2.

.021

1-n

1-n

...)(

.21

23,)1(

21

23

XAAXAAXX

AXb

aLet

b

a

b

a

nnnn

nn

n

n

n



Divide-and-conquer relations

f(n): resources (time or space) needed to solve a problem of size n. Then

f(n) = a f(n/b) + g(n) : where a : the number of subproblems n/b : size of each subproblem g(n) : cost for splitting problem and combining solutions

Problem: How to estimate the size of f(n) ? f(n) = a f(n/b) +g(n) = a2f(n/b2) + a g(n/b) + g(n)

= .... = akf(n/bk) + j=0,k-1 ajg(n/bj).

Hence if n = bk ==> f(n) = akf(1) + j=0,k-1 ajg(n/bj) ----- (1)

Theorem 1: If f(n) = a f(n/b) + c, where a 1, b > 1 and c > 0, is an increasing function, then

f(n) = O(n(logba)) if a > 1 and

= O(log n) if a = 1.



proof:

Pf: If n = bk, by (1), f(n) = akf(1) + j=0,k-1 aj c.

If a = 1 ==> f(n) = f(1) + j=0,k-1 c = f(1) + ck = O(log n).

If a > 1 ==> f(n) = akf(1) + j=0,k-1 aj c

= akf(1) + c(ak-1) /(a-1) = ak[f(1) + c/(a-1)] - c/(a-1)

= c1 algbn + c2 = c1 nlogba + c2 = O(nlogba).

If bk < n < bk+1 is not a power of b.

==> f(n) < f(bk+1) = c1 ak+1 + c2 < c1a alogbn + c2 = O(nlogba ) .

Ex: If f(n) = 5 f(n/2) + 3.

=> a = 5 > 1; c = 3; b = 2. ==> f(n) = O(nlogba ) = O(nlg 5 )

If f(n) = 2 f(n/2) + 2

==> f(n) = O(nlg 2) = O(n) is linear.



divede-and-conquer relations(cont'd)

Theorem 2: If f(n) = a f(n/b) + cnd, where a 1, b > 1 and c,d > 0, is increasing, then

f(n) = O(nd) if a < bd

= O(nd lg n) if a = bd

= O(nlogba ) if a > bd.

Pf: f(bk) = a f(bk-1) + cbkd = a2f(bk-2) + c a bd(k-1) + c bdk.

= .... = akf(1) + j=0,k-1 cbkd aj/bdj

= f(1) ak + cbkd (1-(a/bd)k)/(1 - (a/bd)).

case 1: n = bk. (k = logb n)

Hnece if a < bd ==> f(n) = f(1) ak + cbkd (1-(a/bd)k)/(1 - (a/bd))

O(bkd) = O(bd logbn) = O(nd).

If a = bd =>f(n) = f(1) ak + ckbkd = O(kbkd) = O(nd lg n) .

If a > bd => f(n) = f(1) ak + cbkd (1-(a/bd)k)/(1 - (a/bd)).

f(1) ak + cbkd (a/bd)k = O(ak) = (nlogba).



The master theorem:case 2: bk < n < b k+1:

=> f(bk) f(n) f(bk+1).

But O(f(bk)) = O(f(bk+1)), hence O(f(n)) =O(f(bk)). QED

Ex8:Fast integer multiplication:

A = (a2n-1 a2n-2 ... a1 a0)

B = (b2n-1 b2n-2 ... b1 b0)

AH = (a2n-1...an); AL = (an-1,...,a0); BH = ..., BL = ...

=> A x B = (2n AH + AL) x (2nBH + BL) = 22n AHBH + 2n(AHBL + ALBH) + (AL

BL)

=> f(n) = 4f(n/2) + O(n) => f(n) = O(n lg 4) = O(n2) - no improving !!

But AXB = 22n AHBH + 2n(AHBL + ALBH) + (ALBL)

= 22n AHBH + 2n((AH+AL)(BH+BL)) - AHBH - ALBL) + (ALBL)

==> f(n) = 3f(n/2) + O(n) => f(n) = O(nlg 3) < O(n2).



More examples:

Fast matrix multiplication:(Ex4 & 9)

one nxn matrix multiplication can be divided into

7 (n/2)x(n/2) multiplications + 15 (n/2)x(n/2) additions.

=> f(n) = 7 f(n/2) + 15n2/4

=> a = 7, b = 2, d = 2. => a > bd = 4.

=> f(n) = O(n lg 7) = O(n2.81).

Better than direct multiplication(=O(n3)) !!

Although even better result (O(n2.376 )) is possible.



5.4 Inclusion-Exclusion principle

The principle: A, B : two finite sets

=> |AUB| = |A| + |B| - |A B|.

Ex2: #positive integers < 1000 and dividable by 7 or 11 = ?

sol: let A ={x | x < 1000 and 7 | x}

B = {x | x < 1000 and 11 |x}.

=> |AB| ={x | x < 1000 and 77 | x}

=> |AUB| = |A| + |B| - |AB|

=[1000/7]+[1000/11]-[1000/77]

= 220.

Problem: |AUBUC| = ?

|A| +|B|+|C| -|AB|-|AC||BC|+|ABC|

1 1

1

2

2 23

1 1

1

1

1 10

|A|+|B|+|C|

|A|+|B|+|C| -|AB|-|AC|-|BC|



The general inclusion-exclusion principle

Theorem 1: |A1 U A2 ...UAn | = 1in |Ai| - 1i<jn|AiAj| +

1i<j<kn|AiAjAk| - ... +(-1)n |A1A2...An| ---(*)

pf: Let a be any element belonging to exactly Ad1, Ad2,...,

Adr.

==> It is counted C(r,s) times by the sum:

0<j1<j2<...<js< n+1 |Aj1 Aj2 ...Ajs|.

(note: C(r,s) = 0 if r < s).

==> # a counted by (*) = C(r,1) - C(r,2) + ... +(-1)n C(r,n)

= C(r,1) - C(r,2) + ... +(-1)r C(r,r) = - (1-1)r + C(r,0) = 1



5.5 Applications of the IE principle

Alternative form:

Ai : set of elements having property Pi.

N(Pj1,...,Pjk) = #elements with properties Pj1,...,Pjk.

(i.e. N(Pj1,...,Pjk) = |Aj1 ...Ajk|. )

~Pj : the negation of property Pj.

N(~Pj1,...,~Pjk)) = #elements without any property of Pj1,..,pjk.

= N(~(Pj1\/...\/Pjk)) = |U| - |Aj1.. Ajk|.

=> N(~P1,...~Pn)) = N - N(Pi) + N(PiPj) - N(PiPjPk) +...

+(-1)n N(P1P2...Pn).

=> |~(A1UA2...UAn)| =

|U| - |Ai| + |AiAj| -... + (-1)n |A1...An|.



Examples:

Ex1: x1+x2+x3 = 11, 0 x1 3, 0 x2 4, 0 x3 6.

==> #integer solutions = ?

Sol: let P1 = "x1 > 3"; P2 = "x2 > 4"; P3 = "x3 > 6".

=>#sol = N(~p1/\~p2/\~p3) = N(~(P1\/P2\/P3))

= N - N(p1) - N(P2) - N(P3) +N(p1/\P2) +N(P1/\P3) + N(P2/\P3)

- N(N(p1/\P2/\P3)

=> N = C(11+2, 2);

N(p1) = #sol with x1 > 3 = #sol of "x1' + x2 + x3 = 7" = C(9,2)

N(p2) = C(8,2); N(P3) = C(6,2).

N(p1/\P2) = #slo with X1> 3 and X2 > 4 = #sol of "x1'+x2'+X3 = 2" = C(4,2) = 6.

N(P1/\P3) = C(0+2,2) = 0; N(P2/\P3) = 0.

N(P1/\P2/\P3) = 0.

=> #sol = 78 - 36 - 28 - 15 + 6+ 1 + 0 - 0 = 6.



More example

Ex2: #positive integers < 101 and dividable by 4, 5 or 6.

A = {x : 4|x}; B = {x: 5|x }; C= {x: 6|x}.

=> |A| +|B|+|C| = [100/4] + [100/5] + [100/6] = 25 + 20 + 16 =61.

|AB|+|AC|+|BC| = [100/20] +[100/12] + [100/30] =16

|ABC| = [100/60] = 1. => #sol = 61-16+1 =46.

Ex3': |A|= m, |B| = n, m n. #onto function f:A -> B = ?

Let B = {b1,...,bn} and Pi = "bi is not in the range of the fun"

=> N(Pi) = |Ci|= |{f | f:A->B and bi not in f(A)}|.

#ontos = N(~P1/\~P2.../\~Pn) = N(~(P1\/...\/Pn))

= |U| - |C1 U C2 U... U Cn|

= nm - |Ci| + |Ci Cj| - ... +(-1)n |C1C2...Cn|

= nm - C(n,1)(n-1)m + C(n,2)(n-2)m -... +(-1)n-1 C(n,n-1) 1 m.



more examples (cont'd)

Ex3: #ways to assign 5 jobs to 4 people s.t. each one is assigned at least one job.

sol: m =5, n = 4.

#ways = 45 -C(4,1)35 + C(4,2)25+C(4,3)15

= 1024 - 972 + 192 - 4 = 240.

Ex4: [The hatcheck problem:] n hats randomly returned to the customers. => What is the probability that no one receives his own hat?



Derangementsx1,x2,...,xn: a list

Any permutation a1,...,an of {x1,...,Xn} s.t. aj xj for all j = 1..n is called a derangement of the list.

Ex: 1,2,3,4,5 has derangement 21453 but not 21543

Let Dn = # derangements of n objects.

=> D1 = 0, D2 = 1, D3 = 2. (123 has derangements: 231, 212)

Theorem: Dn = n![1- 1/1! + 1/2! +... + (-1)n /n!]

Pf: let Pi =def "a=a1...an is a perm of x1...xn s.t. ai = xi."

=> Dn = N(~P1/\~P2/\../\~Pn) = N(~(P1\/P2\/..\/Pn))

= N -N(Pi) + N(Pi/\Pj) -... + (-1)nN(p1/\../\Pn) = n! -C(n,1)(n-1)! +C(n,2)(n-2)! + ... +(-1)nC(n,n)(n-n)! = n![1 - 1/1! + 1/2! .. ].

=> solu of the hatcheck problem = Dn / P(n,n) = Dn /n! in [0.3,0.5]

= (1 -1/1! + 1/2!-...) -> e-1 = 0.368 as n -> .

Transparency No. 5-1 Discrete Mathematics Chapter 5 Advanced Counting Techniques.

Documents

Transcript of Transparency No. 5-1 Discrete Mathematics Chapter 5 Advanced Counting Techniques.