Relations (3A) - upload.wikimedia.org · Version 1.2 or any later version published by the Free...

Young Won Lim3/27/18

Relations (3A)


Copyright (c) 2015 – 2018 Young W. Lim.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

Please send corrections (or suggestions) to [email protected].

This document was produced by using LibreOffice and Octave.

Relations (4B) 28 Young Won Lim3/27/18

Relation Examples

R1∈ {(1,a) , (2,b) , (3,a) , (3,b)}

R2∈ {(a , x) , (a , y) , (b , y), (b , z)}

1

2

3

x

y

z

a

b


Composite Relation Examples

R1∈ {(1,a) , (2,b) , (3,a) , (3,b)}

R2∈ {(a , x) , (a , y) , (b , y) , (b , z)}

1

2

3

x

y

z

a

b

R2∘R

1∈ {(1, x) , (1, y ) , (2, y ) , (2, z) , (3, x ), (3, y) , (3, z)}

1

2

3

x

y

z



R1∈ {(1,a) , (2,b) , (3,a) , (3,b)}

R2∈ {(a , x) , (a , y) , (b , y) , (b , z)}

1

2

3

x

y

z

a

b

R2∘R

1∈

{(1, x) , (1, y) , (2, y) , (2, z) , (3, x) , (3, y ) , (3, z)}

1

2

3

x

y

z

A1 = [1 0

0 1

1 1]1

2

3

a b

a

b

x y z

A1 A2 = [1 0

0 1

1 1][1 1 0

0 1 1] = [1 1 0

0 1 1

1 2 1]1

2

3

x y z

A 2 = [1 1 0

0 1 1]


Matrix of a Relation

R1∈ {(1,a) , (2,b) , (3,a) , (3,b)}

R2∈ {(a , x) , (a , y) , (b , y) , (b , z)}

A1 = [1 0

0 1

1 1]1

2

3

a b

a

b

x y z

A1 A2 = [1 0

0 1

1 1][1 1 0

0 1 1] = [1 1 0

0 1 1

1 2 1]1

2

3

x y z

R2∘R

1∈ {(1, x) , (1, y ) , (2, y ) , (2, z) , (3, x ), (3, y) , (3, z)}

A 2 = [1 1 0

0 1 1]


Composite Relation Properties

R1 R

2

A1 A2

R2∘R

1

(i , k ) ∈ R2∘R1aik of A1 A2 ≠ 0

A1 A

2

X Y Z

X Z

i k



R1∈ {(1,a) , (2,b) , (3,a) , (3,b)}

R2∈ {(a , x) , (a , y) , (b , y) , (b , z)}

A1 = [1 0

0 1

1 1]1

2

3

a b

a

b

x y z

A1 A2 = [1 0

0 1

1 1][1 1 0

0 1 1] = [1 1 0

0 1 1

1 2 1]1

2

3

x y z

R2∘R

1∈ {(1, x) , (1, y ) , (2, y ) , (2, z) , (3, x ), (3, y) , (3, z)}

A 2 = [1 1 0

0 1 1]

(i , k) ∈ R2∘R

1aik of A 1 A2 ≠ 0

1

2

3

x

y

z


Composite Relation Property Examples

A1 A2 = [1 0

0 1

1 1] [1 1 0

0 1 1] = [1 1 0

0 1 1

1 2 1]

i

k

i

k

s tu

v

su+tv

s ∈ {0, 1}

t ∈ {0, 1}

u ∈ {0, 1}

v ∈ {0, 1}

A1 = [1 0

0 1

1 1]

A2

= [1 1 0

0 1 1]

1

2

3

a b

a

b

x y z

i ∈ {1, 2, 3}k ∈ {x , y , z}

a ba

b

(i , k) ∈ R2∘R

1

su+tv ≠ 0 nonzero (i , k)th element of A1A

2

(su≠0) ∨ (tv≠0)

(s=1∧u=1) ∨ (t=1∧v=1)


Sufficient Part

A1 A2 = [1 0

0 1

1 1] [1 1 0

0 1 1] = [1 1 0

0 1 1

1 2 1]

i

k

i

k

s tu

v

su+tv

s ∈ {0, 1}

t ∈ {0, 1}

u ∈ {0, 1}

v ∈ {0, 1}

su+tv ≠ 0

su = 1

(s = 1)(u = 1)

(t = 1)(v = 1)

tv = 1

A1 = [1 0

0 1

1 1]

A2

= [1 1 0

0 1 1]

1

2

3

a b

a

b

x y z

i ∈ {1, 2, 3}k ∈ {x , y , z}

a ba

b

(i , a) ∈ R1

(a , k) ∈ R2

(i , b) ∈ R1

(b , k) ∈ R2

(i , k) ∈ R2∘R

1(i , k) ∈ R

2∘R

1

(aik ≠ 0)


Necessary Part

A1 A2 = [1 0

0 1

1 1] [1 1 0

0 1 1] = [1 1 0

0 1 1

1 2 1]

i

k

i

k

s tu

v

su+tv

s ∈ {0, 1}

t ∈ {0, 1}

u ∈ {0, 1}

v ∈ {0, 1}

su+tv ≠ 0

su = 1

(s = 1)(u = 1)

(t = 1)(v = 1)

tv = 1

A1 = [1 0

0 1

1 1]

A2

= [1 1 0

0 1 1]

1

2

3

a b

a

b

x y z

i ∈ {1, 2, 3}k ∈ {x , y , z}

a ba

b

(i , a) ∈ R1

(a , k) ∈ R2

(i , b) ∈ R1

(b , k) ∈ R2

(i , k) ∈ R2∘R

1

(aik ≠ 0)


Transitivity Test Examples

R ∈ {(a ,a) , (b ,b) , (c , c ) , (d ,d) , (b , c) , (c , b)}

A = [1 0 0 0

0 1 1 0

0 1 1 0

0 0 0 1]

a

b

c

a b

A A = [1 0 0 0

0 1 1 0

0 1 1 0

0 0 0 1][1 0 0 0

0 1 1 0

0 1 1 0

0 0 0 1] = [

1 0 0 0

0 2 2 0

0 2 2 0

0 0 0 1]

R ∘R ∈ {(a ,a) , (b ,b) , (c , c) , (d ,d) , (b , c) , (c , b)}

c d

d

a

b

c

a b c d

d


Transitivity Test

a e = 1

A = [∗ ∗ ∗ ∗∗ ∗ ∗ ∗∗ ∗ ∗ ∗∗ ∗ ∗ ∗

]1

2

3

1 2 3

4

4

A2 = [

∗ ∗ ∗ ∗a b c d

∗ ∗ ∗ ∗∗ ∗ ∗ ∗

]1

2

3

1 2 3

4

4

[∗ e ∗ ∗∗ f ∗ ∗∗ g ∗ ∗∗ h ∗ ∗

]1

2

3

1 2 3

4

4

= [∗ ∗ ∗ ∗∗ ∗ ∗ ∗∗ ∗ ∗ ∗∗ ∗ ∗ ∗

]1

2

3

1 2 3

4

4

(2,1) ∈ R

(1,2) ∈ R

b f = 1 c g = 1 d h = 1

(2,2) ∈ R

(2,2) ∈ R

(2,3) ∈ R

(3,2) ∈ R

(2,4) ∈ R

(4,2) ∈ R

(2,2) ∈ R

nonzero (i , j)th element of Anonzero (i , j)th element of A2 ⇒


Binary Relations and Digraphs

A = {0,1,2, 3,4, 5,6}

0 1 2 3

R = [1 0 0 1 0 0 1

0 1 0 0 1 0 0

0 0 1 0 0 1 0

1 0 0 1 0 0 1

0 1 0 0 1 0 0

0 0 1 0 0 1 0

1 0 0 1 0 0 1

]4 5 6

0

1

2

3

4

5

6

R ⊂ A×A

0

364

1 2

5

R = {(a ,b)∣a ≡ b (mod 3)}

http://www.math.fsu.edu/~pkirby/mad2104/SlideShow/s7_1.pdf


Reflexive Relation

0 1 2 3

R = [1 0 0 1 0 0 1

0 1 0 0 1 0 0

0 0 1 0 0 1 0

1 0 0 1 0 0 1

0 1 0 0 1 0 0

0 0 1 0 0 1 0

1 0 0 1 0 0 1

]4 5 6

0

1

2

3

4

5

6

0

364

1 2

5

A = {0,1,2, 3,4, 5,6}

R ⊂ A×A

R = {(a ,b)∣a ≡ b (mod 3)}


Symmetric Relation

0 1 2 3

R = [1 0 0 1 0 0 1

0 1 0 0 1 0 0

0 0 1 0 0 1 0

1 0 0 1 0 0 1

0 1 0 0 1 0 0

0 0 1 0 0 1 0

1 0 0 1 0 0 1

]4 5 6

0

1

2

3

4

5

6

0

364

1 2

5

A = {0,1,2, 3,4, 5,6}

R ⊂ A×A

R = {(a ,b)∣a ≡ b (mod 3)}


Transitive Relation

0 1 2 3

RR = [3 0 0 3 0 0 3

0 2 0 0 2 0 0

0 0 2 0 0 2 0

3 0 0 3 0 0 3

0 2 0 0 2 0 0

0 0 2 0 0 2 0

3 0 0 3 0 0 3

]4 5 6

0

1

2

3

4

5

6

0

36

0

36

0

36

A = {0,1,2, 3,4, 5,6}

R ⊂ A×A

R = {(a ,b)∣a ≡ b (mod 3)}


Transitive Relation


Reflexive and Symmetric Closure

1

2

3

4

5

1 2 3 4 5

1

2

3

4

5

1 2 3 4 5

Not Reflexive R Reflexive Closure of Rthe minimal addition

1

2

3

4

5

1 2 3 4 5

1

2

3

4

5

1 2 3 4 5

1

2

3

4

5

1 2 3 4 5

Not Symmetric R Symmetric Closure of Rthe minimal addition

1

2

3

4

5

1 2 3 4 5


Transitive Closure Examples


Transitive Closure

R

n

n

A R∗ = ∪ n=1

∞R

n

= R∪R2∪ ⋯ ∪R

n

A∨A2∨ ⋯ ∨A

n


Equivalence Relation

A = {0,1,2, 3, 4, 5,6, 7, 8}

R ⊂ A×A

R = {(a ,b)∣a ≡ b (mod 3)}

Reflexive Relation &Symmetric Relation &Transitive Relation

Equivalence Relation

(1,1) ∈ R a ≡ 1 (mod 3)

(1,4) ∈ R 1 ≡ 4 (mod 3)

(4,1) ∈ R 4 ≡ 1 (mod 3)

(1,4) ∈ R 1 ≡ 4 (mod 3)

(4,7) ∈ R 4 ≡ 7 (mod 3)

(1,7) ∈ R 1 ≡ 7 (mod 3)


Equivalence Relation Examples

0

36

A = {0,1,2, 3, 4, 5,6, 7,8}

R ⊂ A×A

R = {(a ,b)∣a ≡ b (mod 3)}

1

47

2

58

0 1 2 3

RR = [1 0 0 1 0 0 1 0 0

0 1 0 0 1 0 0 1 0

0 0 1 0 0 1 0 0 1

1 0 0 1 0 0 1 0 0

0 1 0 0 1 0 0 1 0

0 0 1 0 0 1 0 0 1

1 0 0 1 0 0 1 0 0

0 1 0 0 1 0 0 1 0

0 0 1 0 0 1 0 0 1

]4 5 6

0

1

2

3

4

5

6

7 8

7

8


Equivalence Classes

0

36

1

47

2

58

[1 1 1

1 1 1

1 1 1] [1 1 1

1 1 1

1 1 1] [1 1 1

1 1 1

1 1 1]

Class #1 Class #2 Class #3

0 3 6

0

3

6

1 4 7

1

4

7

2 5 8

2

5

8

P1

= {0,3,6} P2 = {1, 4,7} P3 = {2,5, 8}partition partition partition


Equivalence Class

A = Z+ = {0,1,2,3, 4,5, 6,⋯}

R ⊂ A×A

R = {(a ,b)∣a ≡ b (mod 3)}

{0, 3, 6, 9, ⋯ }

{1, 4, 7, 10, ⋯ }

{2, 5, 8, 11, ⋯ }

[0 ]

[1 ]

[2 ]

[33]

[331]

[3332]

https://www.cse.iitb.ac.in/~nutan/courses/cs207-12/notes/lec7.pdf


References

[1] http://en.wikipedia.org/[2]

Young Won Lim4/4/18

The Growth of Functions (2A)

Young Won Lim4/4/18

Copyright (c) 2015 - 2018 Young W. Lim.





3 Young Won Lim4/4/18

Functions and Ranges

x2

2 x

1

x2+2 x+1 A1 = [0,5]

A2 = [0,100]

A3 = [0,500]



Small Range

Zoom Out

All are distinguishable

x2+2 x+1

x2

2 x

1

A1

for x >−0.5 x2 < x

2+2 x+1



Medium Range

Zoom Out More2 x

similar

x2+2 x+1

x2

A2

for x >−0.5 x2 < x

2+2 x+1

10000+20110000



Large Range

Indistinguishable

x2+2 x+1

x2

A3

for x >−0.5 x2 < x

2+2 x+1

250000+1001250000




2⋅x2

2 x

1

x2+2 x+1

B1 = [0,5]

B2 = [0,100]

B3 = [0,500]



Small Range, 2x2

Zoom Out

distinguishable

x2+2 x+1

2 x2

2 x

1

B1

for x > 2.414 x2+2 x+1 < 2 x

2

for x > k f (n) < C g (n)

25+11

50



Medium Range, 2x2

Zoom Out

distinguishable

x2+2 x+1

2 x2

2 x

1

B2

for x > 2.414 x2+2 x+1 < 2 x

2


10000+201

20000



Large Range, 2x2

Zoom Out

distinguishable

x2+2 x+1

2 x2

2 x

1

B3

for x > 2.414 x2+2 x+1 < 2 x

2


250000+1001

500000




10⋅x2

2 x

1

x2+2 x+1

C1 = [0,5]

C2 = [0,100]

C3 = [0,500]



Small Range, 10x2

Zoom Out

distinguishable

x2+2 x+1

10 x2

2 x1

C1

for x > 0.462 x2+2 x+1 < 10 x

2


25+11

250



Medium Range, 10x2

Zoom Out

distinguishable

x2+2 x+1

10 x2

C2

for x > 0.462 x2+2 x+1 < 10 x

2


10000+201

100000



Large Range, 10x2

distinguishable

x2+2 x+1

10 x2

C3

for x > 0.462 x2+2 x+1 < 10 x

2


250000+1001

2500000




10⋅x

x2

1

x2+2 x+1

D1 = [0,5]

D2 = [0,100]

D3 = [0,500]



Small Range, 10x

Zoom Out

distinguishable

x2+2 x+1

x2

10 x

1

D1

for 0.127 < x < 7.873 x2+2 x+1 < 10 x



Medium Range, 10x

Zoom Out

indistinguishable x2+2 x+1

x2

D2

10 x

for x > 7.873 10 x < x2+2 x+1

for x > k C g(n) < f (n)



Large Range, 10x

indistinguishable

D3

x2+2 x+1

x2

10 x

for x > 7.873 10 x < x2+2 x+1

for x > k C g(n) < f (n)



Big-O Definition

Let f and g be functions (Z→R or R→R)from the set of integers or

the set of real numbersto the set of real numbers.

We say f(x) is O(g(x)) “f(x) is big-oh of g(x)”

If there are constants C and k such that

|f(x)| ≤ C|g(x)| whenever x > k.

g(x) : upper bound of f(x)



Big-Ω Definition

Let f and g be functions (Z→R or R→R)from the set of integers or

the set of real numbersto the set of real numbers.

We say f(x) is Ω(g(x)) “f(x) is big-omega of g(x)”

If there are constants C and k such that

C|g(x)| ≤ |f(x)| whenever x > k.

g(x) : lower bound of f(x)



Big-O Definition

f (x)≤ C|g(x)|

f (x) is O(g(x))

for k < x

x=k

f (n)

C g(n)

x

g(x) has a simpler form than f(x)

is usually a single term

g(x) : upper bound of f(x)



Big-Ω Definition

for k < x

f (x)f (x)≥ C|g(x)|

f (x) is Ω (g(x))

C g(x)

g(x) : lower bound of f(x)

x=kx

g(x) has a simpler form than f(x)

is usually a single term



Big-Θ definition

C|g(x)|≤ f (x)

for k < x

f (x) is Ω (g(x))

f (x)≤ C|g(x)| f (x) is O(g(x))

C1|g(x)|≤ f (x)≤ C2|g(x)| f (x) is Θ (g(x))



Big-Θ = Big-Ω ∩ Big-O

for k < x

Ω(g(x)) ∧ O(g(x))

C1|g(x)|≤ f (x)≤ C2|g(x)| f (x) is Θ (g(x))

Θ (g(x))

Ω (g(x ))

O(g(x ))



Θ(x) and Θ(1)

C x ≤ f (x)

for 0 < k < x

f (x) ≤ C x

C1 x ≤ f (x) ≤ C2 x

f (x) is Ω (x)

f (x) is O (x)

f (x) is Θ (x )

C⋅1 ≤ f (x)

f (x) ≤ C⋅1

C1⋅1 ≤ f (x)≤ C2⋅1

f (x) is Ω (1)

f (x) is O (1)

f (x) is Θ (1)

f (x)

f (x)



Big-O, Big-Ω, Big-Θ Examples

for x >−0.5 1 x2 < x

2+2 x+1

for x > 2.414 x2+2 x+1 < 2 x

2

for x > 0.462 x2+2 x+1 < 10 x

2

for x > 7.873 10 x < x2+2 x+1

x2+2 x+1 is Ω (x2)

x2+2 x+1 is O (x2)

x2+2 x+1 is O (x2)

x2+2 x+1 is Ω (x )

x2+2 x+1 is Θ (x2)

lower bound

lower bound

upper bound

upper bound



Many Larger Upper Bounds

x2+2 x+1 is O (x2)

x2+2 x+1 is O (x3)

x2+2 x+1 is O (x4)

x2+2 x+1

2⋅x2x

3

x4

the least upper bound?



Many Smaller Lower Bound

x2+2 x+1 is Ω (x )

x2+2 x+1 is Ω (√x )

x2+2 x+1 is Ω ( log x )

√x

x

x2+2 x+1

log x

x2+2 x+1 is Ω (x2)

the greatest lower bound?



Many Upper and Lower Bounds

x2+2 x+1 is O (x2)

x2+2 x+1 is O (x3)

x2+2 x+1 is O (x4)

x2+2 x+1 ≤ C x

2

x2+2 x+1 ≤ C x

3

x2+2 x+1 ≤ C x

4

x2+2 x+1 is Ω (x )

x2+2 x+1 is Ω (√x )

x2+2 x+1 is Ω ( log x )

x2+2 x+1 ≥ C x

x2+2 x+1 ≥ C √x

x2+2 x+1 ≥ C log x

lower bound

upper bound

lower bound

lower bound

upper bound

upper bound

the greatest

the least

x2+2 x+1 is O (x2) x

2+2 x+1 ≥ C x2 upper bound



Simultaneously being lower and upper bound

Θ (g(x))

Ω (g(x )) O (g(x )) upper bounds

x2

x3

x2log x

x3√x

x4

x

√x

log x

lower bounds

tight bound

f (x ) = x2+2 x+1



Big-Θ Examples (1)

x2+2 x+1 is Θ (x2)

2⋅x2

1⋅x2

x2+2∗x+1

Zoom Out

0.5⋅x2

x2+2 x+1 is O (x2)

x2+2 x+1 is Ω (x2)



Big-Θ Examples (2)

x2+2 x+1 is Θ (x2)

2⋅x2

1⋅x2

x2+2∗x+1

0.5⋅x2



Big-Θ Examples (3)

6 x3+2 x

2+4 is Θ (x3)

7⋅x3

5⋅x3

6 x3+2 x

2+4 ≈ 6⋅x3

indistinguishable 6⋅x36 x

3+2 x2+45⋅x3 ≤ 6 x

3+2 x2+4 ≤ 7⋅x3



Tight bound Implications

f (x ) is Θ (g(x )) f (x ) is O(g(x ))

f (x ) is Ω(g(x ))f (x ) is Θ (g(x ))

f (x ) is Θ (g(x )) f (x ) is O(g(x ))

f (x ) is Ω(g(x ))f (x ) is Θ (g(x ))

Θ (g( x))

Ω (g( x )) O (g(x ))



Common Growth Functions

x2 x log x x

√x

log x



Upper bounds

x2 x log x x

√x

log x

f 1(x) is O(log x) O (√x) O (x) O (x log x) O (x2)

f 1( x)



Lower bounds

x2 x log x x

√x

log x

f 2(x)

f 2(x) is Ω (x2) Ω(x log x) Ω(x) Ω(√ x) Ω(log x)



Example 1

f (n)=n6+3n

f (n)=2n+12

f (n)=2n+3

n

f (n)=nn+n

f (n)=O(n6)

f (n)=O(2n)

f (n)=O(3n)

f (n)=O (nn)

f (n)=Ω(n)

f (n)=Ω(1)

f (n)=Ω(2n)f (n)=Ω(n)

https://discrete.gr/complexity/



Example 2

f (n)=n6+3n

f (n)=2n+12

f (n)=2n+3

n

f (n)=nn+n

f (n)=O(n6)

f (n)=O(2n)

f (n)=O(3n)

f (n)=O (nn)


f (n)=Ω(n6)

f (n)=Ω(2n)

f (n)=Ω(3n)

f (n)=Ω(nn)

f (n)=Θ(n6)

f (n)=Θ(2n)

f (n)=Θ(3n)

f (n)=Θ(nn)



Example 3

Θ(n)

Θ(n)

Θ(n2)Θ(n)

O(n)

O (n2)

O (n3)O (1)


O(1) Θ(1)

O(n) Θ(1)

Θ(n) → O(n) → O(n2)

Θ(n2) → O (n2) → O(n3)Θ(n) → Ω(n) → O(1)

Θ(1) ≡ Ω(1) ≡ O (1)

log(n) : O (log n) → O (n)log (n) : Ω(logn) → O(1)



Example 4

Θ(n)

Θ(n2)Θ(1)

Θ(n)

O(n)

O(n3)O(n)

O (1)


Θ(n) O (2n)

upper bound

upper bound

upper bound

upper bound

upper bound

tight

wrong

tight O (2n) = O (n)

Θ(n) → Ω(n) → O(1)



Example 5

Θ(1)

Θ(√n)

Θ(n)

Θ(n2)

O (1)

O (√n)

O (n)

O(n2)


Θ(n3) O(n3)

upper bound

upper bound

upper bound

upper bound

upper bound

tight

tight

tight

tight

tight

O (n)

O (n)

O(n2)

O (n3)

O (n4)

upper bound

upper bound

upper bound

upper bound

upper bound

Θ(1)

Θ(√n)

Θ(n)

Θ(n2)

Ω(1)

Ω(√n)

Ω(n)

Ω(n2)

Θ(n3) Ω(n3)

lower bound

lower bound

lower bound

lower bound

lower bound

tight

tight

tight

tight

tight

Ω(1)

Ω(√n)

Ω(n)

Ω(n2)

lower bound

lower bound

lower bound

lower bound

Young Won Lim4/4/18

References


Young Won Lim4/6/18

The Complexity of Algorithms (3A)



Complexity Analysis


● to compare algorithms at the idea level

ignoring the low level details

● To measure how fast a program is

● To explain how an algorithm behaves

as the input grows larger



Counting Instructions


● Assigning a value to a variable

● Accessing a value of a particular array element

● Comparing two values

● Incrementing a value

● Basic arithmetic operations

● Branching is not counted

x= 100;

A[i]

(x > y)

i++

+, –, *, /

if else



Asymptotic Behavior


● avoiding tedious instruction counting

● eliminate all the minor details

● focusing how algorithms behaves when treated badly

● drop all the terms that grow slowly

● only keep the ones that grow fast as n becomes larger



Finding the Maximum


M = A[0];

for (i=0; i<n; ++i) {

if (A[i] >= M) {

M = A[i];

}

}

int A[n]; // n element integer array A

int M; // the current maximum value found so far

// set to the 1st element, initially

// M is set to the 1st element

// if the (i+1)th element is greater than M,

// M is set to that element (new maximum value)



Worst and Best Cases


A[0]

A[1]

A[2]

A[3]

i=0

i=1

i=2

i=3

int A[4];

A[0]=1

A[1]=2

A[2]=3

A[3]=4

Case 1: Worst Case

A[0]=4

A[1]=3

A[2]=2

A[3]=1

Case 2: Best Case

M=1

M=2

M=3

M=4

M=4

for (i=0; i<n; ++i) {

if (A[i] >= M) { // always n comparisons

M = A[i]; // the updating of M depends on the data

} // minimum 1 update, maximum n updates



Assignment


M = A[0]; // 2 instructions

for (i=0; i<n; ++i) {

if (A[i] >= M) {

M = A[i];

}

}

A[0] – 1 instruction

M = – 1 instruction



Loop instructions


for (i=0; i<n; ++i) {

if (A[i] >= M) {

M = A[i];

}

Initialization * 1

i=0 : 1 instruction

i<n : 1 instruction

Update * n

++i : 1 instruction

i<n : 1 instruction

Loop body * n

A[i] : 1 instruction

>= M : 1 instruction

A[i] : 1 instruction

M= : 1 instruction

n

1 ~ n

always

depending on the comparison



Worst case examples


A[0]=1

A[1]=2

A[2]=3

A[3]=4

>= M=1

for (i=0; i<n; ++i) {

if (A[i] >= M) {

M = A[i];

}

M=1

A[0]=1

A[1]=2

A[2]=3

A[3]=4

>= M=1

M=2

A[0]=1

A[1]=2

A[2]=3

A[3]=4

>= M=2

M=3

A[0]=1

A[1]=2

A[2]=3

A[3]=4 >= M=2

M=3

i=0 i=1

i=2 i=32n + 2n = 4n

Instructions

n comparisons

n updates



Best case examples


A[0]=4

A[1]=3

A[2]=2

A[3]=1

>= M=4

for (i=0; i<n; ++i) {

if (A[i] >= M) {

M = A[i];

}

M=4

A[0]=4

A[1]=3

A[2]=2

A[3]=1

< M=4

A[0]=4

A[1]=3

A[2]=2

A[3]=1

< M=4

A[0]=4

A[1]=3

A[2]=2

A[3]=1 < M=4

i=0 i=1

i=2 i=32n + 2

Instructions

n comparisons

1 update



Asymptotic behavior


M = A[0];

for (i=0; i<n; ++i) {

if (A[i] >= M) {

M = A[i];

}

}

2 instructions

2 + 2n instructions (init + update)

2n instructions

2 ~ 2n instructions

6n+4 instructions for the worst case

4n+6 instruction for the best casef(n) =

f(n) = O(n)

f(n) = Ω(n)

f(n) = Θ(n)



O(n) codes

https://stackoverflow.com/questions/11032015/how-to-find-time-complexity-of-an-algorithm

// Here c is a positive integer constant

for (i = 1; i <= n; i += c) {

// some O(1) expressions

}

for (int i = n; i > 0; i -= c) {


}



O(n2) codes


for (i = 1; i <=n; i += c) {

for (j = 1; j <=n; j += c) {


}

}

for (i = n; i > 0; i += c) {

for ( j = i+1; j <=n; j += c) {


}



O(log n) codes


for (int i = 1; i <=n; i *= c) {


}

for (int i = n; i > 0; i /= c) {


}



O(n) vs. O(log n)




O(log n) codes


// Here c is a constant greater than 1

for (int i = 2; i <=n; i = pow(i, c)) { // i = i^c


}

//Here fun is sqrt or cuberoot or any other constant root

for (int i = n; i > 0; i = fun(i)) { // i = i^(1/c)


}

i = i2, i = i

3



O(log log n) codes





}




}

i = i2 (2,22

,24,2

8,2

16,⋯)

i = i1

2 (n ,n1

2 , n1

4 , n1

8 , n1

16 ,⋯)



Some Algorithm Complexities and Examples (1)


O(1) – Constant Time

not affected by the input size n.

O(n) – Linear Time

Proportional to the input size n.

O(log n) – Logarithmic Time

recursive subdivisions of a problem

binary search algorithm

O(n log n) – Linearithmic Time

Recursive subdivisions of a problem and then merge them

merge sort algorithm.



Some Algorithm Complexities and Examples (2)


O(n2) – Quadratic Time

bubble sort algorithm

O(n3) – Cubic Time

straight forward matrix multiplication

O(2^n) – Exponential Time

Tower of Hanoi

O(n!) – Factorial Time

Travel Salesman Problem (TSP)

Young Won Lim4/6/18

References


Relations (3A) - upload.wikimedia.org · Version 1.2 or any later version published by the Free...

Documents

Transcript of Relations (3A) - upload.wikimedia.org · Version 1.2 or any later version published by the Free...