Discrete Mathematics Summer 2006
35
1 Discrete Mathematics Summer 2006 By Dan Barrish-Flood originally for Fundamental Algorithms For use by Harper Langston in D.M.
description
Discrete Mathematics Summer 2006. By Dan Barrish-Flood originally for Fundamental Algorithms For use by Harper Langston in D.M. What is an Algorithm?. A problem is a precise specification of input and desired output - PowerPoint PPT Presentation
Transcript of Discrete Mathematics Summer 2006
1
Discrete Mathematics Summer 2006
By Dan Barrish-Flood originally for Fundamental Algorithms
For use by Harper Langston in D.M.
2
• A problem is a precise specification of input and desired output
• An algorithm is a precise specification of a method for solving a problem.
What is an Algorithm?
3
What We Want in an Algorithm
• Correctness– Always halts– Always gives the right answer
• Efficiency– Time– Space
• Simplicity
4
A Typical Problem
Sorting
Input: A sequence b1, b2, ... bn
Output: A permutation of that sequence b1, b2, ... bn such
that
nbbbb 321
5
A Typical Algorithm
Insertion -Sort(
j 2
2
Insert sorted
sequence
and
6
8
A
length A
key A j
A j
A j
i j
i A i key
A i A i
i i
A i key
)
[ ]
[ ]
[ ] into the
[ .. ]
[ ]
[ ] [ ]
[ ]
1
3
1 1
4 1
5 0
1
7 1
1
for to
do
while
do
6
Running Time
Insertion - Sort(
j 2
2
Insert into the sorted
sequence
and
6
8
A
length A
key A j
A j
A j
i j
i A i key
A i A i
i i
A i key
)
[ ]
[ ]
[ ]
[ .. ]
[ ]
[ ] [ ]
[ ]
1
3
1 1
4 1
5 0
1
7 1
1
for to
do
while
do
1-n
)1(t c
)1(t c
t c
1-n c
1-n 0
1-n c
1 -n
8
n
2j=j7
n
2j=j6
n
2j=j5
4
2
1
c
c
timesCost
tJ is the number of times the “while” loop test in line 5 is executed for that value of j. Total running time is
c1 (n) + c2(n-1) +…+c8(n-1)
7
• Trying to determine the exact running time is tedious, complicated and platform-specific.
• Instead, we’ll bound the worst-case running time to within a constant factor for large inputs.
• Why worst-case?– It’s easier.– It’s a guarantee.– It’s well-defined.– Average case is often no better.
A Better Way to Estimate Running Time
8
Insertion Sort, Redux
• Outer loop happens about n times.
• Inner loop is done j times for each j, worst case.
• j is always < n
• So running time is upper-bounded by about
n n n 2
“Insertion sort runs in (n2) time, worst-case.”
9
• We’ll make all the “about”s and “rough”s precise.
• These so-called asymptotic running times dominate constant factors for large enough inputs.
10
Functions
• constants
• polynomials
• exponentials
• logarithms and polylogarithms
• factorial 1...)1(!
log ,log
3 ,2
52 ,
1 ,0
22
n
3
2
nnn
nn
nnn
11
,
E.g. number. real some is c where
termsof Sum generally, (More
200552 E.g
32/1
32103
0
nnn
an
nnnnn
na
c
d
i
ii
Polynomials
12
f(n) is monotonically increasing iff
I.e. It never “goes down” — always increases or stays the same.
For strictly increasing, replace in the above with <.
I.e. The function is always going up.
impliesm n
f m f n
( ) ( ).
13
Are constant functions monotonically increasing? Yes! (But not strictly increasing.)
If , then is strictly increasing.
So even increases forever to
c nc 0
n n.001 1000 ( ) .
14
Exponentialsf n a
a
a a
a a a a
a a a
a a a
a a
n
n
m n m n
n n n
m n mn n m
n n
n
n
n
( )
/
( ) ( )
,
( )
Facts:
e.g.
e.g.
If is strictly increasing to
0
1
1
1 1 12
1
1
2 2 2 2
2 2 2
1
1
21
2
15
Logarithms
.0log ,10
01log
1log
:Recall
) (log log log log
)(loglog
logln
loglg
:notation Some
.such that number that log
2
xxFor
b
nn
nn
nen
nn
nxbxnb
b
b
b
kk
16
Log Facts1. n = b log
b n (definition)
2. logb(xy)=logbx + logby
3. logban = nlogba
4. logbx = logcx / logcb
5. logb(1/a) = -logba
6. logba = 1/(logab)
7. alogbc= clog
ba
• if b > 1, then logbn is strictly increasing.
17
More About Logs
A polylogarithm is just
For and , is strictly
increasing to
log
log
.
k
k
n
k n n
0 0
18
Factorial
nn
nn
en
n
i
nn
nn
n
i
nnnn
!2 3,>nFor
:seecan weStirling,ut Even witho
))(1()( 2!
:ionapproximat sStirling'
toincreasingstrictly is !
)definition(by 1!0
1)2()1(!
1
1
19
More About Monotonically Increasing Functions
If and are monotonically
increasing, then so are
and if and are also then
is monotonically increasing, too.
Examples:
f n g n
f n g n
f g n
f n g n
f n g n
n n
n
nn
( ) ( )
( ) ( )
( ( ))
( ) ( ) ,
( ) ( )
log
log log
0
2 3
20
Bounding Functions• We are more interested in finding upper and lower
bounds on functions that describe the running times of algorithms, rather than finding the exact function.
• Also, we are only interested in considering large inputs: everything is fast for small inputs.
• So we want asymptotic bounds.• Finally, we are willing to neglect constant factors
when bounding. They are tedious to compute, and are often hardware-dependent anyway.
21
Example
05
1015
20
0 1 2 3
3n+8
n
0
10
20
30
0 1 2 3
3n+8
10n
but when n 2, 3n + 8 is upper -
bounded by
10n.
n
n
by
bounded-uppernot certainly is 83
22
Notation for Function Bounds
choices.]other many are there
;2 and 10 choose :[Proof
)(83
E.g.
allfor )()(
such that and constants positive
exist two thereiff ))(()(say We
)oh"-big(" O :BoundUpper
day.)every themuse llWe'
course. in the sdefinitionimportant most (The
0
0
0
nc
nOn
nnncgnf
nc
ngOnf
23
Big-Oh Facts
• It denotes an upper bound, not necessarily a tight bound. Thus,
O g n f n( ( )) ( )• The “=” is misleading. E.g
is meaningless.
etc. ),2(
)(
)(2
nOn
nOn
nOn
• Transitivity:
))(()(then
))(()( and ))(()( If
nhOnf
nhOngngOnf
24
Examples
We’ve seen that 3n + 8 = O(n).
Is 10n = O(3n + 8)?
Yes: choose c = 10, n0 = 1:
10n n+ 80
25
More Examples
2n2 + 5n - 8 = O(n2)?
Yes:
2n2 + 5n - 8 cn2
n2: 2 + 5/n - 8/n2 c
Choose c = 3; subtracting 2:
5/n - 8/n2 Choose no = 5
26
More Examples
constant.any by bounded-upper becannot it - to
increasingstrictly is )2/3( ,1 3/2 sinceBut
)2/3(
2/3 :2
23
?)2(3 Is
)1 ,1( 32 fact,in :Yes
?)3(2 Is
0
n
n
nnn
nn
nn
nn
nn
c
c
c
O
nc
O
27
Lower Bound Notation
f n g n
c n n n
n n
f n O g n g n f n
n n
( ) ( ( ))
.
( )
( )
( ) ( ( )), ( ) ( ( )),
iff there are positive constants
and such that f(n) cg(n) for all
(Same as definition of O, with replaced by .)
E.g.
In fact:
If then
and vice versa.
0 0
2
3 2
28
“Tight” Bound Notation (Both upper and lower)
iff there are positive constants
and such that
for all
f n g n
c c n
c g n f n c g n n n
( ) ( ( ))
, ,
( ) ( ) ( ) .
1 2 0
1 2 0
0
2
4
6
8c g n2 ( )
f n( )
n0
29
More Facts
is transitive, just like and .
iff
and
iff
and
iff
O
f n g n f n O g n
f n g n
f n g n f n O g n
g n O f n
f n g n g n f n
( ) ( ( )) ( ) ( ( ))
( ) ( ( ))
( ) ( ( )) ( ) ( ( ))
( ) ( ( ))
( ) ( ( )) ( ) ( ( ))
30
Examples
growth. clogarithmi allfor )(lg can write weTherefore,
factor.constant aby differ functions log Since
)(loglog
versa.not vicebut ),(
:because is This ).(
].2 ,1[ )(852 Also,
)(852at earlier th saw We
:because is This ).(852
).(83
so )83( and ),(83 seen that veWe'
21
32
32
022
22
22
n
nn
nOn
nn
ncnnn
nOnn
nnn
nn
nOnnOn
bb
31
Another Upper-Bound Notation
)."( respect to
withnegligible cally)(asymptoti is )("
as ))(()( ofThink
).(not is 2
)(2 E.g.
.0)(
)(
iff ]oh" little[" ))(()(say We
22
2
nlim
ng
nf
ngonf
non
non
ng
nf
ngonf
32
Functions and Bounds
.)0)(
)( so ,)( :(Proof
)).(()( then , toincreasing
strictly is )( andfunction constant a is )( If
.)conventionby 1 use but we etc.,
),3(),2( (also ).1( are functionsconstant All
:functionsConstant
lim
ng
nfng
ngonf
ngnf
n
33
Logarithms
on. so and
)(lg)lg(lg see we(*),in for lg ngSubstituti
1 while)(huge! 2lg
,4for fact that theof spitein
)(lg
-lysurprising perhaps -So
.0 where
, constantsany for )(log (*)
fact,In
)(lg
01.100100
01.100
b
nonnn
nn
n
non
a
banon
non
ab
a
34
Polynomials
)001.1( Thus,
als)(exponenti spolynomial i.e. 1 ),(
),1 (where , to)0 (where ,2 Renaming
2) base log means (recall ))2((
))2(())2(lg(
:have we
0,a ),(lgin for 2 ngSubstituti
class.next for Homework
:Proof
).( is positive ist coefficien
leading whose degree of polynomialAny
100 n
nb
a
nab
anbn
abn
d
on
oaaon
aaa
n lgon
o
nonn
n
d
35
Comparing Functions
vs. n vs.
vs. vs.
vs. vs.
vs. lgn vs.
vs.
n n n
n n n
n n n n n
n n n
n n n
n n
n n
lg
lg
lg lg( )
lg ln lg
/ lg lg
/
lg
2
2
2
2
2 2
