COEN 352 Data Structures and Algorithms: Mathematical...
Transcript of COEN 352 Data Structures and Algorithms: Mathematical...
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
COEN 352 Data Structures and Algorithms:Mathematical Preliminaries
Stuart Thiel
Concordia UniversityDepartment of Electrical Computer Engineering
Summer, 2015
1/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Outline
Notation
Sets
Sets and Relations
Identities
Recursion
Proof Techniques
2/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Size vs. Time
I Size is often a power of 2
I Time is usually decimal
I Shaffer talks about his preferred notation in 2.2, I willattempt to avoid ambiguity
3/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Stirling’s Approximation of n!
I n! ≈√
2πn(ne )n
I We often write n! but are using the approximation forresulting analysis
I proving this not for the faint of heart, but fascinating!
I This and other stuff Stirling did is critical to theunderstanding of advanced algorithm analysis.
4/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Combinatorics
I Permutation S is the members of S arranged in someorder
I n members leads to n! possible permutations
I Often need to generate random permutations foranalysis
I How many do you need to test?
I What does “random” mean?
I How real is “random”?
5/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Notations
I A boolean value is true or false
I A =⇒ B means if A then B
I A ∧ B means A and B
I A ∨ B means A or B
I A means ∼ A means not A, or the negation of AI a%b is the modulus operator, gives remainder of integer
divisionI different languages can do modulus differently, be
careful
6/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
More Notations
I dxe ceiling rounds up to the nearest larger integer
I bxc floor rounds down to the nearest smaller integer
7/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Sets
I useful abstraction for algorithm design
I distinguishable elements of a base type.
I elements are primitive elements of base type
I elements may also be sets
I no duplication
I no order
8/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Example
I P = {7, 11, 42}I elements are 7, 11 and 42
I P is the set
I Integers are the base type
I Sets can be empty: ∅
9/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Definitions
I enumerating each element can be long
I formal definition can communicate well
I “All positive integers”
I “All even integers”
I “All prime numbers”
10/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Definition Example
I P = {x |x is a positive integer}I P = {x |x > 0}I more specific: P = {x |x > 0 ∧ x ∈ Z}
11/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Elements
I in a set
I not in a set
I cardinality of a set
12/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Elements Example
I x ∈ P
I x /∈ P
I Q = {7, 11, 42}I |Q| = 3
13/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Other Sets
I subset
I supersets
I union
I intersection
I difference
14/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Other Sets Example 1
I P = {x |x > 0 ∧ x ∈ Z}I Q = {x |x ∈ Z}I P ⊆ Q
I Q ⊇ P
15/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Other Sets Example 2
I P = {4, 12, 18}I Q = {10, 12, 22}I P ∩ Q = {12}I P ∪ Q = {4, 10, 12, 18, 22}I P \ Q = {4, 18}
16/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Powerset
I All possible subsets of a set
I Includes empty set!
17/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Bags
I Bags are Sets that allow duplication
I using square brackets is a common distinction
I P = [4, 12, 12, 18]
I also called multisets
I useful in advanced analysis!
18/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Sequence
I A bag with an order
I using angle brackets is a common distinction
I P = 〈4, 12, 12, 18〉I They need not be ordered a particular way
I also called tuple or vector
I like our implemented data structures
I 〈4, 12, 12, 18〉 6= 〈12, 4, 12, 18〉
19/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Context
I Set? Bag? Sequence?
I The context is often the guide
I If lucky, brackets distinguish
I Sorting problem? Looks like sets, probably sequences.
20/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Relations
I A relation over a set is a set of order pairs
I A set of sequences of cardinality 2
I elements in the ordered pairs are elements from the set
21/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Relations example
I Set P = {a, b, c}I Relation R1 = {〈a, c〉, 〈b, c〉, 〈c , b〉}I Relation R2 = {〈a, a〉, 〈a, c〉, 〈b, b〉, 〈b, c〉, 〈c , c〉}I How the left item relates to the right item
I lRr
I How many different relations can there be on a set?
22/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Properties of Relations
I Useful to classify relations
I Good for describing, identifying types of problem
I reflexive, ∀a ∈ S : aRa
I symmetric, ∀a, b ∈ S : aRb =⇒ bRa
I antisymmetric, ∀a, b ∈ S : aRb ∧ bRa =⇒ a = b
I transitive, ∀a, b, c ∈ S : aRb ∧ bRc =⇒ aRc
I useful to group of these properties
23/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Properties of Relations Example
I ordinal relation of less than or equal is reflexive
I being a partner is symmetric
I natural numbers are antisymmetric
I being a relative is transitive
24/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Equivalence Relation
I reflexive, symmetric and transitiveI breaks a set into equivalence classesI These partitions are disjoint from all othersI Elements in these partitions are “equivalent”
I “less than or equal” is an ok example
I modulus is a better example
25/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Partial Order Relation
I Partial Ordering Definitions Differ. We use Shaffer.
I antisymmetric and transitive
I “older than” is a good example
26/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Advantage of Identities
I You can replace expressions with other equivalentexpressions
I You can use identities to change the tools you can useI summation may be tricky in math, but it has an identity
that is regular arithmeticI Identities that lead to arithmetic expressions are called
closed-form solutions
I You can make patterns more visible
27/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Outline I
Notation
Sets
Sets and Relations
IdentitiesLogarithmsSummation
Recursion
Proof Techniques
28/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Logarithms
I logarithm operation finds the exponent that a givenbase is raised to in order to equal the parameter
I log usually implies base 2
I ln usually implies base e
I log 2b = b
I ln eb = b
I log3 3b = b
29/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Logarithmic Identities
I log ab = log a + log b
I log a/b = log a− log b
I log xb = b log x
I log b√x = 1
b log x
I eg: log3 81 = log3 92 = 2 log3 9 = 2 log3 32 = 4
30/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Outline I
Notation
Sets
Sets and Relations
IdentitiesLogarithmsSummation
Recursion
Proof Techniques
31/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Analysing Loops and Recurrences
I We want to count costs
I We are looking at algorithms
I algorithms have repeating structures
I count these structures
I evaluate the cost of each
I summation gives us a tool to help
I a powerful tool for analysis
In∑
i=1f (i)
32/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Summation identities
In∑
i=1i = n(n+1)
2
In∑
i=1i2 = n(n+1)(2n+1)
6
33/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Identifying Recurrence Relations
I Looking for closed-form solutions can be done
I Simply, one can try to unfold a summation thatdescribes an algorithm
I You look for a pattern
I Using identities can swap out unfamiliar expressions andhelp you see patterns
I An advanced topic, but we will look at one importantexample analysing a sorting algorithm later this semester
34/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Loops or Recursion
I an algorithm may call itself on a smaller sub-problem
I this makes it a recursive algorithm
I such algorithms have one or more base cases (orterminating cases) that can give a direct solutionwithout recursion
I such algorithms otherwise break the problem down,including the recursive call that will eventually lead to abase case
35/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Recursion Example
I Factorials
I Towers of Hanoi
36/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Proofs
I Do not be sloppy
I Understanding proofs is key
I As Engineers, proofs are the basis for justifyingeverything you do
I You should always be comfortable justifying everythingyou do
I People question you, but you should always be able toprove what you did is correct
37/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Direct Proof
I The easy one. Logical Explanation.
I Logic or arithmetic leads directly there.
I I also include the use of accepted identities here.
I If the exam is tomorrow, it is not today. Today is nottomorrow.
38/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Proof by Contradiction
I Can disprove with one counter-example.
I No number of examples can prove something true.
I Not true, ALL examples are a proof, but that is usuallyintractable.
I We can do something similar:
I Assume what you want to prove is false.
I Find a counter-example for this.
I eg: There is no largest integer!
39/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Proof by Induction
I Looks like Recursion, really
I Not a coincidence!
I Base case(s)
I Induction Step
40/41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Proof by Induction Examples
I Summation
I Coin Example
41/41