Algorithms and Data Structures Conditional...
Transcript of Algorithms and Data Structures Conditional...
Algorithms and Data StructuresFabian Kuhn
Lecture 2
Runtime Analysis, Sorting II
Algorithms and Data StructuresConditional Course
Fabian Kuhn
Algorithms and Complexity
Algorithms and Data StructuresFabian Kuhn
โข How can we analyze the runtime of an algorithm?โ runtime is different on different computersโฆ
โ depends on compiler, programming language, etc.
โข We need an abstract measure to express the runtime
โข Idea: Count the number of (basic) operationsโ instead of directly measuring the time
โ the number of basic operations is independent of computer, compiler
โ It is a good measure for the runtime if all basic operations require about thesame time.
2
Runtime Analysis I
Algorithms and Data StructuresFabian Kuhn
What is a basic operation?
โข Simple arithmetic operations / comparisonsโ +, -, *, /, % (mod), <, >, ==, โฆ
โข One memory accessโ reading or writing a variable
โ not clear if this is really a basic operation?
โข One function callโ Of course only jumping to the function code
โข Intuitively: one line of program code
โข Better: one line of assembly language code
โข Even better (?): one processor cycle
โข We will see: It is only important that the number of basic opertionsis roughly proportional to the actual running time.
3
Basic Operations
Algorithms and Data StructuresFabian Kuhn
RAM = Random Access Machine
โข Standard model to analyze algorithms!
โข Basic operations (as โdefinedโ) all require one time unit
โข In particular, all memory accesses are equally expensive:
Each memory cell (1 machine word) can be read or written in 1 time unitโ In particular ignores memory hierarchies
โ In most cases, it is however a reasonable assumption
โข There are alternative abstract models:โ to explicitly capture memory hierarchies
โ for huge data volumes (cf. big data)
โข e.g.: streaming-models: memory has to be read sequentially
โ for distributed / parallel architectures
โข memory access can be local or over the networkโฆ
4
RAM Model
Algorithms and Data StructuresFabian Kuhn
So far: Number of basic operations is proportional to the runtime
โข We can also achieve this without counting thebasic operations exactly!
Simplification 1: We only calculate an upper bound (or a lowerbound) on the number of basic operations
โ such that the upper / lower bound is still proportional to the runtimeโฆ
โข No. of basic op. can depend on several properties of the inputโ Size/length of input, but, e.g., for sorting also the ordering in the input
Simplification 2: Most important parameter is input size ๐We always consider the runtime ๐(๐) as a function of ๐.
โ And we ignore other properties of the input
5
Runtime analysis II
Algorithms and Data StructuresFabian Kuhn
SelectionSort(A):
1: for i=0 to n-2 do
2: minIdx = i
3: for j=i to n-1 do
4: if A[j] < A[minIdx] then
5: minIdx = j
6: swap(A[i], A[minIdx])
6
Selection Sort: Analysis
#basic op. โค ๐ โ #inner for loop iterations
๐ฅ(๐)
โค ๐1
โค ๐2
โค ๐3
๐ฅ ๐ =
๐=0
๐โ2
๐ โ ๐ =
โ=2
๐
โ โค
โ=1
๐
โ =๐ ๐ + 1
2โค ๐2
Algorithms and Data StructuresFabian Kuhn
SelectionSort(A):
1: for i=0 to n-2 do
2: minIdx = i
3: for j=i to n-1 do
4: if A[j] < A[minIdx] then
5: minIdx = j
6: swap(A[i], A[minIdx])
7
Selection Sort: Analysis
โค ๐1
โค ๐2
โค ๐3
Runtime ๐ป ๐ โค ๐ โ ๐๐ ๐ป ๐ โฅ ๐๐โฒ โ ๐๐
๐(๐)
โฅ ๐2โฒ
#basic op. โค ๐ โ #inner for loop iterations
๐ฅ ๐ โค ๐2
Algorithms and Data StructuresFabian Kuhn
๐ป(๐): Number of basic operations of Selection Sort algorithms for arrays of length ๐
Lemma: There is a constant ๐๐ผ > ๐, such that ๐ป ๐ โค ๐๐ผ โ ๐๐
Lemma: There is a constant ๐๐ณ > ๐, such that ๐ป ๐ โฅ ๐๐ณ โ ๐๐
8
Selection Sort: Analysis
Algorithms and Data StructuresFabian Kuhn
Summary
โข We can only obtain a value that is proportional to the runtime.
โข However, we also do not want anything else:โ Analysis should be independent of computer / compiler / etc.
โ We want to have statements that are valid in 10 / 100 /โฆ years
โข We will always get statements of the following form:
There is a constant ๐ถ, such that
๐ ๐ โค ๐ถ โ ๐ ๐ or ๐ ๐ โฅ ๐ถ โ ๐(๐)
โข The Big-O notation allows to simplify / generalize this kind ofstatementsโฆ
9
Runtime analysis
Algorithms and Data StructuresFabian Kuhn
โข Formalism to describe the asymptotic growth of functions.โ For formal definitions: see next slideโฆ
โข There is a const. ๐ถ > 0, s. t. ๐ ๐ โค ๐ถ โ ๐(๐) becomes:
๐ ๐ โ ๐(๐ ๐ )
โข There is a const. ๐ถ > 0, s. t. ๐ ๐ โฅ ๐ถ โ ๐(๐) becomes:
๐ ๐ โ ฮฉ(๐ ๐ )
โข For Selection Sort:
10
Big-O Notation
๐ป ๐ โ ๐ถ ๐๐
๐ป ๐ โ ๐ ๐๐๐ป ๐ โ ๐ฏ ๐๐
Algorithms and Data StructuresFabian Kuhn
๐ถ ๐ ๐ โ ๐ ๐ | โ๐, ๐๐ > ๐ โ๐ โฅ ๐๐ โถ ๐ ๐ โค ๐ โ ๐(๐)
โข Function ๐ ๐ โ ๐(๐ ๐ ), if there are constants ๐ and ๐0 s. t. ๐ ๐ โค ๐ โ ๐(๐) for all ๐ โฅ ๐0
๐ ๐ ๐ โ ๐ ๐ | โ๐, ๐๐ > ๐ โ๐ โฅ ๐๐ โถ ๐ ๐ โฅ ๐ โ ๐(๐)
โข Function ๐ ๐ โ ฮฉ(๐ ๐ ), if there are constants ๐ and ๐0 s. t. ๐ ๐ โฅ ๐ โ ๐(๐) for all ๐ โฅ ๐0
๐ฏ ๐ ๐ โ ๐ถ ๐ ๐ โฉ ๐ ๐ ๐
โข Function ๐ ๐ โ ฮ(๐ ๐ ), if there are constants ๐1, ๐2 and ๐0 s. t. ๐1 โ ๐ ๐ โค ๐ ๐ โค ๐2 โ ๐(๐) for all ๐ โฅ ๐0, resp. if๐ ๐ โ ๐(๐) and ๐ ๐ โ ฮฉ(๐)
11
Big-O Notation : Definitions
Algorithms and Data StructuresFabian Kuhn
o ๐ ๐ โ ๐ ๐ | โ๐ > ๐ โ๐๐ > ๐ โ๐ โฅ ๐๐ โถ ๐ ๐ โค ๐ โ ๐(๐)
โข Function ๐ ๐ โ ๐(๐ ๐ ), if for all constants ๐ > 0, we have ๐ ๐ โค ๐ โ ๐(๐) (for sufficiently large ๐, indep. of ๐)
๐ ๐ ๐ โ ๐ ๐ | โ๐ > ๐ โ๐๐ > ๐ โ๐ โฅ ๐๐ โถ ๐ ๐ โฅ ๐ โ ๐(๐)
โข Function ๐ ๐ โ ๐(๐ ๐ ), if for all constants ๐ > 0, we have ๐ ๐ โฅ ๐ โ ๐(๐) (for sufficiently large ๐, indep. of ๐)
In particular:
๐ ๐ โ ๐ ๐ ๐ โน ๐ ๐ โ ๐ ๐ ๐
๐ ๐ โ ๐ ๐ ๐ โน ๐ ๐ โ ฮฉ ๐ ๐
12
Big-O Notation : Definitions
Algorithms and Data StructuresFabian Kuhn
๐ ๐ โ ๐ถ ๐ ๐ :
โข ๐ ๐ " โค " ๐(๐), asymptoticallyโฆ
โข ๐(๐) asymptotically grows at most as fast as ๐(๐)
๐ ๐ โ ๐ ๐ ๐ :
โข ๐ ๐ " โฅ " ๐(๐), asymptoticallyโฆ
โข ๐(๐) asymptotically grows at least as fast as ๐(๐)
๐ ๐ โ ๐ฏ ๐ ๐ :
โข ๐ ๐ " = " ๐(๐), asymptoticallyโฆ
โข ๐(๐) asymptotically grows equally fast as ๐(๐)
13
Big-O Notation : Intuitively
Algorithms and Data StructuresFabian Kuhn
๐ ๐ โ ๐ ๐ ๐ :
โข ๐ ๐ " < " ๐(๐), asymptoticallyโฆ
โข ๐(๐) asymptotically grows slower than ๐(๐)
๐ ๐ โ ๐ ๐ ๐ :
โข ๐ ๐ " > " ๐(๐), asymptoticallyโฆ
โข ๐(๐) asymptotically grows faster than ๐(๐)
If ๐(๐) and ๐(๐) grow monotonically, we have:
๐ ๐ โ ๐ ๐ ๐ โบ ๐ ๐ โ ฮฉ ๐ ๐
๐ ๐ โ ๐ ๐ ๐ โบ ๐ ๐ โ ๐ ๐ ๐
14
Big-O Notation : Intuitively
Algorithms and Data StructuresFabian Kuhn
The following definitions hold for monotonically growing functions
๐ ๐ โ ๐ ๐ ๐ , lim๐โโ
๐ ๐
๐ ๐< โ
๐ ๐ โ ฮฉ ๐ ๐ , lim๐โโ
๐ ๐
๐ ๐> 0
๐ ๐ โ ฮ ๐ ๐ , 0 < lim๐โโ
๐ ๐
๐ ๐< โ
๐ ๐ โ ๐ ๐ ๐ , lim๐โโ
๐ ๐
๐ ๐= 0
๐ ๐ โ ๐ ๐ ๐ , lim๐โโ
๐ ๐
๐ ๐= โ
15
Definition by Limits (simplified)
Algorithms and Data StructuresFabian Kuhn
Writing Convention:
โข ๐ ๐ ๐ , ฮฉ ๐ ๐ , โฆ are sets (of functions)
โข Correct way of writing (in principle): ๐ ๐ โ ๐ ๐ ๐
โข Very common way of writing: ๐ ๐ = ๐ ๐ ๐
Examples:
โข ๐ ๐ = ๐(๐2) instead ๐ ๐ โ ๐ ๐2
โข ๐ ๐ = ฮฉ(๐2) instead ๐ ๐ โ ฮฉ ๐2
โข ๐ ๐ = ๐2 + ๐(๐) :
๐ ๐ โ ๐ ๐ โถ โโ ๐ โ ๐ ๐ s. t. ๐ ๐ = ๐2 + โ ๐
โข ๐ ๐ = 1 + ๐ 1 โ ๐ ๐
16
Big-O Notation : Remarks
Algorithms and Data StructuresFabian Kuhn
Writing Convention:
โข ๐ ๐ ๐ , ฮฉ ๐ ๐ , โฆ are sets (of functions)
โข Correct way of writing (in principle): ๐ ๐ โ ๐ ๐ ๐
โข Very common way of writing: ๐ ๐ = ๐ ๐ ๐
Asymptotic Behavior of General Limits:
โข Same notation is used more generally, e.g., ๐(๐ฅ) for ๐ฅ โ 0
โข E.g., Taylor approx.: ๐๐ฅ = 1 + ๐ฅ + ๐(๐ฅ2), or ๐๐ฅ = 1 + ๐ฅ + ๐ ๐ฅ
Alternative Definition for ๐ ๐ ๐ :
๐ ๐ ๐ โ ๐ ๐ | โ๐, ๐๐ > ๐ โ๐ โฅ ๐๐ โถ ๐ ๐ โฅ ๐ โ ๐(๐)
๐ ๐ ๐ โ ๐ ๐ | โ๐ > ๐ โ๐๐ > ๐ โ๐ โฅ ๐๐ โถ ๐ ๐ โฅ ๐ โ ๐(๐)
โ We will use the 1st definition
โ The two definitions are only different for non-monotonic functions
17
Big-O Notation : Remarks
Algorithms and Data StructuresFabian Kuhn
Selection Sort:
โข Runtime ๐ ๐ , there are constants ๐1, ๐2 : ๐1๐2 โค ๐ ๐ โค ๐2๐
2
๐ ๐ โ ๐ ๐2 , ๐ ๐ โ ฮฉ ๐2 , ๐ ๐ โ ฮ ๐2
โข ๐(๐) grows more than linear in ๐: ๐ ๐ โ ๐(๐)
Further examples:
โข ๐ ๐ = 10๐3, ๐ ๐ = ฮค๐3 1000 :
โข ๐ ๐ = ๐๐, ๐ ๐ = ๐100 :
โข ๐ ๐ = ฮค๐ log2 ๐ , ๐ ๐ = ๐ :
โข ๐ ๐ = ๐ ฮค1 256, ๐ ๐ = 10 ln ๐ :
โข ๐ ๐ = log10 ๐ , ๐ ๐ = log2 ๐ :
โข ๐ ๐ = ๐ ๐, ๐ ๐ = 2๐ :
18
Big-O Notation : Examples
lim๐โโ
๐๐
๐100โ โ
๐ ๐
๐ ๐=
๐
log2 ๐
=2 ฮค๐ก 2
๐ก
๐ ๐ โ ฮ ๐ ๐
๐ ๐ โ ๐ ๐ ๐
๐ ๐ โ ๐ ๐ ๐
๐ ๐ โ ๐ ๐ ๐
๐ ๐ โ ฮ ๐ ๐
๐ ๐ โ o ๐ ๐
log ๐ ๐ = ๐ โ log ๐ , log 2๐ = ๐
log10 ๐ =log2 ๐
log2 10
Algorithms and Data StructuresFabian Kuhn
InsertionSort(A):
1: for i = 1 to n-1 do
2: // prefix A[1..i] is already sorted
3: pos = i
4: while (pos > 0) and (A[pos] < A[pos-1]) do
5: swap(A[pos], A[posโ1])
6: pos = pos - 1
19
Analysis Insertion Sort
Algorithms and Data StructuresFabian Kuhn
Worst Case Analysis
โข Analyze runtime ๐(๐) for a worst possible input of size ๐
โข Important / standard way of analyzing algorithms
Best Case Analyse
โข Analyze runtime ๐(๐) for a best possible input of size ๐
โข Usually not very interestingโฆ
Average Case Analyse
โข Analyze runtime ๐(๐) for a typical input of size ๐
โข Problem: what is a typical input?โ Standard approach: use a random input
โ Not clear, how close real inputs and random inputs areโฆ
โ Possible alternative: smoothed analysis (we will not look at this)
20
Worst case, best case, average case
Algorithms and Data StructuresFabian Kuhn
Quadratic = 2x as large input 4x as long runtimeโ For large ๐, this already seems to grow quite fastโฆ
Example calculation:
โข Assume that the number of basic operations ๐ ๐ = ๐2
โข Additionally, assume there is 1 basic operation per processor cycle
โข For a 1Ghz processor, we get 1 ns per basic operation
21
How good is quadratic runtime?
Input size ๐ 4 bytes per number Runtime ๐ป(๐)
103 numbers โ 4KB 103โ 2 โ 10โ9 s = 1 ms
106 numbers โ 4MB 106โ 2 โ 10โ9 s = 16.7 min
109 numbers โ 4GB 109โ 2 โ 10โ9 s = 31.7 years
too slow for large problems!
Algorithms and Data StructuresFabian Kuhn
โข Divide is trivial cost ๐(1)
โข Recursive sorting: We will look at this...
โข Merge: We will look at this first...
22
Analysis Merge Sort
Divide
Sort recursively(by using mergesort)
Merge
Algorithms and Data StructuresFabian Kuhn
MergeSortRecursive(A, start, end, tmp) // sort A[start..end-1]โฎ
5: pos = start; i = start; j = middle
6: while (pos < end) do
7: if (i < middle) and (A[i] < A[j]) then
8: tmp[pos] = A[i]; pos++; i++
9: else
10: tmp[pos] = A[j]; pos++; j++
11: for i = start to end-1 do A[i] = tmp[i]
23
Analysis Merge Step
Algorithms and Data StructuresFabian Kuhn
Runtime ๐(๐) consists of:
โข Divide and Merge: ๐ ๐
โข 2 recursive calls to sort โ ฮค๐ 2โ and ฮค๐ 2 elements
Recursive formulation of ๐ป(๐):
โข There is a constant ๐ > 0, s. t.
๐ ๐ โค ๐๐
2+ ๐
๐
2+ ๐ โ ๐, T 1 โค ๐
โข We simplify a bit and ignore all the rounding:
๐ป ๐ โค ๐ โ ๐ป๐
๐+ ๐ โ ๐, ๐ป ๐ โค ๐
24
Analysis Merge Sort
Algorithms and Data StructuresFabian Kuhn
๐ป ๐ โค ๐ โ ๐ป๐
๐+ ๐ โ ๐, ๐ป ๐ โค ๐
Letโs just try and see what we getโฆ
25
Analysis Merge Sort
Algorithms and Data StructuresFabian Kuhn
Recursive equation: ๐ ๐ โค 2 โ ๐๐
2+ ๐ โ ๐, ๐ 1 โค ๐
Guess: ๐ ๐ โค ๐ โ ๐ โ 1 + log2 ๐
Proof by induction:
26
Analysis Merge Sort
Algorithms and Data StructuresFabian Kuhn
Recursive equation: ๐ ๐ โค 2 โ ๐๐
2+ ๐ โ ๐, ๐ 1 โค ๐
Consider the recursion tree:
27
Alternative Analysis of Merge Sort
Algorithms and Data StructuresFabian Kuhn 28
Merge Sort Measurements
Algorithms and Data StructuresFabian Kuhn 29
Merge Sort Measurements
Algorithms and Data StructuresFabian Kuhn
The runtime of Merge Sort is ๐ป ๐ โ ๐ถ(๐ โ ๐ฅ๐จ๐ ๐).
โข grows almost linearly with the input size ๐โฆ
How good is this?
โข Example calculation:โ Again assume that 1 basic operation = 1 ns
โ We will be a bit more conservative than before and assume that๐ ๐ = 10 โ ๐ log ๐
30
Summary Analysis Merge Sort
Input size ๐ 4 byte numbers Runtime ๐ป ๐ = ๐๐ โ ๐ ๐ฅ๐จ๐ ๐ ๐๐
210 โ 103numbers โ 4KB 10 โ 10 โ 210 โ 10โ9 s โ 0.1 ms 1 ms
220 โ 106 numbers โ 4MB 10 โ 20 โ 220 โ 10โ9 s โ 0.2 s 16.7 min
230 โ 109 numbers โ 4GB 10 โ 30 โ 230 โ 10โ9 s โ 5.4 min 31.7 years
240 โ 1012numbers โ 4TB 10 โ 40 โ 240 โ 10โ9 s โ 122 h > 107 years
Algorithms and Data StructuresFabian Kuhn
โข Runtime depends on how we choose the pivots
โข Runtime to sort array of size ๐ if pivot partitions array into parts of sizes ๐๐ and 1 โ ๐ ๐:
๐ป ๐ = ๐ป ๐๐ + ๐ป ๐ โ ๐ ๐ + "๐ ๐ข๐ง๐ ๐ฉ๐ข๐ฏ๐จ๐ญ + ๐๐ข๐ฏ๐ข๐๐โ
โข Divide:โ We iterate over the array from both sides, ๐ 1 cost per step Time to partition array of length ๐: ๐(๐)
31
Quick Sort : Analysis
๐ฅ
Divide
Sort recursively(by using quicksort)
Algorithms and Data StructuresFabian Kuhn
If we can also find a pivot in time ๐(๐) such that such that the array is partitioned into parts of sizes ๐๐ and 1 โ ๐ ๐:
โข There is a constant ๐ > 0, s. t.
๐ ๐ โค ๐ ๐๐ + ๐ 1 โ ๐ ๐ + ๐ โ ๐, ๐ 1 โค ๐
Extreme case I) ๐ = ฮค1 2 (best case):
๐ ๐ โค 2๐๐
2+ ๐๐, ๐ 1 โค ๐
โข As for Merge Sort: ๐ ๐ โ ๐ ๐ log ๐
Extreme case II) ๐๐ = 1, 1 โ ๐ ๐ = ๐ โ 1 (worst case):
๐ ๐ = ๐ ๐ โ 1 + ๐๐, ๐ 1 โค ๐
32
Quick Sort : Analysis
Algorithms and Data StructuresFabian Kuhn
Extreme case II) ๐๐ = 1, 1 โ ๐ ๐ = ๐ โ 1 (worst case):
๐ ๐ = ๐ ๐ โ 1 + ๐๐, ๐ 1 โค ๐
In this case, we obtain ๐ ๐ โ ฮ ๐2 :
33
Quick Sort : Worst Case Analysis
Algorithms and Data StructuresFabian Kuhn
Partition For Random Pivot:
โข Runtime ๐ป ๐ = ๐ถ(๐ ๐ฅ๐จ๐ ๐) for all inputsโ but only in Erwartungswert and with very high probability
Intuition:
โข With probability ฮค1 2, we get parts of size โฅ ฮค๐ 4, s. t.
๐ ๐ โค ๐๐
4+ ๐
3๐
4+ ๐๐
34
Quick Sort With a Random Pivot
Algorithms and Data StructuresFabian Kuhn
Partition For Random Pivot:
โข Runtime ๐ป ๐ = ๐ถ(๐ ๐ฅ๐จ๐ ๐) for all inputsโ but only in Erwartungswert and with very high probability
Analysis:
โข We will not do this hereโ see, e.g., Cormen et al. or the algorithm theory lecture
โข Possible approach: write recursion in terms of expected values
๐ผ ๐ ๐ โค ๐ผ ๐ ๐๐ฟ + ๐ ๐ โ ๐๐ฟ + ๐๐
35
Quick Sort With a Random Pivot
Algorithms and Data StructuresFabian Kuhn
Task: Sort sequence ๐1, ๐2, โฆ , ๐๐โข Goal: lower bound (worst-case) runtime
Comparison-based sorting algorithms
โข Comparisons are the only allowed way to determine the relative order between elements
โข Hence, the only thing that can influence the sequence of elements in the final sorted sequence are comparisons of the kind
๐๐ = ๐๐ , ๐๐ โค ๐๐ , ๐๐ < ๐๐ , ๐๐ โฅ ๐๐ , ๐๐ > ๐๐
โข If we assume that the elements are pair-wise distinct, we only need comparisons of the form ๐๐ โค ๐๐
โข 1 comparison = 1 basic operation
36
Sorting Lower Bound
Algorithms and Data StructuresFabian Kuhn
Alternative View
โข Every program (for a deterministic, comp.-based sorting alg.) can be brought into a form where every if/while/โฆ-condition is of the following form:
๐ข๐ ๐๐ โค ๐๐ ๐ญ๐ก๐๐ง โฆ
โข In each execution of an algorithm, the results of these comparisons induce a sequence of T/F (true/false) values:
๐๐ ๐ ๐๐๐๐ ๐๐ ๐ ๐๐๐ ๐ ๐ ๐ ๐ ๐๐ ๐๐๐โฆ
โข This sequence uniquely determines how the values of the array are rearranged (permuted) by the algorithm.
โข Different inputs with the same values therefore must lead todifferent T/F sequences.
37
Comparison-Based Sorting Algorithms
Algorithms and Data StructuresFabian Kuhn
Execution tree:
38
Comparison-Based Sorting Algorithms
Algorithms and Data StructuresFabian Kuhn
โข In comparison-based sorting algorithms, the execution depends on the initial ordering of the values in the inputs, but it does not depend on the actual values.โ We restrict to cases where the values are all distinct.
โข W.l.o.g. we can assume that we have to sort the numbers 1,โฆ , ๐.
โข Different inputs have to be handled differently.
โข Different inputs result in different T/F sequences
โข Runtime of an execution โฅ length of the resulting T/F sequence
โข Worst-Case runtime โฅ Length of longest T/F sequence:โ We want a lower bound
โ Count no. of possible inputs we need at least as many T/F sequences...
39
Comp.-Based Sorting: Lower Bound
Algorithms and Data StructuresFabian Kuhn
Number of possible inputs (input orderings):
Number of T/F sequences of length โค ๐:
Theorem: Every comparison-based sorting algorithm requiresฮฉ ๐ โ log ๐ comparisons in the worst case.
40
Comp.-Based Sorting: Lower Bound
Algorithms and Data StructuresFabian Kuhn
โข Not possible with comparison-based algorithmsโ Lower bound also holds for randomized algorithms...
โข Sometimes, we can be fasterโ If we can exploit special properties of the input
โข Example: Sort ๐ numbers ๐๐ โ 0,1 :
1. Count number of zeroes and ones in time ๐(๐)
2. Write solution to array in time ๐ ๐
41
Sorting in Linear Time
Algorithms and Data StructuresFabian Kuhn
Task:
โข Sort integer array ๐ด of length ๐
โข We know that for all ๐ โ {0,โฆ , ๐}, ๐ด ๐ โ {0,โฆ , ๐}
Algorithm:
1: counts = new int[k+1] // new int array of length k
2: for i = 0 to k do counts[i] = 0
3: for i = 0 to n-1 do counts[A[i]]++
4: i = 0;
5: for j = 0 to k do
6: for l = 1 to counts[j] do
7: A[i] = j; i++
42
Counting Sort