Copyright © 2013 by John Wiley & Sons. All rights reserved. SORTING AND SEARCHING CHAPTER Slides by...

Copyright © 2013 by John Wiley & Sons. All rights reserved.

SORTING AND SEARCHING

CHAPTER

Slides by Rick Giles

14

Chapter Goals To study several sorting and searching algorithms To appreciate that algorithms for the same task

can differ widely in performance To understand the big-oh notation To estimate and compare the performance of

algorithms To write code to measure the running time of a

program


Contents Selection Sort Profiling the Selection Sort Algorithm Analyzing the Performance of the Selection

Sort Algorithm Merge Sort Analyzing the Merge Sort Algorithm Searching Problem Solving: Estimating the Running

Time of an Algorithm Sorting and Searching in the Java Library


14.1 Selection Sort Sorts an array by repeatedly finding the

smallest element of the unsorted tail region and moving it to the front

Example: sorting an array of integers


11 9 17 5 12

Sorting an Array of Integers


Find the smallest and swap it with the first element

Find the next smallest. It is already in the correct place

Find the next smallest and swap it with first element of unsorted portion

Repeat

When the unsorted portion is of length 1, we are done

5 9 17 11 12

5 9 17 11 12

5 9 11 17 12

5 9 11 12 17

5 9 11 12 17

SelectionSorter.java


Continued

SelectionSorter.java (cont.)


SelectionSortDemo.java


ArrayUtil.java


Continued

ArrayUtil.java (cont.)


SelectionSortDemo.java


14.2 Profiling the Selection Sort Algorithm

Want to measure the time the algorithm takes to execute

Exclude the time the program takes to load

Exclude output time Create a StopWatch class to measure execution time of

an algorithm

It can start, stop and give elapsed time

Use System.currentTimeMillis method

Create a StopWatch object

Start the stopwatch just before the sort

Stop the stopwatch just after the sort

Read the elapsed time


StopWatch.java


Continued

StopWatch.java (cont.)


Continued

SelectionSortTimer.java


Continued

SelectionSortTimer.java (cont.)


Selection Sort on Various Array Sizes*


n Milliseconds

10,000 786

20,000 2,148

30,000 4,796

40,000 9,192

50,000 13,321

60,000 19,299

* Obtained with a 2GHz Pentium processor, Java 6, Linux

Time Taken by Selection Sort


14.3 Analyzing the Performance of the Selection Sort Algorithm (1)

In an array of size n, count how many times an array element is visited

To find the smallest, visit n elements + 2 visits for the swap

To find the next smallest, visit (n - 1) elements + 2 visits for the swap

The last term is 2 elements visited to find the smallest + 2 visits for the swap



The number of visits:

n + 2 + (n - 1) + 2 + (n - 2) + 2 + ...+ 2 + 2

This can be simplified to n2 /2 + 5n/2 - 3

5n/2 - 3 is small compared to n2 /2 — so ignore it

Also ignore the 1/2 — it cancels out when comparing ratios



The number of visits is of the order n2

Using big-Oh notation: The number of visits is O(n2)

Multiplying the number of elements in an array by 2 multiplies the processing time by 4

Big-Oh notation “f(n) = O(g(n))” expresses that f grows no faster than g

To convert to big-Oh notation: Locate fastest-growing term, and ignore constant coefficient


14.4 Merge Sort Sorts an array by

Cutting the array in half

Recursively sorting each half

Merging the sorted halves

Much more efficient than selection sort


Merge Sort Example


Divide an array in half and sort each half

Merger the two sorted arrays into a single sorted array

sort Method


MergeSorter.java


Continued

MergeSorter.java (cont.)


Continued

MergeSorter.java (cont.)


MergeSortDemo.java


14.5 Analyzing the Merge Sort Algorithm (1)


n Merge Sort (milliseconds) Selection Sort (milliseconds)

10,000 40 786

20,000 73 2,148

30,000 134 4,796

40,000 170 9,192

50,000 192 13,321

60,000 205 19,299

Analyzing the Merge Sort Algorithm (2)




In an array of size n, count how many times an array element is visited

Assume n is a power of 2: n = 2m

Calculate the number of visits to create the two sub-arrays and then merge the two sorted arrays

3 visits to merge each element or 3n visits

2n visits to create the two sub-arrays

total of 5n visits



Let T(n) denote the number of visits to sort an array of n

elements then T(n) = T(n/2) + T(n/2) + 5n or

T(n) = 2T(n/2) + 5n

The visits for an array of size n/2 is:

T(n/2) = 2T(n/4) + 5n/2

So T(n) = 2 × 2T(n/4) +5n + 5n

The visits for an array of size n/4 is:

T(n/4) = 2T(n/8) + 5n/4

So T(n) = 2 × 2 × 2T(n/8) + 5n + 5n + 5n



Repeating the process k times: T(n) = 2kT(n/2k) +5nk

Since n = 2m, when k=m: T(n) = 2mT(n/2m) +5nm

T(n) = nT(1) +5nm

T(n) = n + 5nlog2(n)



To establish growth order

Drop the lower-order term n

Drop the constant factor 5

Drop the base of the logarithm since all logarithms are related by a constant factor

We are left with n log(n)

Using big-Oh notation: Number of visits is O(nlog(n))

Merge Sort vs Selection Sort


Selection sort is an O(n2) algorithm

Merge sort is an O(nlog(n)) algorithm

The nlog(n) function grows much more slowly than n2

14.6 Searching Linear search: Examines all values in an array until it

finds a match or reaches the end

Also called sequential search

Number of visits for a linear search of an array of n elements:

The average search visits n/2 elements

The maximum visits is n

A linear search locates a value in an array in O(n) steps


LinearSearcher.java


LinearSearchDemo.java


Continued

LinearSearchDemo.java (cont.)


Binary Search (1) Locates a value in a sorted array by

Determining whether the value occurs in the first or second half

Then repeating the search in one of the halves


Binary Search (2) To search 15:

15 ≠ 17: We don’t have a match


BinarySearcher.java


Continued

BinarySearcher.java (cont.)


Analyzing Binary Search (1) Count the number of visits to search a sorted array of size

n

We visit one element (the middle element) then search either the left or right subarray

Thus: T(n) = T(n/2) + 1

Using the same equation, T(n/2) = T(n/4) + 1

Substituting into the original equation: T(n) = T(n/4) + 2

This generalizes to: T(n) = T(n/2k) + k


Analyzing Binary Search (2) Assume n is a power of 2, n = 2m, where m = log2(n)

Then: T(n) = 1 + log2(n)

Binary search is an O(log(n)) algorithm


14.7 Problem Solving: Estimating the Running Time of an Algorithm

Differentiating between O(nlog(n)) and O(n2) has great practical implications

Being able to estimate running times of other algorithms is an important skill

Will practice estimating the running times of array algorithms


Linear Time (1) An algorithm to count how many elements of an array

have a particular value:

int count = 0;for (int i = 0; i < a.length; i++){ if (a[i] == value) { count++; }}


Linear Time (2) To visualize the pattern of array element visits:

Imagine the array as a sequence of light bulbs

As the ith element gets visited, imagine the ith light bulb lighting up

When each visit involves a fixed number of actions, the running time is n times a constant, or O(n)


Linear Time (3) When you don’t always run to the end of the array; e.g.:

boolean found = false;for (int i = 0; !found && i < a.length; i++){ if (a[i] == value) { found = true; }}

Loop can stop in middle:

Still O(n), because in some cases the match may be at the very end of the array


Quadratic Time (1) What if we do a lot of work with each visit?

Example: find the most frequent element in an array Obvious for small array

For larger array?

Put counts in a second array of same length

Take the maximum of the counts, 3, look up where the 3 occurs in the counts, and find the corresponding value, 7


Quadratic Time (2) First estimate how long it takes to compute the counts

for (int i = 0; i < a.length; i++){ counts[i] = Count how often a[i] occurs in a}

Still visit each element once, but work per visit is much larger

Each counting action is O(n)

When we do O(n) work in each step, total running time is O(n2)


Quadratic Time (3) Algorithm has three phases:

1. Compute all counts

2. Compute the maximum

3. Find the maximum in the counts

First phase is O(n2), second and third are each O(n)

The big-Oh running time for doing several steps in a row is the largest of the big-Oh times for each step

Thus algorithm for finding the most frequent element is O(n2)


Logarithmic Time (1) An algorithm that cuts the size of work in half in each step

runs in O(log(n)) time

E.g. binary search, merge sort

To improve our algorithm for finding the most frequent element, first sort the array:

Sorting costs O(n log(n)) time

If we can complete the algorithm in O(n) time, have a better algorithm that the pervious O(n2) algorithm


Logarithmic Time (2) While traversing the sorted array:

As long as you find a value that is equal to its predecessor, increment a counter

When you find a different value, save the counter and start counting anew


Logarithmic Time (3)int count = 0;for (int i = 0; i < a.length; i++){ count++; if (i == a.length - 1 || a[i] != a[i + 1]) { counts[i] = count; count = 0; }}


Logarithmic Time (4) Constant amount of work per iteration, even though it

visits two elements:

2n is still O(n)

Entire algorithm is now O(log(n))


14.8 Sorting and Searching in the Java Library

When you write Java programs, you don’t have to implement your own sorting algorithms

Arrays and Collections classes provide sorting and searching methods


Sorting The Arrays class contains static sort methods

To sort an array of integers:

int[] a = ... ;Arrays.sort(a);

That sort method uses the Quicksort algorithm (see

Special Topic 14.3) To sort an array of ArraysList use Collections.sort

method:

ArrayList<String> names = ... ;Collections.sort(names);

That sort method uses the merge sort algorithm


Binary Search Arrays and Collections classes contains static binarySearch methods

These methods implement the binary search algorithm, with a useful enhancement:

If the value is not found in the array, return -k -1, where k is the position before which the element should be inserted

E.g.

int[] a = { 1, 4, 9 };int v = 7;int pos = Arrays.binarySearch(a, v);// Returns –3; v should be inserted before position 2


Comparing Objects (1) Arrays and ArrayLists classes provide sort and

binarySearch methods for objects

These methods cannot know how to compare arbitrary objects

Methods require that the objects belong to a class that implements the Comparable interface with a single method:

public interface Comparable{ int compareTo(Object otherObject);}

These methods cannot know how to compare arbitrary objects


Comparing Objects (2) Call

a.compareTo(b)

must return a negative number if a should come before b, 0 if a and b are the same, and a positive number otherwise

Several classes in the standard Java Library implement the Comparable interface

E.g. String, Date


Comparing Objects (3) You can implement the Comparable interface for your

classes as well

E.g. to sort a collection of countries by area, class Country can implement this interface and supply a compareTo method:


Comparing Objects (3) You can implement the Comparable interface for your

classes as well

E.g. to sort a collection of countries by area, class Country can implement this interface and supply a compareTo method:

public class Country implements Comparable{ public int compareTo(Object otherObject) { Country other = (Country) otherObject; if (area < other.area) { return -1; } else if (area == other.area) { return 0; } else { return 1; } }}


Summary: Selection Sort Algorithm The selection sort algorithm sorts an array by repeatedly

finding the smallest element of the unsorted tail region and moving it to the front.


Summary: Measuring the Running Time of an Algorithm

To measure the running time of a method, get the current time immediately before and after the method call.


Summary: Big-Oh Notation Computer scientists use the big-Oh notation to describe

the growth rate of a function.

Selection sort is an O(n2) algorithm. Doubling the data set means a fourfold increase in processing time.

Insertion sort is an O(n2) algorithm.


Summary: Contrasting Running Times of Merge Sort and Selection Sort Algorithms

Merge sort is an O(n log(n)) algorithm. The n log(n) function grows much more slowly than n2.


Summary: Running Times of Linear Search and Binary Search Algorithms

A linear search examines all values in an array until it finds a match or reaches the end.

A linear search locates a value in an array in O(n) steps.

A binary search locates a value in a sorted array by determining whether the value occurs in the first or second half, then repeating the search in one of the halves.

A binary search locates a value in a sorted array in O(log(n)) steps.


Summary: Practice Developing Big-Oh Estimates of Algorithms

A loop with n iterations has O(n) running time if each step consists of a fixed number of actions.

A loop with n iterations has O(n2) running time if each step takes O(n) time.

The big-Oh running time for doing several steps in a row is the largest of the big-Oh times for each step.

A loop with n iterations has O(n2) running time if the ith step takes O(i) time.

An algorithm that cuts the size of work in half in each step runs in O(log(n)) time.

Copyright © vby John Wiley & Sons. All rights reserved.

Summary: Use the Java Library Methods for Sorting and Searching Data

The Arrays class implements a sorting method that you should use for your Java programs.

The Collections class contains a sort method that can sort array lists.

The sort methods of the Arrays and Collections classes sort objects of classes that implement the Comparable interface.


Copyright © 2013 by John Wiley & Sons. All rights reserved. SORTING AND SEARCHING CHAPTER Slides by...

Documents

Transcript of Copyright © 2013 by John Wiley & Sons. All rights reserved. SORTING AND SEARCHING CHAPTER Slides by...