DSA Review

2011 | Fiona Angelina

Data Structures & Algorithm AnalysisReview

Agenda:• Heap

• Priority Queue• Sorting• Searching• Graph


Heap• Heap is a complete or nearly complete binary

tree in which the value inside the parent is bigger than its children.

• There are actually two types of heap:– Max Heap: parent is bigger than its children– Min Heap: parent is smaller than its children


Heap and Not a Heap

10

9 6

7 5

10

9 6

7 4

Example of a heap because itIs a nearly complete binary tree

And its parent is always greater thanIts children

It is NOT a heap because it is not anearly complete binary tree


Max vs Min Heap

19

15 13

9 14

4

7 9

10 14

Max Heap Min Heap


Building Heap• Suppose you want to build a heap from the

sequence below:80, 40, 30, 60, 91


Building Heap• Step 1

80



80

40



80

40 30



80

40 30

60

80

60 30

40

Heap

up

Compare to its parent. If the children is bigger, swap with parent. This process is called heap up.



80

40 30

60

Heap up

91

80

91 30

60

Heap up

40

After heaping up once, the children is still bigger than its parent. Heap up once again.


Building Heap• Final Result

91

80 30

60 40


Removing a Root• Step 1

91

80 30

60 40

Delete the root and replace it with the last element of the heap.


Removing a Root• Step 2

40

80 30

60

80

40 30

60

Heap down

Heap d

own

Compare the new root with its children. Since the root is only smaller to its left child, swap with its left child. The process is called heap down.


Removing a Root• Final Result

80

60 30

40


Removing a Root• Special Notes:– In max heap: if the root is smaller than both of its

children, swap with its biggest child.– In min heap: if the root is bigger than both of its

children, swap with its smallest child.


Priority Queue• In priority queue, a value with higher priority

value will be put in the front of the line, instead of who actually comes first.


Queue vs Priority Queue• Suppose we want to insert these values:

8, 7, 2, 9, 5

8 7 2 9 5

9 8 7 5 2

Queue

Priority QueueFRONT

FRONT

REAR

REAR

Notes: Assume higher number has higher priority.


Implementing Priority Queue• Priority Queue can be implemented using

heap.• Giving:– O(log n) for enqueue– O(1) for dequeue


Implementing Priority Queue• Suppose we want to insert these values:

8, 7, 2, 9, 5


Enqueue• Step 1: Insert 8

8



8

7



8

7 2



8

7 2

9

Heap up

8

9 2

7

Heap up



9

8 2

7 5

9 8 2 7 5FRONT REAR

Enqueue is DONE here. To output the value, we need to do dequeue.


Dequeue• Step 1: Dequeue 9

8

7 2

5

Here, I do not describe again theprocess of heap down.

Output: 9



7

5 2

Output: 9, 8



5

2

Output: 9, 8, 7



5

Output: 9, 8, 7, 5



Output: 9, 8, 7, 5, 2

Actually, it is the same process as heap sort.

Priority Queue


Sorting Concepts• Several sorting concepts:– Insertion sort– Shell sort– Heap sort– Quick sort– Merge sort


Insertion• Insertion is simple, but more efficient compare

to other simple sorting algorithms such as bubble and selection sort.

• Works well in smaller set and on partially sorted set.

• Worst case: O(n2)


Insertion• Example: Sort the sequence below using

insertion sort in ascending manner.8, 13, 90, 14, 1


Insertion

8 13 90 14 1Original List

sorted unsorted

8 13 90 14 1

Step 1

sorted unsorted

8 13 90 14 1

Step 2

sorted unsorted


Insertion

8 13 14 90 1

Step 4

sorted unsorted

1 8 13 14 90

sorted

Final Result


Shell Sort• Improvement for insertion sort.• The idea is to divide data into segments, and

sort data inside each segment.• Start from large segment, finally to 1-data

segment.


Shell Sort• Sort the sequence below using shell sort:

54, 13, 77, 2, 9, 10, 11, 15

54 13 772 9 10

11 15

Step 1: Increment 3

Segment 1 Segment 2 Segment 3

Sort each segment using insertion sort.


Shell Sort• Segment 1:

54 2 11

2 54 11

2 11 54



13 9 15

9 13 15

9 13 15



77 10

10 77


Shell Sort

54 13 772 9 10

11 15


2 9 1011 13 7754 15


BEFORESORTED

AFTERSORTED


Shell Sort• Step 2: Increment 2

2 910 1113 7754 15

Segment 1 Segment 2

Sort again each segment using insertion sort.


Shell Sort• The new sorted sequence:

2 910 1113 1554 77

Segment 1 Segment 2


Shell Sort• Step 3: Increment 1

29

101113155477

Segment 1

After Sorted

29

101113155477

Segment 1

No value needs to be swapped

since the segment is

already sorted.

The Sorted Sequence:2, 9, 10, 11, 13, 15, 54, 77


Heap Sort• To do heap sort, there are two steps:– Step 1: Build the heap– Step 2: Dequeue the root one-by-one

• The type of heap determines the sequence order:– Min heap results in descending order.– Max heap results in ascending order.


Heap Sort• Suppose you have the sequence:

6, 95, 30, 28, 77, 1• First step is to build the heap.


Building the Heap• Step 1: Insert 6

6



95

6

The process of heap up is not shown in here.

Please refer to previous slide to see the heap up process.



95

6 30



95

28 30

6



95

77 30

6 28



95

77 30

6 28 1


The Sorting• Step 1: Delete 95

77

28 30

6 1

77 28 30 6 1 95

heap sorted

The process of heap down is not shown in here.

Please refer to previous slide to see the heap up process.



30

28 1

6

30 28 1 6 77 95

heap sorted



28

6 1

28 6 1 30 77 95

heap sorted



6

1

6 1 28 30 77 95

heap sorted



1 6 28 30 77 95

The sorted sequence


Quick Sort• To choose the pivot:– Step 1: Compare left & middle value.– Step 2: Compare left & right value.– Step 3: Compare middle & right value.

• After getting the pivot, exchange the left with middle value, and start finding value to be exchanged.


Choosing Pivot• Sort the sequence below using quick sort:

43, 66, 13, 15, 19, 20, 21, 6, 7, 11• First, choose the pivot.• To choose the pivot, you have to identify all

the values:– Left value: 43– Middle value: 19– Right value: 11


Choosing Pivot• Compare the three of them:

43 66 13 15 19 20 21 6 7 11

Compare 43 (left) and 19 (middle)

19 66 13 15 43 20 21 6 7 11

Compare 19 (left) and 11 (right)

11 66 13 15 43 20 21 6 7 19

11 66 13 15 19 20 21 6 7 43Compare 43 (middle) and 19 (right)

pivot


The Sorting• Exchange the pivot with 11 (middle value).

• Put the walker

19 66 13 15 11 20 21 6 7 43

pivot

19 66 13 15 11 20 21 6 7 43

Left walker Right walker

Left walker searches value more than pivot, and right walker searches value less than pivot.


The Sorting

19 66 13 15 11 20 21 6 7 43

19 7 13 15 11 20 21 6 66 43

19 7 13 15 11 6 21 20 66 43

Walker stops here

Left sequence Right sequence


The Sorting• Move back the pivot

6 7 13 15 11 19 21 20 66 43

Walker stops here

Left sequence Right sequence

Do quick sort again on left sequence and right sequence


The Sorting• Left Sequence

6 7 13 15 11 19

6 7 13 15 11 19

After comparing, we found that the pivot is 13

Swap with left value, and put walker

13 7 6 15 11 19


The Sorting

13 7 6 15 11 19

13 7 6 11 15 19

11 7 6 13 15 19

Do quick sort again on left sequence and right sequence


The Sorting• Do the quick sort again and again after all

sequence has been sorted. Merge the result into 1.

6 7 11 13 15 19 20 21 43 63


Merge Sort• In merge sort, you broke the sequence into

smaller sets, and then merge it in order.


Merge Sort• Suppose you want to sort the sequence

below:8, 7, 13, 5, 19, 1, 10, 11


Merge Sort

8 7 13 5 19 1 10 11

8 7 13 5 19 1 10 11

8 7 13 5 19 1 10 11


Merge Sort

8 7 13 5 19 1 10 11

5 7 8 13 1 10 11 19

1 5 7 8 10 11 13 19


Searching• There are two common searching algorithms:– Linear search– Binary search


Linear Search• In linear search, you search the value by

comparing the key to the cell one by one.

78 5 23 51 9 10 1

For example, we want to search 51

FOUND!

1 2 3 4


Binary Search• Binary search only works on sorted array.• The idea is to cut the sequence into two until

finding the correct element.

1 5 9 10 23 51 78

23 51 7851 is bigger than 10

FOUND!

Middle element


Graph• Graph is a collection of vertices and edges.• There are two types of graph:– Directed graph: each edge has direction (arrow

head)– Undirected graph: each edge has no direction


Directed vs Undirected Graph

A

B

C D

A

B

C D

Directed Graph Undirected Graph


Graph Terminologies• A path is a sequence of vertices in which each

vertex is adjacent to the next one.• A cycle is a path consisting of at least three

vertices that starts and ends with the same vertex.

• The degree of a vertex is the number of lines incident to it–Outdegree: leaving the vertex– Indegree: entering the vertex


Graph Terminologies• Graph is said to be strongly connected if all

the vertices are connected.• Graph is said to be weakly connected if not all

vertices are connected.


Strongly vs Weakly Connected

A

B

C D

Weakly connected

A

B

C D

Strongly connected


Depth-First Traversal• All the descendant must be processed before

moving to the adjacent vertices.• Use stack.


Depth-First Traversal

A

B C D

E F Stack

Top

Bottom

Output:

A



A

B C D

E F Stack

Top

Bottom

Output: A

DCB



A

B C D

E F Stack

Top

Bottom

Output: A D

CB



A

B C D

E F Stack

Top

Bottom

Output: A D C

FEB



A

B C D

E F Stack

Top

Bottom

Output: A D C F E B


Breadth-First Traversal• Process all adjacent vertices, before

continuing to the descendants.• Use queue


Breadth-First Traversal

A

B C D

E F

Queue

Front Rear

Output:

A



A

B C D

E F

Queue

Front Rear

Output: A

B C D



A

B C D

E F

Queue

Front Rear

Output: A B

C D



A

B C D

E F

Queue

Front Rear

Output: A B C

D E F



A

B C D

E F

Queue

Front Rear

Output: A B C D E F


Graph Storage Structure• There are two ways to store graphs:– Adjacency matrix– Adjacency list


Adjacency Matrix

A

B C D

E F

Undirected graph

A B C D E FA 0 1 1 1 0 0B 1 0 0 0 0 0C 1 0 0 0 1 1D 1 0 0 0 0 0E 0 0 1 0 0 0F 0 0 1 0 0 0

1 means the vertices are connected.0 means they are not connected.


Adjacency Matrix

A

B C D

E F

Directed graph

A B C D E FA 0 1 1 1 0 0B 0 0 0 0 0 0C 0 0 0 0 1 1D 0 0 0 0 0 0E 0 0 0 0 0 0F 0 0 0 0 0 0


TO

FROM


Adjacency Matrix

A

B C D

E F

Weighted graph

A B C D E FA 0 2 2 3 0 0B 0 0 0 0 0 0C 0 0 0 0 4 1D 0 0 0 0 0 0E 0 0 0 0 0 0F 0 0 0 0 0 0


TO

FROM

2 3

14

2


Adjacency List

A

B C D

E F

A

B

C

D

E

B C D

E F


Minimum Spanning Tree• MST is the most minimum weighted tree that

can be formed from a graph.• If all the weight is unique, there is only one

MST.• If weight duplicate exists, there can be more

than one MST.


Minimum Spanning Tree

A

B

D

F

E

C

4 3

13

6 2

5

Start at A



A

B

D

F

E

C

4 3

13

6 2

5

AF (1) is chosen because it has smaller weight compare

to AB (4) and AC (3).



A

B

D

F

E

C

4 3

13

6 2

5

EF (2) is chosen because it has smaller weight compare to AB

(4), AC (3), or DF (6)



A

B

D

F

E

C

4 3

13

6 2

5

AC (3) is chosen because it has smaller weight compare to EC

(5), AB (4), or DF (6)



A

B

D

F

E

C

4 3

13

6 2

5

AB (4) is chosen because it has smaller weight compare to DF

(6)



A

B

D

F

E

C

4 3

13

6 2

5

BD (3) is chosen because it has smaller weight compare to DF

(6)


Minimum Spanning Tree• Here is the MST

A

B

D

F

E

C

4 3

13

6 2

5


Shortest Path Algorithm

A

B

E

G

F

C

4 3

13

6 2

5 Start at AD

H

8

7

5



A

B

E

G

F

C

4 3

13

6 2

5

D

H

8

7

5



A

B

E

G

F

C

4 3

13

6 2

5

D

H

8

7

5

1

3



A

B

E

G

F

C

4 3

13

6 2

5

D

H

8

7

5

1

3

3



A

B

E

G

F

C

4 3

13

6 2

5

D

H

8

7

5

1

3

3

4



A

B

E

G

F

C

4 3

13

6 2

5

D

H

8

7

5

3

1

3

4

7



A

B

E

G

F

C

4 3

13

6 2

5

D

H

8

7

5

3

1

3

4

7

10



A

B

E

G

F

C

4 3

13

6 2

5

D

H

8

7

5

3

1

3

4

7

10

11



A

B

E

G

F

C

4 3

13

6 2

5

D

H

8

7

5

DSA Review

Education

Transcript of DSA Review