ADA-Unit 1

ANALYSIS AND DESIGN OF ALGORITHMS(CSE43)

TEXT : ANANY LEVITINReference:Coremen T.H

Horowitz E.,Sahani

S., Rajashekarn S

class 14 PESSE 2

Why Study Algorithms? Al Khawarizmi

“A great Iranian mathematician, geographer and astronomer. He introduced the zero, negative numbers, algebra, and the decimal system to the West. He also invented mathematical programming using a set of instructions to perform complex calculations. The term algorithm is named after a variation of his name, Algorithmi. “

What is it? Briefly speaking, algorithms are procedure solution to

problems. Algorithms are not answers but rather precisely defined

procedures for getting answers.

class 14 PESSE 3

Algorithms An algorithm is a sequence of unambiguous instructions for

solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.

“computer”

problem

algorithm

input output

class 14 PESSE 4

Properties of Algorithms What distinguish an algorithm from a recipe, process, method,

technique, procedure, routine…? Finiteness

terminates after a finite number of steps Definiteness

Each step must be rigorously and unambiguously specified.e.g., ”stir until lumpy”

Input Valid inputs must be clearly specified.

OutputThe data that result upon completion of the algorithm must be specified.

EffectivenessSteps must be sufficiently simple and basic.

e.g., check if 2 is the largest integer n for which there is a solution to the equation xn + yn = zn in positive integers x, y, and z

class 14 PESSE 5

Examples of Algorithms – Computing the Greatest Common Divisor of Two Integers

gcd(m, n): the largest integer that divides both non negative and not both zero ints m and n.

First try Euclid’s algorithm: gcd(m, n) = gcd(n, m mod n) Gcd(m,0)=m Ex: gcd(60,24)=gcd(24,12)=gcd(12,0)=12

Step1: If n = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2.

Step2: Divide m by n and assign the value of the remainder to r.

Step 3: Assign the value of n to m and the value of r to n. Go to Step 1.

class 14 PESSE 6

Methods of Specifying an Algorithm

Pseudocode A mixture of a natural language and programming

languagelike structures Pseudocode in this course

omits declarations of variables use indentation to show the scope of such statements as

for, if, and while. use for assignment

class 14 PESSE 7

Pseudocode of Euclid’s AlgorithmAlgorithm Euclid(m, n)//Computes gcd(m, n) by Euclid’s algorithm//Input: Two nonnegative, notbothzero integers m and n //Output: Greatest common divisor of m and n while n ‡ 0 do

r m mod nm nn r

return m Questions:

Finiteness: how do we know that Euclid’s algorithm actually comes to a stop?

Definiteness: nonambiguity Effectiveness: effectively computable.

class 14 PESSE 8

Second Try for gcd(m, n) Consecutive Integer Algorithm

Step1: Assign the value of min{m, n} to t.

Step2: Divide m by t. If the remainder of this division is 0, go to Step3;otherwise, go to Step 4.

Step3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step4.

Step4: Decrease the value of t by 1. Go to Step2. (60,24) t=24, 60 mod 24 =12 then t=23,22,21,20,10,18,17,16,15,14,13,12

Questions Finiteness Definiteness Effectiveness Which algorithm is faster, the Euclid’s or this one?

class 14 PESSE 9

Third try for gcd (m, n) Middleschool procedure

Step1: Find the prime factors of m. Step2: Find the prime factors of n. Step3: Identify all the common factors in the two prime expansions found in Step1 and Step2. (If p is a common factor occurring Pm and Pn times in m and n, respectively, it should be repeated in min{Pm, Pn} times.) Step4: Compute the product of all the common factors and return it as the gcd of the numbers given. For numbers 60 and 24 we get 60=2.2.3.5 24=2.2.2.3Gcd(60,24)=2.2.3=12

class 14 PESSE 10

What can we learn from the previous 3 examples? Each step of an algorithm must be unambiguous. The same algorithm can be represented in several

different ways. (different pseudo codes) There might exists more than one algorithm for a

certain problem. Algorithms for the same problem can be based on

very different ideas and can solve the problem with dramatically different speeds.

class 14 PESSE 11

Sieve of Eratosthenes Algm to generating consecutive primes not exceeding

any given integer n. Starts by initializing a list of prime numbers with

consecutive integers from 2 to n. Mark the very first number as a prime And eliminate from the list all multiples of 2(but

greater then 2) i.e.,4,6,8…..next select 3 and multiples of 3 will be eliminated

Mark next as 5 and eliminate all multiples of 5. √ 25 is 5 5 will the last number.

class 14 PESSE 12

Prime numbers by sieves method2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 252 3 * 5 * 7 * 9 * 11 * 13 * 15 * 17 * 19 * 21 * 23 * 252 3 * 5 * 7 * * * 11 * 13 * * * 17 * 19 * * * 23 * 252 3 * 5 * 7 * * * 11 * 13 * * * 17 * 19 * * * 23 * * 2 3 5 7 11 13 17 19 23 prime numbers between 2 to 25.P largest number whose multiples can still remain on the list and

p*p should not be larger than n

So p can not exceed √ n

class 14 PESSE 13

Algorithm for Sieves methodALGORITHM Sieve(n)// implements the sieve of Eratosthenes//Input: A positive integer n>=2//Output:Array L of all prime numbers less than or equal to nFor p 2 to n do A[p] p // to generate list of numbersFor p 2 to √ n do if A[p] != 0 // p has not been eliminated on previous pass j p * p While j<=n do A[j] 0 // mark an element as eliminated j j+p// copy the remaining elements (non zero elements) of A to array L of the

primesi 0For p 2 to n do if A[p] != 0 L[i] A[p] i i +1Return L

class 14 PESSE 14

Fundamentals of Algorithmic Problem Solving

We can consider algorithms to be procedure solution to problems.

These solutions are not answers but rather precisely defined procedures for getting answers.

class 14 PESSE 15

Fundamentals of Algorithmic Problem Solving Understanding the problem

Asking questions, do a few examples by hand, think about special cases, etc.

Deciding on Exact vs. approximate problem solving Appropriate data structure

Design an algorithm Proving correctness Analyzing an algorithm

Time efficiency : how fast the algorithm runs Space efficiency: how much extra memory the algorithm needs. Simplicity and generality

Coding an algorithm

class 14 PESSE 16

Fundamentals of Algorithmic Problem Solving

Understanding the problem

Deciding on:Computational means,

Exact vs. approximate problem solving, data structure (s),

Algorithm design technique

Design an algorithm

Prove correctness

Analyze the algorithm

Code the algorithm

class 14 PESSE 17

Understanding the problem Before designing an algorithm. Asking questions, do a few examples by

hand, think about special cases, etc. Input to an algm. Specifies an instance of

the problem the algm. Solves. Important to specify exactly the range of instances the algorithm needs to handle. Else it will work correctly for majority of inputs but crash on some “boundary” value.

class 14 PESSE 18

Ascertain the capabilities of a computational device. After understanding need to ascertain the

capabilities of the device. Von Neumann architecturesequential. And are called sequential algorithms. Algorithms which designed to to be executed

on parallel computers called parallel algorithms.

For very complex algorithmsconcentrate on a machine with high speed and more memory.

class 14 PESSE 19

Choosing between exact and approximate problem solving For exact result exact algorithm For approximate result approximation

algorithm. why to select? Obtaining square roots for some numbers

and solving nonlinear equations Deciding on appropriate data structures.. Algorithm + data structure=program.

class 14 PESSE 20

Algorithm design techniques: How to design an algorithm? Methods of specifying an algorithm: Using natural language ambiguity will

be there 2 options: pseudo code and flowchart Pseudo code: mix of natural and

programming language. More precise than NL.

class 14 PESSE 21

Algorithm design techniques Flow chart: method of expressing an

algorithm by collection of connected geometric shapes containing descriptions of the algorithm’s steps.

Inconvenience for large problems.

class 14 PESSE 22

Proving an Algorithm’s Correctness

After specifying an algorithm we have to prove its correctness. (Euclid’s algm)

Proof by mathematical induction is most appropriate for proving the correctness of an algorithm.

class 14 PESSE 23

Analyzing an algorithm After correctness, efficiency Time efficiency and space efficiency Time: how fast the algorithm runs? Space: how much extra memory the

algorithm needs? Simplicity: Generality: If not satisfied with these 3necessary to re

design the algorithm.

class 14 PESSE 24

Code the algorithm Writing program by lang. Selection of programming language

should support the features mentioned in the design phase.

Program testing documentation

class 14 PESSE 25

Important Problem Types Sorting Searching String processing Graph problems Combinatorial problems Geometric problems Numerical problems

class 14 PESSE 26

Sorting (I) Rearrange the items of a given list in

ascending order. Input: A sequence of n numbers <a1, a2, …, an> Output: A reordering <a´

1, a´2, …, a´

n> of the input sequence such that a´

1≤ a´2 ≤ … ≤ a´

n.

Why sorting? Help searching Algorithms often use sorting as a key subroutine.

Sorting key A specially chosen piece of information used to

guide sorting. I.e., sort student records by names.

class 14 PESSE 27

Sorting (II) Examples of sorting algorithms

Selection sort Bubble sort Insertion sort Merge sort Heap sort …

Evaluate sorting algorithm complexity: the number of key comparisons.

Two properties Stability: A sorting algorithm is called stable if it preserves

the relative order of any two equal elements in its input. In place : A sorting algorithm is in place if it does not require

extra memory, except, possibly for a few memory units.

class 14 PESSE 28

Searching Find a given value, called a search key, in a

given set. Examples of searching algorithms

Sequential searching Binary searching…

There is no single lagm to search.Some are faster but requires more space..some

are very fast but requires sorted list and so on.

class 14 PESSE 29

String Processing A string is a sequence of characters

from an alphabet. Text strings: letters, numbers, and

special characters. Bit strings:comprise zeros and ones String matching: searching for a given

word/pattern in a text. Used in assemblers and compilers.

class 14 PESSE 30

Graph Problems Informal definition

A graph is a collection of points called vertices, some of which are connected by line segments called edges.

Modeling reallife problems Modeling WWW communication networks Project scheduling …

Examples of graph algorithms Graph traversal algorithms (how can one visit all the

points in a network?) Shortestpath algorithms (best route b/w 2 cities) Topological sorting for graphs with directed edge.

class 14 PESSE 31

Combinatorial problems Involves permutations, combinations or

subset constructions that satisfy certain conditions by either maximizing the profit or minimizing the profit.

Ex:traveling salesman and graph coloring problems.

Most difficult problems in computing from both theoretical and the practical stand point.

class 14 PESSE 32

Geometric problems Deals with geometric objects such as points, lines

and polygons. These involves closest pair and convex hull problem. Numerical problems Involves solving equations, computing integrations,

evaluating various types of functions and so on… can not be solved exactly

class 14 PESSE 33

Fundamental Data Structures Linear data structures Stacks, queues, and heaps Graphs Trees

class 14 PESSE 34

Linear Data Structures Arrays

A sequence of n items of the same data type that are stored contiguously in computer memory and made accessible by specifying a value of the array’s index.

Linked List A sequence of zero or more nodes

each containing two kinds of information: some data and one or more links called pointers to other nodes of the linked list.

Singly linked list (next pointer) Doubly linked list (next + previous

pointers)

Arrays fixed length (need preliminary

reservation of memory) contiguous memory locations direct access Insert/delete is tedious job

Linked Lists dynamic length arbitrary memory locations access by following links Insert/delete

class 14 PESSE 35

Stacks, Queues, and Heaps (1) Stacks:is a list where insertion and deletion can can be

done only at the end. A stack of plates

insertion/deletion can be done only at the top. LIFO

Two operations (push and pop) Queues: is a list from which elements are deleted from

one end of the structure called front end (dequeue) and new elements are added to the other end called rear (enqueue) A queue of customers waiting for services

Insertion/enqueue from the rear and deletion/dequeue from the front.

FIFO Two operations (enqueue and dequeue)

class 14 PESSE 36

Stacks, Queues, and Heaps (2) Priority queues (implemented using heaps)

A data structure for maintaining a set of elements, each associated with a key/priority, with the following operations

Finding the element with the highest priority Deleting the element with the highest priority Inserting a new element

Scheduling jobs on a shared computer.

class 14 PESSE 37

Graphs Formal definition

A graph G = <V, E> is defined by a pair of two sets: a finite set V of items called vertices and a set E of vertex pairs called edges.

Undirected and directed graphs (digraph). Complete, dense, and sparse graph

A graph with every pair of its vertices connected by an edge is called complete. K|V|

class 14 PESSE 38

Graphs

ca

e

b

d

a c

f

b

fd e

Undirected graph Directed graph

class 14 PESSE 39

Graph Representation Adjacency matrix

n x n boolean matrix if |V| is n. The element on the ith row and jth column is 1 if there’s an

edge from ith vertex to the jth vertex; otherwise 0. The adjacency matrix of an undirected graph is symmetric. a b c d e f

a 0 0 1 1 0 0 b 0 0 1 0 0 1 c 1 1 0 0 1 0 d 1 0 0 0 1 0 e 0 0 1 1 0 1 f 0 1 0 0 1 0

Adjacency matrix

class 14 PESSE 40

Adjacency linked lists A collection of linked lists, one for each

vertex, that contain all the vertices adjacent to the list’s vertex.

A c d B c f C a b e D a e E c d f F b > e Adjacency linked list

class 14 PESSE 41

Weighted Graphs Weighted graphs

Graphs or digraphs with numbers assigned to the edges.

Used in reallife applications, such as finding shortest path b/w 2 points in transportation or network…

5 1 7 4 2

a b

c d

class 14 PESSE 42

Adjacency matrix and linked list for weighted graph a b c d A ∞ 5 1 ∞ B 5 ∞ 7 4 C 1 7 ∞ 2 D ∞ 4 2 ∞

a b,5c,1 B a,5 c,7 d,4 C a,1b,7 d,2 D b,4c,2

class 14 PESSE 43

Graph Properties Paths and Connectivity Paths

A path from vertex u to v of a graph G is defined as a sequence of adjacent (connected by an edge) vertices that starts with u and ends with v.

Simple paths: All edges of a path are distinct. Path lengths: the number of edges, or the number of

vertices – 1. Direcetd path: all the edges are directed.

Connected graphs A graph is said to be connected if for every pair of its

vertices u and v there is a path from u to v. Connected component

The maximum connected subgraph of a given graph.

class 14 PESSE 44

Graph Properties Acyclicity Cycle

A simple path of a positive length that starts and ends a the same vertex.

Acyclic graph A graph without cycles DAG (Directed Acyclic Graph)

class 14 PESSE 45

Trees (I) Trees

A tree (or free tree) is a connected acyclic graph. Forests: a graph that has no cycles but is not

necessarily connected.

a b

c d

gf

a

c

f

b

d e i

h

jg

tree forest

class 14 PESSE 46

Properties of trees

For every two vertices in a tree there always exists exactly one simple path from one of these vertices to the other.

Rooted trees: The above property makes it possible to select an arbitrary vertex in a free tree and consider it as the root of the socalled rooted tree.

Levels of rooted tree.

|E| = |V| 1

class 14 PESSE 47

Trees (II) ancestors

For any vertex v in a tree T, all the vertices on the simple path from the root to that vertex are called ancestors.

descendants All the vertices for which a vertex v is an ancestor are said to be

descendants of v. parent, child and siblings

If (u, v) is the last edge of the simple path from the root to vertex v (and u ‡ v), u is said to be the parent of v and v is called a child of u.

Vertices that have the same parent are called siblings. Leaves

A vertex without children is called a leaf. Subtree

A vertex v with all its descendants is called the subtree of T rooted at v.

class 14 PESSE 48

Trees (III)

Depth of a vertex The length of the simple path from the root to the

vertex. Height of a tree

The length of the longest simple path from the root to a leaf.

class 14 PESSE 49

Ordered Trees

Ordered trees An ordered tree is a rooted tree in which all the children of each

vertex are ordered. Binary trees

A binary tree is an ordered tree in which every vertex has no more than two children and each children is designated s either a left child or a right child of its parent.

Binary search trees Each vertex is assigned a number. A number assigned to each parental vertex is larger than all the

numbers in its left subtree and smaller than all the numbers in its right subtree.

ADA-Unit 1

Documents

Transcript of ADA-Unit 1