1

DATA STRUCTURES AND ALGORITHM

Lecture No. 2

Nabeel Sabir

7th October, 201004/12/2023

07:02 AM

04/12/2023 07:02 AM

2

Review of Previous Lecture

A data structure is a particular way of storing and organizing data in a computer so that it can be used efficiently.

Linear Data Structures Arrays Linked List Stacks Queues

Non Linear Data Structures Graphs Trees Hash Tables

04/12/2023 07:02 AM

A precise rule (or set of rules) specifying how to solve some problem.

Introduction to Algorithms3

04/12/2023 07:02 AM

4

What is an Algorithm?

An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.

Properties Can be represented various forms Unambiguity/clearness Effectiveness Finiteness/termination Correctness

04/12/2023 07:02 AM

5

What is an Algorithm?

Recipe, process, method, technique, procedure, routine,… with the following requirements:

1. Finiteness terminates after a finite number of steps

2. Definiteness rigorously and unambiguously specified

3. Clearly specified input valid inputs are clearly specified

4. Clearly specified/expected output can be proved to produce the correct output given a valid

input

5. Effectiveness steps are sufficiently simple and basic

04/12/2023 07:02 AM

6

Why Study Algorithms?

Algorithms solve problems Good choice: more efficient programs Bad choice: poor programs performance Example:

Problem: Find the largest element ‘k’ out of ‘N’ integers Easy algorithms: sort all integers, then list the first or last

element Better algorithm: sort first element then read through the

list Different algorithms perform better on different

inputs Input size also affect the performance.

04/12/2023 07:02 AM

7

Notion of Algorithm and Problem

“Computer”

Problem

Algorithm

Input Output

04/12/2023 07:02 AM

8

Representation of an Algorithms An algorithm may be represented in

different forms: A description using English/other languages A real computer program, e.g. C++ A pseudo-code, C-like program, program-

language-like program.

Program = algorithms + data structures

04/12/2023 07:02 AM

9

Basic Issues Related to Algorithms How to design algorithms

How to express algorithms

Proving correctness

Efficiency (or complexity) analysis Theoretical analysis

Empirical analysis

Optimality

04/12/2023 07:02 AM

10

Analysis of Algorithms

How good is the algorithm? Correctness Time efficiency Space efficiency

Does there exist a better algorithm? Lower bounds Optimality

04/12/2023 07:02 AM

11

Algorithm Efficiency

There are often many algorithms for a given problem. How do we choose the best? Goals of program design:

Algorithm is to be easy to understand, code, debug

Algorithm makes efficient use of computer’s resources

How to measure the efficiency? Empirical comparison (run the program) Asymptotic algorithm analysis (without running

the program) Critical resources Factors affecting running time (size of the

input)

04/12/2023 07:02 AM

12

Best, Worst and Average Cases Not all inputs of a given size take the

same time. Each algorithm has three cases:

Best case: Worst Case: Average Case:

04/12/2023 07:02 AM

13

Example: Best, Worst and Average Cases

Sequential search for ‘k’ in an array of ‘n’ integers: Best case: ‘k’ is the first element of the

array. Worst case: the search must visit every

element once. This happens when the value being searched for is either the last element in the list, or is not in the list

Average case: on average, assuming the value searched for is in the list and each list element is equally likely to be the value searched for, the search visits only n/2 elements.

04/12/2023 07:02 AM

We analyse algorithms, rather than problems. A problem can be solved with several algorithms, some are more efficient than others.

Algorithm Analysis14

04/12/2023 07:02 AM

15

Why need algorithm analysis?

Just writing a syntax-error-free program is not enough. We need to know whether the algorithm is correct or not, i.e. whether it can give correct answers for the inputs.

If the program is run on a large data set, the running time becomes an issue. We want to know how the algorithm performs when the input size is large. The program may be computationally inefficient, or may need lots of memory. We analyze the resources that the algorithm requires: memory, and computation time. We can also compare algorithms without implementations.

04/12/2023 07:02 AM

16

How to Analyse an Algorithm?

To analyse an algorithm; determine the amount of resources (such as time and storage) necessary to execute it.

Most algorithms are designed to work with inputs of arbitrary length.

Usually the efficiency or running time of an algorithm is stated as a function relating the input length to the number of steps (time complexity) or storage locations (space complexity).

04/12/2023 07:02 AM

17

Analysis of Algorithms

Analysis of Algorithm refers to calculating or guessing resources needful for the algorithm.

Resources means computer memory, processing time etc.

In all these factors, time is most important because the program developed should be fast enough.

04/12/2023 07:02 AM

18

Analysis of Algorithms

Time complexity The amount of time that an algorithm needs to

run to completion Space complexity

The amount of memory an algorithm needs to run

We will occasionally look at space complexity, but we are mostly interested in time complexity in this course.

Thus in this course the better algorithm is the one which runs faster (has smaller time complexity)

04/12/2023 07:02 AM

Running time of an algorithm is stated as a function relating the input length to the number of steps (time complexity) or storage locations (space complexity).

Running Time of An Algorithm

19

04/12/2023 07:02 AM

20

Factors Affecting Running Time There are several factors affecting the

running time Computer Compiler Algorithm Input to the algorithm

The content of the input affects the running time typically, the input size (number of items in the input) is the main consideration

E.g. sorting problem ⇒ the number of items to be sorted.

04/12/2023 07:02 AM

21

Running time of an Algorithm Running time of an algorithm depends upon the

input size and nature of input. Most algorithms transform input objects into

output objects

The running time of an algorithm typically grows with the input size idea: analyze running time as a function of

input size

04/12/2023 07:02 AM

22

Running Time of an Algorithm Even on inputs of the same size, running

time can be very different Example: algorithm that finds the first

prime number in an array by scanning it left to right

Idea: analyze running time in the Best case Worst case Average case

04/12/2023 07:02 AM

23

Finding Running Time of an Algorithm Running time is measured by number of

steps/primitive operations performed

We use RAM (Random Access Model), in which each operation (e.g. +, -, x, /,=) and each memory access take one run-time unit. Loops and functions can take multiple time units.

Count the number of basic operations of an algorithm Read, write, compare, assign, jump, arithmetic

operations (increment, decrement, add, subtract, multiply, divide), open, close, logical operations (not/complement, AND, OR), …

04/12/2023 07:02 AM

24

Example

Lines 1 and 4 count for one unit each. Line 3: executed N times, each time four units. Line 2: (1 for initialization, N+1 for all tests, N for all

increments) total 2N + 2. total time units: 6N + 4. (⇒ O(N), will be discussed

later.)