LECTURE NOTE

ON

DATA STRUCTURE AND ALGORITHMS

COM 213

LEGACY POLYTECNIC

INTRODUCTION

Digital computer can be simply described as a powerful electronic device which has the capability to accept data as input, apply a series of arithmetic and logical operations on the data, and produce the results of these operations as an output (information) at a very fast speed.

A computer can also be defined as an electronic machine that accepts and processes data by following a set of instructions to produce and accurate and efficient result.

COMPUTER AS A SYSTEM: A system can be described as a unit composed of two or more subsystem or subunits that have defined environment and specified interrelationship towards achieving stated goal and objective.

I. Computer display the following characteristics to a greater or lesser extent depending on their type and application.

II. The ability to take in data and to store that data for future retrieval and use.III. The ability to perform calculations at high speeds.IV. The ability to take in and store a sequence of instructions (program) for the

computer to obey.V. The ability to obey a sequence of program instructions provided the

instructions are stored within the computer. These program instructions are executed without manual intervention at each step.

VI. The ability to use simple logical rules to make decisions for their own internal control, for the control of some activity.

ALGORITHMS

Designing a program necessitates the programmer to write detailed logical stages to the solution in the form of descriptive language known as algorithm psuedocode. An algorithm is the step-by-step procedure of accomplishing a task. Flowchart is a diagrammatic representation of an algorithm; that is, it is a visual picture, which

shows the logical steps involve in solving a problem. Computer science is a field of study that deals with the study of algorithms. A good algorithm must satisfy the following:

I. INPUT: An input, zero or more quantities must be supplied.II. OUTPUT: There must be at least one quantity to be produced.

III. EFFECTIVENESS: Every instruction must be sufficiently basic that it can in principle be carried out by a person using only pen and paper.

IV. DEFINITENESS: Each instruction must be clear and unambiguous.V. FINITENESS: If the

Algorithms as opposed to programs

An algorithm for a particular task can be defined as “a finite sequence of instructions, each of which has a clear meaning and can be performed with a finite amount of effort in a finite length of time”. As such, an algorithm must be precise enough to be understood by human beings. However, in order to be executed by a computer, we will generally need a program that is written in a rigorous formal language; and since computers are quite inflexible compared to the human mind; programs usually need to contain more details than algorithms. Here we shall ignore most of those programming details and concentrate on the design of algorithms rather than programs.

The task of implementing the discussed algorithms as computer programs is important, of course, but these notes will concentrate on the theoretical aspects and leave the practical programming aspects to be studied elsewhere. Having said that, we will often find it useful to write down segments of actual programs in order to clarify and test certain theoretical aspects of algorithms and their data structures. It is also worth bearing in mind the distinction between different programming paradigms: Imperative Programming describes computation in terms of instructions that change the program/data state, whereas Declarative Programming specifies what the program should accomplish without describing how to do it. These notes will primarily be concerned with developing algorithms that map easily onto the imperative programming approach.

Algorithms can obviously be described in plain English, and we will sometimes do that. However, for computer scientists it is usually easier and clearer to use something

that comes somewhere in between formatted English and computer program code, but is not runnable because certain details are omitted. This is called pseudocode, which comes in a variety of forms. Often these notes will present segments of pseudocode that are very similar to the languages we are mainly interested in, namely the overlap of C and Java, with the advantage that they can easily be inserted into runnable programs.

Data structures, abstract data types, design patterns

For many problems, the ability to formulate an efficient algorithm depends on being able to organize the data in an appropriate manner. The term data structure is used to denote a particular way of organizing data for particular types of operation. These notes will look at numerous data structures ranging from familiar arrays and lists to more complex structures such as trees, heaps and graphs, and we will see how their choice affects the efficiency of the algorithms based upon them.

Often we want to talk about data structures without having to worry about all the im- plementational details associated with particular programming languages, or how the data is stored in computer memory. We can do this by formulating abstract mathematical models of particular classes of data structures or data types which have common features. These are called abstract data types, and are defined only by the operations that may be performed on them. Typically, we specify how they are built out of more primitive data types (e.g., integers or strings), how to extract that data from them, and some basic checks to control the flow of processing in algorithms. The idea that the implementational details are hidden from the user and protected from outside access is known as encapsulation. We shall see many examples of abstract data types throughout these notes.

At an even higher level of abstraction are design patterns which describe the design of algorithms, rather the design of data structures. These embody and generalize important design concepts that appear repeatedly in many problem contexts. They provide a general structure for algorithms, leaving the details to be added as required for particular problems. These can speed up the development of algorithms by providing familiar proven algorithm structures that can be applied straightforwardly to new problems. We shall see a number of familiar design

patterns throughout these notes.

Language processor

A language processor is a software program designed or used to perform tasks such as processing program code to machine code. Language processors are found in languages such as FORTRAN and COBOL.

Examples of language processors

There are two main types of language processors:

Interpreter - allows a computer to interpret, or understand, what a software program needs the computer to do, what tasks to perform.

Translator - takes a program's code and translates it into machine code, allowing the computer to read and understand what tasks the program needs to be done in its native code. An assembler and a compiler are examples of translators.

The difference between an interpreter and a translator is that an interpreter is telling the computer what to do. A translator takes the program's code and converts it to machine code, allowing the computer to read it. Essentially, the interpreter tells the computer what to do and the translator lets the computer figure out what to do by itself.

BASIC FLOWCHARTING SHAPES AND SYMBOLS

Flowcharts use special shapes to represent different types of actions or steps in a process. Lines and arrows show the sequence of the steps, and the relationships among them.

Start/EndThe terminator symbol marks the starting or ending point of the system. It usually contains the word "Start" or "End."

Action or ProcessA box can represent a single step ("add two cups of flour"), or and entire sub-process ("make bread") within a larger process.

DocumentA printed document or report.

DecisionA decision or branching point. Lines representing different decisions emerge from different points of the diamond.

Input/OutputRepresents material or information entering or leaving the system, such as customer order (input) or a product (output).

ConnectorIndicates that the flow continues where a matching symbol (containing the same letter)

has been placed.

Flow LineLines indicate the sequence of steps and the direction of flow.

DelayIndicates a delay in the process.

MergeIndicates a step where two or more sub-lists or sub-processes become one.

CollateIndicates a step that orders information into a standard format.

SortIndicates a step that organizes a list of items into a sequence or sets based on some pre-determined criteria.

SubroutineIndicates a sequence of actions that perform a specific task embedded within a larger process. This sequence of actions could be described in more detail on a separate flowchart.

Manual LoopIndicates a sequence of commands that will continue to repeat until stopped manually.

Loop LimitIndicates the point at which a loop should stop.

Data storageIndicates a step where data gets stored.

DatabaseIndicates a list of information with a standard structure that allows for searching and sorting.

DisplayIndicates a step that displays information.

Off PageIndicates that the process continues off page.

HOW TO DRAW FLOWCHARTS

Basic Flowchart

A basic flowchart identifies the starting and ending points of a process, the sequence of actions in the process, and the decision or branching points along the way.

The Programming Process Developing a program involves steps similar to any problem-solving task. There are five main ingredients in the programming process:

1. Defining the problem 2. Planning the solution 3. Coding the program

4. Testing the program 5. Documenting the program

Let us discuss each of these in turn.

1. Defining the Problem Suppose that, as a programmer, you are contacted because your services are needed. You meet with users from the client organization to analyze the problem, or you meet with a systems analyst who outlines the project. Specifically, the task of defining the problem consists of identifying what it is you know (input-given data), and what it is you want to obtain (output-the result). Eventually, you produce a written agreement that, among other things, specifies the kind of input, processing, and output required. This is not a simple process.

2. Planning the Solution ;Two common ways of planning the solution to a problem are to draw a flowchart and to write pseudocode, or possibly both. Essentially, a flowchart is a pictorial representation of a step-by-step solution to a problem. It consists of arrows representing the direction the program takes and boxes and other symbols representing actions. It is a map of what your program is going to do and how it is going to do it. The American National Standards Institute (ANSI) has developed a standard set of flowchart symbols. Figure 1 shows the symbols and how they might be used in a simple flowchart of a common everyday act-preparing a letter for mailing.

Pseudocode is an English-like nonstandard language that lets you state your solution with more precision than you can in plain English but with less precision than is required when using a formal programming language. Pseudocode permits you to focus on the program logic without having to be concerned just yet about the precise syntax of a particular programming language. However, pseudocode is not executable on the computer. We will illustrate these later in this chapter, when we focus on language examples.

3. Coding the Program As the programmer, your next step is to code the program-that is, to express your solution in a programming language. You will translate the logic from the flowchart or pseudocode-or some other tool-to a programming language. As we have already noted, a programming language is a set of rules that provides a way of instructing the computer what operations to perform. There are many programming languages: BASIC, COBOL, Pascal, FORTRAN, and C are

some examples. You may find yourself working with one or more of these. We will discuss the different types of languages in detail later in this chapter.

Although programming languages operate grammatically, somewhat like the English language, they are much more precise. To get your program to work, you have to follow exactly the rules-the syntax-of the language you are using. Of course, using the language correctly is no guarantee that your program will work, any more than speaking grammatically correct English means you know what you are talking about. The point is that correct use of the language is the required first step. Then your coded program must be keyed, probably using a terminal or personal computer, in a form the computer can understand.

One more note here: Programmers usually use a text editor, which is somewhat like a word processing program, to create a file that contains the program. However, as a beginner, you will probably want to write your program code on paper first.

4. Testing the Program Some experts insist that a well-designed program can be written correctly the first time. In fact, they assert that there are mathematical ways to prove that a program is correct. However, the imperfections of the world are still with us, so most programmers get used to the idea that their newly written programs probably have a few errors. This is a bit discouraging at first, since programmers tend to be precise, careful, detail-oriented people who take pride in their work. Still, there are many opportunities to introduce mistakes into programs, and you, just as those who have gone before you, will probably find several of them.

Eventually, after coding the program, you must prepare to test it on the computer. This step involves these phases:

o Desk-checking. This phase, similar to proofreading, is sometimes avoided by the programmer who is looking for a shortcut and is eager to run the program on the computer once it is written. However, with careful desk-checking you may discover several errors and possibly save yourself time in the long run. In desk-checking you simply sit down and mentally trace, or check, the logic of the program to attempt to ensure that it is error-free and workable. Many organizations take this phase a step further with a walkthrough, a process in which a group of programmers-your peers-review your program and offer suggestions in a collegial way.

o Translating. A translator is a program that (1) checks the syntax of your program to make sure the programming language was used correctly, giving you all the syntax-error messages, called diagnostics, and (2) then translates your program into a form the computer can understand. A by-product of the process is that the translator tells you if you have improperly used the programming language in some way. These types of mistakes are called syntax errors. The translator produces descriptive error messages. For instance, if in FORTRAN you mistakenly write N=2 *(I+J))-which has two closing parentheses instead of one-you will get a message that says, "UNMATCHED PARENTHESES." (Different translators may provide different wording for error messages.) Programs are most commonly translated by a compiler. A compiler translates your entire program at one time. The translation involves your original program, called a source module, which is transformed by a compiler into an object module. Prewritten programs from a system library may be added during the link/load phase, which results in a load module. The load module can then be executed by the computer.

o Debugging. A term used extensively in programming, debugging means detecting, locating, and correcting bugs (mistakes), usually by running the program. These bugs are logic errors, such as telling a computer to repeat an operation but not telling it how to stop repeating. In this phase you run the program using test data that you devise. You must plan the test data carefully to make sure you test every part of the program.

5. Documenting the Program Documenting is an ongoing, necessary process, although, as many programmers are, you may be eager to pursue more exciting computer-centered activities. Documentation is a written detailed description of the programming cycle and specific facts about the program. Typical program documentation materials include the origin and nature of the problem, a brief narrative description of the program, logic tools such as flowcharts and pseudocode, data-record descriptions, program listings, and testing results. Comments in the program itself are also considered an essential part of documentation. Many programmers document as they code. In a broader sense, program documentation can be part of the documentation for an entire system.

Arrays, Iteration, Invariants

Data is ultimately stored in computers as patterns of bits, though these days most program- ming languages deal with higher level objects, such as characters, integers, and floating point numbers. Generally, we need to build algorithms that manipulate collections of such objects, so we need procedures for storing and sequentially processing them.

Arrays

In computer science, the obvious way to store an ordered collection of items is as an array. Array items are typically stored in a sequence of computer memory locations, but to discuss them, we need a convenient way to write them down on paper. We can just write the items in order, separated by commas and enclosed by square brackets

Loops and Iteration

The standard approach in most programming languages for repeating a process a certain number of times, such as moving sequentially through an array to perform the same operations on each item, involves a loop.

Syntax

the syntax of a computer language is the set of rules that defines the combinations of symbols that are considered to be a correctly structured document or fragment in that language .This applies both to programming languages, where the document represents source code, and to markup languages, where the document represents data.

Semantics

Semantics contrasts with syntax, the study of the combinatorics of units of a language (without reference to their meaning), and pragmatics, the study of the relationships between the symbols of a language, their meaning, and the users of the language. Semantics as a field of study also has significant ties to various representational theories of meaning including truth theories of meaning, coherence theories of meaning, and correspondence theories of meaning.

DATA

In computing, data is information that has been translated into a form that is efficient for movement or processing. Relative to today's computers and transmission media, data is information converted into binary digital form. It is acceptable for data to be used as a singular subject or a plural subject. Raw data is a term used to describe data in its most basic digital format.

The concept of data in the context of computing has its roots in the work of Claude Shannon, an American mathematician known as the father of information theory.  He ushered in binary digital concepts based on applying two-value Boolean logic to electronic circuits. Binary digit formats underlie the CPUs, semiconductor memories and disk drives, as well as many of the peripheral devices common in computing today. Early computer input for both control and data took the form of punch cards, followed by magnetic tape and the hard disk. 

Early on, data's importance in business computing became apparent by the popularity of the terms "data processing" and "electronic data processing," which, for a time, came to encompass the full gamut of what is now known as information technology.  Over the history of corporate computing, specialization occurred, and a distinct data profession emerged along with growth of corporate data processing.

How data is stored

Computers represent data, including video, images, sounds and text, as binary values using patterns of just two numbers: 1 and 0. A bit is the smallest unit of data, and

represents just a single value. A byte is eight binary digits long. Storage and memory is measured in megabytes and gigabytes.

The units of data measurement continue to grow as the amount of data collected and stored grows. The relatively new term "brontobyte," for example, is data storage that is equal to 10 to the 27th power of bytes.

TOOLS FOR IEDENTIFRICATION OF DATA

Data can be described by the five W’s. Who, what, where, when and why.

Who created the data? Presumably, someone created the data for a reason. This person, the data owner, has the initial responsibility for storing the data in an appropriate location and for granting access to the data so it is important to know who these people are.

What information does the data contain? Classify the data so that you can understand if it should be protected against loss or disclosure and how much effort should be expended in protecting it.

Where is the data located? The location of the data determines the level of organizational control that can be enacted over the data. An organization would have little control over data on a social network, but they may have a great deal of control over data in an Enterprise Resource Planning (ERP) tool.

When was the data created? When was it accessed? When was it archived? This common metadata of creation, access and archive date is important because it can show how important the data is to the company. Less frequently used data is generally considered less important. It is also important to know when the data was last archived or backed up since this determines whether the data can be recovered if it is lost, stolen or corrupted.

Why does the data exist? This is one of the most important questions because data that is not needed should be deleted. There is no reason to protect data that provides no value. This data is only a liability as the loss of the data could impact the organization. Even if the loss is inconsequential, storing, indexing and managing data takes time and money, so organizations would be well served to remove nonessential data.

Why waste time and money implementing security that does not address the data itself? This, all too common approach, often results in some data being under

protected or not protected at all while other data is overprotected. Furthermore, since the organization does not know of some data, a breach of that data is more likely to go unnoticed. Understand the five W’s and create security controls, policies and procedures to govern how the data is used, stored, shared and deleted.

Data Structures - Algorithms BasicsAlgorithm is a step-by-step procedure, which defines a set of instructions to be executed in a certain order to get the desired output. Algorithms are generally created independent of underlying languages, i.e. an algorithm can be implemented in more than one programming language.

A data structure is a particular way of organizing data in a computer so that it can be used effectively.

A database is a collection of information that is organized so that it can be easily accessed, managed and updated. Computer databases typically contain aggregations of data records or files, containing information about sales transactions or interactions with specific customers.

A field is an area in a fixed or known location in a unit of data such as a record, message header, or computer instruction that has a purpose and usually a fixed size. In some contexts, a field can be subdivided into smaller fields.

A “universe” of data is a data set that encompasses the entirety of collectable data.

A character is a unit of information that roughly corresponds to a grapheme, grapheme-like unit, or symbol, such as in an alphabet or syllabary in the written form of a natural language.

From the data structure point of view, following are some important categories of algorithms −

Search − Algorithm to search an item in a data structure.

Sort − Algorithm to sort items in a certain order.

Insert − Algorithm to insert item in a data structure.

Update − Algorithm to update an existing item in a data structure.

Delete − Algorithm to delete an existing item from a data structure.

Characteristics of an Algorithm

Not all procedures can be called an algorithm. An algorithm should have the following characteristics −

Unambiguous − Algorithm should be clear and unambiguous. Each of its steps (or phases), and their inputs/outputs should be clear and must lead to only one meaning.

Input − An algorithm should have 0 or more well-defined inputs.

Output − An algorithm should have 1 or more well-defined outputs, and should match the desired output.

Finiteness − Algorithms must terminate after a finite number of steps.

Feasibility − Should be feasible with the available resources.

Independent − An algorithm should have step-by-step directions, which should be independent of any programming code.

How to Write an Algorithm?

There are no well-defined standards for writing algorithms. Rather, it is problem and resource dependent. Algorithms are never written to support a particular programming code.

As we know that all programming languages share basic code constructs like loops (do, for, while), flow-control (if-else), etc. These common constructs can be used to write an algorithm.

We write algorithms in a step-by-step manner, but it is not always the case. Algorithm writing is a process and is executed after the problem domain is well-defined. That is, we should know the problem domain, for which we are designing a solution.

Example

Let's try to learn algorithm-writing by using an example.

Problem − Design an algorithm to add two numbers and display the result.

Step 1 − START

Step 2 − declare three integers a, b & c

Step 3 − define values of a & b

Step 4 − add values of a & b

Step 5 − store output of step 4 to c

Step 6 − print c

Step 7 − STOP

Algorithms tell the programmers how to code the program. Alternatively, the algorithm can be written as −

Step 1 − START ADD

Step 2 − get values of a & b

Step 3 − c ← a + b

Step 4 − display c

Step 5 − STOP

In design and analysis of algorithms, usually the second method is used to describe an algorithm. It makes it easy for the analyst to analyze the algorithm ignoring all unwanted definitions. He can observe what operations are being used and how the process is flowing.

Writing step numbers, is optional.

We design an algorithm to get a solution of a given problem. A problem can be solved in more than one ways.

Hence, many solution algorithms can be derived for a given problem. The next step is to analyze those proposed solution algorithms and implement the best suitable solution.

Algorithm Analysis

Efficiency of an algorithm can be analyzed at two different stages, before implementation and after implementation. They are the following −

A Priori Analysis − This is a theoretical analysis of an algorithm. Efficiency of an algorithm is measured by assuming that all other factors, for example, processor speed, are constant and have no effect on the implementation.

A Posterior Analysis − This is an empirical analysis of an algorithm. The selected algorithm is implemented using programming language. This is then executed on target computer machine. In this analysis, actual statistics like running time and space required, are collected.

We shall learn about a priori algorithm analysis. Algorithm analysis deals with the execution or running time of various operations involved. The running time of an operation can be defined as the number of computer instructions executed per operation.

Algorithm Complexity

Suppose X is an algorithm and n is the size of input data, the time and space used by the algorithm X are the two main factors, which decide the efficiency of X.

Time Factor − Time is measured by counting the number of key operations such as comparisons in the sorting algorithm.

Space Factor − Space is measured by counting the maximum memory space required by the algorithm.

The complexity of an algorithm f(n) gives the running time and/or the storage space required by the algorithm in terms of n as the size of input data.

Space Complexity

Space complexity of an algorithm represents the amount of memory space required by the algorithm in its life cycle. The space required by an algorithm is equal to the sum of the following two components −

A fixed part that is a space required to store certain data and variables, that are independent of the size of the problem. For example, simple variables and constants used, program size, etc.

A variable part is a space required by variables, whose size depends on the size of the problem. For example, dynamic memory allocation, recursion stack space, etc.

Space complexity S(P) of any algorithm P is S(P) = C + SP(I), where C is the fixed part and S(I) is the variable part of the algorithm, which depends on instance characteristic I. Following is a simple example that tries to explain the concept −

Algorithm: SUM(A, B)

Step 1 - START

Step 2 - C ← A + B + 10

Step 3 - Stop

Here we have three variables A, B, and C and one constant. Hence S(P) = 1 + 3. Now, space depends on data types of given variables and constant types and it will be multiplied accordingly.

Time Complexity

Time complexity of an algorithm represents the amount of time required by the algorithm to run to completion. Time requirements can be defined as a numerical function T(n), where T(n) can be measured as the number of steps, provided each step consumes constant time.

For example, addition of two n-bit integers takes n steps. Consequently, the total computational time is T(n) = c n, where c is the time taken for the addition of two ∗bits. Here, we observe that T(n) grows linearly as the input size increases.