Advance Computing for Electrical Engineers
Transcript of Advance Computing for Electrical Engineers
![Page 1: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/1.jpg)
Advance Computing for Electrical Engineers An Introduction
Virendra SinghAssociate ProfessorComputer Architecture and Dependable Systems LabDepartment of Electrical EngineeringIndian Institute of Technology Bombay http://www.ee.iitb.ac.in/~viren/E-mail: [email protected]
EE717/453:Advance Computing for Electrical Engineers
Lecture0
![Page 2: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/2.jpg)
EE 717/453
• Pre-requisite● Knowledge of programming (Any language C/C+
+/Java, ..)
• Attendance– Not mandatory (except first week)– Submission of assignments (firm deadlines)
• Timing– M Th (Slot 13): 6:30 pm to 8:00 pm– Office Hours: Wed 4 pm to 6 pm
18 July 2013 EE-717/453@IITB 2
![Page 3: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/3.jpg)
Course Outline• Data Structures and Algorithm
– About 6 - 8 classes– Cover basic data structures (Stack, Queue,
Linked List, Tree, Graph, Hashing – Basic Algorithm Design
• Computer Architecture and Compiler Design
– About 2-4 lectures– Basic architectures and compiler design
phases
18 July 2013 EE-717/453@IITB 3
![Page 4: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/4.jpg)
Course Outline• Operating System
– About 6-8 classes– Basics of operating system– PintOS– Implementation of basic OS functions
(Threads, user program, virtual memory, file system)
Parallel Programming– About 6 – 8 lectures– Basics of parallel programming– CUDA
18 July 2013 EE-717/453@IITB 4
![Page 5: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/5.jpg)
Course Outline• Miscellaneous
– About 1-2 lectures
• Summary
18 July 2013 EE-717/453@IITB 5
![Page 6: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/6.jpg)
Books• Data Structures, Algorithms, and its applications in
C++– Sartaj Sahni
• Introduction to Algorithms– Cormen et al.
• Modern Operating Systems– Tanenbaum
• Introduction to parallel programming– Thomas Rauber
18 July 2013 EE-717/453@IITB 6
![Page 7: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/7.jpg)
Evaluation
• Mid semester Examination– 15 Marks
• Final Examination– 30 Marks
• Assignments– 40 Marks
• Continuous Evaluation– 15 Marks
18 July 2013 EE-717/453@IITB 7
![Page 8: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/8.jpg)
GradesAbsolute• AA: > 94• AB : 85 - 94• BB : 75 - 84• BC : 65 – 74• CC : 55 – 74• CD : 45 – 54• DD : 40 – 45• FR: < 40
18 July 2013 EE-717/453@IITB 8
![Page 9: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/9.jpg)
Problem Solving: Main Steps
1. Problem definition
2. Algorithm design/ Algorithm specification
3. Algorithm analysis
4. Implementation
5. Testing
6. [Maintenance]
18 July 2013 EE-717/453@IITB 9
![Page 10: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/10.jpg)
Running Program on Processor
Processor Performance = ---------------Time
Program
Architecture --> Implementation --> Realization
Compiler Designer Processor Designer Chip Designer
Instructions
Cycles
Program
Instruction
TimeCycle
(code size)
= X X
(CPI)
(cycle time)
EE-717/453@IITB18 July 2013
10
![Page 11: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/11.jpg)
From Source to Executable
Compiler
main()sub1()data
source program
foo.c
mainsub1data
objectmodules
foo.o
printfscanfgets
fopenexitdata
...
static librarylibc.a
LinkageEditor
mainsub1dataprintfexitdata
loadmodule
a.out
otherprograms
...
mainsub1dataprintfexitdata
other......
kernel
Machine memory
?
(systemcalls)
Loader
(Run Time)
Dynamic library case not shown
“Load time”
18 July 2013 EE-717/453@IITB 11
![Page 12: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/12.jpg)
Problem Definition
• What is the task to be accomplished?– Calculate the average of the grades for a
given student – Understand the talks given out by politicians
and translate them in Chinese
What are the time / space / speed performance requirements ?
18 July 2013 EE-717/453@IITB 12
![Page 13: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/13.jpg)
Algorithm Design/Specifications• Algorithm: Finite set of instructions that, if followed,
accomplishes a particular task. • Describe: in natural language / pseudo-code / diagrams /
etc • Criteria to follow:
– Input: Zero or more quantities (externally produced) – Output: One or more quantities – Definiteness: Clarity, precision of each instruction – Finiteness: The algorithm has to stop after a finite (may
be very large) number of steps – Effectiveness: Each instruction has to be basic enough
and feasible● Understand speech● Translate to Chinese
18 July 2013 EE-717/453@IITB 13
![Page 14: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/14.jpg)
Algorithm Design/Specifications• Algorithm: Finite set of instructions that, if followed,
accomplishes a particular task. • Describe: in natural language / pseudo-code / diagrams /
etc • Criteria to follow:
– Input: Zero or more quantities (externally produced) – Output: One or more quantities – Definiteness: Clarity, precision of each instruction – Finiteness: The algorithm has to stop after a finite (may
be very large) number of steps – Effectiveness: Each instruction has to be basic enough
and feasible● Understand speech● Translate to Chinese
18 July 2013 EE-717/453@IITB 14
![Page 15: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/15.jpg)
Implementation, Testing, and Maintenance• Implementation
– Decide on the programming language to use ● C,C++,Lisp,Java,Perl,Prolog,Assembly etc.
– Write clean, well documented code
• Test, test, test
• Integrate feedback from users, fix bugs, ensure compatibility across different versions Maintenance
18 July 2013 EE-717/453@IITB 15
![Page 16: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/16.jpg)
Algorithm Analysis
• Space complexity– How much space is required
Time complexity– How much time does it take to run the
algorithm
Often, we deal with estimates!
18 July 2013 EE-717/453@IITB 16
![Page 17: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/17.jpg)
Space Complexity (1/3)• Space complexity = The amount of
memory required by an algorithm to run to completion
– [Core dumps = the most often encountered cause is “memory leaks” – the amount of memory required larger than the memory available on a given system]
Some algorithms may be more efficient if data completely loaded into memory
– Need to look also at system limitations– E.g. Classify 10 GB of text in various categories
[politics, tourism, sport, natural disasters, etc.] – can I afford to load the entire collection? 18 July 2013 EE-717/453@IITB 17
![Page 18: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/18.jpg)
Space complexity (2/3 )• Fixed part: The size required to store
certain data/variables, that is independent of the size of the problem:
– e.g. name of the data collection – same size for classifying 2GB or 1MB of texts
• Variable part: Space needed by variables, whose size is dependent on the size of the problem:
– e.g. actual text– load 2GB of text VS. load 1MB of text
18 July 2013 EE-717/453@IITB 18
![Page 19: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/19.jpg)
Space Complexity (3/3)
• S(P) = c + S(instance characteristics)• Example: void float sum (float* a, int n){
float s = 0;
for (int i = 0; I < n; i++){
s += a[i]
}
return s;
}
• Space? One word for n, one for a [passed by reference], one for i Constant space
18 July 2013 EE-717/453@IITB 19
![Page 20: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/20.jpg)
Time Complexity
• Often more important than space complexity
– space available (for computer programs!) tends to be larger and larger
– time is still a problem for all of us
3-4 GHz multi-core processors are in the market
– still researchers estimate that the computation of various transformations for 1 single DNA chain for one single protein on 10 GHz computer would take about 1 year to run to completion
Algorithms running time is an important issue
18 July 2013 EE-717/453@IITB 20
![Page 21: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/21.jpg)
Running Time• Problem: Prefix average
– Given an array X – Compute the array A such that A[i] is the average of
elements X[0] ... X[i], for i=0..n-1
Solution 1:– At each step i, compute the element X[i] by
traversing the array A and determining the sum of its elements, respectively the average
Solution 2:– At each step i update a sum of the elements in the
array A – Compute the element X[i] as sum/I
Big question: Which solution to choose?18 July 2013 EE-717/453@IITB 21
![Page 22: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/22.jpg)
Running Time
• Suppose the program includes an if-then statement that may execute or not variable running time
• Typically algorithms are measured by their worst case
18 July 2013 EE-717/453@IITB 22
Worst case
Best case
Average case
![Page 23: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/23.jpg)
Experimental Approach
• Write a program that implements the algorithm
• Run the program with data sets of varying size.
• Determine the actual running time using a system call to measure time (e.g. system (date) );
• Problems?
18 July 2013 EE-717/453@IITB 23
![Page 24: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/24.jpg)
Experimental Approach
• It is necessary to implement and test the algorithm in order to determine its running time
• Experiments can be done only on a limited set of inputs, and may not be indicative of the running time of the other inputs
• The same hardware and software should be used in order to compare two algorithms – condition very hard to achieve!
18 July 2013 EE-717/453@IITB 24
![Page 25: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/25.jpg)
Theoretical Approach
• Based on high-level description of the algorithms, rather than language dependent implementation
• Makes possible an evaluation of the algorithms that is independent of the hardware and software environment
Generality
18 July 2013 EE-717/453@IITB 25
![Page 26: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/26.jpg)
Algorithm Description• How to describe algorithms independent of a programming
language• Pseudo-Code = a description of an algorithm that
– more structured than usual prose but– less formal than a programming language
• (Or diagram)• Example: find the maximum element of an array
Algorithm ArrayMax (A, n):
Input: An array A storing n intergers
Output: The maximum element in A
currentMax A[0]
for I 1 to n-1 do
if currentMax < A[i] then currentMax A[i]
return currentMax18 July 2013 EE-717/453@IITB 26
![Page 27: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/27.jpg)
Pseudo Code• Expression: use standard mathematical symbols
– use for assignment– use = for the equality relationship
• Method declarations: Algorithm name (parameters)• Programming constructs:
– Decision structures: if … then … [else ..]– While-loop: while .. Do– Repeat-loops: repeat … until …– For-loop: for … do– Array indexing: A[i]
• Methods– Calls: object methods (arguments) ; - return: return value
• Use comments• Instructions have to be basic enough and feasible
18 July 2013 EE-717/453@IITB 27
![Page 28: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/28.jpg)
Low Level Algorithm Analysis• Based on primitive operations (low-level
computations independent from the programming language)
• For example:– Make an addition = 1 operation– Calling a method or returning from a method = 1
operation– Index in an array = 1 operation– Comparison = 1 operation etc.
• Method: Inspect the pseudo-code and count the number of primitive operations executed by the algorithm
18 July 2013 EE-717/453@IITB 28
![Page 29: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/29.jpg)
Example
• Algorithm ArrayMax (A, n):Input: An array A storing n intergers
Output: The maximum element in A
currentMax A[0]
for I 1 to n-1 do
if currentMax < A[i] then currentMax A[i]
return currentMax
• How many operations ?
18 July 2013 EE-717/453@IITB 29
![Page 30: Advance Computing for Electrical Engineers](https://reader030.fdocuments.in/reader030/viewer/2022040822/624ddb75b898c037a8393505/html5/thumbnails/30.jpg)
Asymptotic Notation
• Need to abstract further• Give an idea of how the algorithm
performs• n steps vs. n+5 steps• n steps vs. n2 steps
18 July 2013 EE-717/453@IITB 30