1GWU CS 259 Brad Taylor Spring 2004 Systems Programming Meeting 1: Systems Technology: Program,...

1GWU CS 259 Brad Taylor Spring 2004

Systems Programming

Meeting 1:Meeting 1:

Systems Technology: Systems Technology: Program, Process & Thread Program, Process & Thread

OverviewOverview


Objectives

•Value / Motivation (Experiment to Value / Motivation (Experiment to Start) Start)

•Course MechanicsCourse Mechanics

•Course Topic OverviewCourse Topic Overview

•Defining Systems ProgrammingDefining Systems Programming


Thought Experiment

•Everyone chose role: Everyone chose role: hardware (I/O, CPU, memory, system bus hardware (I/O, CPU, memory, system bus & clock), OS (processes, signals), & clock), OS (processes, signals), application (pseudo code), operatorapplication (pseudo code), operator•Need: A card from deckNeed: A card from deck•Walk thru (clock cadence): Walk thru (clock cadence): - Operator ‘runs’ processes (stored in - Operator ‘runs’ processes (stored in memory), supplying parameter(s)memory), supplying parameter(s)- Operands read from file (input), - Operands read from file (input), processes retrieved from memory files, processes retrieved from memory files, ALU perform calculations in registers, ALU perform calculations in registers, writes results to separate file (output) … writes results to separate file (output) … •What if some hurdle was added (i.e., What if some hurdle was added (i.e., process file relocation)?process file relocation)?•Lessons Learned? Lessons Learned? •This is Systems Programming!This is Systems Programming!


Systems Programming Motivation

•Familiar with the ‘dominant OS’ Familiar with the ‘dominant OS’ incompatibilities?incompatibilities?•Exploding growth of web uses/ Exploding growth of web uses/ users (linear?) exacerbates issueusers (linear?) exacerbates issue•Systems & Applications: designed Systems & Applications: designed to perform to perform specific tasksspecific tasks and and functionsfunctions. . Do they?Do they?•Goes wrong examples: IEEE, Goes wrong examples: IEEE, NASA/ESA, Power Grids & Plant, NASA/ESA, Power Grids & Plant, costs to $BBscosts to $BBs•Fix it sooner or later?Fix it sooner or later?(Easier, faster vs. ‘hero’)(Easier, faster vs. ‘hero’){Web version: sorry, no movie clip}{Web version: sorry, no movie clip}


Systems Programming Motivation (more …)

•At Operating Systems & Application Programming At Operating Systems & Application Programming intersection: Learn to develop precise code where intersection: Learn to develop precise code where required or systems library use where availablerequired or systems library use where available•FundamentalsFundamentals (UNIX/LINUX programs, processes, files, (UNIX/LINUX programs, processes, files, directories, I/O, programming tools, token ring)directories, I/O, programming tools, token ring) •Asynchronous EventsAsynchronous Events (signals, timers)(signals, timers) •ConcurrencyConcurrency (threads, synchronization, critical (threads, synchronization, critical sections, semaphores, inter-process communication, sections, semaphores, inter-process communication, SMP/Cray)SMP/Cray) •Communication Communication (connection-oriented, wireless, (connection-oriented, wireless, MANET/ProxiNet)MANET/ProxiNet) •ProjectProject (develop actual release modules for open source (develop actual release modules for open source reusable library for scripting programs—expert in our midst!)reusable library for scripting programs—expert in our midst!)


Course Mechanics

Course Web Page: Course Web Page: www.seas.gwu.www.seas.gwu.eduedu/~csci259/~csci259//

(materials, links, assignments, exams)(materials, links, assignments, exams)Email to: Email to:

btaylorbtaylor@[email protected] (questions, anytime) (questions, anytime)CSCI259CSCI259@@gwugwu..eduedu (assignments, exams) (assignments, exams)

Office Hours: Monday 5 - 6pm, Phillips Hall Office Hours: Monday 5 - 6pm, Phillips Hall 720 (or by appt.)720 (or by appt.)

Prereqs: CSci 210; ‘C’ & UNIX experience; Prereqs: CSci 210; ‘C’ & UNIX experience; SEAS (or other) UNIX account accessSEAS (or other) UNIX account access

A little background?A little background?


Course Mechanics (more)

Text: Text: UNIX Systems Programming: Communication, UNIX Systems Programming: Communication, Concurrency, and ThreadsConcurrency, and Threads, by Kay A. & Steven , by Kay A. & Steven Robbins (skim before, study after class), Robbins (skim before, study after class), supplementary materials provided and can be supplementary materials provided and can be found (web links, bookstore)found (web links, bookstore)

Class time is discussion; not simply lectureClass time is discussion; not simply lectureHomework from web page, due by email: Homework from web page, due by email:

collaboration allowed in study; final product, your collaboration allowed in study; final product, your own; course project ~ 3 person groups.own; course project ~ 3 person groups.

Programming assignments require archive file with Programming assignments require archive file with ‘readme’ (contents, instructions), source code, ‘readme’ (contents, instructions), source code, executable (SEAS UNIX), test cases & results executable (SEAS UNIX), test cases & results

Coding Guidelines (3 D’s: Document, document…)Coding Guidelines (3 D’s: Document, document…)


Relationship to other coursework

• ProgrammingProgramming (learn to use those involving (learn to use those involving aspects of hardware and system calls)aspects of hardware and system calls)

• Operating SystemsOperating Systems (addresses how (addresses how distributed systems programs use OS services)distributed systems programs use OS services)

• NetworksNetworks (natural progression from Internet (natural progression from Internet Protocols’ specifics to general case) Protocols’ specifics to general case)

• ArchitectureArchitecture (how components interact)(how components interact)

• Formal MethodsFormal Methods (critical systems calls (critical systems calls either robust or complicit in system crashes)either robust or complicit in system crashes)


But What Would I Use It For?

•Single Computer ApplicationsSingle Computer Applications•Networked, Complex SystemsNetworked, Complex Systems•Team & Individual Projects Team & Individual Projects •Safety Critical ApplicationsSafety Critical Applications•Timing Critical AssignmentsTiming Critical Assignments


What will I get out of this?

• Understand & apply course concepts & logic, their Understand & apply course concepts & logic, their strengths & weaknessesstrengths & weaknesses

• Choose appropriate methods for particular system classesChoose appropriate methods for particular system classes• Describe relations between various levels of abstractionDescribe relations between various levels of abstraction• OS ‘Primeval mud’ of computer system: demystifyOS ‘Primeval mud’ of computer system: demystify• OS instance of Complex System: similar to biological (ants, OS instance of Complex System: similar to biological (ants,

DNA, neurons), weather, social (cities, bf/gf), internet DNA, neurons), weather, social (cities, bf/gf), internet examples; all emergent, huge, parallel, not well understood examples; all emergent, huge, parallel, not well understood

• Strategy: Use abstraction, modularity & iteration Strategy: Use abstraction, modularity & iteration {fail early, often, grow from working simple “Hello World” {fail early, often, grow from working simple “Hello World” model!} model!}


Operating Systems

•Operating systems: the Operating systems: the applications’ interface to applications’ interface to hardwarehardware•Even if hardware doesn’t Even if hardware doesn’t change (though it will), every change (though it will), every patch, version update, etc. patch, version update, etc. may appear as hardware may appear as hardware change to applications change to applications •Challenges/Benefits of Challenges/Benefits of multiple applications/usersmultiple applications/users•Adds security (or insecurity)Adds security (or insecurity)


Acquiring Systems Programming Tools

Links will be available to Links will be available to academically available academically available materialsmaterials


Systems Programming

•Key Characteristics? (Brainstorm)Key Characteristics? (Brainstorm)•Reliability, Robustness, MaintainabilityReliability, Robustness, Maintainability•Reliability: Specification provided & validated, Reliability: Specification provided & validated, within resources (budget) {Note: the latter is within resources (budget) {Note: the latter is promoted by early definition of the former – promoted by early definition of the former – using methods we will cover – but multiple using methods we will cover – but multiple iterations expected}iterations expected}•Robustness: Graceful degradation response to Robustness: Graceful degradation response to unanticipated exceptions {Software failure rate unanticipated exceptions {Software failure rate inversely (vs. hardware, directly) related to time}inversely (vs. hardware, directly) related to time}•Maintainability: Handle circumstances changing Maintainability: Handle circumstances changing with time (operators, hardware, operating with time (operators, hardware, operating systems, archive media; ex: storage access vs. systems, archive media; ex: storage access vs. processor speed gap widens)processor speed gap widens)


Embedded Processor OS

•Critical / Safety Critical / Safety ApplicationsApplications•Cars: networks Cars: networks < $10 per node< $10 per node•Chemical plantsChemical plants•AvionicsAvionics•Remote spacecraftRemote spacecraft

Robustness and Security issues not confined to Robustness and Security issues not confined to this context, but clear herethis context, but clear here

Consider remote ‘OnStar’ login & buffer overflowConsider remote ‘OnStar’ login & buffer overflow


Design Process:Abstractions & Mathematics

• Mathematical models (abstractions) represent Mathematical models (abstractions) represent real world systems; OS abstracts hardware & real world systems; OS abstracts hardware & provides greater platform portabilityprovides greater platform portability

• Formal methods & reasoning improve system Formal methods & reasoning improve system design & production ability (reveal design flaws, design & production ability (reveal design flaws, define requirements precisely, implement define requirements precisely, implement rational production)rational production)

• Various logical models have differing strengths Various logical models have differing strengths and weaknessesand weaknesses

• Abstraction: focus on important problem aspectsAbstraction: focus on important problem aspects• Mathematics: reason about problem solutionsMathematics: reason about problem solutions


Where did these systems begin?

• Evolution from: abacus, slide rule, mechanical calculators Evolution from: abacus, slide rule, mechanical calculators & difference engines& difference engines

• Turing machine realized with room of vacuum tubes; Turing machine realized with room of vacuum tubes; programming by physical switches, patch wire and lights; programming by physical switches, patch wire and lights; “time sharing” by assigned time (hour) on machine “time sharing” by assigned time (hour) on machine

• Enhanced input methods: (paper) card & tape readers, Enhanced input methods: (paper) card & tape readers, magnetic tape and disk storage, now optical and soon magnetic tape and disk storage, now optical and soon holographic means; interface evolved first along holographic means; interface evolved first along typewriter path, then GUItypewriter path, then GUI

• Output improvements followed similar pathOutput improvements followed similar path• Meanwhile time sharing progressed from batch Meanwhile time sharing progressed from batch

processing to multitasking mainframes with ‘dumb’ processing to multitasking mainframes with ‘dumb’ terminals; then stand-alone single-user PCs, now terminals; then stand-alone single-user PCs, now networked thru Internetnetworked thru Internet

• Resource sharing of certain network elements (e.g., Resource sharing of certain network elements (e.g., printers, gateways)printers, gateways)


What OS Tools handle Concurrency?

• Interrupts (handling, context switches, device drivers for Interrupts (handling, context switches, device drivers for particular I/O) particular I/O)

• Signals (synchronous or asynchronous) as software Signals (synchronous or asynchronous) as software event notification event notification

• I/O with significantly different handling characteristics I/O with significantly different handling characteristics (cache access vs. user keyboard entry)(cache access vs. user keyboard entry)

• Processes (how different from programs?) started using Processes (how different from programs?) started using forkfork; communication through pipes (common ancestor) ; communication through pipes (common ancestor) or signals, FIFOs/named pipes, semaphores, shared or signals, FIFOs/named pipes, semaphores, shared memory, or messagesmemory, or messages

• Threads (how different from processes?) provide a Threads (how different from processes?) provide a further level of concurrency & further potential resource further level of concurrency & further potential resource conflictsconflicts

• Multiple processors (SMP or ASMP) with distributed Multiple processors (SMP or ASMP) with distributed [network] or shared [Cray/IBM] memory (consider ant [network] or shared [Cray/IBM] memory (consider ant colonies & brain neurons)colonies & brain neurons)


Programs, Processes & Threads

• Compiled Compiled programprogram organized into functions, linked with organized into functions, linked with necessary libraries, with certain defined parameters, stored necessary libraries, with certain defined parameters, stored on diskon disk

• ProcessProcess is a program instance copied by OS into particular is a program instance copied by OS into particular memory space, allowing for tracking variable values (memory space, allowing for tracking variable values (process process statestate), and assigns a unique ), and assigns a unique process ID process ID to distinguish itto distinguish it

• Each flow of control thru the process is a Each flow of control thru the process is a threadthread; each of ; each of which will have a subset of memory space assigned, unique which will have a subset of memory space assigned, unique state & identificationstate & identification

• Threads each have an execution stack, program counter, set Threads each have an execution stack, program counter, set of registers and variable state; synchronization to share of registers and variable state; synchronization to share resources is keyresources is key


Program Image

• Executable appears in contiguous memory blockExecutable appears in contiguous memory block• Diagram Diagram on board …on board …• Environment & command line parametersEnvironment & command line parameters• Activation records for each function call on stackActivation records for each function call on stack• Heap memory allocationsHeap memory allocations• Uninitialized static dataUninitialized static data• Initialized static dataInitialized static data• NOTE: static variables may make program unsafe NOTE: static variables may make program unsafe

for threads (Key: Handling re-entrant functions)for threads (Key: Handling re-entrant functions)


System Function Calls, Process Environment & Termination

• System libraries provide wealth of functions to System libraries provide wealth of functions to interface hardware with application; properly used interface hardware with application; properly used and handling all errors, powerful; improperly and handling all errors, powerful; improperly handled: system crash (Ex: Koopman & MS VP)handled: system crash (Ex: Koopman & MS VP)

• Environmental variables provide an array of Environmental variables provide an array of pointers to strings to pass system- and/or user- pointers to strings to pass system- and/or user- specific information upon starting a new process specific information upon starting a new process for initializationfor initialization

• Proper termination of a process includes notifying Proper termination of a process includes notifying parent process (or parent process (or initinit if a if a zombiezombie, no parent) and , no parent) and releasing resources, resetting OS status inforeleasing resources, resetting OS status info


Conclusion

•The cost of poor systems programming is significant The cost of poor systems programming is significant and rising (MS?)and rising (MS?)•Systems programming alternatives:Systems programming alternatives: - poor design and significant modification during - poor design and significant modification during testing, validation and operation; or testing, validation and operation; or - good design and minor changes during system - good design and minor changes during system validation validation •Course will develop your knowledge of operating Course will develop your knowledge of operating systems fundamentals and concepts of asynchronous systems fundamentals and concepts of asynchronous event notification, concurrency & communication; event notification, concurrency & communication; applicable to most, primarily focus on Unix (POSIX)applicable to most, primarily focus on Unix (POSIX)•Plenty of interaction in class, demonstration of skills Plenty of interaction in class, demonstration of skills on assignments, projects & examson assignments, projects & exams•Questions? Comments?Questions? Comments?

1GWU CS 259 Brad Taylor Spring 2004 Systems Programming Meeting 1: Systems Technology: Program,...

Documents

Transcript of 1GWU CS 259 Brad Taylor Spring 2004 Systems Programming Meeting 1: Systems Technology: Program,...