Post on 10-May-2021
OSHistoryandOSStructures
KarthikDantuCSE421/521:Opera>ngSystems
SlidesadoptedfromCS162classatBerkeley,CSE451atU-WashingtonandCSE421byProfKosaratUB
Ac>onItemsFromLastClass• JoinPiazza• Lookthroughassignment#0• Setupdevelopmentenvironment:VirtualBox+Ubuntu16.04
• Implementassignmentandtestintheenvironment• Formgroups
WhatisanOS?
• SoYwaretomanageacomputer’sresourcesforitsusersandapplica>ons
TCP/IP Networking
Virtual Memory
Hardware-Specific Software and Device Drivers
File System
Scheduling
Graphics Processor
Address TranslationProcessors
Network
Hardware
Users
User-mode
Kernel-modeKernel-user Interface
(Abstract virtual machine)
Hardware Abstraction Layer
APP
SystemLibrary
APP
SystemLibrary
APP
SystemLibrary
Disk
ComputerPerformanceOverTime1.3 Operating Systems: Past, Present, and Future 27
1981 1997 2014 Factor(2014/1981)
Uniprocessor speed (MIPS) 1 200 2500 2.5K
CPUs per computer 1 1 10+ 10+
Processor MIPS/$ $100K $25 $0.20 500K
DRAM Capacity (MiB)/$ 0.002 2 1K 500K
Disk Capacity (GiB)/$ 0.003 7 25K 10M
Home Internet 300 bps 256 Kbps 20 Mbps 100K
Machine room network10 Mbps(shared)
100 Mbps(switched)
10 Gbps(switched) 1000
Ratio of usersto computers
100:1 1:1 1:several 100+
Figure 1.8: Approximate computer server performance over time, reflecting the most widely used servers ofeach era: in 1981, a minicomputer; in 1997, a high-end workstation; in 2014, a rack-mounted multicoreserver. MIPS stands for “millions of instructions per second,” a measure of processor performance. The VAX11/782 was introduced in 1982; it achieved 1 MIP. DRAM prices are from Hennessey and Patterson,“Computer Architecture: A Quantitative Approach.” Disk drive prices are from John McCallum. The Hayessmartmodem, introduced in 1981, ran at 300bps. The 10 Mbps shared Ethernet standard was alsointroduced in 1981. One of the authors built his first operating system in 1982, used a VAX at his first job,and owned a Hayes to work from home.
from expensive to cheap devices occurred with telephones over the pasthundred years. Initially, telephone lines were very expensive, and a singleline was shared among everyone in a neighborhood. Over time, of course,both computers and telephones have become cheap enough to sit idle untilwe need them.
Despite these changes, operating systems still face the same conceptualchallenges as they did fifty years ago. To manage computer resources for ap-plications and users, they must allocate resources among applications, providefault isolation and communication services, abstract hardware limitations, andso forth. Tremendous progress has been made towards improving the reliabil-ity, security, efficiency, and portability of operating systems, but much more isneeded. Although we do not know precisely how computing technology orapplication demand will evolve over the next 10-20 years, it is highly likelythat these fundamental operating system challenges will persist.
Early Operating SystemsComputers wereexpensive; users
would wait.The first operating systems were runtime libraries intended to simplify theprogramming of early computer systems. Rather than the tiny, inexpensiveyet massively complex hardware and software systems of today, the firstcomputers often took up an entire floor of a warehouse, cost millions of
EarlyOpera>ngSystems:SerialOpera>ons
• Oneapplica>onata>me– Hadcompletecontrolofhardware– OSwasrun>melibrary– Userswouldstandinlinetousethecomputer
• Batchsystems– KeepCPUbusybyhavingaqueueofjobs– OSwouldloadnextjobwhilecurrentoneruns– Userswouldsubmitjobs,andwait,andwait,and
Time-SharingOpera>ngSystems:Client-ServerAge
• Mul>pleusersoncomputeratsame>me– Mul>programming:runmul>pleprogramsatsame>me– Interac>veperformance:trytocompleteeveryone’stasksquickly
– Ascomputersbecamecheaper,moreimportanttoop>mizeforuser>me,notcomputer>me
Today’sOpera>ngSystems:ComputersCheap
• Smartphones• Embeddedsystems• Laptops• Tablets• Virtualmachines• Datacenterservers
Tomorrow’sOpera>ngSystems
• Giant-scaledatacenters• Increasingnumbersofprocessorspercomputer• Increasingnumbersofcomputersperuser• Verylargescalestorage• MarkWeiser:UbiquitousandPervasiveCompu>ng
OSHistory
UnixHistory• Firstdevelopedin1969byKenThompsonandDennisRitchieof
theResearchGroupatBellLaboratories;incorporatedfeaturesofotheropera>ngsystems,especiallyMULTICS
• ThethirdversionwaswrideninC,whichwasdevelopedatBellLabsspecificallytosupportUNIX
• Themostinfluen>alofthenon-BellLabsandnon-AT&TUNIXdevelopmentgroups—UniversityofCaliforniaatBerkeley(BerkeleySoYwareDistribu>ons-BSD)
• 4BSDUNIXresultedfromDARPAfundingtodevelopastandardUNIXsystemforgovernmentuse
• DevelopedfortheVAX,4.3BSDisoneofthemostinfluen>alversions,andhasbeenportedtomanyotherplanorms
• Severalstandardiza>onprojectsseektoconsolidatethevariantflavorsofUNIXleadingtooneprogramminginterfacetoUNIX
TimelineofUnixversions
WhatisanOS?
• SoYwaretomanageacomputer’sresourcesforitsusersandapplica>ons
TCP/IP Networking
Virtual Memory
Hardware-Specific Software and Device Drivers
File System
Scheduling
Graphics Processor
Address TranslationProcessors
Network
Hardware
Users
User-mode
Kernel-modeKernel-user Interface
(Abstract virtual machine)
Hardware Abstraction Layer
APP
SystemLibrary
APP
SystemLibrary
APP
SystemLibrary
Disk
Opera>ngSystemRoles
• Referee:– Resourcealloca>onamongusers,applica>ons– Isola>onofdifferentusers,applica>onsfromeachother– Communica>onbetweenusers,applica>ons
• Illusionist– Eachapplica>onappearstohavetheen>remachinetoitself– Infinitenumberofprocessors,(near)infiniteamountofmemory,reliablestorage,reliablenetworktransport
• Glue– Libraries,userinterfacewidgets,…
Example:FileSystems
• Referee– Preventusersfromaccessingeachother’sfileswithoutpermission
– EvenaYerafileisdele>nganditsspacere-used• Illusionist– Filescangrow(nearly)arbitrarilylarge– Filespersistevenwhenthemachinecrashesinthemiddleofasave
• Glue– Nameddirectories,prinn,…
Ques>on
• What(hardware,soYware)doyouneedtobeabletorunanuntrustworthyapplica>on?
OSChallenges-Correctness• Reliability– Doesthesystemdowhatitwasdesignedtodo?
• Availability– Whatpor>onofthe>meisthesystemworking?– MeanTimeToFailure(MTTF),MeanTimetoRepair
• Security– Canthesystembecompromisedbyanadacker?
• Privacy– Dataisaccessibleonlytoauthorizedusers
OSChallenges–WideApplicability
• Portability– Forprograms:
• Applica>onprogramminginterface(API)
• Abstractvirtualmachine(AVM)
– Fortheopera>ngsystem• Hardwareabstrac>onlayer
TCP/IP Networking
Virtual Memory
Hardware-Specific Software and Device Drivers
File System
Scheduling
Graphics Processor
Address TranslationProcessors
Network
Hardware
Users
User-mode
Kernel-modeKernel-user Interface
(Abstract virtual machine)
Hardware Abstraction Layer
APP
SystemLibrary
APP
SystemLibrary
APP
SystemLibrary
Disk
OSChallenges-Performance
• Latency/response>me– Howlongdoesanopera>ontaketocomplete?
• Throughput– Howmanyopera>onscanbedoneperunitof>me?
• Overhead– HowmuchextraworkisdonebytheOS?
• Fairness– Howequalistheperformancereceivedbydifferentusers?
• Predictability– Howconsistentistheperformanceover>me?
OPERATINGSYSTEMSSTRUCTURES
Today:FourFundamentalOSConcepts• Thread
– Singleuniqueexecu>oncontext:fullydescribesprogramstate– ProgramCounter,Registers,Execu>onFlags,Stack
• Addressspace(withtransla>on)– Programsexecuteinanaddressspacethatisdis>nctfromthememoryspaceofthephysicalmachine
• Process– Aninstanceofanexecu>ngprogramisaprocessconsis,ngofanaddressspaceandoneormorethreadsofcontrol
• Dualmodeopera>on/Protec>on– Onlythe“system”hastheabilitytoaccesscertainresources– TheOSandthehardwareareprotectedfromuserprogramsanduserprogramsareisolatedfromoneanotherbycontrollingthetransla,onfromprogramvirtualaddressestomachinephysicaladdresses
OSBodomLine:RunPrograms
int main() { … ; }
edito
rProgram Source
foo.c
Load
&
Exec
ute M
emory
PC:
Processor
registers
0x000…
0xFFF…
instructions
data
heap
stack
OS
com
pile
r
Executable
a.out
data
instructions
• Loadinstruc>onanddatasegmentsofexecutablefileintomemory
• Createstackandheap• “Transfercontroltoprogram”• Provideservicestoprogram• Whileprotec>ngOSandprogram
Instruc>onFetch/Decode/ExecuteCycleTheinstruc>oncycle
PC:
Instruction fetch
Registers
ALU
Execute
Memory
instruction
Decode decode
next
data
Processor
FetchExec
R0…
R31F0…F30PC
…Data1Data0Inst237Inst236
…Inst5Inst4Inst3Inst2�Inst1Inst0
Addr 0
Addr 232-1
Whathappensduringprogramexecu>on?
• Execu>onsequence:– FetchInstruc>onatPC– Decode– Execute(possiblyusingregisters)– Writeresultstoregisters/mem– PC=NextInstruc>on(PC)– Repeat
PCPCPCPC
FirstOSConcept:ThreadofControl• Certainregistersholdthecontextofthread– Stackpointerholdstheaddressofthetopofstack
• Otherconven>ons:Framepointer,Heappointer,Data– Maybedefinedbytheinstruc>onsetarchitectureorbycompilerconven>ons
• Thread:Singleuniqueexecu>oncontext– ProgramCounter,Registers,Execu>onFlags,Stack
• Athreadisexecu>ngonaprocessorwhenitisresidentintheprocessorregisters.
• PCregisterholdstheaddressofexecu>nginstruc>oninthethread
• Registersholdtherootstateofthethread.– Therestis“inmemory”
SecondOSConcept:Program’sAddressSpace
0x000…
0xFFF…
code
Static Data
heap
stack• Addressspace⇒thesetofaccessibleaddresses+stateassociatedwiththem:– Fora32-bitprocessorthereare232=4billionaddresses
• Whathappenswhenyoureadorwritetoanaddress?– Perhapsnothing– Perhapsactslikeregularmemory– Perhapsignoreswrites– PerhapscausesI/Oopera>on
• (Memory-mappedI/O)– Perhapscausesexcep>on(fault)
AddressSpace:InaPicture
Processorregisters
PC:
0x000…
0xFFF…
Code Segment
Static Data
heap
stack
instruction
SP:
• What’sinthecodesegment?Sta>cdatasegment?• What’sintheStackSegment?
– Howisitallocated?Howbigisit?• What’sintheHeapSegment?
– Howisitallocated?Howbig?
Mul>programming-Mul>pleThreadsofControl
OS
Proc 1
Proc 2
Proc n…
codeStatic Data
heap
stack
codeStatic Data
heap
stack
codeStatic Data
heap
stack
CPU
Howcanwegivetheillusionofmul>pleprocessors?
vCPU3vCPU2vCPU1
Shared Memory
• Assumeasingleprocessor.Howdoweprovidetheillusionofmul>pleprocessors?– Mul>plexin>me!
• Eachvirtual“CPU”needsastructuretohold:– ProgramCounter(PC),StackPointer(SP)– Registers(Integer,Floa>ngpoint,others…?)
• HowswitchfromonevirtualCPUtothenext?– SavePC,SP,andregistersincurrentstateblock– LoadPC,SP,andregistersfromnewstateblock
• Whattriggersswitch?– Timer,voluntaryyield,I/O,otherthings
vCPU1 vCPU2 vCPU3 vCPU1 vCPU2
Time
TheBasicProblemofConcurrency• Thebasicproblemofconcurrencyinvolvesresources:– Hardware:singleCPU,singleDRAM,singleI/Odevices– Mul>programmingAPI:processesthinktheyhaveexclusiveaccesstosharedresources
• OShastocoordinateallac>vity– Mul>pleprocesses,I/Ointerrupts,…– Howcanitkeepallthesethingsstraight?
• BasicIdea:UseVirtualMachineabstrac>on– Simplemachineabstrac>onforprocesses– Mul>plextheseabstractmachines
Proper>esofthissimplemul>programmingtechnique
• AllvirtualCPUssharesamenon-CPUresources– I/Odevicesthesame– Memorythesame
• Consequenceofsharing:– Eachthreadcanaccessthedataofeveryotherthread(goodforsharing,badforprotec>on)
– Threadscanshareinstruc>ons(goodforsharing,badforprotec>on)
– CanthreadsoverwriteOSfunc>ons?• This(unprotected)modeliscommonin:– Embeddedapplica>ons– Windows3.1/EarlyMacintosh(switchonlywithyield)– Windows95—ME(switchwithbothyieldand>mer)
Protec>on• Opera>ngSystemmustprotectitselffromuserprograms– Reliability:compromisingtheopera>ngsystemgenerallycausesittocrash
– Security:limitthescopeofwhatprocessescando– Privacy:limiteachprocesstothedataitispermidedtoaccess– Fairness:eachshouldbelimitedtoitsappropriateshareofsystemresources(CPU>me,memory,I/O,etc)
• ItmustprotectUserprogramsfromoneanother• PrimaryMechanism:limitthetransla>onfromprogramaddressspacetophysicalmemoryspace– Canonlytouchwhatismappedintoprocessaddressspace
• Addi>onalMechanisms:– Privilegedinstruc>ons,in/outinstruc>ons,specialregisters– syscallprocessing,subsystemimplementa>on
• (e.g.,fileaccessrights,etc)
ThirdOSConcept:Process• Process: execu>onenvironmentwithRestrictedRights
– Address Space with One or More Threads– Ownsmemory(addressspace)– Ownsfiledescriptors,filesystemcontext,…– Encapsulateoneormorethreadssharingprocessresources
• Whyprocesses?– Protectedfromeachother!– OSProtectedfromthem– Processesprovidesmemoryprotec>on– Threadsmoreefficientthanprocesses(later)
• Fundamentaltradeoffbetweenprotec>onandefficiency• Communica>oneasierwithinaprocess• Communica>onharderbetweenprocesses
• Applica>oninstanceconsistsofoneormoreprocesses– E.g.,Facebookapponyourphone
SingleandMul>threadedProcesses
• Threadsencapsulateconcurrency:“Ac>ve”component• Addressspacesencapsulateprotec>on:“Passive”part– Keepsbuggyprogramfromtrashingthesystem
• Whyhavemul>plethreadsperaddressspace?– E.g.,webserver
FourthOSConcept:DualModeOpera>on• Hardwareprovidesatleasttwomodes:
– “Kernel”mode(or“supervisor”or“protected”)– “User”mode:Normalprogramsexecuted
• Whatisneededinthehardwaretosupport“dualmode”opera>on?– Abitofstate(user/systemmodebit)– Certainopera>ons/ac>onsonlypermidedinsystem/kernelmode
• Inusermodetheyfailortrap
– UseràKerneltransi>onsetssystemmodeANDsavestheuserPC• Opera>ngsystemcodecarefullyputsasideuserstatethenperformsthenecessaryopera>ons
– KernelàUsertransi>onclearssystemmodeANDrestoresappropriateuserPC• return-from-interrupt
User/Kernel(Privileged)Mode
User Mode
Kernel Mode
Full HW accessLimited HW access
exec
syscall
exitrtn
interrupt
rfi
exception
SimpleProtec>on:BaseandBound(B&B)
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
0000…
1100…
0100…
Bound
1100…
1000…
Base
>=
<
Programaddress
0010…
1010…
SimpleProtec>on:BaseandBound(B&B)
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
0000…
1100…
0100…
Bound
1100…
1000…
Base
>=
<
Programaddress
0010…
1010…
• Requiresreloca>ngloader• S>llprotectsOSandisolatesprogram
• Noaddi>ononaddresspath
Addresses translated when program is loaded
Anotheridea:AddressSpaceTransla>on• Programoperatesinanaddressspacethatisdis>nctfromthephysicalmemoryspaceofthemachine
Processor Memory
0x000…
0xFFF…
translator
Asimpleaddresstransla>onwithBaseandBound
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
0000…
Programaddress
Base Address
Bound <
1000…
1100…0100…
• CantheprogramtouchOS?• Canittouchotherprograms?
0010…0010…
Addresses translated �on-the-fly
Tyingittogether:SimpleB&B:OSloadsprocess
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
1100…
3000…
3080…
Base xxxx …
xxxx…Bound
xxxx…uPC
regs
sysmode
…
1
PC
0000…
FFFF…
SimpleB&B:OSgetsreadytoexecuteprocess
• PrivilegedInst:setspecialregisters
• RTU
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
1100…
3000…
3080…
Base 1000 …
1100…Bound
0001…uPC
regs
sysmode
…
1
PC
0000…
FFFF…
00FF…
RTU
SimpleB&B:UserCodeRunning
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
1100…
3000…
3080…
Base 1000 …
1100…Bound
xxxx…uPC
regs
sysmode
…
0
PC
0000…
FFFF…
00FF…
• Howdoeskernelswitchbetweenprocesses?
• Firstques>on:Howtoreturntosystem?
0001…
3typesofModeTransfer• Syscall
– Processrequestsasystemservice,e.g.,exit– Likeafunc>oncall,but“outside”theprocess– Doesnothavetheaddressofthesystemfunc>ontocall– LikeaRemoteProcedureCall(RPC)–forlater– Marshallthesyscallidandargsinregistersandexecsyscall
• Interrupt– Externalasynchronouseventtriggerscontextswitch– e.g.,Timer,I/Odevice– Independentofuserprocess
• TraporExcep>on– Internalsynchronouseventinprocesstriggerscontextswitch– e.g.,Protec>onviola>on(segmenta>onfault),Dividebyzero,…
• All3areanUNPROGRAMMEDCONTROLTRANSFER– Wheredoesitgo?
Howdowegetthesystemtargetaddressofthe“unprogrammedcontroltransfer?”
InterruptVector
interrupt number (i)
intrpHandler_i () { …. }
Address and properties of each interrupt handler
SimpleB&B:User=>Kernel
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
1100…
3000…
3080…
Base 1000 …
1100…Bound
xxxx…uPC
regs
sysmode
…
0
PC
0000…
FFFF…
00FF…
• Howtoreturntosystem?
0000 1234
SimpleB&B:Interrupt
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
1100…
3000…
3080…
Base 1000 …
1100 …Bound
0000 1234uPC
regs
sysmode
…
1
PC
0000…
FFFF…
00FF…
• Howtosaveregistersandsetupsystemstack?
IntrpVector[i]
SimpleB&B:SwitchUserProcess
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
1100…
3000…
3080…
Base 3000 …
0080 …Bound
0000 0248uPC
regs
sysmode
…
1
PC
0000…
FFFF…
00D0…
• Howtosaveregistersandsetupsystemstack?
0001 0124
1000 …
1100 …
0000 1234
regs
00FF…
RTU
SimpleB&B:“resume”
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
1100…
3000…
3080…
Base 3000 …
0080 …Bound
xxxx xxxxuPC
regs
sysmode
…
0
PC
0000…
FFFF…
00D0…
• Howtosaveregistersandsetupsystemstack?
000 0248
1000 …
1100 …
0000 1234
regs
00FF…
RTU
Conclusion:FourfundamentalOSconcepts
• Thread– Singleuniqueexecu>oncontext– ProgramCounter,Registers,Execu>onFlags,Stack
• AddressSpacewithTransla>on– Programsexecuteinanaddressspacethatisdis>nctfromthememoryspaceofthephysicalmachine
• Process– Aninstanceofanexecu>ngprogramisaprocessconsis,ngofanaddressspaceandoneormorethreadsofcontrol
• DualModeopera>on/Protec>on– Onlythe“system”hastheabilitytoaccesscertainresources– TheOSandthehardwareareprotectedfromuserprogramsanduserprogramsareisolatedfromoneanotherbycontrollingthetransla,onfromprogramvirtualaddressestomachinephysicaladdresses