Operating Systems Principles and Practice
-
Upload
andrea-bongy-quarta -
Category
Documents
-
view
855 -
download
163
Transcript of Operating Systems Principles and Practice
-
OperatingSystems
Principles&Practice
VolumeI:KernelsandProcessesSecondEdition
ThomasAndersonUniversityofWashington
MikeDahlinUniversityofTexasandGoogle
RecursiveBooks
recursivebooks.com
-
OperatingSystems:PrinciplesandPractice(SecondEdition)VolumeI:KernelsandProcessesbyThomasAndersonandMichaelDahlinCopyrightThomasAndersonandMichaelDahlin,2011-2015.
ISBN978-0-9856735-3-6Publisher:RecursiveBooks,Ltd.,http://recursivebooks.com/Cover:ReflectionLake,Mt.RainierCoverdesign:CameronNeatIllustrations:CameronNeatCopyeditors:SandyKaplan,WhitneySchmidtEbookdesign:RobinBriggsWebdesign:AdamAnderson
SUGGESTIONS,COMMENTS,andERRORS.Wewelcomesuggestions,commentsanderrorreports,[email protected]
Noticeofrights.Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformbyanymeanselectronic,mechanical,photocopying,recording,orotherwisewithoutthepriorwrittenpermissionofthepublisher.Forinformationongettingpermissionsforreprintsandexcerpts,[email protected]
Noticeofliability.TheinformationinthisbookisdistributedonanAsIsbasis,withoutwarranty.NeithertheauthorsnorRecursiveBooksshallhaveanyliabilitytoanypersonorentitywithrespecttoanylossordamagecausedorallegedtobecauseddirectlyorindirectlybytheinformationorinstructionscontainedinthisbookorbythecomputersoftwareandhardwareproductsdescribedinit.
Trademarks:Throughoutthisbooktrademarkednamesareused.Ratherthanputatrademarksymbolineveryoccurrenceofatrademarkedname,westateweareusingthenamesonlyinaneditorialfashionandtothebenefitofthetrademarkownerwithnointentionofinfringementofthetrademark.Alltrademarksorservicemarksarethepropertyoftheirrespectiveowners.
http://recursivebooks.com/
-
ToRobin,Sandra,Katya,andAdamTomAnderson
ToMarla,Kelly,andKeithMikeDahlin
-
Contents
Preface
IKernelsandProcesses1Introduction
1.1WhatIsAnOperatingSystem?
1.1.1ResourceSharing:OperatingSystemasReferee1.1.2MaskingLimitations:OperatingSystemasIllusionist1.1.3ProvidingCommonServices:OperatingSystemasGlue1.1.4OperatingSystemDesignPatterns
1.2OperatingSystemEvaluation
1.2.1ReliabilityandAvailability1.2.2Security1.2.3Portability1.2.4Performance1.2.5Adoption1.2.6DesignTradeoffs
1.3OperatingSystems:Past,Present,andFuture
1.3.1ImpactofTechnologyTrends1.3.2EarlyOperatingSystems1.3.3Multi-UserOperatingSystems1.3.4Time-SharingOperatingSystems1.3.5ModernOperatingSystems1.3.6FutureOperatingSystems
Exercises
2TheKernelAbstraction
2.1TheProcessAbstraction
2.2Dual-ModeOperation
2.2.1PrivilegedInstructions2.2.2MemoryProtection2.2.3TimerInterrupts
2.3TypesofModeTransfer
2.3.1UsertoKernelMode2.3.2KerneltoUserMode
2.4ImplementingSafeModeTransfer
2.4.1InterruptVectorTable2.4.2InterruptStack
-
2.4.3TwoStacksperProcess2.4.4InterruptMasking2.4.5HardwareSupportforSavingandRestoringRegisters
2.5PuttingItAllTogether:x86ModeTransfer
2.6ImplementingSecureSystemCalls
2.7StartingaNewProcess
2.8ImplementingUpcalls
2.9CaseStudy:BootinganOperatingSystemKernel
2.10CaseStudy:VirtualMachines
2.11SummaryandFutureDirections
Exercises
3TheProgrammingInterface
3.1ProcessManagement
3.1.1WindowsProcessManagement3.1.2UNIXProcessManagement
3.2Input/Output
3.3CaseStudy:ImplementingaShell
3.4CaseStudy:InterprocessCommunication
3.4.1Producer-ConsumerCommunication3.4.2Client-ServerCommunication
3.5OperatingSystemStructure
3.5.1MonolithicKernels3.5.2Microkernel
3.6SummaryandFutureDirections
Exercises
II:Concurrency4.ConcurrencyandThreads
5.SynchronizingAccesstoSharedObjects
6.Multi-ObjectSynchronization
7.Scheduling
III:MemoryManagement8.AddressTranslation
-
9.CachingandVirtualMemory
10.AdvancedMemoryManagement
IV:PersistentStorage11.FileSystems:IntroductionandOverview
12.StorageDevices
13.FilesandDirectories
14.ReliableStorage
References
Glossary
AbouttheAuthors
-
Preface
PrefacetotheeBookEdition
OperatingSystems:PrinciplesandPracticeisatextbookforafirstcourseinundergraduateoperatingsystems.Inuseatover50collegesanduniversitiesworldwide,thistextbookprovides:
Apathforstudentstounderstandhighlevelconceptsallthewaydowntoworkingcode.Extensiveworkedexamplesintegratedthroughoutthetextprovidestudentsconcreteguidanceforcompletinghomeworkassignments.Afocusonup-to-dateindustrytechnologiesandpractice
TheeBookeditionissplitintofourvolumesthattogethercontainexactlythesamematerialasthe(2nd)printeditionofOperatingSystems:PrinciplesandPractice,reformattedforvariousscreensizes.Eachvolumeisself-containedandcanbeusedasastandalonetext,e.g.,atschoolsthatteachoperatingsystemstopicsacrossmultiplecourses.
Volume1:KernelsandProcesses.ThisvolumecontainsChapters1-3oftheprintedition.Wedescribetheessentialstepsneededtoisolateprogramstopreventbuggyapplicationsandcomputervirusesfromcrashingortakingcontrolofyoursystem.Volume2:Concurrency.ThisvolumecontainsChapters4-7oftheprintedition.Weprovideaconcretemethodologyforwritingcorrectconcurrentprogramsthatisinwidespreaduseinindustry,andweexplainthemechanismsforcontextswitchingandsynchronizationfromfundamentalconceptsdowntoassemblycode.Volume3:MemoryManagement.ThisvolumecontainsChapters8-10oftheprintedition.Weexplainboththetheoryandmechanismsbehind64-bitaddressspacetranslation,demandpaging,andvirtualmachines.Volume4:PersistentStorage.ThisvolumecontainsChapters11-14oftheprintedition.Weexplainthetechnologiesunderlyingmodernextent-based,journaling,andversioningfilesystems.
Amoredetaileddescriptionofeachchapterisgivenintheprefacetotheprintedition.
PrefacetothePrintEdition
WhyWeWroteThisBook
Manyofourstudentstellusthatoperatingsystemswasthebestcoursetheytookasanundergraduateandalsothemostimportantfortheircareers.Wearenotalonemanyofourcolleaguesreportreceivingsimilarfeedbackfromtheirstudents.
Partoftheexcitementisthatthecoreideasinamodernoperatingsystemprotection,concurrency,virtualization,resourceallocation,andreliablestoragehavebecome
-
widelyappliedthroughoutcomputerscience,notjustoperatingsystemkernels.WhetheryougetajobatFacebook,Google,Microsoft,oranyotherleading-edgetechnologycompany,itisimpossibletobuildresilient,secure,andflexiblecomputersystemswithouttheabilitytoapplyoperatingsystemsconceptsinavarietyofsettings.Inamodernworld,nearlyeverythingauserdoesisdistributed,nearlyeverycomputerismulti-core,securitythreatsabound,andmanyapplicationssuchaswebbrowsershavebecomemini-operatingsystemsintheirownright.
Itshouldbenosurprisethatformanycomputersciencestudents,anundergraduateoperatingsystemsclasshasbecomeadefactorequirement:atickettoaninternshipandeventuallytoafull-timeposition.
Unfortunately,manyoperatingsystemstextbooksarestillstuckinthepast,failingtokeeppacewithrapidtechnologicalchange.Severalwidely-usedbookswereinitiallywritteninthemid-1980s,andtheyoftenactasiftechnologystoppedatthatpoint.Evenwhennewtopicsareadded,theyaretreatedasanafterthought,withoutpruningmaterialthathasbecomelessimportant.Theresultaretextbooksthatareverylong,veryexpensive,andyetfailtoprovidestudentsmorethanasuperficialunderstandingofthematerial.
Ourviewisthatoperatingsystemshavechangeddramaticallyoverthepasttwentyyears,andthatjustifiesafreshlookatbothhowthematerialistaughtandwhatistaught.Thepaceofinnovationinoperatingsystemshas,ifanything,increasedoverthepastfewyears,withtheintroductionoftheiOSandAndroidoperatingsystemsforsmartphones,theshifttomulticorecomputers,andtheadventofcloudcomputing.
Topreparestudentsforthisnewworld,webelievestudentsneedthreethingstosucceedatunderstandingoperatingsystemsatadeeplevel:
Conceptsandcode.Webelieveitisimportanttoteachstudentsbothprinciplesandpractice,conceptsandimplementation,ratherthaneitheralone.Thistextbooktakesconceptsallthewaydowntothelevelofworkingcode,e.g.,howacontextswitchworksinassemblycode.Inourexperience,thisistheonlywaystudentswillreallyunderstandandmasterthematerial.Allofthecodeinthisbookisavailablefromtheauthorswebsite,ospp.washington.edu.
Extensiveworkedexamples.Inourview,studentsneedtobeabletoapplyconceptsinpractice.Tothatend,wehaveintegratedalargenumberofexampleexercises,alongwithsolutions,throughoutthetext.Weusestheseexercisesextensivelyinourownlectures,andwehavefoundthemessentialtochallengingstudentstogobeyondasuperficialunderstanding.
Industrypractice.Toshowstudentshowtoapplyoperatingsystemsconceptsinavarietyofsettings,weusedetailed,concreteexamplesfromFacebook,Google,Microsoft,Apple,andotherleading-edgetechnologycompaniesthroughoutthetextbook.Becauseoperatingsystemsconceptsareimportantinawiderangeofcomputersystems,wetaketheseexamplesnotonlyfromtraditionaloperatingsystemslikeLinux,Windows,andOSXbutalsofromothersystemsthatneedtosolveproblemsofprotection,concurrency,virtualization,resourceallocation,andreliablestoragelikedatabases,webbrowsers,webservers,mobileapplications,andsearchengines.
-
Takingafreshperspectiveonwhatstudentsneedtoknowtoapplyoperatingsystemsconceptsinpracticehasledustoinnovateineverymajortopiccoveredinanundergraduate-levelcourse:
KernelsandProcesses.Thesafeexecutionofuntrustedcodehasbecomecentraltomanytypesofcomputersystems,fromwebbrowserstovirtualmachinestooperatingsystems.YetexistingtextbookstreatprotectionasasideeffectofUNIXprocesses,asiftheyaresynonyms.Instead,westartfromfirstprinciples:whataretheminimumrequirementsforprocessisolation,howcansystemsimplementprocessisolationefficiently,andwhatdostudentsneedtoknowtoimplementfunctionscorrectlywhenthecallerispotentiallymalicious?
Concurrency.Withtheadventofmulti-corearchitectures,moststudentstodaywillspendmuchoftheircareerswritingconcurrentcode.Existingtextbooksprovideablizzardofconcurrencyalternatives,mostofwhichwereabandoneddecadesagoasimpractical.Instead,wefocusonprovidingstudentsasinglemethodologybasedonMesamonitorsthatwillenablestudentstowritecorrectconcurrentprogramsamethodologythatisbyfarthedominantapproachusedinindustry.
MemoryManagement.Evenasdemand-paginghasbecomelessimportant,virtualizationhasbecomeevenmoreimportanttomoderncomputersystems.Weprovideadeeptreatmentofaddresstranslationhardware,sparseaddressspaces,TLBs,andon-chipcaches.Wethenusethoseconceptsasaspringboardfordescribingvirtualmachinesandrelatedconceptssuchascheckpointingandcopy-on-write.
PersistentStorage.Reliablestorageinthepresenceoffailuresiscentraltothedesignofmostcomputersystems.Existingtextbookssurveythehistoryoffilesystems,spendingmostoftheirtimeadhocapproachestofailurerecoveryandde-fragmentation.Yetnomodernfilesystemsstillusethoseadhocapproaches.Instead,ourfocusisonhowfilesystemsuseextents,journaling,copy-on-write,andRAIDtoachievebothhighperformanceandhighreliability.
IntendedAudience
OperatingSystems:PrinciplesandPracticeisatextbookforafirstcourseinundergraduateoperatingsystems.Webelieveoperatingsystemsshouldbetakenasearlyaspossibleinanundergraduatescourseofstudy;manystudentsusethecourseasaspringboardtoaninternshipandacareer.Tothatend,wehavedesignedthetextbooktoassumeminimalpre-requisites:specifically,studentsshouldhavetakenadatastructurescourseandoneoncomputerorganization.Thecodeexamplesarewritteninacombinationofx86assembly,C,andC++.Inparticular,wehavedesignedthebooktointerfacewellwiththeBryantandOHallorantextbook.Wereviewandcoverinmuchmoredepththematerialfromthesecondhalfofthatbook.
Weshouldnotewhatthistextbookisnot:itisnotintendedtoteachtheAPIorinternalsofanyspecificoperatingsystem,suchasLinux,Android,Windows8,OSX,oriOS.Weusemanyconcreteexamplesfromthesesystems,butourfocusisonthesharedproblemsthese
-
systemsfaceandthetechnologiesthesesystemsusetosolvethoseproblems.
AGuidetoInstructors
Oneofourgoalsisenableinstructorstochooseanappropriatelevelofdepthforeachcoursetopic.Eachchapterbeginsataconceptuallevel,withimplementationdetailsandthemoreadvancedmaterialtowardstheend.Themoreadvancedmaterialcanbeomittedwithoutcompromisingtheabilityofstudentstofollowlatermaterial.Nosingle-quarterorsingle-semestercourseislikelytobeabletocovereverytopicwehaveincluded,butwethinkitisagoodthingforstudentstocomeawayfromanoperatingsystemscoursewithanappreciationthatthereisalwaysmoretolearn.
Foreachtopic,weattempttoconveyitatthreelevels:
Howtoreasonaboutsystems.Wedescribecoresystemsconcepts,suchasprotection,concurrency,resourcescheduling,virtualization,andstorage,andweprovidepracticeapplyingtheseconceptsinvarioussituations.Inourview,thisprovidesthebiggestlong-termpayofftostudents,astheyarelikelytoneedtoapplytheseconceptsintheirworkthroughouttheircareer,almostregardlessofwhatprojecttheyendupworkingon.
Powertools.Weintroducestudentstoanumberofabstractionsthattheycanapplyintheirworkinindustryimmediatelyaftergraduation,andthatweexpectwillcontinuetobeusefulfordecadessuchassandboxing,protectedprocedurecalls,threads,locks,conditionvariables,caching,checkpointing,andtransactions.
Detailsofspecificoperatingsystems.Weincludenumerousexamplesofhowdifferentoperatingsystemsworkinpractice.However,thismaterialchangesrapidly,andthereisanorderofmagnitudemorematerialthancanbecoveredinasinglesemester-lengthcourse.Thepurposeoftheseexamplesistoillustratehowtousetheoperatingsystemsprinciplesandpowertoolstosolveconcreteproblems.WedonotattempttoprovideacomprehensivedescriptionofLinux,OSX,oranyotherparticularoperatingsystem.
Thebookisdividedintofiveparts:anintroduction(Chapter1),kernelsandprocesses(Chapters2-3),concurrency,synchronization,andscheduling(Chapters4-7),memorymanagement(Chapters8-10),andpersistentstorage(Chapters11-14).
Introduction.ThegoalofChapter1istointroducetherecurringthemesfoundinthelaterchapters.Wedefinesomecommonterms,andweprovideabitofthehistoryofthedevelopmentofoperatingsystems.
TheKernelAbstraction.Chapter2coverskernel-basedprocessprotectiontheconceptandimplementationofexecutingauserprogramwithrestrictedprivileges.Giventheincreasingimportanceofcomputersecurityissues,webelieveprotectedexecutionandsafetransferacrossprivilegelevelsareworthtreatingindepth.Wehavebrokenthedescriptionintosections,toallowinstructorstochooseeitheraquickintroductiontotheconcepts(upthroughSection2.3),orafulltreatmentofthekernelimplementationdetailsdowntothelevelofinterrupthandlers.Someinstructorsstart
-
withconcurrency,andcoverkernelsandkernelprotectionafterwards.Whileourtextbookcanbeusedthatway,wehavefoundthatstudentsbenefitfromabasicunderstandingoftheroleofoperatingsystemsinexecutinguserprograms,beforeintroducingconcurrency.
TheProgrammingInterface.Chapter3isintendedasanimpedancematchforstudentsofdifferingbackgrounds.Dependingonstudentbackground,itcanbeskippedorcoveredindepth.Thechaptercoverstheoperatingsystemfromaprogrammersperspective:processcreationandmanagement,device-independentinput/output,interprocesscommunication,andnetworksockets.Ourgoalisthatstudentsshouldunderstandatadetailedlevelwhathappenswhenauserclicksalinkinawebbrowser,astherequestistransferredthroughoperatingsystemkernelsanduserspaceprocessesattheclient,server,andbackagain.Thischapteralsocoverstheorganizationoftheoperatingsystemitself:howdevicedriversandthehardwareabstractionlayerworkinamodernoperatingsystem;thedifferencebetweenamonolithicandamicrokerneloperatingsystem;andhowpolicyandmechanismareseparatedinmodernoperatingsystems.
ConcurrencyandThreads.Chapter4motivatesandexplainstheconceptofthreads.Becauseoftheincreasingimportanceofconcurrentprogramming,anditsintegrationwithmodernprogramminglanguageslikeJava,manystudentshavebeenintroducedtomulti-threadedprogramminginanearlierclass.Thisisabitdangerous,asstudentsatthisstagearepronetowritingprogramswithraceconditions,problemsthatmayormaynotbediscoveredwithtesting.Thus,thegoalofthischapteristoprovideasolidconceptualframeworkforunderstandingthesemanticsofconcurrency,aswellashowconcurrentthreadsareimplementedinboththeoperatingsystemkernelandinuser-levellibraries.Instructorsneedingtogomorequicklycanomittheseimplementationdetails.
Synchronization.Chapter5discussesthesynchronizationofmulti-threadedprograms,acentralpartofalloperatingsystemsandincreasinglyimportantinmanyothercontexts.Ourapproachistodescribeoneeffectivemethodforstructuringconcurrentprograms(basedonMesamonitors),ratherthantoattempttocoverseveraldifferentapproaches.Inourview,itismoreimportantforstudentstomasteronemethodology.Monitorsareaparticularlyrobustandsimpleone,capableofimplementingmostconcurrentprogramsefficiently.Theimplementationofsynchronizationprimitivesshouldbeincludedifthereistime,sostudentsseethatthereisnomagic.
Multi-ObjectSynchronization.Chapter6discussesadvancedtopicsinconcurrencyspecifically,thetwinchallengesofmultiprocessorlockcontentionanddeadlock.Thismaterialisincreasinglyimportantforstudentsworkingonmulticoresystems,butsomecoursesmaynothavetimetocoveritindetail.
Scheduling.Thischaptercoverstheconceptsofresourceallocationinthespecificcontextofprocessorscheduling.Withtheadventofdatacentercomputingandmulticorearchitectures,theprinciplesandpracticeofresourceallocationhaverenewedimportance.Afteraquicktourthroughthetradeoffsbetweenresponsetimeandthroughputforuniprocessorscheduling,thechaptercoversasetofmore
-
advancedtopicsinaffinityandmultiprocessorscheduling,power-awareanddeadlinescheduling,aswellasbasicqueueingtheoryandoverloadmanagement.Weconcludethesetopicsbywalkingstudentsthroughacasestudyofserver-sideloadmanagement.
AddressTranslation.Chapter8explainsmechanismsforhardwareandsoftwareaddresstranslation.Thefirstpartofthechaptercovershowhardwareandoperatingsystemscooperatetoprovideflexible,sparseaddressspacesthroughmulti-levelsegmentationandpaging.Wethendescribehowtomakememorymanagementefficientwithtranslationlookasidebuffers(TLBs)andvirtuallyaddressedcaches.WeconsiderhowtokeepTLBsconsistentwhentheoperatingsystemmakeschangestoitspagetables.Weconcludewithadiscussionofmodernsoftware-basedprotectionmechanismssuchasthosefoundintheMicrosoftCommonLanguageRuntimeandGooglesNativeClient.
CachingandVirtualMemory.Cachesarecentraltomanydifferenttypesofcomputersystems.Moststudentswillhaveseentheconceptofacacheinanearlierclassonmachinestructures.Thus,ourgoalistocoverthetheoryandimplementationofcaches:whentheyworkandwhentheydonot,aswellashowtheyareimplementedinhardwareandsoftware.Wethenshowhowtheseideasareappliedinthecontextofmemory-mappedfilesanddemand-pagedvirtualmemory.
AdvancedMemoryManagement.Addresstranslationisapowerfultoolinsystemdesign,andweshowhowitcanbeusedforzerocopyI/O,virtualmachines,processcheckpointing,andrecoverablevirtualmemory.Asthisismoreadvancedmaterial,itcanbeskippedbythoseclassespressedfortime.
FileSystems:IntroductionandOverview.Chapter11framesthefilesystemportionofthebook,startingtopdownwiththechallengesofprovidingausefulfileabstractiontousers.WethendiscusstheUNIXfilesysteminterface,themajorinternalelementsinsideafilesystem,andhowdiskdevicedriversarestructured.
StorageDevices.Chapter12surveysblockstoragehardware,specificallymagneticdisksandflashmemory.Thelasttwodecadeshaveseenrapidchangeinstoragetechnologyaffectingbothapplicationprogrammersandoperatingsystemsdesigners;thischapterprovidesasnapshotforstudents,asabuildingblockforthenexttwochapters.Ifstudentshavepreviouslyseenthismaterial,thischaptercanbeskipped.
FilesandDirectories.Chapter13discussesfilesystemlayoutondisk.Ratherthansurveyallpossiblefilelayoutssomethingthatchangesrapidlyovertimeweusefilesystemsasaconcreteexampleofmappingcomplexdatastructuresontoblockstoragedevices.
ReliableStorage.Chapter14explainstheconceptandimplementationofreliablestorage,usingfilesystemsasaconcreteexample.Startingwiththeadhoctechniquesusedinearlyfilesystems,thechapterexplainscheckpointingandwriteaheadloggingasalternateimplementationstrategiesforbuildingreliablestorage,anditdiscusseshowredundancysuchaschecksumsandreplicationareusedtoimprovereliabilityandavailability.
-
Wewelcomeandencouragesuggestionsforhowtoimprovethepresentationofthematerial;pleasesendanycommentstothepublisherswebsite,[email protected].
Acknowledgements
Wehavebeenincrediblyfortunatetohavethehelpofalargenumberofpeopleintheconception,writing,editing,andproductionofthisbook.
WestartedonthejourneyofwritingthisbookoverdinnerattheUSENIXNSDIconferencein2010.Atthetime,wethoughtperhapsitwouldtakeusthesummertocompletethefirstversionandperhapsayearbeforewecoulddeclareourselvesdone.Wewereverywrong!Itisnoexaggerationtosaythatitwouldhavetakenusalotlongerwithoutthehelpwehavereceivedfromthepeoplewementionbelow.
Perhapsmostimportanthavebeenourearlyadopters,whohavegivenusenormouslyusefulfeedbackaswehaveputtogetherthisedition:
Carnegie-Mellon DavidEckhardtandGarthGibson
Clarkson JeannaMatthews
Cornell GunSirer
ETHZurich MothyRoscoe
NewYorkUniversity LaskshmiSubramanian
PrincetonUniversity KaiLi
SaarlandUniversity PeterDruschel
StanfordUniversity JohnOusterhout
UniversityofCaliforniaRiverside HarshaMadhyastha
UniversityofCaliforniaSantaBarbara BenZhao
UniversityofMaryland NeilSpring
UniversityofMichigan PeteChen
UniversityofSouthernCalifornia RameshGovindan
UniversityofTexas-Austin LorenzoAlvisi
-
UniverstiyofToronto DingYuan
UniversityofWashington GaryKimuraandEdLazowska
Indevelopingourapproachtoteachingoperatingsystems,bothbeforewestartedwritingandafterwardsaswetriedtoputourthoughtstopaper,wemadeextensiveuseoflecturenotesandslidesdevelopedbyotherfaculty.OfparticularhelpwerethematerialscreatedbyPeteChen,PeterDruschel,SteveGribble,EddieKohler,JohnOusterhout,MothyRoscoe,andGeoffVoelker.Wethankthemall.
Ourillustratorforthesecondedition,CameronNeat,hasbeenajoytoworkwith.
WearealsogratefultoLorenzoAlvisi,AdamAnderson,PeteChen,SteveGribble,SamHopkins,EdLazowska,HarshaMadhyastha,JohnOusterhout,MarkRich,MothyRoscoe,WillScott,GunSirer,IonStoica,LakshmiSubramanian,andJohnZahorjanfortheirhelpfulcommentsandsuggestionsastohowtoimprovethebook.
WethankJoshBerlin,MarlaDahlin,SandyKaplan,JohnOusterhout,WhitneySchmidt,andMikeWalfishforhelpingusidentifyandcorrectgrammaticalortechnicalbugsinthetext.
WethankJeffDean,GarthGibson,MarkOskin,SimonPeter,DaveProbert,AminVahdat,andMarkZbikowskifortheirhelpinexplainingtheinternalworkingsofsomeofthecommercialsystemsmentionedinthisbook.
WewouldliketothankDaveWetherall,DanWeld,MikeWalfish,DavePatterson,OlavKvern,DanHalperin,ArmandoFox,RobinBriggs,KatyaAnderson,SandraAnderson,LorenzoAlvisi,andWilliamAdamsfortheirhelpandadviceontextbookeconomicsandproduction.
TheHelenRiaboffWhiteleyCenteraswellasDonandJeanneDahlinwerekindenoughtolendusaplacetoescapewhenweneededtogetchapterswritten.
Finally,wethankourfamilies,ourcolleagues,andourstudentsforsupportingusinthislarger-than-expectedeffort.
-
IKernelsandProcesses
-
1.Introduction
AllIreallyneedtoknowIlearnedinkindergarten.RobertFulgham
Howdoweconstructreliable,portable,efficient,andsecurecomputersystems?Anessentialcomponentisthecomputersoperatingsystemthesoftwarethatmanagesacomputersresources.
First,thebadnews:operatingsystemsconceptsareamongthemostcomplexincomputerscience.Amodern,general-purposeoperatingsystemcanexceed50millionlinesofcode,orinotherwords,morethanathousandtimeslongerthanthistextbook.Newoperatingsystemsarebeingwrittenallthetime:ifyouuseane-bookreader,tablet,orsmartphone,anoperatingsystemismanagingyourdevice.Giventhisinherentcomplexity,welimitourfocustotheessentialconceptsthateverycomputerscientistshouldknow.
Nowthegoodnews:operatingsystemsconceptsarealsoamongthemostaccessibleincomputerscience.Manytopicsinthisbookwillseemfamiliartoyouifyouhaveevertriedtodotwothingsatonce,orpickedthewronglineatagrocerystore,ortriedtokeeparoommateorsiblingfrommessingwithyourthings,orsucceededatpullingoffanAprilFoolsjoke.Eachoftheseactivitieshasananalogueinoperatingsystems.Itisthisfamiliaritythatgivesushopethatwecanexplainhowoperatingsystemsworkinasingletextbook.Allweassumeofthereaderisabasicunderstandingoftheoperationofacomputerandtheabilitytoreadpseudo-code.
Webelievethatunderstandinghowoperatingsystemsworkisessentialforanystudentinterestedinbuildingmoderncomputersystems.Ofcourse,everyonewhousesacomputerorasmartphoneorevenamoderntoasterusesanoperatingsystem,sounderstandingthefunctionofanoperatingsystemisusefultomostcomputerscientists.Thisbookaimstogomuchdeeperthanthat,toexplainoperatingsysteminternalsthatwerelyoneverydaywithoutrealizingit.
Softwareengineersusemanyofthesametechnologiesanddesignpatternsasthoseusedinoperatingsystemstobuildothercomplexsystems.Whetheryourgoalistoworkontheinternalsofanoperatingsystemkernelortobuildthenextgenerationofsoftwareforcloudcomputing,securewebbrowsers,gameconsoles,graphicaluserinterfaces,mediaplayers,databases,ormulticoresoftwaretheconceptsandabstractionsneededforreliable,portable,efficientandsecuresoftwarearemuchthesame.Inourexperience,thebestwaytolearntheseconceptsistostudyhowtheyareusedinoperatingsystems,butwehopeyouwillapplythemtoamuchbroaderrangeofcomputersystems.
Togetstarted,considerthewebserverinFigure1.1.Itsbehaviorisamazinglysimple:itreceivesapacketcontainingthenameofthewebpagefromthenetwork,asanHTTPGETrequest.Thewebserverdecodesthepacket,readsthefilefromdisk,andsendsthecontentsofthefilebackoverthenetworktotheusersmachine.
-
Figure1.1:Theoperationofawebserver.TheclientmachinesendsanHTTPGETrequesttothewebserver.Theserverdecodesthepacket,readsthefile,andsendsthecontentsbacktotheclient.
Partofanoperatingsystemsjobistomakeiteasytowriteapplicationslikewebservers.Butdiggingabitdeeper,thissimplestoryquicklyraisesasmanyquestionsasitanswers:
Manywebrequestsinvolvebothdataandcomputation.Forexample,theGooglehomepagepresentsasimpletextbox,buteachsearchqueryenteredinthatboxconsultsdataspreadovermanymachines.Tokeeptheirsoftwaremanageable,webserversofteninvokehelperapplications,e.g.,tomanagetheactualsearchfunction.Themainwebservermustbeabletocommunicatewiththehelperapplicationsforthistowork.Howdoestheoperatingsystemenablemultipleapplicationstocommunicatewitheachother?
Whatiftwousers(oramillion)requestawebpagefromtheserveratthesametime?Asimpleapproachmightbetohandleeachrequestinturn.Ifanyindividualrequesttakesalongtime,however,everyotherrequestmustwaitforittocomplete.Afaster,butmorecomplex,solutionistomultitask:tojugglethehandlingofmultiplerequestsatonce.Multitaskingisespeciallyimportantonmodernmulticorecomputers,whereeachprocessorcanhandleadifferentrequestatthesametime.Howdoestheoperatingsystemenableapplicationstodomultiplethingsatonce?
Forbetterperformance,thewebservermightwanttokeepacopy,sometimescalledacache,ofrecentlyrequestedpages.Inthisway,ifmultipleusersrequestthesamepage,theservercanrespondtosubsequentrequestsmorequicklyfromthecache,ratherthanstartingeachrequestfromscratch.Thisrequiresthewebservertocoordinate,orsynchronize,accesstothecachesdatastructuresbypossiblythousandsofwebrequestsatthesametime.Howdoestheoperatingsystemsynchronizeapplicationaccesstoshareddata?
Tocustomizeandanimatetheuserexperience,webserverstypicallysendclientsscriptingcodealongwiththecontentsofthewebpage.Butthismeansthatclickingonalinkcancausesomeoneelsescodetorunonyourcomputer.Howdoestheclientoperatingsystemprotectitselffromcompromisebyacomputervirus
-
surreptitiouslyembeddedintothescriptingcode?
Supposethewebsiteadministratorusesaneditortoupdatethewebpage.Thewebservermustbeabletoreadthisfile.Howdoestheoperatingsystemstorethebytesondisksothatthewebservercanfindandreadthem?
Takingthisastepfurther,theadministratormaywanttomakeaconsistentsetofchangestothewebsitesothatembeddedlinksarenotleftdangling,eventemporarily.Howcantheoperatingsystemletusersmakeasetofchangestoawebsite,sothatrequestsseeeithertheoldornewpages,butnotacombinationofthetwo?
Whathappenswhentheclientbrowserandthewebserverrunatdifferentspeeds?Iftheservertriestosendawebpagetotheclientfasterthantheclientcanrenderthepageonthescreen,wherearethecontentsofthefilestoredinthemeantime?Cantheoperatingsystemdecoupletheclientandserversothateachcanrunatitsownspeedwithoutslowingtheotherdown?
Asdemandonthewebservergrows,theadministratormayneedtomovetomorepowerfulhardware,withmorememory,moreprocessors,fasternetworkdevices,andfasterdisks.Totakeadvantageofnewhardware,mustthewebserverbere-writteneachtime,orcanitbewritteninahardware-independentfashion?Whatabouttheoperatingsystemmustitbere-writtenforeverynewpieceofhardware?
Wecouldgoon,butyougettheidea.Thisbookwillhelpyouunderstandtheanswerstotheseandmanymorequestions.
Chapterroadmap:
Therestofthischapterdiscussesthreetopicsindetail:
OperatingSystemDefinition.Whatisanoperatingsystem,andwhatdoesitdo?(Section1.1)
OperatingSystemEvaluation.Whatdesigngoalsshouldwelookforinanoperatingsystem?(Section1.2)
OperatingSystems:Past,Present,andFuture.Howhaveoperatingsystemsevolved,andwhatnewfunctionalityarewelikelytoseeinfutureoperatingsystems?(Section1.3)
1.1WhatIsAnOperatingSystem?
Anoperatingsystem(OS)isthelayerofsoftwarethatmanagesacomputersresourcesforitsusersandtheirapplications.Operatingsystemsruninawiderangeofcomputersystems.Theymaybeinvisibletotheenduser,controllingembeddeddevicessuchastoasters,gamingsystems,andthemanycomputersinsidemodernautomobilesandairplanes.Theyarealsoessentialtomoregeneral-purposesystemssuchassmartphones,desktopcomputers,andservers.
Ourdiscussionwillfocusongeneral-purposeoperatingsystemsbecausethetechnologies
-
theyneedareasupersetofthoseneededforembeddedsystems.Increasingly,operatingsystemstechnologiesdevelopedforgeneral-purposecomputingaremigratingintotheembeddedsphere.Forexample,earlymobilephoneshadsimpleoperatingsystemstomanagetheirhardwareandtorunahandfulofprimitiveapplications.Today,smartphonesphonescapableofrunningindependentthird-partyapplicationsarethefastestgrowingsegmentofthemobilephonebusiness.Thesedevicesrequiremuchmorecompleteoperatingsystems,withsophisticatedresourcemanagement,multi-tasking,securityandfailureisolation.
Likewise,automobilesareincreasinglysoftwarecontrolled,raisingahostofoperatingsystemissues.Cananyonewritesoftwareforyourcar?Whatifthesoftwarefailswhileyouaredrivingdownthehighway?Canacarsoperatingsystembehijackedbyacomputervirus?Althoughthismightseemfar-fetched,researchersrecentlydemonstratedthattheycouldremotelyturnoffacarsbrakingsystemthroughacomputervirusintroducedintothecarscomputersviaahackedcarradio.Agoalofthisbookistoexplainhowtobuildmorereliableandsecurecomputersystemsinavarietyofcontexts.
Figure1.2:Ageneral-purposeoperatingsystemisalayerofsoftwarethatmanagesacomputersresourcesforitsusersandapplications.
Forgeneral-purposesystems,usersinteractwithapplications,applicationsexecuteinanenvironmentprovidedbytheoperatingsystem,andtheoperatingsystemmediatesaccesstotheunderlyinghardware,asshowninFigure1.2andexpandedinFigure1.3.Howcananoperatingsystemrunmultipleapplications?Forthis,operatingsystemsneedtoplaythreeroles:
-
Figure1.3:Thisshowsthestructureofageneral-purposeoperatingsystem,asanexpansiononthesimpleviewpresentedinFigure1.2.Atthelowestlevel,thehardwareprovidesprocessors,memory,andasetofdevicesforstoringdataandcommunicatingwiththeoutsideworld.Thehardwarealsoprovidesprimitivesthattheoperatingsystemcanuseforfaultisolationandsynchronization.Theoperatingsystemrunsasthelowestlayerofsoftwareonthecomputer.Itcontainsbothadevice-specificlayerformanagingthemyriadhardwaredevicesandasetofdevice-independentservicesprovidedtoapplications.Sincetheoperatingsystemmustisolatemaliciousandbuggyapplicationsfromotherapplicationsortheoperatingsystemitself,muchoftheoperatingsystemrunsinaseparateexecutionenvironmentprotectedfromapplicationcode.Aportionoftheoperatingsystemcanalsorunasasystemlibrarylinkedintoeachapplication.Inturn,applicationsruninanexecutioncontextprovidedbytheoperatingsystemkernel.Theapplicationcontextismuchmorethanasimpleabstractionontopofhardwaredevices:applicationsexecuteinavirtualenvironmentthatismoreconstrained(topreventharm),morepowerful(tomaskhardwarelimitations),andmoreuseful(viacommonservices)thantheunderlyinghardware.
1. Referee.Operatingsystemsmanageresourcessharedbetweendifferentapplicationsrunningonthesamephysicalmachine.Forexample,anoperatingsystemcanstop
-
oneprogramandstartanother.Operatingsystemsisolateapplicationsfromeachother,soabuginoneapplicationdoesnotcorruptotherapplicationsrunningonthesamemachine.Anoperatingsystemmustalsoprotectitselfandotherapplicationsfrommaliciouscomputerviruses.Andsincetheapplicationssharephysicalresources,theoperatingsystemneedstodecidewhichapplicationsgetwhichresourcesandwhen.
2. Illusionist.Operatingsystemsprovideanabstractionofphysicalhardwaretosimplifyapplicationdesign.TowriteaHelloworld!program,youdonotneed(orwant!)tothinkabouthowmuchphysicalmemorythesystemhas,orhowmanyotherprogramsmightbesharingthecomputersresources.Instead,operatingsystemsprovidetheillusionofnearlyinfinitememory,despitehavingalimitedamountofphysicalmemory.Likewise,theyprovidetheillusionthateachprogramhasthecomputersprocessorsentirelytoitself.Obviously,therealityisquitedifferent!Theseillusionsletyouwriteapplicationsindependentlyoftheamountofphysicalmemoryonthesystemorthephysicalnumberofprocessors.Becauseapplicationsarewrittentoahigherlevelofabstraction,theoperatingsystemcaninvisiblychangetheamountofresourcesassignedtoeachapplication.
3. Glue.Operatingsystemsprovideasetofcommonservicesthatfacilitatesharingamongapplications.Asaresult,cutandpasteworksuniformlyacrossthesystem;afilewrittenbyoneapplicationcanbereadbyanother.Manyoperatingsystemsprovidecommonuserinterfaceroutinessoapplicationscanhavethesamelookandfeel.Perhapsmostimportantly,operatingsystemsprovidealayerseparatingapplicationsfromhardwareinputandoutput(I/O)devicessoapplicationscanbewrittenindependentlyofthespecifickeyboard,mouse,anddiskdriveinuseonaparticularcomputer.
Wenextdiscussthesethreerolesingreaterdetail.
1.1.1ResourceSharing:OperatingSystemasReferee
Sharingiscentraltomostusesofcomputers.Rightnow,mylaptopisrunningabrowser,podcastlibrary,texteditor,emailprogram,documentviewer,andnewspaper.Theoperatingsystemmustsomehowkeepalloftheseactivitiesseparate,yetalloweachthefullcapacityofthemachineiftheothersarenotrunning.Ataminimum,whenoneprogramstopsrunning,theoperatingsystemshouldletmerunanother.Betterstill,theoperatingsystemshouldletmultipleapplicationsrunatthesametime,soIcanreademailwhileIdownloadasecuritypatchtothesystemsoftware.
Evenindividualapplicationscandomultipletasksatonce.Forinstance,awebserversresponsivenessimprovesifithandlesmultiplerequestsconcurrentlyratherthanwaitingforeachtocompletebeforestartingthenextone.Thesameholdsforthebrowseritismoreresponsiveifitcanstartrenderingapagewhiletherestofthepageistransferring.Onmultiprocessors,thecomputationinsideaparallelapplicationcanbesplitintoseparateunitsthatcanberunindependentlyforfasterexecution.Theoperatingsystemitselfisanexampleofsoftwarewrittentodomultipletasksatonce.Aswewillillustratethroughoutthebook,theoperatingsystemisacustomerofitsownabstractions.
-
Sharingraisesseveralchallengesforanoperatingsystem:
Resourceallocation.Theoperatingsystemmustkeepallsimultaneousactivitiesseparate,allocatingresourcestoeachasappropriate.Acomputerusuallyhasonlyafewprocessorsandafiniteamountofmemory,networkbandwidth,anddiskspace.Whentherearemultipletaskstodoatthesametime,howshouldtheoperatingsystemdecidehowmanyresourcestogivetoeach?Seeminglytrivialdifferencesinhowresourcesareallocatedcanimpactuser-perceivedperformance.AswewillseeinChapter9,anoperatingsystemthatallocatestoolittlememorytoaprogramslowsdownnotonlythatparticularprogram,butoftenotherapplicationsaswell.
Toillustratethedifferencebetweenexecutiononaphysicalmachineversusontheabstractmachineprovidedbytheoperatingsystem,whatshouldhappenifanapplicationexecutesaninfiniteloop?
Ifprogramsrandirectlyonrawhardware,thiscodefragmentwouldlockupthecomputer,makingitcompletelynon-responsivetouserinput.Iftheoperatingsystemensuresthateachprogramgetsitsownsliceofthecomputersresources,aspecificapplicationmightlockup,butotherprogramscouldproceedunimpeded.Additionally,theusercouldasktheoperatingsystemtoforcetheloopingprogramtoexit.
Isolation.Anerrorinoneapplicationshouldnotdisruptotherapplications,oreventheoperatingsystemitself.Thisiscalledfaultisolation.Anyonewhohastakenanintroductorycomputerscienceclassknowsthevalueofanoperatingsystemthatcanprotectitselfandotherapplicationsfromprogrammerbugs.Debuggingwouldbevastlyharderifanerrorinoneprogramcouldcorruptdatastructuresinotherapplications.Likewise,downloadingandinstallingascreensaverorotherapplicationshouldnotcrashunrelatedprograms,provideawayforamaliciousattackertosurreptitiouslyinstallacomputervirus,orletoneuseraccessorchangeanothersdatawithoutpermission.
Faultisolationrequiresrestrictingthebehaviorofapplicationstolessthanthefullpoweroftheunderlyinghardware.Otherwise,anyapplicationdownloadedofftheweb,oranyscriptembeddedinawebpage,couldcompletelycontrolthemachine.Anyapplicationcouldinstallspywareintotheoperatingsystemtologeverykeystrokeyoutype,orrecordthepasswordtoeverywebsiteyouvisit.Withoutfaultisolationprovidedbytheoperatingsystem,anybuginanyprogrammightirretrievablycorruptthedisk.Error-proneormalignantapplicationscouldcauseallsortsofhavoc.
Communication.Theflipsideofisolationistheneedforcommunicationbetweendifferentapplicationsanddifferentusers.Forexample,awebsitemaybe
-
implementedbyacooperatingsetofapplications:onetoselectadvertisements,anothertocacherecentresults,yetanothertofetchandmergedatafromdisk,andseveralmoretocooperativelyscanthewebfornewcontenttoindex.Forthistowork,thevariousprogramsmustcommunicatewithoneanother.Iftheoperatingsystempreventsbugsandmalicioususersandapplicationsfromaffectingotherusersandtheirapplications,howdoesitalsosupportcommunicationtoshareresults?Insettingupboundaries,anoperatingsystemmustalsoallowthoseboundariestobecrossedincarefullycontrolledwayswhentheneedarises.
Initsroleasreferee,anoperatingsystemissomewhatakintothatofaparticularlypatientkindergartenteacher.Itbalancesneeds,separatesconflicts,andfacilitatessharing.Oneusershouldnotbeallowedtomonopolizesystemresourcesortoaccessorcorruptanotherusersfileswithoutpermission;abuggyapplicationshouldnotbeabletocrashtheoperatingsystemorotherunrelatedapplications;andyet,applicationsmustalsoworktogether.Enforcingandbalancingtheseconcernsisacentralroleoftheoperatingsystem.
1.1.2MaskingLimitations:OperatingSystemasIllusionist
Asecondimportantroleofanoperatingsystemistomasktherestrictionsinherentincomputerhardware.Physicalconstraintslimithardwareresourcesacomputerhasonlyalimitednumberofprocessorsandalimitedamountofphysicalmemory,networkbandwidth,anddisk.Further,sincetheoperatingsystemmustdecidehowtodivideitsfixedresourcesamongthevariousapplicationsrunningateachmoment,aparticularapplicationcanhavedifferingamountsofresourcesfromtimetotime,evenwhenrunningonthesamehardware.Whilesomeapplicationsaredesignedtotakeadvantageofacomputersspecifichardwareconfigurationandresourceassignment,mostprogrammersprefertouseahigherlevelofabstraction.
Virtualizationprovidesanapplicationwiththeillusionofresourcesthatarenotphysicallypresent.Forexample,theoperatingsystemcanprovidetheabstractionthateachapplicationhasadedicatedprocessor,eventhoughataphysicalleveltheremaybeonlyasingleprocessorsharedamongalltheapplicationsrunningonthecomputer.
Withtherighthardwareandoperatingsystemsupport,mostphysicalresourcescanbevirtualized.Forexample,hardwareprovidesonlyasmall,finiteamountofmemory,whiletheoperatingsystemprovidesapplicationstheillusionofanearlyinfiniteamountofvirtualmemory.Wirelessnetworksdroporcorruptpackets;theoperatingsystemmasksthesefailurestoprovidetheillusionofareliableservice.Ataphysicallevel,magneticdiskandflashRAMsupportblockreadsandwrites,wherethesizeoftheblockdependsonthephysicaldevicecharacteristics,addressedbyadevice-specificblocknumber.Mostprogrammersprefertoworkwithbyte-addressablefilesorganizedbynameintohierarchicaldirectories.Eventhetypeofprocessorcanbevirtualizedtoallowthesame,unmodifiedapplicationtorunonasmartphone,tablet,andlaptopcomputer.
-
Figure1.4:Aguestoperatingsystemrunninginsideavirtualmachine.
Pushingthisonestepfurther,someoperatingsystemsvirtualizetheentirecomputer,runningtheoperatingsystemasanapplicationontopofanotheroperatingsystem(seeFigure1.4).Thisiscalledcreatingavirtualmachine.Theoperatingsystemrunninginthevirtualmachine,calledtheguestoperatingsystem,thinksitisrunningonareal,physicalmachine,butthisisanillusionpresentedbythetrueoperatingsystemrunningunderneath.
Onebenefitofavirtualmachineisapplicationportability.Ifaprogramrunsonlyonanoldversionofanoperatingsystem,itcanstillworkonanewsystemrunningavirtualmachine.Thevirtualmachinehoststheapplicationontheoldoperatingsystem,runningatopthenewone.Virtualmachinesalsoaiddebugging.Ifanoperatingsystemcanberunasanapplication,thenitsdeveloperscansetbreakpoints,stopthekernel,andsinglesteptheircodejustastheywouldwhendebugginganapplication.
Throughoutthebook,wediscusstechniquesthattheoperatingsystemusestoaccomplishtheseandotherillusions.Ineachcase,theoperatingsystemprovidesamoreconvenientandflexibleprogrammingabstractionthanthatprovidedbytheunderlyinghardware.
1.1.3ProvidingCommonServices:OperatingSystemasGlue
Operatingsystemsplayathirdkeyrole:providingasetofcommon,standardservicestoapplicationstosimplifyandstandardizetheirdesign.Anexampleisthewebserverdescribedearlierinthischapter.Theoperatingsystemhidesthespecificsofhowthenetworkanddiskdeviceswork,providingasimplerabstractionbasedonreceiving/sendingreliablestreamsofbytesandreading/writingnamedfiles.Thisletsthewebserverfocusonitscoretaskdecodingincomingrequestsandfillingthemratherthanonformattingdataintoindividualnetworkpacketsanddiskblocks.
-
Animportantreasonfortheoperatingsystemtoprovidecommonservices,ratherthanlettingeachapplicationprovideitsown,istofacilitatesharingamongapplications.Thewebservermustbeabletoreadthefilethatthetexteditorwrote.Forapplicationstosharefiles,theymustbestoredinastandardformat,withastandardsystemformanagingfiledirectories.Mostoperatingsystemsalsoprovideastandardwayforapplicationstopassmessagesandtosharememory.
Thechoiceofwhichservicesanoperatingsystemshouldprovideisoftenjudgmentcall.Forexample,computerscancomeconfiguredwithablizzardofdifferentdevices:differentgraphicsco-processorsandpixelformats,differentnetworkinterfaces(WiFi,Ethernet,andBluetooth),differentdiskdrives(SCSI,IDE),differentdeviceinterfaces(USB,Firewire),anddifferentsensors(GPS,accelerometers),nottomentiondifferentversionsofeach.Mostapplicationscanignorethesedifferences,byusingonlyagenericinterfaceprovidedbytheoperatingsystem.Forotherapplications,suchasadatabase,thespecificdiskdrivemaymatterquiteabit.Forapplicationsthatcanoperateatahigherlevelofabstraction,theoperatingsystemservesasaninteroperabilitylayersothatbothapplicationsanddevicescanevolveindependently.
Anotherstandardserviceinmostmodernoperatingsystemsisthegraphicaluserinterfacelibrary.BothMicrosoftsandApplesoperatingsystemsprovideasetofstandarduserinterfacewidgets.Thisfacilitatesacommonlookandfeeltouserssothatfrequentoperationssuchaspulldownmenusandcutandpastecommandsarehandledconsistentlyacrossapplications.
Mostofthecodeinanoperatingsystemimplementsthesecommonservices.However,muchofthecomplexityofoperatingsystemsisduetoresourcesharingandthemaskingofhardwarelimits.Becausecommonservicecodeusestheabstractionsprovidedbytheothertwooperatingsystemroles,thisbookwillfocusprimarilyontheoperatingsystemasarefereeandasanillusionist.
1.1.4OperatingSystemDesignPatterns
Thechallengesthatoperatingsystemsaddressarenotuniquetheyapplytomanydifferentcomputerdomains.Manycomplexsoftwaresystemshavemultipleusers,runprogramswrittenbythird-partydevelopers,and/orneedtocoordinatemanysimultaneousactivities.Theseposequestionsofresourceallocation,faultisolation,communication,abstractionsofphysicalhardware,andhowtoprovideausefulsetofcommonservicesforsoftwaredevelopers.Notonlyarethechallengesthesame,butoftenthesolutionsare,aswell:thesesystemsusemanyofthedesignpatternsandtechniquesdescribedinthisbook.
Wenextdescribesomeofthesystemswithdesignchallengessimilartothosefoundinoperatingsystems:
-
Figure1.5:Cloudcomputingsoftwareprovidesaconvenientabstractionofserverresourcestocloudapplications.
Cloudcomputing(Figure1.5)isamodelofcomputingwhereapplicationsrunonsharedcomputingandstorageinfrastructureinlarge-scaledatacentersinsteadofontheusersowncomputers.Cloudcomputingmustaddressmanyofthesameissuesasinoperatingsystemsintermsofsharing,abstraction,andcommonservices.
Referee.Howareresourcesallocatedbetweencompetingapplicationsrunninginthecloud?Howarebuggyormaliciousapplicationspreventedfromdisruptingotherapplications?
Illusionist.Thecomputingresourcesinthecloudarecontinuallyevolving;whatabstractionsareprovidedtoisolateapplicationdevelopersfromchangesintheunderlyinghardware?
Glue.Cloudservicesoftendistributetheirworkacrossdifferentmachines.Whatabstractionsshouldcloudsoftwareprovidetohelpservicescoordinateandsharedatabetweentheirvariousactivities?
-
Figure1.6:Awebbrowserisolatesscriptsandplug-insfromaccessingprivilegedresourcesonthehostoperatingsystem.
Webbrowsers(Figure1.6),suchasChrome,InternetExplorer,Firefox,andSafari,playarolesimilartoanoperatingsystem.Browsersloadanddisplaywebpages,but,aswementionedearlier,manypagesembedscriptingprogramsthatthebrowsermustexecute.Thesescriptscanbebuggyormalicious;hackershaveusedthemtotakeovervastnumbersofhomecomputers.Likeanoperatingsystem,thebrowsermustisolatetheuser,otherwebsites,andeventhebrowseritselffromerrorsormaliciousactivitybythesescripts.Similarly,mostbrowsershaveaplug-inarchitectureforsupportingextensions,andtheseextensionsmustalsobeisolatedtopreventthemfromcausingharm.
Referee.Howcanabrowserensureresponsivenesswhenauserhasmultipletabsopenwitheachtabrunningascriptfromadifferentwebsite?Howcanwelimitwebscriptsandplug-instopreventbugsfromcrashingthebrowserandmaliciousscriptsfromaccessingsensitiveuserdata?
Illusionist.Manywebservicesaregeographicallydistributedtoimprovetheuserexperience.Notonlydoesthisputserversclosertousers,butifoneservercrashesoritsnetworkconnectionhasproblems,abrowsercanconnecttoadifferentsite.Theuserinmostcasesdoesnotnoticethedifference,evenwhenupdatingashoppingcartorwebform.Howdoesthebrowsermakeserverchangestransparenttotheuser?
Glue.Howdoesthebrowserachieveaportableexecutionenvironmentforscriptsthatworksconsistentlyacrossoperatingsystemsandhardware
-
platforms?
Mediaplayers,suchasFlashandSilverlight,areoftenpackagedasbrowserplug-ins,buttheythemselvesprovideanexecutionenvironmentforscriptingprograms.Thus,thesesystemsfacemanyofthesameissuesasbothbrowsersandoperatingsystemsonwhichtheyrun:isolationofbuggyormaliciouscode,concurrentbackgroundandforegroundtasks,andplug-inarchitectures.
Referee.Mediaplayersareofteninthenewsforbeingvulnerabletosomenew,maliciousattack.Howshouldmediaplayerssandboxmaliciousorbuggyscriptstopreventthemfromcorruptingthehostmachine?
Illusionist.Mediaapplicationsareoftenbothcomputationallyintensiveandhighlyinteractive.Howdotheycoordinateforegroundandbackgroundactivitiestomaintainresponsiveness?
Glue.High-performancegraphicshardwarerapidlyevolvesinresponsetothedemandsofthevideogamemarket.HowdomediaplayersprovideasetofstandardAPIsforscriptstoworkacrossadiversityofgraphicsaccelerators?
MultiplayergamesoftenhaveextensibilityAPIstoallowthirdpartysoftwarevendorstoextendthegameinsignificantways.Oftentheseextensionsareminiaturegamesintheirownright,yetgameextensionsmustalsobepreventedfrombreakingtheoverallrulesofthegame.
Referee.Manygamestrytooffloadworktoclientmachinestoreduceserverloadandimproveresponsiveness,butthisopensupgamestothethreatofusersinstallingspecializedextensionstogainanunfairadvantage.Howdogamedesignerssetlimitsforextensionsandgameplayerstoensurealevelplayingfield?
Illusionist.Ifobjectsinthegamearespreadacrossclientandservermachines,isthatdistinctionvisibletoextensioncodeoristheinterfaceatahigherlevel?
Glue.Mostsuccessfulgameshavealargenumberofextensions;howshouldagamedesignersetuptheirAPIstomakeiteasiertofosteracommunityofdevelopers?
-
Figure1.7:Databasesperformmanyofthetasksofanoperatingsystem:theyallocateresourcesamonguserqueriestoensureresponsiveness,theymaskdifferencesintheunderlyingoperatingsystemandhardware,andtheyprovideaconvenientprogrammingabstractiontodevelopers.
Multi-userdatabasesystems(Figure1.7),suchasOracleandMicrosoftsSQLServer,allowlargeorganizationstostore,query,andupdatelargedatasets,suchasdetailedrecordsofeverypurchaseevermadeatAmazonorWalmart.Largescaledataanalysisgreatlyoptimizesbusinessoperations,but,asaconsequence,databasesfacemanyofthesamechallengesasoperatingsystems.Theyaresimultaneouslyaccessedbymanydifferentusersinmanydifferentlocations.Theythereforemustallocateresourcesamongdifferentuserrequests,isolateconcurrentupdatestoshareddata,andensurethatdataisstoredconsistentlyondisk.Infact,severalofthetechniqueswediscussinChapter14wereoriginallydevelopedfordatabasesystems.
Referee.Howshouldresourcesbeallocatedamongthevarioususersofadatabase?Howdoesthedatabaseenforcedataprivacysothatonlyauthorizedusersaccessrelevantdata?
Illusionist.Howdoesthedatabasemaskmachinefailuressothatdataisalwaysstoredconsistentlyregardlessofwhenthefailureoccurs?
Glue.Whatcommonservicesmakeiteasiertodevelopdatabaseapplications?
Parallelapplicationsareprogramsdesignedtotakeadvantageofmultipleprocessorsonasinglecomputer.Eachapplicationdividesitsworkontoafixednumberofprocessorsandmustensurethataccessestoshareddatastructuresarecoordinatedtopreserveconsistency.Whilesomeparallelprogramsdirectlyusetheservicesprovidedbytheunderlyingoperatingsystem,othersneedcarefulcontroloftheassignmentofworktoprocessorstoachievegoodperformance.Thesesystems
-
interposearuntimesystemontopoftheoperatingsystemtomanageuser-levelparallelism,essentiallybuildingamini-operatingsystemontopoftheunderlyingone.
Referee.Whentherearemoretaskstoperformthanprocessors,howdoestheruntimesystemdecidewhichtaskstoperformfirst?
Illusionist.Howdoestheruntimesystemhidephysicaldetailsofthehardwarefromtheprogrammer,suchasthenumberofprocessorsortheinterprocessorcommunicationlatency?
Glue.Highlyconcurrentdatastructurescanmakeiteasiertowriteefficientparallelprograms;howdoweprogramtrees,hashtables,andlistssothattheycanbeusedbymultipleprocessorsatthesametime?
TheInternetisusedeverydaybyahugenumberofpeople,butatthephysicallayer,thoseuserssharethesameunderlyingresources.HowshouldtheInternethandleresourcecontention?Becauseofitsdiverseuserbase,theInternetisrifewithmaliciousbehavior,suchasdenial-of-serviceattacksthatfloodtrafficoncertainlinkstopreventlegitimateusersfromcommunicating.VariousattemptsareunderwaytodesignsolutionsthatwilllettheInternetcontinuetofunctiondespitesuchattacks.
Referee.ShouldtheInternettreatallusersidentically(e.g.,networkneutrality)orshouldISPsbeabletofavorsomeusesoverothers?CantheInternetbere-designedtopreventdenial-of-service,spam,phishing,andothermaliciousbehaviors?
Illusionist.TheInternetprovidestheillusionofasingleworldwidenetworkthatcandeliverapacketfromanymachineontheInternettoanyothermachine.However,networkhardwareiscomposedofmanydiscretenetworkelementswith:(i)theabilitytotransmitlimitedsizepacketsoveralimiteddistance,and(ii)somechancethatpacketswillbegarbledintheprocess.TheInternettransformsthenetworkintosomethingmoreusefulforapplicationslikethewebafacilitytoreliablytransmitdataofarbitrarylength,anywhereintheworld.
Glue.TheInternetprotocolsuitewasexplicitlydesignedtoactasaninteroperabilitylayerthatletsnetworkapplicationsevolveindependentlyofchangesinnetworkhardware,andviceversa.DoesthesuccessoftheInternetholdanylessonsforoperatingsystemdesign?
Manyofthesesystemsusethesametechniquesanddesignpatternsasoperatingsystems.Studyingoperatingsystemsisagreatwaytounderstandhowtheseotherssystemswork.Inafewcases,differentmechanismsareusedtoachievethesamegoals,but,evenhere,theboundariesarefuzzy.Forexample,browsersoftenusecompile-timecheckstopreventscriptsfromgainingcontroloverthem,whilemostoperatingsystemsusehardware-basedprotectiontolimitapplicationprogramsfromtakingoverthemachine.Morerecently,however,somesmartphoneoperatingsystemshavebeguntousethesamecompile-timetechniquesasbrowserstoprotectagainstmaliciousmobileapplications.Inturn,somebrowsershavebeguntouseoperatingsystemhardware-basedprotectiontoimprovetheisolationtheyprovide.
-
Toavoidspreadingourdiscussiontoothinly,thisbookfocusesonhowoperatingsystemswork.Justasitiseasiertolearnasecondcomputerprogramminglanguageafteryoubecomefluentinthefirst,itisbettertoseehowoperatingsystemsprinciplesapplyinonecontextbeforelearninghowtheycanbeappliedinothersettings.Wehopeandexpect,however,thatyouwillbeabletoapplytheconceptsinthisbookmorewidelythanjustoperatingsystemdesign.
1.2OperatingSystemEvaluation
Havingdefinedwhatanoperatingsystemdoes,howshouldwechooseamongalternativedesigns?Wediscussseveraldesirablecriteriaforoperatingsystems:
ReliabilityandAvailability.Doestheoperatingsystemdowhatyouwant?
Security.Cantheoperatingsystembecorruptedbyanattacker?
Portability.Istheoperatingsystemeasytomovetonewhardwareplatforms?
Performance.Istheuserinterfaceresponsive,ordoestheoperatingsystemimposetoomuchoverhead?
Adoption.Howmanyotherusersarethereforthisoperatingsystem?
Inmanycases,tradeoffsbetweenthesecriteriaareinevitableimprovingasystemalongonedimensionmayhurtitalonganother.Weconcludethissectionwithsomeconcreteexamplesofdesigntradeoffs.
1.2.1ReliabilityandAvailability
Perhapsthemostimportantcharacteristicofanoperatingsystemisitsreliability.Reliabilitymeansthatasystemdoesexactlywhatitisdesignedtodo.Asthelowestlevelofsoftwarerunningonthesystem,operatingsystemerrorscanhavedevastatingandhiddeneffects.Iftheoperatingsystembreaks,youmaynotbeabletogetworkdone,andinsomecases,youmayevenlosepreviouswork,e.g.,ifthefailurecorruptsfilesondisk.Bycontrast,applicationfailurescanbemuchmorebenign,preciselybecauseoperatingsystemsprovidefaultisolationandarapidandcleanrestartafteranerror.
Makinganoperatingsystemreliableischallenging.Operatingsystemsoftenoperateinahostileenvironment,onewherecomputervirusesandothermaliciouscodetrytotakecontrolofthesystembyexploitingdesignorimplementationerrorsintheoperatingsystemsdefenses.
Unfortunately,themostcommonwaystoimprovesoftwarereliability,suchasrunningtestcasesforcommoncodepaths,arelesseffectivewhenappliedtooperatingsystems.Sincemaliciousattackscantargetaspecificvulnerabilitypreciselytocauseexecutiontofollowararecodepath,everythingmustworkcorrectlyfortheoperatingsystemtobereliable.Evenwithoutintentionallymaliciousattacks,extremelyrarecornercasescanoccurregularly:foranoperatingsystemwithamillionusers,aonceinabillioneventwilleventuallyoccurtosomeone.
-
Arelatedconceptisavailability,thepercentageoftimethatthesystemisusable.Abuggyoperatingsystemthatcrashesfrequently,losingtheuserswork,isbothunreliableandunavailable.Abuggyoperatingsystemthatcrashesfrequentlybutneverlosestheusersworkandcannotbesubvertedbyamaliciousattackisreliablebutunavailable.Anoperatingsystemthathasbeensubvertedbutcontinuestoappeartorunnormallywhileloggingtheuserskeystrokesisunreliablebutavailable.
Thus,bothreliabilityandavailabilityaredesirable.Availabilityisaffectedbytwofactors:thefrequencyoffailures,measuredasthemeantimetofailure(MTTF),andthetimeittakestorestoreasystemtoaworkingstateafterafailure(forexample,toreboot),calledthemeantimetorepair(MTTR).AvailabilitycanbeimprovedbyincreasingtheMTTForreducingtheMTTR.
Throughoutthisbook,wewillpresentvariousapproachestoimprovingoperatingsystemreliabilityandavailability.Inmanycases,theabstractionsmayseematfirstglanceoverlyrigidandformulaic.Itisimportanttorealizethisisdoneonpurpose!Onlypreciseabstractionsprovideabasisforconstructingreliableandavailablesystems.
1.2.2Security
Twoconceptscloselyrelatedtoreliabilityaresecurityandprivacy.Securitymeansthecomputersoperationcannotbecompromisedbyamaliciousattacker.Privacyisanaspectofsecurity:datastoredonthecomputerisonlyaccessibletoauthorizedusers.
Alas,nousefulcomputerisperfectlysecure!Anycomplexpieceofsoftwarehasbugs,andseeminglyinnocuousbugscanbeexploitedbyanattackertogaincontrolofthesystem.Orthecomputerhardwaremightbetamperedwith,toprovideaccesstotheattacker.Orthecomputersadministratormightbeuntrustworthy,usinghisorhercredentialstostealuserdata.OranOSsoftwaredevelopermightbeuntrustworthy,insertingabackdoorfortheattackertogainaccesstothesystem.
Nevertheless,anoperatingsystemcanbe,andshouldbe,designedtominimizeitsvulnerabilitytoattack.Forexample,strongfaultisolationcanpreventthirdpartyapplicationsfromtakingoverthesystem.Downloadingandinstallingascreensaverorotherapplicationshouldnotprovideawayforanattackertosurreptitiouslyinstallacomputervirusonthesystem.Acomputerprogramthatmodifiesanoperatingsystemorapplicationtocopyitselffromcomputertocomputerwithoutthecomputerownerspermissionorknowledge.Onceinstalledonacomputer,avirusoftenprovidestheattackercontroloverthesystemsresourcesordata.Anexamplecomputervirusisakeylogger:aprogramthatmodifiestheoperatingsystemtorecordeverykeystrokeenteredbytheuserandsendthembacktotheattackersmachine.Inthisway,theattackercouldgainaccesstotheuserspasswords,bankaccountnumbers,andotherprivateinformation.Likewise,amaliciousscreensavermightsurreptitiouslyscanthediskforfilescontainingpersonalinformationorturnthesystemintoanemailspamserver.
Evenwithstrongfaultisolation,asystemcanbeinsecureifitsapplicationsarenotdesignedforsecurity.Forexample,theInternetemailstandardprovidesnostrongassuranceofthesendersidentity;itispossibletoformanemailmessagewithanyonesemailaddressinthefromfield,notnecessarilytheactualsenders.Thus,anemail
-
messagecanappeartobefromsomeone(perhapssomeoneyoutrust),wheninrealityitisfromtheattackerandcontains,asanattachment,amaliciousvirusthattakesoverthecomputerwhentheattachmentisopened.Bynow,youarehopefullysuspiciousofclickingonanyemailattachment.Steppingback,theissuecouldbeseenasalimitationoftheinteractionbetweentheemailsystemandtheoperatingsystem.Iftheoperatingsystemprovidedacheapandeasywaytoprocessanattachmentinanisolatedexecutionenvironmentwithlimitedcapabilities,thenevenattachmentscontainingviruseswoulddonoharm.
Complicatingmattersisthattheoperatingsystemmustnotonlypreventunwantedaccesstoshareddata,itmustalsoallowaccessinmanycases.Usersandprogramsmustbeabletointeractwitheachother,sothatitispossibletocutandpastetextbetweendifferentapplications,andtosharedatawrittentodiskoroverthenetwork.Ifeachprogramwerecompletelystandaloneandneverneededtointeractwithanyotherprogram,thenfaultisolationbyitselfwouldbesufficient.However,wenotonlywanttoisolateprogramsfromoneanother,buttoeasilysharedatabetweenprogramsandbetweenusers.
Thus,anoperatingsystemneedsbothanenforcementmechanismandasecuritypolicy.Enforcementishowtheoperatingsystemensuresthatonlypermittedactionsareallowed.Thesecuritypolicydefineswhatispermittedwhoisallowedtoaccesswhatdata,andwhocanperformwhatoperations.
Maliciousattackerscantargetvulnerabilitiesineitherenforcementmechanismsorsecuritypolicies.Anerrorinenforcementcanallowanattackertoevadethepolicy;anerrorinthepolicycanallowtheattackeraccesswhenitshouldhavebeenprohibited.
1.2.3Portability
Alloperatingsystemsprovideapplicationswithanabstractionoftheunderlyingcomputerhardware;aportableabstractionisonethatdoesnotchangeasthehardwarechanges.AprogramwrittenforMicrosoftsWindows8shouldruncorrectlyregardlessofwhetheraspecificgraphicscardisbeingused,whetherpersistentstorageisprovidedviaflashmemoryorrotatingmagneticdisk,orwhetherthenetworkisBluetooth,WiFi,orgigabitEthernet.
Portabilityalsoappliestotheoperatingsystemitself.Aswehavenoted,operatingsystemsareamongthemostcomplexsoftwaresystemseverinvented,makingitimpracticaltore-writethemfromscratcheverytimenewhardwareisproducedoranewapplicationisdeveloped.Instead,newoperatingsystemsareoftenderived,atleastinpart,fromoldones.Asoneexample,iOS,theoperatingsystemfortheiPhoneandiPad,wasderivedfromtheMacOSXcodebase.
Asaresult,mostsuccessfuloperatingsystemshavealifetimemeasuredindecades.MicrosoftWindows8originallybeganwiththedevelopmentofWindowsNTstartingin1988.Atthattime,thetypicalcomputerwas10000timeslesspowerful,andwith10000timeslessmemoryanddiskstorage,thanisthecasetoday.Operatingsystemsthatlastdecadesarenoanomaly.Microsoftsprioroperatingsystem,MS/DOS,wasintroducedin1981.ItlaterevolvedintotheearlyversionsofMicrosoftWindowsbeforefinallybeingphasedoutaround2000.
-
Thismeansthatoperatingsystemsmustbedesignedtosupportapplicationsthathavenotyetbeenwrittenandtorunonhardwarethathasnotyetbeendeveloped.Likewise,developersdonotwanttore-writeapplicationswhentheoperatingsystemisportedfrommachinetomachine.Sometimes,theimportanceoffuture-proofinganoperatingsystemisdiscoveredonlyinretrospect.Microsoftsfirstoperatingsystem,MS/DOS,wasdesignedin1981assumingthatpersonalcomputerswouldneverhavemorethan640KBofmemory.Thislimitationwasacceptableatthetime,buttoday,evencellphoneshaveordersofmagnitudemorememorythanthat.
Howmightwedesignanoperatingsystemtoachieveportability?AsweillustratedearlierinFigure1.3,ithelpstohaveasimple,standardwayforapplicationstointeractwiththeoperatingsystem,theabstractvirtualmachine(AVM).Thisistheinterfaceprovidedbyoperatingsystemstoapplications,including:(i)theapplicationprogramminginterface(API),thelistoffunctioncallstheoperatingsystemprovidestoapplications,(ii)thememoryaccessmodel,and(iii)whichinstructionscanbelegallyexecuted.Forexample,aninstructiontochangewhetherthehardwareisexecutingtrustedoperatingsystemcode,oruntrustedapplicationcode,mustbeavailabletotheoperatingsystembutnottoapplications.
Awell-designedoperatingsystemAVMprovidesafixedpointacrosswhichbothapplicationcodeandhardwarecanevolveindependently.ThisissimilartotheroleoftheInternetProtocol(IP)standardinnetworking.Distributedapplicationssuchasemailandtheweb,writtenusingIP,areinsulatedfromchangesintheunderlyingnetworktechnology(Ethernet,WiFi,optical).Equallyimportantisthatchangesinapplications,fromemailtoinstantmessagingtofilesharing,donotrequiresimultaneouschangesintheunderlyinghardware.
Thisnotionofaportablehardwareabstractionissopowerfulthatoperatingsystemsusethesameideainternally:theoperatingsystemitselfcanlargelybeimplementedindependentlyofthehardwarespecifics.Theinterfacethatmakesthispossibleiscalledthehardwareabstractionlayer(HAL).ItmightseemthattheoperatingsystemAVMandtheoperatingsystemHALshouldbeidentical,ornearlysoafterall,bothareportablelayersdesignedtohidehardwaredetails.TheAVMmustdomore,however.Aswenoted,applicationsexecuteinarestricted,virtualizedcontextandwithaccesstohigh-levelcommonservices,whiletheoperatingsystemitselfusesaproceduralabstractionmuchclosertotheactualhardware.
Today,Linuxisanexampleofahighlyportableoperatingsystem.Ithasbeenusedastheoperatingsystemforwebservers,personalcomputers,tablets,netbooks,e-bookreaders,smartphones,settopboxes,routers,WiFiaccesspoints,andgameconsoles.LinuxisbasedonanoperatingsystemcalledUNIX,whichwasoriginallydevelopedintheearly1970s.UNIXwaswrittenbyasmallteamofdevelopers.Itwasdesignedtobecompact,simpletoprogram,andhighlyportable,butatsomecostinperformance.Overtheyears,UNIXsandLinuxsportabilityandconvenientprogrammingabstractionshavebeenkeystotheirsuccess.
1.2.4Performance
-
Whiletheportabilityofanoperatingsystembecomesapparentovertime,theperformanceofanoperatingsystemisoftenimmediatelyvisibletoitsusers.Althoughweoftenassociateperformancewitheachindividualapplication,theoperatingsystemsdesigncangreatlyaffecttheapplicationsperceivedperformance.Theoperatingsystemdecideswhenanapplicationcanrun,howmuchmemoryitcanuse,andwhetheritsfilesarecachedinmemoryorclusteredefficientlyondisk.Theoperatingsystemalsomediatesapplicationaccesstomemory,thenetwork,andthedisk.Itmustavoidslowingdownthecriticalpathwhilestillprovidingneededfaultisolationandresourcesharingbetweenapplications.
Performanceisnotasinglequantity.Rather,itcanbemeasuredinseveraldifferentways.Oneperformancemetricistheoverhead,theaddedresourcecostofimplementinganabstractionpresentedtoapplications.Arelatedconceptisefficiency,thelackofoverheadinanabstraction.Onewaytomeasureoverhead(orinversely,efficiency)isthedegreetowhichtheabstractionimpedesapplicationperformance.Supposeyoucouldruntheapplicationdirectlyontheunderlyinghardwarewithouttheoverheadoftheoperatingsystemabstraction;howmuchwouldthatimprovetheapplicationsperformance?
Operatingsystemsalsoneedtoallocateresourcesamongapplications,andthiscanaffecttheperformanceofthesystemasperceivedbytheenduser.Oneissueisfairnessbetweendifferentusersorapplicationsrunningonthesamemachine.Shouldresourcesbedividedequallybetweendifferentusersorapplications,orshouldsomegetpreferentialtreatment?Ifso,howdoestheoperatingsystemdecidewhattasksgetpriority?
Tworelatedconceptsareresponsetimeandthroughput.Responsetime,sometimescalleddelay,ishowlongittakesforasingletasktorun,fromthetimeitstartstothetimeitcompletes.Forexample,ahighlyvisibleresponsetimefordesktopcomputersisthetimefromwhentheusermovesthehardwaremouseuntilthepointeronthescreenreflectstheusersaction.Anoperatingsystemthatprovidespoorresponsetimecanbeunusable.Throughputistherateatwhichthesystemcompletestasks.Throughputisameasureofefficiencyforagroupoftasksratherthanasingleone.Whileitmightseemthatdesignsthatimproveresponsetimewouldalsonecessarilyimprovethroughput,thisisnotthecase,aswediscussinChapter7.
Arelatedconsiderationisperformancepredictability:whetherthesystemsresponsetimeorothermetricisconsistentovertime.Predictabilitycanoftenbemoreimportantthanaverageperformance.Ifauseroperationsometimestakesaninstantbutsometimesmuchlonger,theusermayfinditdifficulttoadapt.Consider,forexample,twosystems.Inone,eachkeystrokeisusuallyinstantaneous,but1%ofthetime,ittakes10secondstotakeeffect.Intheothersystem,akeystrokealwaystakesexactly0.1secondstoappearonthescreen.Averageresponsetimeisthesameinbothsystems,butthesecondismorepredictable.Whichdoyouthinkwouldbemoreuser-friendly?
EXAMPLE:Toillustratetheconceptsofefficiency,overhead,fairness,responsetime,throughput,andpredictability,consideracardrivingtoitsdestination.Ifnoothercarsorpedestrianswereeverontheroad,thecarcouldgoquitequickly,neverneedingtoslowdownforstoplights.Stopsignsandstoplightsenablemultiplecarstosharetheroad,atsomecostinoverheadandresponsetimeforeachindividualdriver.Asthesystembecomesmorecongested,predictabilitysuffers.Throughputofthesystemimproveswithcarpooling.Withdedicatedcarpoollanes,carpoolingcanevenreducedelaydespite
-
carpoolersneedingtocoordinatetheirpickups.Scrappingthecarandbuildingmasstransitcanimprovepredictability,throughput,andfairness.
1.2.5Adoption
Inadditiontoreliability,portabilityandperformance,thesuccessofanoperatingsystemdependsontwofactorsoutsideitsimmediatecontrol:thewideavailabilityofapplicationsportedtothatoperatingsystem,andthewideavailabilityofhardwarethattheoperatingsystemcansupport.AniPhonerunsiOS,butwithoutthepre-installedapplicationsandthecontentsoftheAppStore,theiPhonewouldbejustanothercellphone.
Thenetworkeffectoccurswhenthevalueofsometechnologydependsnotonlyonitsintrinsiccapabilities,butalsoonthenumberofotherpeoplewhohaveadoptedit.Applicationandhardwaredesignersspendtheireffortsonthoseoperatingsystemplatformswiththemostusers,whileusersfavorthoseoperatingsystemswiththebestapplicationsorthecheapesthardware.Ifthissoundscircular,itis!Moreusersimplymoreapplicationsandcheaperhardware;moreapplicationsandcheaperhardwareimplymoreusers,inavirtuouscycle.
Considerhowyoumightdesignanoperatingsystemtotakeadvantageofthenetworkeffect,oratleasttoavoidbeingcrushedbyit.Anobviousstepwouldbetodesignthesystemtomakeiteasytoaccommodatenewhardwareandforapplicationstobeportedacrossdifferentversionsofthesameoperatingsystem.
Amoresubtleissueisthechoiceofwhethertheoperatingsystemprogramminginterface(API),ortheoperatingsystemsourcecodeitself,isopenorproprietary.Aproprietarysystemisoneunderthecontrolofasinglecompany;itcanbechangedatanytimebyitsprovidertomeettheneedsofitscustomers.Anopensystemisonewherethesystemssourcecodeispublic,givinganyonetheabilitytoinspectandchangethecode.Often,anopensystemhasanAPIthatcanbechangedonlywiththeagreementofapublicstandardsbody.AdherencetostandardsprovidesassurancetoapplicationdevelopersthattheAPIwillnotbechangedexceptbygeneralagreement;ontheotherhand,standardsbodiescanmakeitdifficulttoquicklyaddnew,desiredfeatures.
Neitheropennorproprietarysystemsareintrinsicallybetterforadoption.Windows8andMacOSareproprietaryoperatingsystems;Linuxisanopenoperatingsystem.Allthreearewidelyused.Opensystemsareeasiertoadapttoawidevarietyofhardwareplatforms,buttheyriskdevolvingintomultipleversions,impairingthenetworkeffect.Purveyorsofproprietaryoperatingsystemsarguethattheirsystemsaremorereliableandbetteradaptedtotheneedsoftheircustomers.Interoperabilityproblemscanbereducedifthesamecompanycontrolsboththehardwareandthesoftware,butlimitinganoperatingsystemtoonehardwareplatformimpairsthenetworkeffectandrisksalienatingconsumers.
Makingiteasytoportapplicationsfromexistingsystemstoanewoperatingsystemcanhelpanewsystembecomeestablished;conversely,designinganoperatingsystemAPIthatmakesitdifficulttoportapplicationsawayfromtheoperatingsystemcanhelppreventcompetitionfrombecomingestablished.Thus,thereareoftencommercialpressuresforoperatingsysteminterfacestobecomeidiosyncratic.Throughoutthisbook,wediscussoperatingsystemsissuesataconceptuallevel,butrememberthatthedetails
-
mayvaryconsiderablyforanyspecificoperatingsystemduetoimportant,butsometimeschaotic,commercialinterests.
1.2.6DesignTradeoffs
Mostpracticaloperatingsystemdesignsstrikeabalancebetweenthegoalsofreliability,security,portability,performance,andadoption.Designchoicesthatimproveportabilityforexample,preservinglegacyinterfacesoftenmakethesystemasawholelessreliableandlesssecure.Similarly,itisoftenpossibletoincreasesystemperformancebybreakinganabstraction.However,suchperformanceoptimizationsmayaddcomplexityandthereforepotentiallyhurtreliability.Theoperatingsystemdesignermustcarefullyweighthesecompetinggoals.
EXAMPLE:Toillustratethetradeoffbetweenperformanceandcomplexity,considerthefollowingtruestory.Aresearchoperatingsystemdevelopedinthelate1980susedatype-safelanguagetoreducetheincidenceofprogrammererrors.Forspeed,themostfrequentlyusedroutinesatthecoreoftheoperatingsystemwereimplementedinassemblycode.Inoneoftheseroutines,theimplementationteamdecidedtouseasequenceofinstructionsthatshavedasingleinstructionoffaveryfrequentlyusedcodepath,butthatwouldsometimesbreakiftheoperatingsystemexceededaparticularsize.Atthetime,theoperatingsystemwasnowherenearthislimit.Afterafewyearsofproductionuse,however,thesystemstartedmysteriouslycrashing,apparentlyatrandom,andonlyaftermanydaysofexecution.Manyweeksofpainstakinginvestigationrevealedtheproblem:theoperatingsystemhadgrownbeyondthelimitassumedintheassemblycodeimplementation.Thefixwaseasy,oncetheproblemwasfound,butthequestionis:doyouthinktheoriginaloptimizationwasworththerisk?
1.3OperatingSystems:Past,Present,andFuture
Weconcludethischapterbydiscussingtheoriginsofoperatingsystems,inordertoillustratewherethesesystemsareheadinginthefuture.Asthelowestlayerofsoftwarerunningontopofcomputerhardware,operatingsystemsdatebacktothefirstcomputers,evolvingnearlyasrapidlyascomputerhardware.
1.3.1ImpactofTechnologyTrends
1981 1997 2014 Factor(2014/1981)
Singleprocessorspeed(MIPS) 1 200 2500 2.5K
CPUspercomputer 1 1 10+ 10+
Processor$/MIP $100K $25 $0.20 500K
-
DRAMcapacity(MiB)/$ 0.002 2 1K 500K
Diskcapacity(GiB)/$ 0.003 7 25K 10M
HomeInternet 300bps 256Kbps 20Mbps 100K
Machineroomnetwork 10Mbpsshared100Mbpsswitched
10Gbpsswitched 1000+
Ratioofuserstocomputers 100:1 1:1 1:several 100+
Figure1.8:Approximatecomputerserverperformanceovertime,reflectingwidelyusedserversofeachera:in1981,aminicomputer;in1997,ahigh-endworkstation;in2014,arack-mountedmulticoreserver.MIPSstandsformillionsofinstructionspersecond,aroughmeasureofprocessorperformance.TheVAX11/782wasintroducedin1982;itachieved1MIP.DRAMpricesarefromHennesseyandPatterson,ComputerArchitecture:AQuantitativeApproach.DiskdrivepricesarefromJohnMcCallum.TheHayessmartmodem,introducedin1981,ranat300bps.The10MbpssharedEthernetstandardwasalsointroducedin1981.Oneoftheauthorsbuilthisfirstoperatingsystemin1982,usedaVAXathisfirstjob,andownedaHayestoworkfromhome.
ThemoststrikingaspectofthelastfiftyyearsincomputingtechnologyhasbeenthecumulativeeffectofMooresLawandthecomparableadvancesinrelatedtechnologies,suchasmemoryanddiskstorage.MooresLawstatesthattransistordensityincreasesexponentiallyovertime;similarexponentialimprovementshaveoccurredinmanyothercomponenttechnologies.Figure1.8providesanoverviewofthepastthreedecadesoftechnologyimprovementsincomputerhardware.Thecostofprocessingandmemoryhasdecreasedbyalmostsixordersofmagnitudeoverthisperiod;thecostofdiskcapacityhasdecreasedbysevenordersofmagnitude.Notalltechnologieshaveimprovedatthesamerate;disklatency(notshowninthetable)hasimproved,butatamuchslowerratethandiskcapacity.Theserelativechangeshaveradicallyalteredboththeuseofcomputersandthetradeoffsfacedbyoperatingsystemdesigners.
Itishardtoimaginehowthingsusedtobe.Today,youprobablycarryasmartphoneinyourpocket,withanincrediblypowerfulcomputerinside.Thousandsofservercomputerswaitpatientlyforyoutotypeinasearchquery;whenthequeryarrives,theycansynthesizearesponseinafractionofasecond.Intheearlyyearsofcomputing,however,thecomputersweremoreexpensivethanthesalariesofthepeoplewhousedthem.Userswouldqueueup,oftenfordays,fortheirturntorunaprogram.Asimilarprogressionfromexpensivetocheapdevicesoccurredwithtelephonesoverthepasthundredyears.Initially,telephonelineswereveryexpensive,withasinglesharedlineamongeveryoneinaneighborhood.Overtime,ofcourse,bothcomputersandtelephoneshavebecomecheap
-
enoughtositidleuntilweneedthem.
Despitethesechanges,operatingsystemsstillfacethesameconceptualchallengesastheydidfiftyyearsago.Tomanagecomputerresourcesforapplicationsandusers,theymustallocateresourcesamongapplications,providefaultisolationandcommunicationservices,abstracthardwarelimitations,andsoforth.Wehavemadetremendousprogresstowardsimprovingthereliability,security,efficiency,andportabilityofoperatingsystems,butmuchmoreisneeded.Althoughwedonotknowpreciselyhowcomputingtechnologyorapplicationdemandwillevolveoverthenext10-20years,itishighlylikelythatthesefundamentaloperatingsystemchallengeswillpersist.
1.3.2EarlyOperatingSystems
Thefirstoperatingsystemswereruntimelibrariesintendedtosimplifytheprogrammingofearlycomputersystems.Ratherthanthetiny,inexpensiveyetmassivelycomplexhardwareandsoftwaresystemsoftoday,thefirstcomputersoftentookupanentirefloorofawarehouse,costmillionsofdollars,andyetwerecapableofbeingusedonlybyasinglepersonatatime.Theuserwouldfirstresetthecomputer,loadtheprogrambytogglingitintothesystemonebitatatime,andhitgo,producingoutputtobeporedoverduringthenextusersturn.Iftheprogramhadabug,theuserwouldneedtowaittotrytherunoveragain,oftenthenextday.
Itmightseemliketherewasnoneedforanoperatingsysteminthissetting.However,sincecomputerswereenormouslyexpensive,reducingthelikelihoodofprogrammererrorwasparamount.Thefirstoperatingsystemsweredevelopedasawaytoreduceerrorsbyprovidingastandardsetofcommonservices.Forexample,earlyoperatingsystemsprovidedstandardinput/output(I/O)routinesthateachusercouldlinkintotheirprograms.Theseservicesmadeitmorelikelythatausersprogramwouldproduceusefuloutput.
Althoughtheseinitialoperatingsystemswereahugestepforward,theresultwasstillextremelyinefficient.ItwasaroundthistimethattheCEOofIBMfamouslypredictedthatwewouldonlyeverneedfivecomputersintheworld.Ifcomputerstodaycostmillionsofdollarsandcouldonlyruntinyapplicationsbyonepersonatatime,hemighthavebeenright.
1.3.3Multi-UserOperatingSystems
Thenextstepforwardwassharing,introducingmanyoftheadvantages,andchallenges,thatweseeintodaysoperatingsystems.Whenprocessortimeisvaluable,restrictingthesystemtooneuseratatimeiswasteful.Forexample,inearlysystemstheprocessorremainedidlewhiletheuserloadedtheprogram,eveniftherewasalonglineofpeoplewaitingtheirturn.
Abatchoperatingsystemworksonaqueueoftasks.Itrunsasimpleloop:load,run,andunloadeachjobinturn.Whileonejobwasrunning,theoperatingsystemsetsuptheI/Odevicestodobackgroundtransfersforthenext/previousjobusingaprocesscalleddirectmemoryaccess(DMA).WithDMA,theI/Odevicetransfersitsdatadirectlyintomemoryatalocationspecifiedbytheoperatingsystem.WhentheI/Otransfercompletes,the
-
hardwareinterruptstheprocessor,transferringcontroltotheoperatingsysteminterrupthandler.TheoperatingsystemstartsthenextDMAtransferandthenresumesexecutionoftheapplication.Theinterruptappearstotheapplicationasifnothinghadhappened,exceptforsomedelaybetweenoneinstructionandthenext.
Batchoperatingsystemsweresoonextendedtorunmultipleapplicationsatonce,calledmultitaskingorsometimesmultiprogramming.Multipleprogramsareloadedintomemoryatthesametime,eachreadytousetheprocessorifforanyreasontheprevioustaskneededtopause,forexample,toreadadditionalinputorproduceoutput.Multitaskingincreasesprocessorefficiencytonearly100%;ifthequeueoftasksislongenough,andasufficientnumberofI/Odevicescankeepfeedingtheprocessor,thereisnoneedfortheprocessortowait.
However,processorsharingraisestheneedforprogramisolation,tolimitabuginoneprogramfromcrashingorcorruptinganother.Duringthisperiod,computerdesignersaddedhardwarememoryprotection,toreducetheoverheadoffaultisolation.
Apracticalchallengewithbatchcomputing,however,ishowtodebugtheoperatingsystemitself.Unlikeanapplicationprogram,abatchoperatingsystemassumesitisindirectcontrolofthehardware.Newversionscanonlybetestedbystoppingeveryapplicationandrebootingthesystem,essentiallyturningthecomputerbackintoasingle-usersystem.Needlesstosay,thiswasanexpensiveoperation,oftenscheduledforthedeadofthenight.
Virtualmachinesaddressthislimitation(seeFigure1.4).Insteadofrunningatestoperatingsystemdirectlyonthehardware,virtualmachinesrunanoperatingsystemasanapplication.Thehostoperatingsystem,alsocalledavirtualmachinemonitor,exportsanabstractvirtualmachine(AVM)thatisidenticaltotheunderlyinghardware.Thetestoperatingsystemrunningontopofthevirtualmachinedoesnotneedtoknowthatitisrunninginavirtualenvironmentitexecutesinstructions,accesseshardwaredevices,andrestoresapplicationstateafteraninterruptjustasifitwererunningonrealhardware.
Virtualmachinesarenowwidelyusedforoperatingsystemdevelopment,backwardcompatibility,andcross-platformsupport.Applicationsoftwarethatrunsonlyonanoldversionofanoperatingsystemcansharehardwarewithentirelynewapplications.Thevirtualmachinemonitorrunstwovirtualmachinesoneforthenewoperatingsystemforcurrentapplicationsandaseparateoneforlegacyapplications.Asanotherexample,MacOSuserswhoneedtorunWindowsorLinuxapplicationscandosobyrunningtheminsideavirtualmachine.
1.3.4Time-SharingOperatingSystems
-
Figure1.9:Genealogyofseveralmodernoperatingsystems.
Eventually,thecumulativeeffectofMooresLawmeantthatthecostofcomputingdroppedtowheresystemscouldbeoptimizedforusersratherthanforefficientuseoftheprocessor.UNIX,forexample,wasdevelopedintheearly70sonasparecomputerthatnoonewasusingatthetime.UNIXbecamethebasisforApplesMacOSX,Linux,VMware(awidelyusedvirtualmachinemonitor),andGoogleAndroid.Figure1.9tracesthelineageoftheseoperatingsystems.
Time-sharingoperatingsystemssuchasWindows,MacOS,orLinuxaredesignedtosupportinteractiveuseofthecomputerratherthanthebatchmodeprocessingofearliersystems.Withtime-sharing,theusertypesinputonakeyboardorotherinputdevicedirectlyconnectedtothecomputer.Eachkeystrokeormouseactioncausesaninterrupttotheprocessorsignalingtheevent;theinterrupthandlerreadstheeventfromthedeviceandqueuesitinsidetheoperatingsystem.Whentheuserswordprocessor,game,orotherapplicationresumes,itfetchestheeventfromtheoperatingsystem,processesit,andaltersthedisplayappropriatelybeforefetchingthenextevent.Hundredsoreventhousandsofsucheventscanbeprocessedpersecond,requiringboththeoperatingsystemandtheapplicationtobedesignedforfrequent,veryshortburstsofactivityratherthanthesustainedexecutionmodelofbatchprocessing.
Thebasicoperationofawebserverissimilartoatime-sharingsystem.Thewebserverwaitsforapackettoarrive,torequestawebpage,websearch,orbookpurchase.ThenetworkhardwarecopiesthearrivingpacketintomemoryusingDMA.Oncethetransferiscomplete,thehardwaresignalsthepacketsarrivalbyinterruptingtheprocessor.Thistriggerstheservertoperformtherequestedtask.Likewise,theprocessorisinterruptedaseachblockofawebpageisreadfromdiskintomemory.Likeatime-sharingsystem,serveroperatingsystemsmustbedesignedtohandleverylargenumbersofshortactionspersecond.
-
Theearliesttime-sharingsystemssupportedmanysimultaneoususers,buteventhiswasjustaphase.Eventually,computersbecamecheapenoughthatpeoplecouldaffordtheirowndedicatedpersonalcomputers,whichwouldsitpatientlyunusedformuchoftheday.Accesstoshareddatabecameparamount,cementingtheshifttoclient-servercomputing.
1.3.5ModernOperatingSystems
Today,wehaveavastdiversityofcomputingdevices,withmanydifferentoperatingsystemsrunningonthem.Thetradeoffsfacedbyanoperatingsystemdesignerdependonthephysicalcapabilitiesofthehardwareaswellasapplicationanduserneeds.Herearesomeexamplesofoperatingsystemsthatyoumayhaveusedrecently:
Desktop,laptop,andnetbookoperatingsystems.ExamplesincludeWindows8,MacOSX,andLinux.Thesesystemsaresingleuser,runmanyapplications,andhavevariousI/Odevices.Onemightthinkthatwithonlyoneuser,therewouldbenoneedtodesignthesystemtosupportsharing,andindeedtheinitialpersonalcomputeroperatingsystemstookthisapproach.Theyhadaverylimitedabilitytoisolatedifferentpartsofthesystemfromeachother.Overtime,however,itbecameclearthatstricterfaultisolationwasneededtoimprovesystemreliabilityandresilienceagainstcomputerviruses.Otherkeydesigngoalsforthesesystemsincludeadoption(tosupportarichsetofapplications)andinteractiveperformance.
Smartphoneoperatingsystems.Asmartphoneisacellphonewithanembeddedcomputercapableofrunningthirdpartyapplications.ExamplesofsmartphoneoperatingsystemsincludeiOS,Android,Symbian,WebOS,BlackberryOSandWindowsPhone.Whilesmartphoneshaveonlyoneuser,theymustsupportmanyapplications.Keydesigngoalsincluderesponsiveness,supportforawidevarietyofapplications,andefficientuseofthebattery.Anotherdesigngoalisuserprivacy.Becausethird-partyapplicationsmightsurreptitiouslygatherprivatedatasuchastheuserscontactlistformarketingpurposes,theoperatingsystemmustbedesignedtolimitaccesstoprotecteduserdata.
Serveroperatingsystems.Searchengines,webmedia,e-commercesites,andemailsystemsarehostedoncomputersindatacenters;eachofthesecomputersrunsanoperatingsystem,oftenanindustrialstrengthversionofoneofthedesktopsystemsdescribedabove.Usually,onlyasingleapplication,suchasawebserver,runspermachine,buttheoperatingsystemmustcoordinatethousandsofsimultaneousincomingnetworkconnections.Throughputinhandlingalargenumberofrequestspersecondisakeydesigngoal.Atthesametime,thereisapremiumonresponsiveness:AmazonandGooglebothreportthataddingeven100millisecondsofdelaytoeachwebrequestcansignificantlyaffectrevenue.Serversalsooperateinahostileenvironment,wheremaliciousattackersmayattempttosubvertorblocktheservice;resistancetoattackisanessentialrequirement.
Virtualmachines.Aswenoted,avirtualmachinemonitorisanoperatingsystemthatcanrunanotheroperatingsystemasifitwereanapplication.ExamplesincludeVMWare,Xen,andWindowsVirtualPC.Virtualmachinemonitorsfacemanyofthe
-
samechallengesasotheroperatingsystems,withtheaddedchallengeposedbycoordinatingasetofcoordinators.Aguestoperatingsystemrunninginsideavirtualmachinemakesresourceallocationandfaultisolationdecisionsasifitwereincompletecontrolofitsresources,eventhoughitissharingthesystemwithotheroperatingsystemsandapplications.
Acommerciallyimportantuseofvirtualmachinesistotoallowasingleservermachinetorunasetofindependentservices.Eachvirtualmachinecanbeconfiguredasneededbythatparticularservice.Forexample,thisallowsmultipleunrelatedwebserverstosharethesamephysicalhardware.Theprimarydesigngoalforvirtualmachinesisthusefficiencyandlowoverhead.
Embeddedsystems.Overtime,computershavebecomecheapenoughtointegrateintoanynumberofconsumerdevices,fromcableTVset-topboxes,tomicrowaveovens,thecontrolsystemsforautomobilesandairplanes,LEGOrobots,andmedicaldevices,suchasMRImachinesandWiFi-basedintravenoustitrationsystems.Embeddeddevicestypicallyrunacustomizedoperatingsystembundledwiththetask-specificsoftwarethatcontrolsthedevice.Althoughyoumightthinkthesesystemsastoosimpletomeritmuchattention,softwareerrorsinthemcanhavedevastatingeffects.OneexampleistheTherac-25,anearlycomputer-controlledradiologydevice.Programmingerrorsintheoperatingsystemcodecausedthesystemtomalfunction,leadingtoseveralpatientdeaths.
Serverclusters.Forfaulttolerance,scale,andresponsiveness,websitesareincreasinglyimplementedondistributedclustersofcomputershousedinoneormoregeographicallydistributeddatacenterslocatedclosetousers.Ifonecomputerfailsduetoahardwarefault,softwarecrash,orpowerfailure,anothercomputercantakeoveritsrole.Ifdemandforthewebsiteexceedswhatasinglecomputercanaccommodate,webrequestscanbepartitionedamongmultiplemachines.Aswithnormaloperatingsystems,serverclusterapplicationsrunontopofanabstractclusterinterfacetoisolatetheapplicationfromhardwarechangesandtoisolatefaultsinoneapplicationfromaffectingotherapplicationsinthesamedatacenter.Likewise,resourcescanbesharedbetween:(1)variousapplicationsonthesamewebsite(suchasGoogleSearch,GoogleEarth,andGmail),and(2)multiplewebsiteshostedonthesameclusterhardware(suchaswithAmazonsElasticComputeCloudorGooglesComputeEngine).
1.3.6FutureOperatingSystems
Whereareoperatingsystemsheadingfromhereoverthenextdecade?Operatingsystemshavebecomedramaticallybetteratresistingmaliciousattacks,buttheystillhavequiteawaystogo.Providedsecurityandreliabilitychallengescanbemet,hugepotentialbenefitswouldresultfromhavingcomputerstightlycontrolandcoordinatephysicalinfrastructure,suchasthepowergrid,thetelephonenetwork,andahospitalsmedicaldevicesandmedicalrecordsystems.Thousandsoflivesarelostannuallythroughtrafficaccidentsthatcouldpotentiallybepreventedthroughcomputercontrolofautomobiles.Ifwearetorelyoncomputersforthesecriticalsystems,weneedgreaterassurancethatoperatingsystems
-
areuptothetask.
Second,underlyinghardwarechangeswilloftentriggernewworkinoperatingsystemdesign.Thefutureofoperatingsystemsisalsothefutureofhardware:
Verylargescaledatacenters.Operatingsystemswillneedtocoordinatethehundredsofthousandsorevenmillionsofcomputersindatacenterstosupportessentialonlineservices.
Verylargescalemulticoresystems.Computerarchitecturesalreadycontainseveralprocessorsperchip;thistrendwillcontinue,yieldingsystemswithhundredsorpossiblyeventhousandsofprocessorspermachine.
Ubiquitousportablecomputingdevices.Withtheadventofsmartphones,tablets,ande-bookreaders,computersandtheiroperatingsystemswillbecomeuntetheredfromthekeyboardandthescreen,respondingtovoice,gestures,andperhapsevenbrainwaves.
Veryheterogeneoussystems.Aseverydevicebecomesprogrammable,operatingsystemswillbeneededforahugevarietyofdevices,fromsupercomputerstorefrigeratorstoindividuallightswitches.
Verylargescalestorage.Alldatathatcanbestored,willbe;theoperatingsystemwillneedtostoreenormousamountsofdatareliably,sothatitcanberetrievedatanypoint,evendecadeslater.
Managingallthisisthejoboftheoperatingsystem.
Exercises
1. Whatisanexampleofanoperatingsystemas:a. Referee?b. Illusionist?c. Glue?
2. Whatisthedifference,ifany,betweenthefollowingterms:a. Reliabilityvs.availability?b. Securityvs.privacy?c. Securityenforcementvs.securitypolicy?d. Throughputvs.responsetime?e. Efficiencyvs.overhead?f. Applicationprogramminginterface(API)vs.abstractvirtualmachine(AVM)?g. Abstractvirtualmachine(AVM)vs.hardwareabstractionlayer(HAL)?h. Proprietaryvs.openoperatingsystem?i. Batchvs.interactiveoperatingsystem?j. Hostvs.guestoperatingsystem?
3. Definetheterm,directmemoryaccess(DMA).
Forthefollowingquestions,takeamomenttospeculate.Weprovideanswerstothesequestionsthroughoutthebook,but,givenwhatyouknownow,howwouldyou
-
answerthem?Beforetherewereoperatingsystems,someoneneededtodevelopsolutionswithoutbeingabletolookthemup!Howwouldyouhavedesignedthefirstoperatingsystem?
4. Supposeacomputersystemandallofitsapplicationswerecompletelybugfree.Supposefurtherthateveryoneintheworldwerecompletelyhonestandtrustworthy.Inotherwords,weneednotconsiderfaultisolation.a. Howshouldanoperatingsystemallocatetimeontheprocessor?Shoulditgive
theentireprocessortoeachapplicationuntilitnolongerneedsit?Ifthereweremultipletasksreadytogoatthesametime,shoulditschedulefirstthetaskwiththeleastamountofworktodoortheonewiththemost?Justifyyouranswer.
b. Howshouldtheoperatingsystemallocatephysicalmemorytoapplications?Whatshouldhappenifthesetofapplicationsdoesnotfitinmemoryatthesametime?
c. Howshouldtheoperatingsystemallocateitsdiskspace?Shouldthefirstusertoaskacquireallofthefreespace?Whatwouldthelikelyoutcomebeforthatpolicy?
5. Nowsupposethecomputersystemneedstosupportfaultisolation.Whathardwareand/oroperatingsupportdoyouthinkw