11
Operating Systems and Operating Systems and SecuritySecurity
22
OS SecurityOS Security
OSs are large, complex programsOSs are large, complex programs– Many bugs in any such programMany bugs in any such program– We have seen that bugs can be security We have seen that bugs can be security
threatsthreats
Here we are concerned with security Here we are concerned with security provided by OSprovided by OS– Not concerned with threat of bad OS softwareNot concerned with threat of bad OS software
Concerned with OS as security Concerned with OS as security enforcerenforcer
33
IntroductionIntroduction Operating systems provide the lowest layer Operating systems provide the lowest layer
of software visible to usersof software visible to users
Operating systems are close to the hardwareOperating systems are close to the hardware – – Often have complete hardware accessOften have complete hardware access
If the operating system isn’t protected, the If the operating system isn’t protected, the machine isn’t protectedmachine isn’t protected
Flaws in the OS generally compromise all Flaws in the OS generally compromise all security at higher levelssecurity at higher levels
44
Why Is OS Security So Important?Why Is OS Security So Important?
The OS controls access to application memoryThe OS controls access to application memory
The OS controls scheduling of the processorThe OS controls scheduling of the processor
The OS ensures that users receive the resources The OS ensures that users receive the resources they ask forthey ask for
If the OS isn’t doing these things securely, If the OS isn’t doing these things securely, practically anything can go wrongpractically anything can go wrong
So almost all other security systems must assume So almost all other security systems must assume a secure OS at the bottoma secure OS at the bottom
55
OS Security ChallengesOS Security Challenges Modern OS isModern OS is multi-user multi-user and and multi-multi-
tasking tasking OS must deal withOS must deal with
– MemoryMemory– I/O devices (disk, printer, etc.)I/O devices (disk, printer, etc.)– Programs, threadsPrograms, threads– Network issuesNetwork issues– Data, etc.Data, etc.
OS must protect processes from other OS must protect processes from other processes and users from other usersprocesses and users from other users– Whether accidental or maliciousWhether accidental or malicious
66
Single User Vs. Multiple UserSingle User Vs. Multiple UserMachinesMachines
The majority of today’s computers usually The majority of today’s computers usually support a single usersupport a single user
– – Sometimes one at a time, sometimes only one Sometimes one at a time, sometimes only one everever
Some computers are still multi-userSome computers are still multi-user – – MainframesMainframes – – ServersServers – – Network-of-workstation machinesNetwork-of-workstation machines
Single user machines often run multiple Single user machines often run multiple processes, thoughprocesses, though
77
Server Machines Vs. GeneralServer Machines Vs. GeneralPurpose MachinesPurpose Machines
Most server machines provide only Most server machines provide only limited serviceslimited services
– – Web page accessWeb page access
– – File accessFile access
– – DNS lookupDNS lookup
Security problems are simpler for Security problems are simpler for them them
88
Downloadable Code and SingleDownloadable Code and SingleUser MachinesUser Machines
Applets and other downloaded code Applets and other downloaded code should run in a constrained modeshould run in a constrained mode
Using access control on a finer Using access control on a finer granularity than the usergranularity than the user
Essentially the same protection Essentially the same protection problem as multiple usersproblem as multiple users
99
Mechanisms for SecureMechanisms for SecureOperating SystemsOperating Systems
Most operating system security is Most operating system security is based on separationbased on separation
– – Keep the bad guys away from the Keep the bad guys away from the good stuffgood stuff
– – Since you don’t know who’s bad, Since you don’t know who’s bad, separate most thingsseparate most things
1010
Separation MethodsSeparation Methods
Physical separationPhysical separation
– – Different machinesDifferent machines Temporal separationTemporal separation
– – Same machine, different timesSame machine, different times Logical separationLogical separation
– – HW/software enforcementHW/software enforcement Cryptographic separationCryptographic separation
1111
The Problem of SharingThe Problem of Sharing Separating stuff is actually pretty easySeparating stuff is actually pretty easy
The hard problem is allowing The hard problem is allowing controlled sharingcontrolled sharing
How can the OS allow users to share How can the OS allow users to share exactly what they intend to share?exactly what they intend to share?
– – In exactly the ways they intendIn exactly the ways they intend
1212
OS Security FunctionsOS Security Functions
Memory protectionMemory protection– Protect memory from users/processesProtect memory from users/processes
File protectionFile protection– Protect user and system resourcesProtect user and system resources
AuthenticationAuthentication– Determines and enforce authentication resultsDetermines and enforce authentication results
AuthorizationAuthorization– Determine and enforces access controlDetermine and enforces access control
1313
Memory ProtectionMemory Protection
Fundamental problemFundamental problem– How to keep users/processes separate?How to keep users/processes separate?
SeparationSeparation– Physical separation Physical separation separate devices separate devices– Temporal separation Temporal separation one at a time one at a time– Logical separation Logical separation with with
hardware/softwarehardware/software– Cryptographic separation Cryptographic separation make make
information unintelligible to outsiderinformation unintelligible to outsider– Or any combination of the aboveOr any combination of the above
1414
Memory ProtectionMemory Protection
Base/bounds register Base/bounds register lower and lower and upper address limitupper address limit
Assumes contiguous spaceAssumes contiguous space
Like a FenceLike a Fence users cannot cross a users cannot cross a specified addressspecified address
1515
Memory ProtectionMemory Protection
Tagging Tagging specify protection of each specify protection of each addressaddress++ Extremely fine-grained protection Extremely fine-grained protection-- High overhead High overhead can be reduced by tagging can be reduced by tagging
sections instead of individual addressessections instead of individual addresses
More common is segmentation and/or More common is segmentation and/or pagingpaging– Protection is not as flexibleProtection is not as flexible– But much more efficientBut much more efficient
1616
SegmentationSegmentation Divide memory into logical units, such asDivide memory into logical units, such as
– Single procedureSingle procedure– Data in one array, etc.Data in one array, etc.
Can enforce different access restrictions on Can enforce different access restrictions on different segmentsdifferent segments
Any segment can be placed in any memory Any segment can be placed in any memory location (if location is large enough)location (if location is large enough)
OS keeps track of actual locationsOS keeps track of actual locations
1717
SegmentationSegmentation
program
memory
1818
SegmentationSegmentation
OS can place segments anywhereOS can place segments anywhere OS keeps track of segment locations OS keeps track of segment locations
as as <segment,offset><segment,offset> Segments can be moved in memorySegments can be moved in memory Segments can move out of memorySegments can move out of memory All address references go thru OSAll address references go thru OS
1919
Segmentation AdvantagesSegmentation Advantages
Every address reference can be Every address reference can be checkedchecked– Possible to achieve Possible to achieve complete complete
mediationmediation Different protection can be applied to Different protection can be applied to
different segmentsdifferent segments Users can share access to segmentsUsers can share access to segments Specific users can be restricted to Specific users can be restricted to
specific segmentsspecific segments
2020
Segmentation DisadvantagesSegmentation Disadvantages How to reference How to reference <segment,offset><segment,offset> ? ?
– OS must know OS must know segmentsegment sizesize to verify access is to verify access is within within segmentsegment
– But some segments can grow during execution (for But some segments can grow during execution (for example, dynamic memory allocation)example, dynamic memory allocation)
– OS must keep track of OS must keep track of variablevariable segment sizessegment sizes
Memory fragmentation is also a problemMemory fragmentation is also a problem– Compacting memory changes tablesCompacting memory changes tables
A lot of work for the OSA lot of work for the OS
More complex More complex more chance for mistakes more chance for mistakes
2121
PagingPaging
Like segmentation, but fixed-size Like segmentation, but fixed-size segmentssegments
Access via Access via <page,offset><page,offset> Plusses and minusesPlusses and minuses
++ Avoids fragmentation, improved efficiency Avoids fragmentation, improved efficiency
++ OS need not keep track of variable segment OS need not keep track of variable segment sizessizes
-- No logical unity to pages No logical unity to pages
-- What protection to apply to a given page? What protection to apply to a given page?
2222
PagingPaging
program
memory
Page 1
Page 0
Page 2
Page 3
Page 4
Page 2
Page 1
Page 0
Page 3
Page 4
2323
Protecting InterprocessProtecting InterprocessCommunicationsCommunications
Operating systems provide various kinds Operating systems provide various kinds of interprocess communicationsof interprocess communications
– – MessagesMessages
– – SemaphoresSemaphores
– – Shared memoryShared memory
– – SocketsSockets
How can we be sure they’re used How can we be sure they’re used properly?properly?
2424
IPC Protection IssuesIPC Protection Issues
How hard it is depends on what you’re How hard it is depends on what you’re worried aboutworried about
For the moment, let’s say we’re worried For the moment, let’s say we’re worried about one process improperly using IPC to about one process improperly using IPC to get info from anotherget info from another
– – Process A wants to steal information from Process A wants to steal information from process Bprocess B
How would process A do that?How would process A do that?
2525
Message SecurityMessage Security
Can process B use message based IPC to steal the secret?
2626
How Can B Get the Secret?How Can B Get the Secret?
He can convince the system he’s AHe can convince the system he’s A
– – A problem for authenticationA problem for authentication He can break into A’s memoryHe can break into A’s memory
– – That doesn’t use message IPCThat doesn’t use message IPC
– – And is handled by page tablesAnd is handled by page tables He can forge a message from someone He can forge a message from someone
else to get the secretelse to get the secret He can “eavesdrop” on someone else who He can “eavesdrop” on someone else who
gets the secretgets the secret
2727
Forging An IdentityForging An Identity
2828
Operating System ProtectionsOperating System Protections The operating system knows who The operating system knows who
each process belongs toeach process belongs to
It can tag the message with the It can tag the message with the identity of the senderidentity of the sender
If the receiver cares, he can know the If the receiver cares, he can know the identityidentity
2929
How About Eavesdropping?How About Eavesdropping?
3030
What’s Really Going on Here?What’s Really Going on Here? On a single machine, what is a message send, On a single machine, what is a message send,
really?really?
A message is copied from a process buffer to A message is copied from a process buffer to an OS bufferan OS buffer
– – Then from the OS buffer to another process’ Then from the OS buffer to another process’ bufferbuffer
• • If attacker can’t get at processes’ internal If attacker can’t get at processes’ internal buffers and can’t get at OS buffers, he can’t buffers and can’t get at OS buffers, he can’t “eavesdrop”“eavesdrop”
3131
File ProtectionFile Protection How do we apply these access protection How do we apply these access protection
mechanisms to a real system resource?mechanisms to a real system resource?
Files are a common example of a typically Files are a common example of a typically shared resourceshared resource
If an OS supports multiple users, it needs to If an OS supports multiple users, it needs to address the question of file protectionaddress the question of file protection
3232
Unix File ProtectionUnix File Protection
A model for protecting files A model for protecting files developed in the 1970sdeveloped in the 1970s
Still in very wide use todayStill in very wide use today
– – With relatively few modificationsWith relatively few modifications
But not very flexibleBut not very flexible
3333
Unix File Protection Unix File Protection PhilosophyPhilosophy
Essentially, Unix uses a limited ACLEssentially, Unix uses a limited ACL
Only three subjects per fileOnly three subjects per file – – OwnerOwner – – GroupGroup – – OtherOther
Limited set of rights specifiableLimited set of rights specifiable – – Read, write, executeRead, write, execute – – Special meanings for some file typesSpecial meanings for some file types
3434
Unix GroupsUnix Groups A set of Unix users can be joined into a A set of Unix users can be joined into a
groupgroup
All users in that group receive common All users in that group receive common privilegesprivileges
– – Except file owners always get the owner Except file owners always get the owner privilegesprivileges
A user can be in multiple groupsA user can be in multiple groups
But a file has only one groupBut a file has only one group
3535
Setuid and SetgidSetuid and Setgid Unix mechanisms for changing your user Unix mechanisms for changing your user
identity and group identityidentity and group identity
Either indefinitely or for the run of a single Either indefinitely or for the run of a single programprogram
Created to deal with inflexibilities of the Created to deal with inflexibilities of the Unix access control modelUnix access control model
But the source of endless security problemsBut the source of endless security problems
3636
Unix File Access Control andUnix File Access Control andComplete MediationComplete Mediation
Unix doesn’t offer complete mediationUnix doesn’t offer complete mediation
File access is checked on open to a fileFile access is checked on open to a file – – For the requested modes of accessFor the requested modes of access
Opening program can use the file in the Opening program can use the file in the open mode for as long as it wantsopen mode for as long as it wants
– – Even if the file’s access permissions Even if the file’s access permissions changechange
Substantially cheaper in performanceSubstantially cheaper in performance
3737
Pros and Cons of Unix FilePros and Cons of Unix FileProtection ModelProtection Model
+ Low cost+ Low cost + Simple and easy to understand+ Simple and easy to understand + Time tested+ Time tested
– – Lacking in flexibilityLacking in flexibility • • In granularity of controlIn granularity of control – – Subject and objectSubject and object • • In what controls are possibleIn what controls are possible – – No complete mediationNo complete mediation
3838
Other OS Security FunctionsOther OS Security Functions OS must enforce access controlOS must enforce access control AuthenticationAuthentication
– Passwords, biometricsPasswords, biometrics– Single sign-on, etc.Single sign-on, etc.
AuthorizationAuthorization– ACLACL– CapabilitiesCapabilities
OS is an attractive target for attack!OS is an attractive target for attack!
3939
Desired Security Features of aDesired Security Features of aNormal OSNormal OS
Authentication of usersAuthentication of users Memory protectionMemory protection File and I/O access controlFile and I/O access control General object access controlGeneral object access control Enforcement of sharingEnforcement of sharing Fairness guaranteesFairness guarantees Secure IPC and synchronizationSecure IPC and synchronization Security of OS protection mechanismsSecurity of OS protection mechanisms
4040
Extra Features for a Trusted OSExtra Features for a Trusted OS
Mandatory and discretionary access Mandatory and discretionary access controlcontrol
Object reuse protectionObject reuse protection Complete mediationComplete mediation Audit capabilitiesAudit capabilities Intruder detection capabilitiesIntruder detection capabilities
4141
Trusted Operating SystemTrusted Operating System
An OS is An OS is trustedtrusted if we rely on it for if we rely on it for– Memory protectionMemory protection– File protectionFile protection– AuthenticationAuthentication– AuthorizationAuthorization
Every OS does these thingsEvery OS does these things But if a trusted OS fails to provide these, But if a trusted OS fails to provide these,
our security failsour security fails
4242
Trust vs SecurityTrust vs Security SecuritySecurity is a judgment is a judgment
of effectivenessof effectiveness Judged based on Judged based on
specified policyspecified policy Security depends on Security depends on
trust relationshipstrust relationships
Trust Trust implies implies reliancereliance
Trust is binaryTrust is binary Ideally, only trust Ideally, only trust
secure systemssecure systems All trust All trust
relationships should relationships should be explicitbe explicit
4343
Trusted Operating SystemsTrusted Operating Systems
Trust Trust implies relianceimplies reliance A trusted system is relied on for securityA trusted system is relied on for security An untrusted system is not relied on for An untrusted system is not relied on for
securitysecurity If all untrusted systems are If all untrusted systems are
compromised, your security is unaffectedcompromised, your security is unaffected Ironically,Ironically, only a trusted system can only a trusted system can
break your security!break your security!
4444
Trusted OSTrusted OS
OS mediates interactions between OS mediates interactions between subjects (users) and objects subjects (users) and objects (resources)(resources)
Trusted OS must decideTrusted OS must decide– Which objects to protect and howWhich objects to protect and how– Which subjects are allowed to do whatWhich subjects are allowed to do what
4545
General Security PrinciplesGeneral Security Principles Least privilege Least privilege like “low watermark” like “low watermark” SimplicitySimplicity Open design (Kerchoffs Principle)Open design (Kerchoffs Principle) Complete mediationComplete mediation White listing (preferable to black White listing (preferable to black
listing)listing) SeparationSeparation Ease of useEase of use But commercial OSs emphasize But commercial OSs emphasize
featuresfeatures– Results in complexity and poor securityResults in complexity and poor security
4646
MAC and DACMAC and DAC
Mandatory Access Control (MAC)Mandatory Access Control (MAC)– Access not controlled by owner of objectAccess not controlled by owner of object– Example: User does not decide who holds Example: User does not decide who holds
a a TOP SECRET TOP SECRET clearanceclearance Discretionary Access Control (DAC)Discretionary Access Control (DAC)
– Owner of object determines accessOwner of object determines access– Example: UNIX/Windows file protectionExample: UNIX/Windows file protection
If DAC and MAC both apply, MAC wins If DAC and MAC both apply, MAC wins
4747
Object Reuse ProtectionObject Reuse Protection
OS must prevent leaking of infoOS must prevent leaking of info
ExampleExample– User creates a fileUser creates a file– Space allocated on diskSpace allocated on disk– But same space previously usedBut same space previously used– ““Leftover” bits could leak informationLeftover” bits could leak information– Magnetic remanence is a related issueMagnetic remanence is a related issue
4848
How To Achieve OS SecurityHow To Achieve OS Security
Kernelized designKernelized design
Separation and isolation mechanismsSeparation and isolation mechanisms
VirtualizationVirtualization
Layered designLayered design
4949
Advantages of KernelizationAdvantages of Kernelization
Smaller amount of trusted codeSmaller amount of trusted code
Easier to check every accessEasier to check every access
Separation from other complex pieces of Separation from other complex pieces of the systemthe system
Easier to maintain and modify security Easier to maintain and modify security featuresfeatures
5050
Security KernelSecurity Kernel KernelKernel is the lowest-level part of the OS is the lowest-level part of the OS Kernel is responsible forKernel is responsible for
– SynchronizationSynchronization– Inter-process communicationInter-process communication– Message passingMessage passing– Interrupt handlingInterrupt handling
The The security kernelsecurity kernel is the part of the is the part of the kernel that deals with securitykernel that deals with security
Security kernel contained within the Security kernel contained within the kernelkernel
5151
Security KernelSecurity Kernel
Why have a security kernel?Why have a security kernel? All accesses go thru kernelAll accesses go thru kernel
– Ideal place for access controlIdeal place for access control Security-critical functions in one Security-critical functions in one
locationlocation– Easier to analyze and test Easier to analyze and test – Easier to modifyEasier to modify
More difficult for attacker to get in More difficult for attacker to get in “below” security functions“below” security functions
5252
Reference MonitorReference Monitor An important security concept for OS designAn important security concept for OS design
A A reference monitor reference monitor is a subsystem that controls is a subsystem that controls access to objectsaccess to objects
– – It provides (potentially) complete mediationIt provides (potentially) complete mediation
Must be tamper-resistant Must be tamper-resistant
Must be analyzableMust be analyzable– SmallSmall– Simple, etc.Simple, etc.
5353
Trusted Computing BaseTrusted Computing Base
TCBTCB everything in the OS that we everything in the OS that we rely on to enforce securityrely on to enforce security
If everything outside TCB is subverted, If everything outside TCB is subverted, trusted OS would still be trustedtrusted OS would still be trusted
TCB protects users from each otherTCB protects users from each other– Context switching between usersContext switching between users– Shared processesShared processes– Memory protection for usersMemory protection for users– I/O operations, etc.I/O operations, etc.
5454
TCB ImplementationTCB Implementation
Security may occur many places within Security may occur many places within OSOS
Ideally, design security kernel first, and Ideally, design security kernel first, and build the OS around itbuild the OS around it– Reality is usually the other way aroundReality is usually the other way around
Example of a trusted OS: Example of a trusted OS: SCOMPSCOMP– Developed by HoneywellDeveloped by Honeywell– Less than 10,000 LOC in SCOMP security Less than 10,000 LOC in SCOMP security
kernelkernel– Win XP has 40,000,000 lines of code! Win XP has 40,000,000 lines of code!
5555
TCB DesignTCB Design
HardwareSecurity kernelOperating systemUser space
Security kernel is the security layer
5656
Assurance of Trusted Operating Assurance of Trusted Operating SystemsSystems
How do I know that I should trust How do I know that I should trust someone’s operating system?someone’s operating system?
What methods can I use to achieve What methods can I use to achieve the level of trust I require?the level of trust I require?
5757
Assurance MethodsAssurance Methods
TestingTesting
Formal verificationFormal verification
ValidationValidation
5858
Secure Operating SystemSecure Operating SystemStandardsStandards
If I want to buy a secure operating If I want to buy a secure operating system, how do I compare options?system, how do I compare options?
Use established standards for OS Use established standards for OS securitysecurity
Several standards existSeveral standards exist
5959
Some Security StandardsSome Security Standards
U.S. Orange BookU.S. Orange Book
European ITSECEuropean ITSEC
U.S. Combined Federal CriteriaU.S. Combined Federal Criteria
Common Criteria for Information Common Criteria for Information Technology Security EvaluationTechnology Security Evaluation
6060
The U.S. Orange BookThe U.S. Orange Book
The earliest evaluation standard for The earliest evaluation standard for trusted operating systemstrusted operating systems
Defined by the Department of Defined by the Department of Defense in the late 1970sDefense in the late 1970s
Now largely a historical artifactNow largely a historical artifact
6161
Purpose of the Orange BookPurpose of the Orange Book
To set standards by which OS security To set standards by which OS security could be evaluatedcould be evaluated
Fairly strong definitions of what features Fairly strong definitions of what features and capabilities an OS had to have to and capabilities an OS had to have to achieve certain levelsachieve certain levels
Allowing “head-to-head” evaluation of Allowing “head-to-head” evaluation of security of systemssecurity of systems
– – And specification of requirementsAnd specification of requirements
6262
The Common CriteriaThe Common Criteria
Modern international standards for Modern international standards for computer systems securitycomputer systems security
Covers more than just operating systemsCovers more than just operating systems
Design based on lessons learned from Design based on lessons learned from earlier security standardsearlier security standards
Lengthy documents describe the Common Lengthy documents describe the Common CriteriaCriteria
6363
Basics of Common CriteriaBasics of Common CriteriaApproachApproach
Highly detailed methodology for Highly detailed methodology for specifying specifying
1. What security goals a system has1. What security goals a system has
2. What environment it operates in2. What environment it operates in
3. What mechanisms it uses to 3. What mechanisms it uses to
achieve its security goalsachieve its security goals
4. Why anyone should believe it does 4. Why anyone should believe it does soso
6464
Logging and AuditingLogging and Auditing
An important part of a complete An important part of a complete security solutionsecurity solution
Practical security depends on knowing Practical security depends on knowing what is happening in your systemwhat is happening in your system
Logging and auditing is required for Logging and auditing is required for that purposethat purpose
6565
LoggingLogging No security system will stop all No security system will stop all
attacksattacks – – Logging what has happened is vitalLogging what has happened is vital to dealing with the holesto dealing with the holes
Logging also tells you when someone Logging also tells you when someone is trying to break inis trying to break in
– – Perhaps giving you a chance to closePerhaps giving you a chance to close possible holespossible holes
6666
AuditingAuditing
Security mechanisms are greatSecurity mechanisms are great
– – If you have proper policies to use themIf you have proper policies to use them
Security policies are greatSecurity policies are great
– – If you follow themIf you follow them
For practical systems, proper policies and For practical systems, proper policies and consistent use are a major security consistent use are a major security problemproblem
6767
AuditingAuditing A formal (or semi-formal) process of A formal (or semi-formal) process of
verifying system securityverifying system security
A requirement if you really want your A requirement if you really want your systems to run securelysystems to run securely
Auditing should be done Auditing should be done – – Periodically Periodically – – After making major system changesAfter making major system changes – – When problems ariseWhen problems arise
6868
What Does an Audit Cover?What Does an Audit Cover?
Conformance to policyConformance to policy Review of control structuresReview of control structures Examination of audit trail (logs)Examination of audit trail (logs) User awareness of securityUser awareness of security Physical controlsPhysical controls Software licensing and intellectual Software licensing and intellectual
property issuesproperty issues
Top Related