CS194-24 OS Structure (Con’t): The Linux Kernel Advanced ...

CS194-24Advanced Operating Systems

Structures and Implementation Lecture 4

OS Structure (Con’t)Modern Architecture

February 6th, 2013Prof. John Kubiatowicz

http://inst.eecs.berkeley.edu/~cs194-24

Lec 4.22/6/13 Kubiatowicz CS194-24 ©UCB Fall 2013

Goals for Today

• OS Structure (Con’t): The Linux Kernel• Modern Computer Architecture

Interactive is important!Ask Questions!

Note: Some slides and/or pictures in the following areadapted from slides ©2013


Recall: OS Resources – at the center of it all!• What do modern OSs do?

– Why all of these pieces running together?

– Is this complexity necessary?• Control of Resources

– Access/No Access/Partial Access» Check every access to see if it

is allowed– Resource Multiplexing

» When multiple valid requests occur at same time – how to multiplex access?

» What fraction of resource can requester get?

– Performance Isolation» Can requests from one entity

prevent requests from another?• What or Who is a requester???

– Process? User? Public Key?– Think of this as a “Principle”

Access Control and Multiplexing

IndependentRequesters


Recall: Microkernel Structure

• Moves as much from the kernel into “user” space– Small core OS running at kernel level– OS Services built from many independent user-level processes– Communication between modules with message passing

• Benefits:– Easier to extend a microkernel– Easier to port OS to new architectures– More reliable (less code is running in kernel mode)– Fault Isolation (parts of kernel protected from other parts)– More secure

• Detriments:– Performance overhead severe for naïve implementation

Figure ©Wikipedia

Mon

olithic

Kern

el

Microkernel


Recall: Modules-based Structure

• Most modern operating systems implement modules– Uses object-oriented approach– Each core component is separate– Each talks to the others over known interfaces– Each is loadable as needed within the kernel

• Overall, similar to layers but with more flexible


Recall: ExoKernel

• Provide extremely thin layer to present hardware resources directly to users– As little abstraction as possible– Only Protection and Multiplexing of resources

• On top of Exokernel is the LibraryOS which provides much of the traditional functionality of OS in Library

• Low-level abstraction layer


Concurrency

• “Thread” of execution– Independent Fetch/Decode/Execute loop– Operating in some Address space

• Uniprogramming: one thread at a time– MS/DOS, early Macintosh, Batch processing– Easier for operating system builder– Get rid concurrency by defining it away– Does this make sense for personal computers?

• Multiprogramming: more than one thread at a time– Multics, UNIX/Linux, OS/2, Windows NT/2000/XP, Mac OS X

– Often called “multitasking”, but multitasking has other meanings (talk about this later)

• ManyCore Multiprogramming, right?


The Basic Problem of Concurrency

• The basic problem of concurrency involves resources:– Hardware: single CPU, single DRAM, single I/O devices– Multiprogramming API: users think they have exclusive access to shared resources

• OS Has to coordinate all activity– Multiple users, I/O interrupts, …– How can it keep all these things straight?

• Basic Idea: Use Virtual Machine abstraction– Decompose hard problem into simpler ones– Abstract the notion of an executing program– Then, worry about multiplexing these abstract machines

• Dijkstra did this for the “THE system”– Few thousand lines vs 1 million lines in OS 360 (1K bugs)


FetchExec

R0…

R31F0…

F30PC

…Data1Data0

Inst237Inst236

…Inst5Inst4Inst3Inst2Inst1Inst0

Addr 0

Addr 232-1

Recall (61C): What happens during execution?

• Execution sequence:– Fetch Instruction at PC – Decode– Execute (possibly using registers)– Write results to registers/mem– PC = Next Instruction(PC)– Repeat

PCPCPCPC


How can we give the illusion of multiple processors?

CPU3CPU2CPU1

Shared Memory

• Assume a single processor. How do we provide the illusion of multiple processors?– Multiplex in time!

• Each virtual “CPU” needs a structure to hold:– Program Counter (PC), Stack Pointer (SP)– Registers (Integer, Floating point, others…?)

• How switch from one CPU to the next?– Save PC, SP, and registers in current state block– Load PC, SP, and registers from new state block

• What triggers switch?– Timer, voluntary yield, I/O, other things

CPU1 CPU2 CPU3 CPU1 CPU2

Time


Properties of this simple multiprogramming technique

• All virtual CPUs share same non-CPU resources– I/O devices the same– Memory the same

• Consequence of sharing:– Each thread can access the data of every other thread (good for sharing, bad for protection)

– Threads can share instructions(good for sharing, bad for protection)

– Can threads overwrite OS functions? • This (unprotected) model common in:

– Embedded applications– Windows 3.1/Machintosh (switch only with yield)– Windows 95—ME? (switch with both yield and timer)


What needs to be saved in Modern X86?64-bit Register Set Traditional 32-bit subset

EFLAGS Register


Modern Technique: SMT/Hyperthreading• Hardware technique

– Exploit natural propertiesof superscalar processorsto provide illusion of multiple processors

– Higher utilization of processor resources

• Can schedule each threadas if were separate CPU– However, not linearspeedup!

– If have multiprocessor,should schedule eachprocessor first

• Original technique called “Simultaneous Multithreading”– See http://www.cs.washington.edu/research/smt/– Alpha, SPARC, Pentium 4 (“Hyperthreading”), Power 5


Chip-scale features of SandyBridge

• Significant pieces:– Four OOO cores with Hyperthreads

» New Advanced Vector eXtensions (256-bit FP)

» AES instructions» Instructions to help with Galois-Field mult» 4 -ops/cycle

– Integrated GPU– System Agent (Memory and Fast I/O)– Shared L3 cache divided in 4 banks– On-chip Ring bus network

» Both coherent and non-coherent transactions» High-BW access to L3 Cache

• Integrated I/O– Integrated memory controller (IMC)

» Two independent channels of DDR3 DRAM– High-speed PCI-Express (for Graphics cards)– DMI Connection to SouthBridge (PCH)


How to protect threads from one another?

• Need three important things:1. Protection of memory

» Every task does not have access to all memory2. Protection of I/O devices

» Every task does not have access to every device3. Protection of Access to Processor:

Preemptive switching from task to task» Use of timer» Must not be possible to disable timer from

usercode


Program A

ddress Space

Recall: Program’s Address Space

• Address space the set of accessible addresses + state associated with them:– For a 32-bit processor there are 232 = 4 billion addresses

• What happens when you read or write to an address?– Perhaps Nothing– Perhaps acts like regular memory– Perhaps ignores writes– Perhaps causes I/O operation

» (Memory-mapped I/O)– Perhaps causes exception (fault)


Providing Illusion of Separate Address Space:Load new Translation Map on Switch

Prog 1VirtualAddressSpace 1

Prog 2VirtualAddressSpace 2

CodeDataHeapStack

CodeDataHeapStack

Data 2

Stack 1

Heap 1

OS heap & Stacks

Code 1

Stack 2

Data 1

Heap 2

Code 2

OS code

OS dataTranslation Map 1 Translation Map 2

Physical Address SpaceLec 4.182/6/13 Kubiatowicz CS194-24 ©UCB Fall 2013

X86 Memory model with segmentation


The Six x86 Segment Registers• CS - Code Segment• SS - Stack Segment

– “Stack segments are data segments which must be read/write segments. Loading the SS register with a segment selector for a nonwritable data segment generates a general-protection exception (#GP)”

• DS - Data Segment• ES/FS/GS - Extra (usually data) segment registers

– FS and GS used for thread-local storage/by glibc• The “hidden part” is like a cache so that segment

descriptor info doesn’t have to be looked up each time.


UNIX Process

• Process: Operating system abstraction to represent what is needed to run a single program– Originally: a single, sequential stream of execution in its own address space

– Modern Process: multiple threads in same address space!

• Two parts:– Sequential Program Execution Streams

» Code executed as one or more sequential stream of execution (threads)

» Each thread includes its own state of CPU registers» Threads either multiplexed in software (OS) or

hardware (simultaneous multithrading/hyperthreading)– Protected Resources:

» Main Memory State (contents of Address Space)» I/O state (i.e. file descriptors)

• This is a virtual machine abstraction– Some might say that the only point of an OS is to support a clean Process abstraction


ProcessControlBlock

How do we multiplex processes?• The current state of process held in a

process control block (PCB):– This is a “snapshot” of the execution and protection environment

– Only one PCB active at a time• Give out CPU time to different

processes (Scheduling):– Only one process “running” at a time– Give more time to important processes

• Give pieces of resources to different processes (Protection):– Controlled access to non-CPU resources– Sample mechanisms:

» Memory Mapping: Give each process their own address space

» Kernel/User duality: Arbitrary multiplexing of I/O through system calls


Modern “Lightweight” Process with Threads

• Thread: a sequential execution stream within process (Sometimes called a “Lightweight process”)– Process still contains a single Address Space– No protection between threads

• Multithreading: a single program made up of a number of different concurrent activities – Sometimes called multitasking, as in Ada…

• Why separate the concept of a thread from that of a process?– Discuss the “thread” part of a process (concurrency)– Separate from the “address space” (Protection)– Heavyweight Process Process with one thread


Single and Multithreaded Processes

• Threads encapsulate concurrency: “Active” component• Address spaces encapsulate protection: “Passive” part

– Keeps buggy program from trashing the system• Why have multiple threads per address space?


Preview: System-Level Control of x86

• Full support for Process Abstraction involves a lot of system-level state– This is state that can only be accessed in kernel mode!

– We will be talking about a number of these pieces as we go through the term…

• There is a tradeoff between amount of system state and cost of switching from thread to thread!


Additional system state for I/O

• Platform Controller Hub– Used to be

“SouthBridge,” but no “NorthBridge” now

– Connected to processor with proprietary bus» Direct Media

Interface– Code name “Cougar

Point” for SandyBridgeprocessors

• Types of I/O on PCH:– USB– Ethernet– Audio– BIOS support– More PCI Express (lower

speed than on Processor)– Sata (for Disks)

SandyBridgeSystem Configuration


Linux Structure


Layout of Linux Sources• Layout of basic linux sources:

kubitron@kubi(16)% lsarch/ drivers/ Kbuild modules.builtin samples/ usr/block/ firmware/ kernel/ modules.order scripts/ virt/COPYING fs/ lib/ Module.symvers security/ vmlinux*CREDITS include/ MAINTAINERS net/ sound/ vmlinux.ocrypto/ init/ Makefile README System.mapDocumentation/ ipc/ mm/ REPORTING-BUGS tools/kubitron@kubi(17)%

• Specific Directories:– arch: Architecture-specific source– block: Block I/O layer– crypto: Crypto API– Documentation: Kernel source Documentation– drivers: Device drivers– firmware: Device firmware needed to use certain drivers– fs: The VFS and individual filesystems– include: Kernel headers– init: Kernel boot and initialization– ipc: Interprocess Communication code– kernel: Core subsystems, such as the scheduler– lib: Helper routines– mm: Memory management subsystem and the vm– net: Networking subsystem– samples: Sample, demonstrative code– scripts: Scripts used to build the kernel– security: Linux Security Module– sound: Sound subsystem– usr: Early user-space code (called initramfs)– tools: Tools helpful for developing Linux– virt: Virtualization infrastructure


System Calls

• Challenge: Interaction Despite Isolation– How to isolate processes and their resources…

» While still permitting them to request help from the kernel» Letting them interact with resources while maintaining usage policies such as

security, QoS, etc– Letting processes interact with one another in a controlled way

» Through messages, shared memory, etc• Enter the System Call interface

– Layer between the hardware and user-space processes– Programming interface to the services provided by the OS

• Mostly accessed by programs via a high-level Application Program Interface (API) rather than directly– Get at system calls by linking with libraries in glibc

• Three most common APIs are:– Win32 API for Windows– POSIX API for POSIX-based systems (including virtually all versions of UNIX,

Linux, and Mac OS X)– Java API for the Java virtual machine (JVM)

Call toprintf()

Printf() in the C library

Write() system call


Example of System Call usage• System call sequence to copy the contents of one

file to another file:

• Many crossings of the User/Kernel boundary!– The cost of traversing this boundary can be high


Example: Use strace to trace syscalls• prompt% strace wc production.log

execve("/usr/bin/wc", ["wc", "production.log"], [/* 52 vars */]) = 0brk(0) = 0x1987000mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff24b8f7000access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)open("/etc/ld.so.cache", O_RDONLY) = 3fstat(3, {st_mode=S_IFREG|0644, st_size=137151, ...}) = 0mmap(NULL, 137151, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff24b8d5000close(3) = 0open("/lib64/libc.so.6", O_RDONLY) = 3read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\355\241,0\0\0\0"..., 832) = 832fstat(3, {st_mode=S_IFREG|0755, st_size=1922112, ...}) = 0mmap(0x302ca00000, 3745960, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x302ca00000mprotect(0x302cb89000, 2097152, PROT_NONE) = 0mmap(0x302cd89000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x189000) = 0x302cd89000mmap(0x302cd8e000, 18600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x302cd8e000close(3) = 0mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff24b8d4000mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff24b8d3000mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff24b8d2000arch_prctl(ARCH_SET_FS, 0x7ff24b8d3700) = 0mprotect(0x302cd89000, 16384, PROT_READ) = 0mprotect(0x302c81f000, 4096, PROT_READ) = 0munmap(0x7ff24b8d5000, 137151) = 0brk(0) = 0x1987000brk(0x19a8000) = 0x19a8000open("/usr/lib/locale/locale-archive", O_RDONLY) = 3fstat(3, {st_mode=S_IFREG|0644, st_size=99158576, ...}) = 0mmap(NULL, 99158576, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff245a41000close(3) = 0stat("production.log", {st_mode=S_IFREG|0644, st_size=526550, ...}) = 0open("production.log", O_RDONLY) = 3read(3, "# Logfile created on Fri Dec 28 "..., 16384) = 16384open("/usr/lib64/gconv/gconv-modules.cache", O_RDONLY) = 4fstat(4, {st_mode=S_IFREG|0644, st_size=26060, ...}) = 0mmap(NULL, 26060, PROT_READ, MAP_SHARED, 4, 0) = 0x7ff24b8f0000close(4) = 0read(3, "m: cannot remove `/tmp/fixrepo/g"..., 16384) = 16384read(3, "a36de93203e0b4972c1a3c81904e': P"..., 16384) = 16384read(3, "xrepo/git-tess/gitolite-admin/.g"..., 16384) = 16384

Many repetitions of these readsread(3, "ixrepo/git-tess/gitolite-admin\n "..., 16384) = 16384read(3, "ite/redmine/vendor/plugins/redmi"..., 16384) = 16384read(3, "ited with positive recursionChec"..., 16384) = 16384read(3, "ting changes to gitolite-admin r"..., 16384) = 2262read(3, "", 16384) = 0fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 3), ...}) = 0mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff24b8ef000write(1, " 4704 28993 526550 production."..., 36 4704 28993 526550 production.log) = 36close(3) = 0close(1) = 0munmap(0x7ff24b8ef000, 4096) = 0close(2) = 0exit_group(0)


Example of Standard API


System Call Implementation• Typically, a number associated with each system call

– System-call interface maintains a table indexed according to these numbers

– The fact that the call is by “number”, is essential for security reasons!• The system call interface invokes intended system call in OS kernel

and returns status of the system call and any return values– Return value: often a long (integer)

» Return of zero is usually a sign of success, but not always» Return of -1 is almost always reflects an error

– On error – return code placed into global “errno” variable» Can translate into human-readable errors with the “perror()” call

• The caller need know nothing about how the system call is implemented– Just needs to obey API and understand what OS will do as a result call– Most details of OS interface hidden from programmer by API

» Managed by run-time support library (set of functions built into libraries included with compiler)


API – System Call – OS Relationship


System Call Parameter Passing

• Often, more information is required than simply identity of desired system call– Exact type and amount of information vary according to OS

and call

• Three general methods used to pass parameters to the OS– Simplest: pass the parameters in registers

» In some cases, may be more parameters than registers– Parameters stored in a block, or table, in memory, and

address of block passed as a parameter in a register » This approach taken by Linux and Solaris

– Parameters placed, or pushed, onto the stack by the program and popped off the stack by the operating system

– Block and stack methods do not limit the number or length of parameters being passed


Parameter Passing via Table

• Kernel must always verify parameters passed to it by the user– Are parameters in a reasonable range?– Are memory addresses actually owned by the calling user (rather than bogus addresses)


Types of System Calls

• Process control– end, abort– load, execute– create process, terminate process– get process attributes, set process attributes– wait for time– wait event, signal event– allocate and free memory

– Dump memory if error– Debugger for determining bugs, single step execution

– Locks for managing access to shared data between processes


Types of System Calls

• File management– create file, delete file– open, close file– read, write, reposition– get and set file attributes

• Device management– request device, release device– read, write, reposition– get device attributes, set device attributes– logically attach or detach devices


Types of System Calls (Cont.)

• Information maintenance– get time or date, set time or date– get system data, set system data– get and set process, file, or device attributes

• Communications– create, delete communication connection– send, receive messages if message passing model to host name or process name» From client to server

– Shared-memory model create and gain access to memory regions

– transfer status information– attach and detach remote devices


Types of System Calls (Cont.)

• Protection– Control access to resources– Get and set permissions– Allow and deny user access


POSIX standard

• Portable Operating System Interface for UNIX» An attempt to standardize a “UNIXy” interface

• Conformance: IEEE POSIX 1003.1 and ISO/IEC 9945 – Latest version from 2008– Originally one document consisting of a core programming inteface –

now 19 separate docs– Many OSes provide “partial conformance” (including Linux)

• What does POSIX define?– POSIX.1: Core Services

» Process Creation and Control» Signals» Floating Point Exceptions, Segmentation/memory violations, illegal

instructions, Bus Erors» Timers» File and Directory Operations» Pipes» C Library (Standard C)» I/O Port Interface and Control » Process Triggers

– POSIX.1b: Realtime Extensions– POSIX.2: Shell and Utilities


POSIX (cont)• Process Primitives:

– fork, execl, execlp, execv, execve, execvp, wit, waitpid– _exit, kill, sigxxx, alarm, pause, sleep….

• Example file access primitives:– opendir, readdir, rewinddir, closedir, chdir, getcwd, open, creat, umask, link,

mkdir, unlink, rmdir, rename, stat, fstat, access, fchmod, chown, utime, ftruncate,pathconf,fpathconf

• I/O primitives:– pipe, dup, dup2, close, read, write, fcntl, lseek, fsync

• C-Language primitives:– abort, exit, fclose, fdopen, fflush, fgetc, fgets, fileno, fopen, fprintf, fputc,

fputs, fread, freopen, fscanf, fseek, ftell, fwrite, getc, getchar, gets, perror, printf, putc, putchar, puts, remove, rewind, scanf, setlocale, siglongjmp, sigsetjmp, tmpfile, tmpnam, tzset

• Synchronization:– sem_init, sem_destroy, sem_wait, sem_trywait, sem_post, pthread_mutex_init,

pthread_mutex_destroy, pthread_mutex_lock, pthread_mutex_trylock, pthread_mutex_unlock

• Memory Management– mmap, mprotect, msync, munmap

• ETC……• How to get information on a system call?

– Type “man callname”, i.e. “man open”– System calls are in section “2” of the man pages


Portability

• POSIX does provide some portability– But is still pretty high level– Does not specify file systems, network interfaces, power management, other important things

– Many variations in compilers, user programs, libraries, other build environment aspects

• UNIX Portability:– C-preprocessor conditional compilation– Conditional and multi-target Makefile Rules– GNU configure scripts to generate Makefiles– Shell environment variables (LD_LIBRARY_PATH, LD_PRELOAD, others)


Examples of Windows and Unix System Calls


Standard C Library Example

• C program invoking printf() library call, which calls write() system call


Summary• Processes have two parts

– Threads (Concurrency)– Address Spaces (Protection)

• Concurrency accomplished by multiplexing CPU Time:– Unloading current thread (PC, registers)– Loading new thread (PC, registers)– Such context switching may be voluntary (yield(), I/O

operations) or involuntary (timer, other interrupts)• Protection accomplished restricting access:

– Memory mapping isolates processes from each other– Dual-mode for isolating I/O, other resources

• System-Call interface– This is the I/O for the process “virtual machine”– Accomplished with special trap instructions which vector off

a table of system calls– Usually programmers use the standard API provided by the C

library rather than direct system-call interface• POSIX interface

– An attempt to standardize “unixy” Oses

CS194-24 OS Structure (Con’t): The Linux Kernel Advanced ...

Documents

Transcript of CS194-24 OS Structure (Con’t): The Linux Kernel Advanced ...