Recitation 8 (Nov. 1)

Outline Lab 5 hints Virtual Memory

Reminder Shell Lab:

Due THIS Thursday

TA: Kun Gao

Modified from

Minglong Shao’sRecitation, Fall 2004

Lab 5

A tiny shell (tsh) with job control & I/O redirection Key points:

Understand process tree Executing programs Reap all child processes Handle SIGCHLD, SIGTSTP, SIGINT Avoid race hazard I/O redirection

Process tree for shell

Fore-ground

job

Back-groundjob #1

Back-groundjob #2

Shell

Child Child

pid=10pgid=10

Foregroundprocess group 20

Backgroundprocess group 32

Backgroudprocess group 40

pid=20pgid=20

pid=32pgid=32

pid=40pgid=40

pid=21pgid=20

pid=22pgid=20

Each job has a unique process group idint setpgid(pid_t pid, pid_t pgid);setpgid(0, 0);

Process tree for shell (part 2)

Fore-ground

job

Back-groundjob #1

Back-groundjob #2

tsh

Child Child

pid=10pgid=10

Foregroundprocess group 20

Backgroundprocess group 32

Backgroudprocess group 40

pid=20pgid=20

pid=32pgid=32

pid=40pgid=40

pid=21pgid=20

pid=22pgid=20

UNIXshell

pid=5pgid=5

Foreground jobreceives SIGINT, SIGTSTP, when you type ctrl-c, ctrl-z

Forward signals

int kill(pid_t pid, int sig)pid > 0: send sig to specified processpid = 0: send sig to all processes in same grouppid < -1: send sig to group -pid

Execute program

int execve(const char *fname,

char *const argv[],

char *const envp[]);

Examples:

execve(“/bin/ls”, NULL, NULL);

execve(“./mytest”, argv, envp);

Reaping child processwaitpid(pid_t pid, int *status, int options)

pid: wait for child process with pid (process id or group id)-1: wait for any child process

status: tell why child terminated

options:WNOHANG: return immediately if no children zombiedWUNTRACED: report status of terminated or stopped children

In Lab 5, usewaitpid(-1, &status, WNOHANG|WUNTRACED)

In sigchld_handler():while ((c_pid = waitpid(…)) > 0) { … }

Reaping child process (part 2) int status;waitpid(pid, &status, WNOHANG|WUNTRACED)

What to check in sigchld_handler: WIFEXITED(status): child exited normally (the child

finished, and quit normally on its own) WEXITSTATUS(status): returns code when child

exits

WIFSIGNALED(status): child exited because a signal is not caught (SIGINT, or typing CTRL-C) WTERMSIG(status): gives the terminating signal

number

WIFSTOPPED(status): child is stopped by the receipt of a signal (SIGSTOP, or typing CTRL-Z) WSTOPSIG(status): gives the stop signal number

Reaping child process (part 3)

Where to put waitpid(…)

In sigchld_handler(): for both tsh should still wait for fg job to complete, how?

Busy wait for foreground job

In eval():

if(fork() != 0) { /* parent */

addjob(…);

while(fg process still alive){

/* do nothing */

}

}

Sleep

In eval():

if(fork() != 0) { /* parent */

addjob(…);

while(fg process still alive){

sleep(1);

}

}

Race hazard

A data structure is shared by two pieces of code that can run concurrently

Different behaviors of program depending upon how the schedule interleaves the execution of code.

An example of race hazard

sigchld_handler() { while ((pid = waitpid(…)) > 0){ deletejob(pid); }}eval() { pid = fork(); if(pid == 0) { /* child */ execve(…); } /* parent */ /* signal handler may run BEFORE addjob()*/ addjob(…);}

An okay schedule

Shell Signal Handler Child

fork()

addjob()

execve()

exit()

sigchld_handler()

deletejobs()

time

A problematic schedule

Shell Signal Handler Child

fork()

execve()

exit()

sigchld_handler()

deletejobs()

time

addjob()

Job added to job list after the signal handler tried to delete it!

Solution: blocking signalssigchld_handler() { pid = waitpid(…); deletejob(pid);}eval() { sigprocmask(SIG_BLOCK, …) pid = fork(); if(pid == 0) { /* child */ sigprocmask(SIG_UNBLOCK, …) execve(…); } /* parent */ /* signal handler might run BEFORE addjob() */ addjob(…); sigprocmask(SIG_UNBLOCK, …)}

Solution: blocking signals (part 2) What should our block set be?

SIGSTP SIGINT SIGCHLD

I/O redirection Do it before call execve() in child processeval() { pid = fork(); if(pid == 0) { /* child */ /* Redirect I/O */ if (redirect_input) dup2(…);/* redirect STDIN */

if (redirect_output) dup2(…);/* redirect STDOUT */ execve(…); } addjob(…);}

dup2(int oldfd, int newfd) Covered in Chapter 11oldfd: old file descriptornewfd: new file descriptor

Dup2 makes newfd be a copy of the oldfd (i.e. operations on newfd is done on oldfd instead)

Some examples:Get input from in_fd instead of standard inputdup2(in_fd, STDIN_FILENO);

Print output to out_fd instead of standard outputdup2(out_fd, STDOUT_FILENO);

Reminders Some important system calls:

fork(), execve(), waitpid(), sigprocmask(), setpgid(), kill() …

Check man pages for details about system calls man 2 kill

Check return values of all system calls

STEP by STEP Test your shell by typing commands first Each trace worth the same (work on easy ones

first!) Start now!

Virtual memory

One of the most important concepts in CS Why use virtual memory (VM)

Use RAM as a cache for disk Easier memory management Access protection Enable “partial swapping” Share memory efficiently

Virtual memory

CPU

0:1:

N-1:

Memory

0:1:

P-1:

Page Table

Disk

VirtualAddresses

PhysicalAddresses

Page Page table Page hits, Page faults Demand Paging

Per Process:

Conceptual address translation

Higher bits of address (page number) mapped from virtual addr. to physical addr.

Lower bits (page offset) stay the same.

virtual page number (VPN) page offset virtual address

physical page number (PPN) page offset physical address

0p–1

address translation

pm–1

n–1 0p–1p

Virtual Memory: up to 2n -1 bytes

Physical Memory: up to 2m -1 bytes

Address translation through page table Translation

Separate (set of) page table(s) per process VPN forms index into page table (points to a page table entry)

virtual page number (VPN) page offset

virtual address

physical page number (PPN) page offset

physical address

0p–1pm–1

n–1 0p–1ppage table base register

if valid=0then pagenot in memory

valid physical page number (PPN)access

VPN acts astable index

virtual page number (VPN) page offset

virtual address

physical page number (PPN) page offset

physical address

0p–1pm–1

n–1 0p–1ppage table base register

if valid=0then pagenot in memory

valid physical page number (PPN)access

VPN acts astable index

Address translation with TLB

virtual addressvirtual page number page offset

physical address

n–1 0p–1p

valid physical page numbertag

valid tag data

data=

cache hit

tag byte offsetindex

=

TLB hit

TLB

Cache

. ..

Integrating VM and cache

Most caches are physically addressed Why?

Perform address translation before cache lookup, could potentially go to memory =*(

TLB keeps things manageable

TLB

Cache/Memory

1. VA 4. PAMMU/

Translation

2. VPN 3. PTE

5. Data

CPU

Integrating VM and cache (TLB hit)

Most caches are physically addressed Why?

Perform address translation before cache lookup, could potentially go to memory =*(

TLB keeps things manageable

TLB

Cache/Memory

1. VA 4. PAMMU/

Translation

2. VPN 3. PTE

5. Data

CPU

Integrating VM and cache (TLB miss)

TLB miss, but page table is in cache/memory

TLB

Cache/Memory

1. VA3. PTEA (PTBP + VPN)

MMU/Translation

2. VPN

6. Data

CPU

4. PTE5. PA

What is the worst case mem read (most delay)? TLB miss, page fault on page table, then page

fault on memory read

TLB

Cache/Memory

1. VA3. PTEA (PTBP + VPN)

MMU/Translation

2. VPN

6. Data

CPU

4. PTE5. PA

DISK

Page Fault/Page Table

Page Fault/Mem Read

Example

Understand end-to-end addr. translation 20-bit virtual addresses 18-bit physical addresses Page size is 1024 bytes TLB is 2-way associative with 16 total entries

Part 1

A. Virtual address

B. Physical address

16 15 14 13 12 11 10 9 8 7 6 5 4 31718 2 1 019

VPN VPO

TLBT TLBI

16 15 14 13 12 11 10 9 8 7 6 5 4 317 2 1 0

PPN PPO

Example: TLB and page table

Part 2

Virtual address 0x04AA4 A. 04AA4 = 0000 0100 1010 1010 0100 B. Address translation

C. Physical address

01 1010 0010 1010 0100

parameter Value Parameter Value

VPN 0x012 TLB hit? Y

TLB Index 0x2 Page fault? N

TLB Tag 0x02 PPN 0x68

Part 2

Virtual address 0x78E6 A. 078E6 = B. Address translation

C. Physical address

parameter Value Parameter Value

VPN 0x01E TLB hit? N

TLB Index 0x6 Page fault? N

TLB Tag 0x03 PPN 0x57

0000 0111 1000 1110 0110

01 0101 1100 1110 0110

End-to-end address translation

Section 10.6.4: a concrete example Read carefully and solve practice problem 10.4