Computer Systems Principles Virtual Memory & Paging

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTS ASSACHUSETTS AAMHERST • MHERST • Department of Computer Science Department of Computer Science

Computer Systems PrinciplesVirtual Memory & Paging

Emery Berger and Mark CornerUniversity of Massachusetts

Amherst

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTS ASSACHUSETTS AAMHERST • MHERST • Department of Computer Science Department of Computer Science 2

Virtual vs. Physical Memory Apps don’t access physical memory

– Well, not directly Apps use virtual memory

– Addresses start at 0– One level of indirection– Address you see is not “real” address

Any ideas why?


Memory Pages Programs use memory as individual bytes OS manages groups of bytes: pages

– typically 4kB, 8kB– Why? (think Tetris with all squares)– Applies this to virtual and physical memory

• Physical pages usually called frames

A


Mapping Virtual to Physical

Note this is simplified and the data here includes the heap, not the typical data segment…


Why Virtual Memory? Why?

– Simpler• Everyone gets illusion

of whole address space

– Isolation• Every process

protected from every other

– Optimization • Reduces space

requirements


Typical Virtual Memory Layout Some things grow

– Must leave room! Mmap and heap spaces

– Mmap increases mmap– Brk increases heap

Other layouts possible


Quick Quiz! Are arrays contiguous in memory?

– Physical memory?– Virtual memory?

Where does the code for a program live?


Memory Management Unit Programs issue loads and stores What kind of addresses are these? MMU Translates virtual to physical addresses

– Maintains page table (big hash table):– Almost always in HW… Why?

MMU

Physical

Address

Virtual Addres

s

Program

Memory

Page Table


Page Tables Table of translations

– virtual pages -> physical pages One page table per process One page table entry per virtual page How?

– Programs issue virtual address– Find virtual page (how?)– Lookup physical page, add offset


Page Table Entries Do all virtual pages -> physical page?

– Valid and Invalid bits PTEs have lots of other information

– For instance some pages can only be read


Address Translation Powers of 2:

– Virtual address space: size 2^m– Page size 2^n

Page#: High m-n bits of virtual address Lower n bits select offset in page


Quick Activity How much mem does a page table need?

– 4kB pages, 32 bit address space– page table entry (PTE) uses 4 bytes

2^32/2^12*4=2^22 bytes=4MB– Is this a problem?– Isn’t this per process?– What about a 64 bit address space?

Any ideas how to fix this?


Multi-Level Page Tables Use a multi-level page table

A A

A

Level 0 Table

Level 1 Table

Level 1 Table


Quick Activity How much mem does a page table need?

– 4kB pages, 32 bit address space– Two level page table– 20bits = 10 bits each level– page table entry (PTE) uses 4 bytes– Only first page of program is valid

2^10*4+2^10*4=2^13 bytes=8kB

Isn’t this slow?


Translation Lookaside Buffer (TLB) TLB: fast, fully associative memory

– Caches page table entries– Stores page numbers (key) and frame (value)

in which they are stored Assumption: locality of reference

– Locality in memory accesses =locality in address translation

TLB sizes: 8 to 2048 entries– Powers of 2 simplifies translation

of virtual to physical addresses


Virtual Memory is an Illusion! How much memory does a process have?

– Do all processes have this? Key idea: use RAM as cache for disk

– OS transparently moves pages Requires locality:

– Working set must fit in RAM• memory referenced recently

– If not: thrashing (nothing but disk traffic)


Paging


A

B

AB

Paging + Locality Most programs obey

90/10 “rule”– 90% of time spent

accessing 10% of memory

Exploit this rule:– Only keep “live” parts

of process in memory


Key Policy Decisions Two key questions: (for any cache):

– When do we read page from disk?– When do we write page to disk?


Reading Pages Read on-demand:

– OS loads page on its first reference– May force an eviction of page in RAM– Pause while loading page = page fault

Can also perform pre-paging:– OS guesses which page will next be needed,

and begins loading it Most systems just do demand paging What about writes?


Demand Paging On every reference, check if page is in memory

(resident bit in page table)– Who is doing this?

If not: trap to OS– How does this work in HW?

OS:– Selects victim page to be replaced– Writes victim page if necessary, marks non-resident– Begins loading new page from disk– OS can switch to another process

• more on this later


Swap Space Swap space = where victim pages go

– Partition or special file reserved on disk

Size of reserved swap space limits what?


Tricks with Page Tables Do all pages of memory end up in swap? Parts of address space mapped into files

– see man pages for mmap


Overview A Day in the Life of a Page

– Allocation– Use– Eviction– Reuse

Terms: Resident and Non-resident


char * x = new char[16];

25

virtual memory layout

Allocate some memory

0x40001000

0x40001040 → 0x4000104F

A Day in the Life of a Page


char * x = new char[16];

26


Update page tables

0x40001000

0x40001040 → 0x4000104F

physicalmemory

layout



strcpy(x, “hello”);

27


Write contents – dirty page

0x40001000

0x40001040 → 0x4000104F

physicalmemory

layout




Other processes fill up memory…

physicalmemory

layout




Forcing our page to be evicted (paged out)

physicalmemory

layout

swapspace(disk)




Now page nonresident & protected

physicalmemory

layout

swapspace(disk)



y[0] = x[0];

31


Touch page – swap it in

0x40001000

0x40001040 → 0x4000104F

physicalmemory

layout

swapspace(disk)



y[0] = x[0];

32


Touch page – swap it in

0x40001000

0x40001040 → 0x4000104F

physicalmemory

layout

swapspace(disk)



Tricks with Page Tables: Sharing Paging allows sharing

of memory across processes– Reduces memory

requirements Shared stuff includes

code, data– Code typically R/O


Tricks with Page Tables: COW Copy on write (COW)

– Just copy page tables– Make all pages read-only

What if process changes mem?

All processes are created this way!


Allocating Pages ultimately from sbrk or mmap Sbrk increases # of valid pages

– Increases the heap Mmap maps address space to file

– Increases the mmap space Oddity:

– Allocators can use either mmap or brk to get pages– You will use mmap

What does mmap /dev/zero mean?– Think about COW


Overview

Replacement policies– Comparison


A, B, C, D, E, F, G, H, I, J, A...

size of available memory

Cost of Paging Usually in algorithms, we pick algorithm

with best asymptotic worst-case– Paging: worst-case analysis useless!– Easy to construct adversary:

every page requires page fault


A A, B, C, D, E, F, G, H, I, J, A...


Cost of Paging Worst-case analysis – useless

– Easy to construct adversary example:every page requires page fault


A B A, B, C, D, E, F, G, H, I, J, A...





A B C A, B, C, D, E, F, G, H, I, J, A...





A B C D A, B, C, D, E, F, G, H, I, J, A...





A B C D E A, B, C, D, E, F, G, H, I, J, A...





F B C D E A, B, C, D, E, F, G, H, I, J, A...





F G H I J A, B, C, D, E, F, G, H, I, J, A...





Optimal Replacement (MIN/OPT) Evict page accessed furthest in future

– Optimal page replacement algorithm• Invented by Belady (“MIN”), a.k.a. “OPT”

Provably optimal policy– Just one small problem...

Requires predicting the future– Useful point of comparison

• How far from optimal


sequence of page accesses

contents of page frames

Quick Activity: OPT

Page faults: 5


Least-Recently Used (LRU) Evict page not used in longest time

(least-recently used)– Approximates OPT

• If recent past ≈ predictor of future– Variant used in all real operating systems


Quick Activity: LRU example

Page faults: ?


LRU example

Page faults: 5


LRU, example II

Page faults: ?


LRU, example II

Page faults: 12!– Loop: well-known worst-case for LRU


A, B, C, D, A, B, C, D, ...


Most-Recently Used (MRU) Evict most-recently used page Shines for LRU’s worst-case: loop that exceeds RAM size


A A, B, C, D, A, B, C, D, ...




A B A, B, C, D, A, B, C, D, ...




A B C A, B, C, D, A, B, C, D, ...




A B D A, B, C, D, A, B, C, D, ...




FIFO First-in, first-out: evict oldest page As competitive as LRU, but

performs miserably in practice!– Ignores locality– Suffers from Belady’s anomaly:

• More memory can mean more paging!– LRU & similar algs. do not

• Stack algorithms – more memory means ≥ hits


Virtual Memory in Reality Implementing exact LRU Approximating LRU

– Hardware Support– Clock– Segmented queue

Multiprogramming– Global LRU– Working Set


A, B, C, B, C, C, D

Implementing Exact LRU On each reference, time stamp page When we need to evict: select oldest page

= least-recently used


A1

A, B, C, B, C, C, D




A1

B2

A, B, C, B, C, C, D




A1

B2

C3

A, B, C, B, C, C, D




A1

B4

C3

A, B, C, B, C, C, D




A1

B4

C5

A, B, C, B, C, C, D




A1

B4

C6

A, B, C, B, C, C, D




A1

B4

C6

A, B, C, B, C, C, DD7

LRU page

How should we implement this?




Implementing Exact LRU Could keep pages in order

– optimizes eviction– Priority queue:

update = O(log n), eviction = O(log n)

Optimize for common case!– Common case: hits, not misses– Hash table:

update = O(1), eviction = O(n)


Cost of Maintaining Exact LRU Hash tables: too expensive

– On every reference:• Compute hash of page address• Update time stamp

– Unfortunately: 10x – 100x more expensive!


Cost of Maintaining Exact LRU Alternative: doubly-linked list

– Move items to front when referenced– LRU items at end of list– Still too expensive

• 4-6 pointer updates per reference

Can we do better?


A1

B1

C1

A, B, C, B, C, C, D

Hardware Support Maintain reference bits for every page

– On each access, set reference bit to 1– Page replacement algorithm periodically

resets reference bits


A0

B0

C0

A, B, C, B, C, C, D

reset reference bits





A0

B1

C0

A, B, C, B, C, C, D





A0

B1

C1

A, B, C, B, C, C, D





A0

B1

C1

A, B, C, B, C, C, DD1



resets reference bits– Evict page with reference bit = 0

–

Cost per miss = O(n)


B1

C1

A1

D1

A, B, C, D, B, C, E, F, C, G

The Clock Algorithm Variant of FIFO & LRU Keep frames in circle On page fault, OS:

– Checks reference bit of next frame

– If reference bit = 0, replace page, set bit to 1

– If reference bit = 1, set bit to 0, advance pointer to next frame


B1

C1

A1

D1

A, B, C, D, B, C, E, F, C, G






B1

C1

A0

D1

A, B, C, D, B, C, E, F, C, G






B0

C1

A0

D1

A, B, C, D, B, C, E, F, C, G






B0

C0

A0

D1

A, B, C, D, B, C, E, F, C, G






B0

C0

A0

D0

A, B, C, D, B, C, E, F, C, G






B0

C0

A0

D0

A, B, C, D, B, C, E, F, C, G

E1






B0

C0

A0

D0

A, B, C, D, B, C, E, F, C, G

E0






B0

C0

A0

D0

A, B, C, D, B, C, E, F, C, G

E0

F1






B0

C0

A0

D0

A, B, C, D, B, C, E, F, C, G

E0

F1

C1






B0

C0

A0

D0

A, B, C, D, B, C, E, F, C, G

E0

F0

C1






B0

C0

A0

D0

A, B, C, D, B, C, E, F, C, G

E0

F0

C1C0






B0

C0

A0

D0

A, B, C, D, B, C, E, F, C, G

E0

F0

C1C0

G1






clock exact LRU

Segmented Queue Real systems: segment queue into two

– approximate for frequently-referenced pages• e.g., first 1/3 page frames – fast

– exact LRU for infrequently-referenced pages• last 2/3 page frames; doubly-linked list – precise

How do we move between the two?


Enhancing Clock Recall: don’t write back unmodified pages

– Idea: favor eviction of unmodified pages– Extend hardware to keep another bit:

modified bit Total order of tuples: (ref bit, mod bit)

– (0,0), (0,1), (1,0), (1,1)– Evict page from lowest nonempty class


Replacement, Enhanced Clock OS scans at most three times

– Page (0,0) – replace that page– Page (0,1) – write out page, clear mod bit– Page (1,0), (1,1) – clear reference bit

Passes:– all pages (0,0) or (0,1)– all pages (0,1) - write out pages– all pages (0,0) – replace any page

Fast, but still coarse approximation of LRU


Multiprogramming & VM Multiple programs compete for memory

– Processes move memory from and to disk– Pages needed by one process may get

squeezed out by another process– thrashing - effective cost of memory access

= cost of disk access = really really bad Must balance memory across processes

– avoid thrashing


Global LRU Put all pages from all procs in one pool

– Manage with LRU (Segmented Queue)– Used by Linux, BSD, etc.

Advantages:– Easy

Disadvantages:– Many


Global LRU Disadvantages No isolation between processes

– One process touching many pages can force another process’ pages to be evicted

Priority ignored, or inverted– All processes treated equally

Greedy (or wasteful) processes rewarded– Programs with poor locality squeeze out

those with good locality– Result: more page faults


Global LRU Disadvantages “Sleepyhead” problem

– Intermittent, important process– Every time it wakes up – no pages! – back to sleep...– Think ntpd

Susceptible to denial of service– Non-paying “guest”, lowest priority, marches over

lots of pages – gets all available memory

Alternatives?– Pinning?


Working Set Denning: Only run processes whose

working set fits in RAM– Other processes: deactivate (suspend)

Classical definition:working set = pages touched in last references

Provides isolation– Process’s reference behavior only affects

itself


Working Set Problems Algorithm relies on key parameter,

– How do we set ?– Is there one correct ?

• Different processes have different timescales over which they touch pages

Not acceptable (or necessarily possible) to suspend processes altogether

Not really used– Very rough variant used in Windows


Solution: CRAMM New VM management alg:

Cooperative Robust Automatic Memory Management[OSDI 2006, Yang et al.]

Redefine working set size =pages required to spend < n% time paging– CRAMM default = 5%


d e f g h i j k l m n c k l m ncb c d e f g h i j k l m n c k l m n

aba abc defghijklmn abc defghijklm n abdefghijckl n m abc defghijk mn l abc defghijlmn k abdefghijklmn c

4

n

321 1

Memory reference sequence

LRU QueuePages in Least Recently

Used order

Hit Histogram

Fault Curve

0000000000 0000

m n

1 14

l m nk l m nc k l m na b c d e f g h i j k l m n c k l m n

5

1

1 14114

Associated with each LRU position

pages

faults

Calculating WSS w.r.t 5%

[ ]∑∞

+=i

i ihistfault 1


Computing hit histogram Not possible in standard VM:

– Global LRU queues– No per process/file information or control

• Difficult to estimate app’s WSS / available memory

CRAMM VM:– Per process/file page management:

• Page list: Active, Inactive, Evicted• Add & maintain histogram


Active (CLOCK) Inactive (LRU) Evicted (LRU)

Major fault

EvictedRefill & Adjustment

Minor fault

Pages protected by turning off permissions (minor fault)

Pages evicted to disk. (major fault)

Header

Page Des

AVL node

HistogramPages

faults

Managing pages per process


Buffer

Active (CLOCK) Inactive (LRU) Evicted (LRU)

Pages protected by turning off permissions (minor fault)

Pages evicted to disk. (major fault)

Header

Page Des

AVL node

HistogramPages

faults

control the boundary: 1% of execution time

Controlling overhead


Competitive Analysis I removed this slide because I don’t get it. This is

the worst case analysis…

Instead of worst-case, Compare replacement policy (OPT)– How much worse is algorithm than optimal?

Result: LRU & FIFO both “k-competitive”– k = size of queue– Can incur k times more misses than OPT


FIFO & Belady’s Anomaly


LRU: No Belady’s Anomaly

Why no anomaly for LRU?

Computer Systems Principles Virtual Memory & Paging

Documents

Transcript of Computer Systems Principles Virtual Memory & Paging