CSCC69: Operating Systems

Assignment 2Some slides are borrowed from CSCC69 offered in winter 2012

Virtual Memory

• Allowing a program to be designed as though there is only one kind of memory, "virtual" memory, which behaves like directly addressable read/write memory (RAM).– Need to manage what is in

memory (TLB, page table, coremap)

– Swap pages on request

Virtual Memory

• Translation Look aside Buffer (TLB): is a cache that memory management hardware uses to improve virtual address translation speed.– The search key is the

virtual address and the search result is a physical address.

Virtual Memory

• TLB hit: if the requested address is present in the TLB; the retrieved physical address can be used to access memory.

• TLB miss: if the requested address is not in the TLB; the translation proceeds by looking up the page table.

TLB Miss

• When a page is requested but not in memory?– Page fault

• When there is no more space in main memory to bring in pages?– Replace, evict

Core Map

• The core map is a table containing an entry for every physical page frame in the system.

• For each page frame, a core map entry keeps track of:– Is the page frame allocated or free?– Which address space is using this page?– Which virtual page number within space?– Plus possibly other flags, such as whether the page

is currently locked in memory for I/O purposes.

Swap Space

• Swap space is used when the amount of physical memory (RAM) is full.

• If the system needs more memory resources and the RAM is full, inactive pages in memory are moved to the swap space.

• Swap space is located on hard drives, which have a slower access time than physical memory.

OS 161 Page Tables

• OS/161 paging uses virtual memory objects• struct vm_object defined in src/kern/include/vmprivate.h

– A VM object defines a region of an address space– Contains a base virtual address and an array of pages – Redzone - A possible guard band against other vm_objects

struct vm_object { struct lpage_array *vmo_lpages; vaddr_t vmo_base; size_t vmo_lower_redzone;};

OS 161 Page Tables• Each VM object has an array of logical pages (lpages), one for each

virtual page in the region• lpage stores where the page is in physical memory (lp_paddr), and

where the page is stored in swap when not in main memory (lp_swapaddr)– If the page is not in RAM, lp_paddr is INVALID_PADDR.– If no swap has been allocated, lp_swapaddr is INVALID_SWAPADDR.

• Low bits of lp_addr used to hold flags (DIRTY, PINNED) • Read comments in src/kern/include/vmprivate.h

struct lpage { volatile paddr_t lp_paddr; off_t lp_swapaddr; struct spinlock lp_spinlock;

};

Lpage Operations

• lpage_create - creates an lpage object. • lpage_destroy - deallocates an lpage, releases

any RAM or swap pages involved • lpage_lock/unlock - for exclusive access to an

lpage • lpage_copy - clones an lpage, including the

contents • lpage_zerofill - materializes an lpage and zero-

fill it

Coremap

• Logical pages are nice, but we ultimately need to work with physical memory

• Need to keep track of physical pages • Coremap contains an entry per physical page

frame to indicate its status

Coremap

• Inverted page table: Maps pages in memory to their virtual addresses– It allows you to use the physical address to find the logical

page that inhabits it (NULL if empty)– Has bit flags that indicate if pages are kernel pages, pinned

(busy), etc.struct coremap_entry { struct lpage *cm_lpage;/* logical page we hold, or NULL */ ... /*flags*/ };

Coremap Functions

• coremap_alloc_one_page(lp, pin): called when a page is needed. None free? Call do_page_replace

• coremap_{pin, unpin}: pin/unpin a page• page_replace: returns number of the page to be

evicted – Replacement algorithm

• do_evict: performs the page eviction • do_page_replace: starting point for page

replacements

Coremap_entry vs. lpage

• Each lpage entry is a logical piece of memory– That memory may be in memory – It may also be in swap (on disk) – Each lpage points to the location of its data

• The coremap maps physical memory to virtual– When you need physical memory, consult the

coremap to see what memory is free – Each entry points to an lpage.

Coremap_entry vs. lpage

MIPS TLB Entry

• src/kern/arch/mips/include/tlb.h• TLB keeps track of mapping from virtual to physical pages• High-order word

– Virtual page number for lookup (TLBHI_VPAGE) : 20 bits (mask 0xffff000)– Also has 6 bits for PID; 6 bits unused in OS/161

• Low-order word– Physical page number (TLBLO_PPAGE) : 20 bits – Also has 4 status bits, and 8 unused bits – Eg: V for “valid”, D for “dirty” (“writable”/”referenced”)

TLB functions

• Can be found:– src/kern/arch/mips/include/tlb.h

• tlb_write• tlb_read• Tlb_probe

Address Translation Process

MIPS TLB

• In our case the TLB is software-managed (by the OS)

• On memory read/write, checks the entries in the TLB in parallel: – Entry is found - TLB hit – Entry not found - TLB miss

• Causes EX_TLBL for loads (reads)• Causes EX_TLBS for stores (writes)

– Protection fault - trying to write to read-only memory causes EX_MOD (modify)

TLB Exceptions• vm_fault() is called from mips_trap in src/kern/arch/mips/locore/trap.c for

any TLB exception– Different types of VM_FAULT_* are passed on– vm_fault() in src/kern/include/vm.h and src/kern/arch/mips/vm/vm.c

• vm_fault() calls as_fault() in src/kern/vm/addrspace.c and kern/include/addrspace.h

• Eventually gets to lpage_fault() in src/kern/vm/lpage.c (this is where you come in)

• On a TLB miss:– Look up the page in the page table

• Implemented in src/kern/vm/addrspace.c: as_fault()

– Choose an entry in the TLB to replace it • In src/kern/arch/mips/vm/coremap.c: tlb_replace()

– Update TLB entry with PTE from page table • In src/kern/arch/mips/vm/coremap.c: mmu_map()

Page Faults – High Level

Page Faults

• Minor fault: TLB does not contain a requested PTE (but it is in memory)– Find the Page Table Entry for it and insert the new

mapping into the TLB and the coremap– See coremap.c for a function you can use! (mmu_map)

• Major fault: the desired page is not in main memory (it’s either in swap space, or hasn’t been created yet) – How do we know if it’s a major fault? – lp_paddr field of the lpage struct will tell you – lp_paddr is INVALID_PADDR if the page is not in memory

Page Faults

• Major fault: desired page is not in memory – Page hasn’t been created yet

• A new page is allocated to the process and initialized (zero-filled) in src/kern/vm/addrspace.c: as_fault()

– Page is in swap • We need to swap the page into memory from swap space • Need a page of physical memory for the page

– look at lpage_copy for ideas• Set lp_*addr to INVALID_* when appropriate - to

indicate page is not in main memory or swap

Assignment 2

• Implement paging by writing the following functions: – lpage_fault --- handles a page fault – lpage_evict --- evicts an lpage from physical

memory – page_replace --- implements page replacement

policy• sequential replacement• random replacement

• Much of the system is already provided

Page Eviction

• lpage_evict – evict an lpage from physical memory

• Evicts the contents of the page at lp_paddr by writing it to lp_swapaddr on the swap device (if it is dirty), and mark lp_paddr invalid

• Called by do_evict (in coremap.c) when a physical page is chosen for eviction

Page Replacement

• Updating the victim’s PTE to show that it is in swap – swap functions src/kern/vm/swap.c

• Copying it to disk (iff it is dirty) • Evicting (invalidating) victim’s PTE from the TLB • Loading the new page into memory • Updating the new page’s PTE and inserting it

into the TLB

Synchronization

• OS161 assumes that lpages, vm_objects and address spaces are not shared.– But one thread may access an lpage belonging to

another thread, in order to evict a page– Thus you need not use locks when accessing

address spaces and vm_objects, but lpages do need synchronization• Bit lock is used to save space (see

lpage_lock/lpage_unlock)

Synchronization

• global_paging_lock, limits number of pages pinned at any one time

• swaplock: used by swap_alloc, swap_free, swap_reserve, swap_unreserve

• cm_pinned locks pages that are in transit. • one bit lock per lpage • Lock Ordering (i.e you should acquire in this order): – global_paging_lock BEFORE coremap pages

BEFORElpages