Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the...

Computer Architecture CSE 3322

Lecture 23

Multilevel Caches

• Microprocessors are getting faster and including a smallhigh speed cache on the same chip

Multilevel Caches


• A Second level cache is off chip that is larger and slowerthat interfaces with the Main Memory

Multilevel Caches



• Same cache techniques can be used

Multilevel Caches




Ave Memory Access Time = Hit time(L1)+ Miss rate (L1) * Miss Penalty( L1)

Multilevel Caches





Miss penalty (L1) = Hit time (L2) + Miss rate (L2) * Miss Penalty( L2)

Multilevel Caches





Miss penalty (L1) = Hit time (L2) + Miss rate (L2) * Miss Penalty( L2)

Ave Memory Access Time = Hit time(L1)+ Miss rate (L1) * Hit time (L2)+ Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2)

Ave Memory Access Time = Hit time(L1)

+ Miss rate (L1) * Hit time (L2)

+ Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2)

Ex: Hit time (L1) = 1 clock cycleMiss rate(L1) = 10%Hit time ( L2) = 10 Clock CyclesMiss rate (L2) = 20%Miss penalty (L2) = 100 clock cyles

If no secondary cacheAve Memory Access = 1 + 0.1 * 100 = 11 clock cycles

Ave Memory Access Time = Hit time(L1)

+ Miss rate (L1) * Hit time (L2)

+ Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2)

Ex: Hit time (L1) = 1 clock cycleMiss rate(L1) = 10%Hit time ( L2) = 10 Clock CyclesMiss rate (L2) = 20%Miss penalty (L2) = 100 clock cyles

If no secondary cacheAve Memory Access = 1 + 0.1 * 100 = 11 clock cycles

With secondary cacheAve Memory Access = 1 + 0.1*10 + 0.1*0.2*100

1 + 1 + 2 = 4 clock cycles

Characteristics of programs that makes efficient use of cache memory


• Code has tight loops with lots of reuse



• Code minimizes jumps and branches to far away



• Code minimizes jumps and branches to far away

• Data marches through arrays

Processor

Memory Hierarchy

Cache Memory

Main Memory

Transfer in 1 clockcycle

Transfer > 10 clockcycles

SRAM

DRAM

Processor

Virtual Memory Hierarchy

Cache Memory

Main Memory

Transfer in 10 + clock cycles

Transfer > 100,000 clock cycles

DRAM

HardDisk

Virtual MemoryThe Illusion of Unlimited Amount of Memory

1. Invented to allow programmers to use the full addressingcapability of the processor. Not limited by physical RAM



2. Enabled Multiprocessing. Computer running multipleprocesses ( programs and environments) each with it’sown address space. ( Time Sharing )




3. Provides a structure for protection. Pages can have restricted use by assigned processes. Types and Levels

of access.




3. Provides a structure for protection. Pages can have restricted use by assigned processes. Types and Levels

of access.

4. Provides simple relocation by mapping to pages inphysical memory

Virtual Address – 4 GB

31 30 29 . . . 14 13 12 11 10 9 8 . . . 2 1 0

Virtual page number Page offset

28 27 26 . . . 14 13 12 11 10 9 8 . . . 2 1 0

Physical page number Page offset

Page = 212 = 4KBTranslation

Physical Address – 512 MB

Virtual Address

Virtual Page No. Page Offset

PageTable Main

Memory

Hard Disk

PhysicalAddress

Page = Block

What is the miss penalty?Called Page Fault

1. Where can a page ( block) be placed in Main Memory?


• The miss penalty is very high, so minimize miss rate



• The long access time of the hard disk enables the OS to control placing the pages in Main Memory



• The long access time of the hard disk enables the OS to control placing the pages in Main Memory

• The lowest miss rate results from locating pagesanywhere in Main Memory ( Fully Associative )

2. How is a page ( block) found if it is in Main Memory?


• A page is located by the page table that contains thephysical address of the page in Main Memory inthe entry addressed by the virtual page number.( The index is mapped by the software using the page table and no tag is required.)



• The page table has an entry for every virtual pagenumber which can be 1 M entries. So, the page table is typically stored in Main Memory.



• The page table has an entry for every virtual pagenumber which can be 1 M entries. So, the page table is typically stored in Main Memory.

• Accessing Main Memory twice each time takes to long,so another cache must be added.

3. Which page ( block) should be replaced on a VirtualMemory Miss ( Page Fault )


• Under OS control, so clever algorithms are feasible.



• Usually try to approximate least recently used (LRU)replacement strategy.



• Usually try to approximate least recently used (LRU)replacement strategy.

• May keep a use bit, which is set whenever a page isaccessed to help estimate LRU.

4. What happens on a Write?


• Write Through is not feasible due to the very slowspeed of the Hard Disk



• The write strategy is always Write Back



• The write strategy is always Write Back

• Write Back# The word is written only to the page in Main

Memory (cache).# The modified page is written to the Hard Disk only when it is replaced. A Dirty Bit

indicates if a page has been altered.

Selecting a Page Size

• Reasons Pages should be larger:1. Minimize miss rate, but too large can get

no improvement or even increase.2. Reduces the size of the page table3. Transfers to / from Hard Disk are more efficient

( also over networks)





• Reasons Pages should be smaller:1. Wastes space because each process has several

primary pages required.2. Start up time is longer for small processes.





• Reasons Pages should be smaller:1. Wastes space because each process has several

primary pages required.2. Start up time is longer for small processes.

• Page size is 4 KB to 64 KB

Virtual Address


PageTable Main

Memory

Hard Disk

PhysicalAddress

Page = Block

Physical Page Address

Valid Use

Dirty

Page TableMaps Virtual PageNo. to Disk Addr

The Page Table is• Large• Usually in DRAM Main Memory• Too Slow

The Page Table is• Large• Usually in DRAM Main Memory• Too Slow

So, provide a cache for the Page Table

TLB – translation-lookaside buffer



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



PageTable

MainMemory

Hard Disk

TLB

Page Table Register

+

PhysicalPage Addr

Physical Page Addr

Tag

Valid, Dirty, Use

TLB – translation-lookaside bufferVirtual Page No. Page Offset

PageTable

MainMemory

Hard Disk

TLB

Page Table Register

+

PhysicalPage Addr

Physical Page Addr

Tag

Valid, Dirty, Use


PageTable

MainMemory

Hard Disk

TLB

Page Table Register

+

PhysicalPage Addr

Physical Page Addr

Tag

Valid, Dirty, Use

Hit


PageTable

MainMemory

Hard Disk

TLB

Page Table Register

+

PhysicalPage Addr

Physical Page Addr

Tag

Valid, Dirty, Use

Hit• Read Main Set Use Bit TLB


PageTable

MainMemory

Hard Disk

TLB

Page Table Register

+

PhysicalPage Addr

Physical Page Addr

Tag

Valid, Dirty, Use

Hit• Write Main Set Dirty Bit TLB


PageTable

MainMemory

Hard Disk

TLB

Page Table Register

+

PhysicalPage Addr

Physical Page Addr

Tag

Valid, Dirty, Use

Miss


PageTable

MainMemory

Hard Disk

TLB

Page Table Register

+

PhysicalPage Addr

Physical Page Addr

Tag

Valid, Dirty, Use

MissTLB Miss or Page Fault

TLB Miss –

TLB Miss –

1. Select entry to be replaced (LRU or Random)

TLB Miss –

1. Select entry to be replaced (LRU or Random)2. Write Back Use and Dirty Bits to Page Table

replaced entry

TLB Miss –

1. Select entry to be replaced (LRU or Random)2. Write Back Use and Dirty Bits to Page Table

replaced entry3. Access requested entry in Page Table

If Page is in Main Memory ( Valid), load thetranslation from the Page Table to TLB andtry again

TLB Miss – 1. Select entry to be replaced (LRU or Random)2. Write Back Use and Dirty Bits to Page Table

replaced entry3. Access requested entry in Page Table

If Page is in Main Memory ( Valid), load thetranslation from the Page Table to TLB andtry again

If Page is not in Main Memory ( Valid), then it isa Page Fault :

Write Back Replaced Page to Disk if DirtyMove page from Disk to Main MemoryUpdate Page Table & TLB and try again

Virtual Address

31 30 29 . . . 14 13 12 11 10 9 . . . . 2 1 0Virtual Page Number Page Offset

==

==

Tag Physical Page Number20

TLB

Hit

Valid

Dirty

To Main Memory

2012

Virtual Address


==

==


TLB

Hit

Valid

Dirty

To Main Memory

2012

Design GoalsHit Time – 1 clock cycleTLB Miss Penalty – 10 - 20 clock cyclesTLB Miss Rate - < 1%

Virtual Address


==

==


TLB

Hit

Valid

Dirty

Tag - m Index-k

20 12Block Offset

Byte Offset

Main Memory Cache

PhysicalAddress



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use

Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use


Size of page table assuming no disk address and 4 bits V,P,D,U?



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



14



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



1426



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



1426

262



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



1426

26222



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



1426

26222

x 26



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use


TLB 2-way Associative with 256 entries. Index and Tag?

1426

26222

x 26



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



1426

26222

x 26

Tag Index



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



1426

26222

x 26

Tag Index-8



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



1426

26222

x 26

Tag-18 Index-8



PageTable

MainMemory

Hard Disk

TLB

Physical Page Addr

Tag

Valid, Dirty, Use



1426

26222

x 26

Tag-18 Index-8

4 18

88

CPU

TLB

On Chip Cache

Secondary Cache

DRAM Main Memory

Hard Disk

Virtual Address 32-64 bits4KB pages

Microprocessors

CPU

TLB

On Chip Cache

Secondary Cache

DRAM Main Memory

Hard Disk

Virtual Address 32-64 bits4KB pagesTLB Split Data and Instruction 4-Way Set Associative

64 – 128 entries

Microprocessors

CPU

TLB

On Chip Cache

Secondary Cache

DRAM Main Memory

Hard Disk

Virtual Address 32-64 bits4KB pagesTLB Split Data and Instruction 4-Way Set Associative

64 – 128 entriesOn Chip Cache Split Data and Instruction 8KB – 16KB each 4-Way Associative 32 Bytes / Block

Microprocessors

Implementing Protection with Virtual Memory

• Add protection bits to the TLB / Page TableRead / Write AccessUser(s) / Supervisor



• CPU supplies Read / Write and User / Supervisorsignals for each access




• Comparisons can be made in the TLB. Non-comparecan cause an exception




• Comparisons can be made in the TLB. Non-comparecan cause an exception

• User programs cannot modify the protectionbits in Page Table / TLB

Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the...

Documents

Transcript of Multilevel Caches Microprocessors are getting faster and including a small high speed cache on the...