Memory Hierarchy Memory Hierarchy and and

Cache MemoryCache Memory

Jennifer TsayJennifer TsayCS 147CS 147Section 3Section 3October 8, 2009October 8, 2009

Two Basic Types of MemoryTwo Basic Types of Memory

RAM (Random Access RAM (Random Access Memory)Memory) Used to store programs and Used to store programs and

data that computer needs data that computer needs when executing programswhen executing programs

Volatile and loses Volatile and loses information once power is information once power is turned offturned off

2 Basic Types of Memory2 Basic Types of Memory

ROM (Read-Only Memory)ROM (Read-Only Memory) Stores critical information necessary to operate Stores critical information necessary to operate

the system, such as program necessary to the system, such as program necessary to boot computerboot computer

Not volatile and always retains its dataNot volatile and always retains its data

Also embedded in systems where Also embedded in systems where programming does not need to changeprogramming does not need to change

Memory HierarchyMemory Hierarchy

Hierarchal MemoryHierarchal Memory Approach in which computer systems use Approach in which computer systems use

combination of memory types to provide best combination of memory types to provide best performance at best costperformance at best cost

Basic types that constitute hierarchal memory Basic types that constitute hierarchal memory system include registers, cache, main system include registers, cache, main memory and secondary memorymemory and secondary memory

Memory HierarchyMemory Hierarchy

Today’s computers each have small amount of Today’s computers each have small amount of very high-speed memory, called very high-speed memory, called cachecache where where data from frequently used memory locations may data from frequently used memory locations may be temporarily storedbe temporarily stored

Cache is connected to main memory, which is Cache is connected to main memory, which is typically medium-speed memorytypically medium-speed memory

Main memory is complemented by secondary Main memory is complemented by secondary memory, composed of hard disk and various memory, composed of hard disk and various removable mediaremovable media

The Memory HierarchyThe Memory Hierarchy

How Do We Classify Memory?How Do We Classify Memory?

Based on its “distance” from the processorBased on its “distance” from the processor Distance is measured by the number of Distance is measured by the number of

machine cycles required for accessmachine cycles required for access The closer memory is to the processor, the The closer memory is to the processor, the

faster it should befaster it should be

Locality of ReferenceLocality of Reference

Clustering of memory references into Clustering of memory references into groupsgroups

Important because through use of Important because through use of cache technique, locality of reference cache technique, locality of reference can optimize performancecan optimize performance

Cache MemoryCache Memory

Small, temporary, but fast (and thus Small, temporary, but fast (and thus high-cost) memoryhigh-cost) memory

Processor uses cache memory for Processor uses cache memory for information it is likely to need again information it is likely to need again in very near futurein very near future

Non-computer ExamplesNon-computer Examples Cell Phone Address Book vs. Phone BookCell Phone Address Book vs. Phone Book Desk vs. File CabinetDesk vs. File Cabinet

MAIN MEMORYMAIN MEMORY CACHECACHE

Cache Mapping SchemesCache Mapping Schemes

For cache to be functional, it must store For cache to be functional, it must store useful datauseful data

This data becomes useless if CPU can’t This data becomes useless if CPU can’t find itfind it

When accessing data or instructions, the When accessing data or instructions, the CPU first generates main memory addressCPU first generates main memory address

If data has been copied to cache, address If data has been copied to cache, address of data in cache is not same as main of data in cache is not same as main memory addressmemory address

Types of Mapping SchemesTypes of Mapping Schemes

Direct Mapped CacheDirect Mapped Cache Each main memory block has specific location to which it Each main memory block has specific location to which it

maps in cachemaps in cache If block already occupies cache location where new block If block already occupies cache location where new block

must be placed, block currently in cache is removedmust be placed, block currently in cache is removed

Types of Mapping SchemesTypes of Mapping Schemes

Fully Associative CacheFully Associative Cache Allows main memory block to be placed anywhere in cacheAllows main memory block to be placed anywhere in cache When cache is full, need replacement algorithm to decide which When cache is full, need replacement algorithm to decide which

block to throw out of cacheblock to throw out of cache

Replacement AlgorithmsReplacement Algorithms

LRU (Least Recently Used)LRU (Least Recently Used) Keep track of the last time each block was accessedKeep track of the last time each block was accessed Select the block that has been used least recently to Select the block that has been used least recently to

be the block that gets removedbe the block that gets removed Requires system to keep history of accesses for every Requires system to keep history of accesses for every

cache block which requires significant space and cache block which requires significant space and slows down operation of cacheslows down operation of cache

FIFO (First-in, First out)FIFO (First-in, First out) Block that has been in cache the longest (regardless Block that has been in cache the longest (regardless

of how recently it has been used) would be selected of how recently it has been used) would be selected as block to be removed from cache memoryas block to be removed from cache memory

Measuring PerformanceMeasuring Performance

How do we measure performance of How do we measure performance of two-level hierarchical memory?two-level hierarchical memory?

By its By its Effective Access Time (EAT)Effective Access Time (EAT), or average , or average time per accesstime per access

EAT = H x AccessEAT = H x AccessCC + (1 – H) x Access + (1 – H) x AccessMMMM

H = cache hit rateH = cache hit rate AccessAccessMMMM = main memory access time = main memory access time

AccessAccessC C = cache access time= cache access time

ExampleExample

Let’s say that:Let’s say that:

AccessAccessC C = = 10ns10ns

AccessAccessMMMM = = 200ns200ns

H = 99%H = 99%

EAT = H x AccessEAT = H x AccessCC + (1 – H) x Access + (1 – H) x AccessMMMM

EAT = 0.99 x 10ns + (0.01) x 200ns ~ 12nsEAT = 0.99 x 10ns + (0.01) x 200ns ~ 12ns

Cache Writing PoliciesCache Writing Policies

Designers have to determine what to do with blocks Designers have to determine what to do with blocks that have been modifiedthat have been modified

Cache write policy determines when actual main Cache write policy determines when actual main memory block is updated to match cache blockmemory block is updated to match cache block

Write-ThroughWrite-Through Updates both cache and main memory Updates both cache and main memory

simultaneously on every writesimultaneously on every write Write-backWrite-back

Only updates blocks in main memory when cache Only updates blocks in main memory when cache block is selected to be removed from cacheblock is selected to be removed from cache

Data and Instruction CachesData and Instruction Caches

Data CacheData Cache Instruction CacheInstruction Cache Advantages of separatingAdvantages of separating

Unified cache has only one port for data and Unified cache has only one port for data and instructions, resulting in conflicts between the instructions, resulting in conflicts between the twotwo

Allows accesses to be less random and more Allows accesses to be less random and more clusteredclustered

Levels of CacheLevels of Cache Multilevel Cache MemoryMultilevel Cache Memory

Caches using caches for increased Caches using caches for increased performanceperformance

Level 1 (L1) cache is term used for cache Level 1 (L1) cache is term used for cache resident on chip and is fastest, smallest cacheresident on chip and is fastest, smallest cache

L1 is always checked first, but if the data is L1 is always checked first, but if the data is not found in L1, L2 cache is checkednot found in L1, L2 cache is checked

L3 cache refers to extra cache between L3 cache refers to extra cache between processor and memory on processorsprocessor and memory on processors

ReferencesReferences http://en.wikipedia.org/wiki/Locality_of_referencehttp://en.wikipedia.org/wiki/Locality_of_reference http://images.google.com/url?source=imgres&ct=ref&q=http://http://images.google.com/url?source=imgres&ct=ref&q=http://

www.viewpoints.com/HON-210-Series-28-1-2-Deep-Vertical-File-Cabinet-www.viewpoints.com/HON-210-Series-28-1-2-Deep-Vertical-File-Cabinet-review-a897&usg=AFQjCNFogrb2UbcND7bTPbDviC8DvlTpeQreview-a897&usg=AFQjCNFogrb2UbcND7bTPbDviC8DvlTpeQ

http://i.ehow.com/images/GlobalPhoto/Articles/4799688/IMGP1527edited-1-http://i.ehow.com/images/GlobalPhoto/Articles/4799688/IMGP1527edited-1-main_Full.jpg main_Full.jpg

www.leoncountyfl.govwww.leoncountyfl.gov/recycling//recycling/ www.calgary-city-maps.com/Calgary-phone-book.htmlwww.calgary-city-maps.com/Calgary-phone-book.html tjliu.myweb.hinet.net/COA_CH_6.htmtjliu.myweb.hinet.net/COA_CH_6.htm http://www.kidscreen.com/planetpreschool/wp-content/uploads/2009/08/2_http://www.kidscreen.com/planetpreschool/wp-content/uploads/2009/08/2_

mel-at-messy-desk.jpgmel-at-messy-desk.jpg

http://images.google.com/url?source=imgres&ct=ref&q=http://http://images.google.com/url?source=imgres&ct=ref&q=http://www.novopc.com/2008/09/ram-random-access-memory/www.novopc.com/2008/09/ram-random-access-memory/&usg=AFQjCNF9SaCi-7adm2Vu2fdpZF-hNVu4sw &usg=AFQjCNF9SaCi-7adm2Vu2fdpZF-hNVu4sw

Null, L. & Lobur, J. Null, L. & Lobur, J. The Essentials of Computer Organization and The Essentials of Computer Organization and Architecture. Architecture. 2006: Sudbury.2006: Sudbury.