Lec 5Memory Hierarchy and Cache
Memory
Memory Hierarchy: Terminology Hit: data appears in some block in the upper level
(example: Block X) Hit Rate: the fraction of memory access found in the upper
level Hit Time: Time to access the upper level which consists of
RAM access time + Time to determine hit/miss Miss: data needs to be retrieve from a block in the lower level
(Block Y) Miss Rate = 1 - (Hit Rate) Miss Penalty: Time to replace a block in the upper level +
Time to deliver the block the processor Hit Time << Miss Penalty
Lower LevelMemoryUpper Level
MemoryTo Processor
From ProcessorBlk X
Blk Y
General Principles of Memory Locality
Temporal Locality: referenced memory is likely to be referenced again soon (e.g. code within a loop)
Spatial Locality: memory close to referenced memory is likely to be referenced soon (e.g., data in a sequentially access array)
Definitions Upper: memory closer to processor Block: minimum unit that is present or not present Block address: location of block in memory Hit: Data is found in the desired location Hit time: time to access upper level Miss rate: percentage of time item not found in upper level
Locality + smaller HW is faster = memory hierarchy Levels: each smaller, faster, more expensive/byte than level
below Inclusive: data found in upper level also found in the lower
level
What is Cache?“A computer memory with very short access
time used for storage of frequently used instructions or data” – webster.com
Types of CacheL1/L2/L3 CacheRAM CacheDisk CacheSoftware Level Cache
Memory Hierarchy Comparison
L1/L2/L3 Cache (Cache Memory)Cache closer to the CPU that stores
recently accessed data from RAMHolds instructions to be executed next
and variables for the CPU
L2 Cache
COAST: Cache on a Stick
Cache Memory ContinuedLevel 1 Cache is stored on the CPUL2/L3 Cache stored near, but not on the CPUL1/L2/L3 Cache is more expensive than RAM
Different CPU Caches:
Cache MemoryReplacement Policies
CPU requests data from cached memoryCache memory has data, sends to CPU (Hit)RAM has data, sends to CPU (Miss)
Cache performance measured by hits/misses where the ideal algorithm has a ratio close to 1.
Replacement PoliciesOn a miss the data from RAM is loaded in
to CacheDue to the nature of most programs
(Variables are accessed multiple times in the same program) this cached data will have a high chance of being accessed often.
One possible replacement algorithm is a FIFO based one where the oldest cached data is replaced first.
CPU Cache - Write PoliciesCPU writes data in cache (hit)
Write-through – Cache and RAM updatedWrite-back – Cache updated, RAM updated
when new data is going to replace that data in cache Dirty bit keeps track of if a part of cache has
changed
CPU Cache - Write PoliciesCPU writes but data not in cache (Miss)
Write-no-allocate – RAM updatedWrite-allocate – RAM updated, data loaded into
cache
CPU Cache – SMP ProblemsMultiprocessor systems have trouble with
write-through and write-backIE. One processor updates data cached by both
processorsPossible Solutions:
Cache only unshared memory Update all cache simultaneously
RAM CacheSame as Cache Memory or CPU Cache,
except instead of the L1/L2 cache caching RAM, RAM caches data from the hard disk.
For example, Windows loads multiple required libraries for running in memory, but code isn’t executed from them constantly.
Disk CacheLike cache memory, disk cache holds data
recently read or written to from a hard disk. This is to speed up subsequent reads from recently accessed data.
Most newer hard drives have around 8mb of cache memory.
Software CacheSoftware also needs to cache data into
RAM that it uses often – however, more often than not, this data isn’t directly from the hard disk.
The data software caches is often processed data from a file translated by the program into another data structure. The process of translating may take a long time, so the cache in RAM is used to speed up the rest of the programs run time.
Software Cache - ExampleGames such as Half-Life and Unreal
Tournament 2004 have a loading time before each game where the map, models, sounds, and other important data are all loaded into RAM.
Example: Website loading lots of data from an SQL server each page load may take a few seconds, but if the first load caches all the data, the load may only take a few milliseconds.
Software Cache - ExampleInternet Explorer
History – Keeps a cache of recently viewed websites for easy access
Cache – Keeps images and other files on disk so websites load faster the second time you visit them. These files are updated when there is a newer version of them on the server, or the cache is cleared.
Interesting FactsThe first Celeron CPU had no caching,
however it performed almost as fast as the PII with cache.
Because some chipsets couldn’t cache more than the first 64mb of RAM a rumor was spread that Windows 98 could only use up to 64mb of RAM, however in reality it supports up to 2GB of RAM.In these chipsets, adding more than 64MB of
RAM decreased performance.
Top Related