Simulations of Memory Hierarchy

LAB 2: CACHE LAB

OVERVIEW• Objectives

• Cache Set-Up

• Command line parsing

• Least Recently Used (LRU)

• Matrix Transposition

• Cache-Friendly Code

OBJECTIVE• There are two parts to this lab:

• Part A: Cache Simulator

• Simulate a cache table using the LRU algorithm

• Part B: Optimizing Matrix Transpose

• Write “cache-friendly” code in order to optimize cache hits/misses in the implementation of a matrix transpose function

• When submitting your lab, please submit the handin.tar file as described in the instructions.

MEMORY HIERARCHY• Pick your poison: smaller, faster, and costlier, or larger,

slower, and cheaper

CACHE ADDRESSING• X-bit memory addresses (in Part A, X <= 64 bits)

• Block offset: b bits

• Set index: s bits

• Tag bits: X – b – s

• Cache is a collection of S=2^s cache sets

• Cache set is a collection of E cache lines

• E is the associativity of the cache

• If E=1, the cache is called “direct-mapped”

• Each cache line stores a block of B=2^b bytes of data

ADDRESS ANATOMY

CACHE TABLE BASICS• Conditions:

• Set size (S)

• Block size (B)

• Line size (E)

• Note that the total capacity of this cache would be S*B*E

• Blocks are the fundamental units of the cache

CACHE TABLE CORRESPONDENCE WITH ADDRESS

Example for 32 bit address

CACHE SET LOOK-UP• Determine the set index and the tag bits based on the

memory address

• Locate the corresponding cache set and determine whether or not there exists a valid cache line with a matching tag

• If a cache miss occurs:

• If there is an empty cache line, utilize it

• If the set is full then a cache line must be evicted

TYPES OF CACHE MISSES• Compulsory Miss:

• First access to a block has to be a miss

• Conflict Miss:

• Level k cache is large enough, but multiple data objects all map to the same level k block

• Capacity Miss:

• Occurs when the working set of blocks (blocks of memory being used) is larger than the cache

PART A:CACHE SIMULATION

YOUR OWN CACHE SIMULATOR• NOT a real cache

• Block offsets are NOT used but are important in understanding the concept of a cache

• s, b, and E given at runtime

FUNCTIONS TO USE FOR COMMAND LINE PARSING• int getopt(int argc, char*const* argv, const char*

options)

• See: http://www.gnu.org/software/libc/manual/html_node/Example-of-Getopt.html#Example-of-Getopt

• long long int strtoll(const char* str, char** endptr, int base)

• See: http://www.cplusplus.com/reference/cstdlib/strtoll/

LEAST RECENTLY USED (LRU) ALGORITHM

• A least recently used algorithm should be used to determine which cache lines to evict in what order

• Each cache line will need some sort of “time” field which should be update each time that cache line is referenced

• If a cache miss occurs in a full cache set, the cache line with the least relevant time field should be evicted

PART B:OPTIMIZING MATRIX TRANSPOSE

WHAT IS A MATRIX TRANSPOSITION?

• The transpose of a matrix A is denoted as AT

• The rows of AT are the columns of A, and the columns of AT are the rows of A

• Example:

GENERAL MATRIX TRANSPOSITION

CACHE-FRIENDLY CODE• In order to have fewer cache misses, you must make

good use of:

• Temporal locality: reuse the current cache block if possible (avoid conflict misses [thrashing])

• Spatial locality: reference the data of close storage locations

• Tips:

• Cache blocking

• Optimized access patterns

• Your code should look ugly if done correctly

CACHE BLOCKING• Partition the matrix in question into sub-matrices

• Divide the larger problem into smaller sub-problems

• Main idea:

• Iterate over blocks as you perform the transpose as opposed to the simplistic algorithm which goes index by index, row by row

• Determining the size of these blocks will take some amount of thought and experimentation

QUESTIONS TO PONDER• What would happen if instead of accessing each index in row

order you alternated with jumping from row to row within the same column?

• What would happen if you declared only 4 local variables as opposed to 12 local variables?

• Is it possible to get rid of the local variables all together?

• What happens when accessing elements along the diagonal?

• What happens when the program is run in a different directory?

(XKCD)