Predicting cache contents by abstract interpretation

A hierarchy independent approach (ongoing work)

Michael Monerau Chris HankinCourant Institute, NYU

Ecole Normale Supérieure de Paris, France

Imperial College London

2

Overview

Problem description Abstract Interpretation Cache prediction Related & Future work Conclusion

3

Cache mechanicsQuick introduction: How do cache work?

4

Cache mechanics: Cache hit

CPUs

Cache hierarchy

Read / Write

L1

L2

L2

L3

L3

Virtual Memory

Off-chip

On-chip

5

Virtual Memory

Cache mechanics: Cache miss

CPUs

Cache hierarchy

Read / Write

L1

L2

L2

L3

L3Ask for data

Sends data

Off-chip

On-chip

6

Cache Level

Cache mechanisms: Low-Level

Cache line...........

An example: 4-Way Associative Cache0.1.2.3.4.5.6.7.

8.9.10.11.

A cache line contains a copy of a Virtual Memory

Block

VM Address % 3 == 0

VM Address % 3 == 1

VM Address % 3 == 2

7

Cache hierarchies

Cache hierarchies may vary a lot Number of levels Internals of each level Replacement strategies (each level) Write strategies Inter-level strategies Several CPUs, shared/unshared level

Combinatorial explosion !

8

Predict it!

Knowing cache contents can help WCET (Worst Case Execution Time) Clever allocation on scratchpads

(software-managed small caches in embedded systems)

GoalsModular and hierarchy independent

prediction framework

9

Abstract InterpretationA quick overview of Abstract Interpretation

10

Abstract Interpretation: Principle

CONCRETE WORLD ABSTRACT WORLD

11

Abstract Interpretation: Principle

CONCRETE WORLD ABSTRACT WORLD

T

T

T

T

12

Abstract Interpretation: Galois

Given two posets and

Best abstraction (ie. most precise) iff Galois Connection :

,C ,A ô

, , ( ) ( )P C Q A P Q P Q ô –

Soundness

is optimal

13

Abstract Interpretation: Analysis

GoalFind an overapproximation of the abstract state at each control point

Equations (fixpoint)

x = Rand(1, 10)

y = 1y = 0

Print y

x > 5 x <= 5

1.

2. 3.

4.

2 15x

4 2 3 ò3 15x

14

Abstract Interpretation: Analysis

e.g. with intervals:

x = Rand(1, 10)

y = 1y = 0

Print y

x > 5 x <= 5

1.

2. 3.

4.

1 1[1,10], ] , [x y

2 2[6,10], [0,0]x y

3 3[1,5], [1,1]x y

4 [1,5] [6,10] [1,10]x ò

4 [0,0] [1,1] [0,1]y ò

15

Cache predictionDescription of the abstract domain for cache prediction

16

Cache Prediction: Dispatch

Cache Level

Relation Strategy

Stock Strategy

Stock Management

(low-level data organization)

Other levelOther level

Receive / Forward

Receive / Forward

Requests

Report updates

Make Address Available

Candidate locations

L3 L1

L2

17

Cache Prediction: Priority Goal: Modeling the replacement dynamic

Instead of abstracting locations contentsabstract addresses priorities

Priority for each Virtual Memory Address Or : « the address is not in the cache »

Meaning: Victim is the lowest priority

[0, 1]p n

0p

18

Cache Prediction: ChangePriority

Main ingredient to implement strategies:

Change Priority(VMAddress, p)

LRU: Just call ChangePriority(.., Highest+1)

FIFO: Call ChangePriority(.., Highest+1) only if the address is not in the cache level

Pseudo-LRU: More complicated, but same idea

19

Cache Prediction: AbstractionABSTRACTION FOR A LINES CACHEn

({ : [0, 1] { }

/ is injective})

p nVM

p

C=P ™}){ { }:p VM n A P™ ([0, - 1]

P : ( )p P

p ad p ad

p[0, 1] { }

/ is injective

an

{ :

}d , ( ) ( )

n

p

ad VM p ad p ad

p VM

™

Galois Connection

20

a4

Cache Prediction: ChangePriority#

ChangePriority#(Ad, pr)

.a1a2a3

a5a6

.

.

Priority

pr.

a1a2a3

a5a6

.

.

Priority

pr

1( )p pr 1( )p pr

Ad a4

Ad a7Ad a7

Ad a4

Moved 0 or 1 down

Moved 1 down

Soundness : #CP CP ô

21

Cache Prediction: Hits & Misses

CONCRETE WORLD

Miss p(Address)

Hit p(Address)

ABSTRACT WORLD

Let

Miss

Hit/Miss?

Hit

{ } P

{ } P Ö

P

(Address)P p

22

Cache Prediction: Scratchpad

Virtual Memory

CPU

Cache hierarchy

L1L2L3

Scratchpad

Hardware managed

Analysis gives information:

Possible contents of the cache

Simulate Scratchpad as a « L0 » cache

Heuristic

Software manage

d

Scratchpad allocationstrategy

If (unlikely) y=0

don’t put y on the

scratchpad

23

Related & Future Work

Related Work [Wilhelm et al. VMCAI’10] and [Reineke et al. SAS’09]:

clever abstract domains, no generic proof, no hierarchy

[Marvadel et al.]: profiling (for scratchpad allocation)

Future work Safe initial states for some replacement strategies

(domino effect, … cf. [SAS’09], [Berg ’06]) Improvements proposed in [SAS’09] Implementation & benchmarks Clever strategies for scratchpads

24

Conclusion

Modular analysis: Hierarchy-independent Replacement strategy-independent Easy-to-build transfer functions

Formal and modular proofs of soundness

Thank you!