Data Criticality in Network-On-Chip Design...Data Criticality in Network-On-Chip Design Joshua San...

Data Criticality in Network-On-Chip Design

Joshua San Miguel

Natalie Enright Jerger

Network-On-Chip Efficiency

2

Efficiency is the ability to produce results with the least amount of waste.

Wasted time NoC accounts for 30-70% of on-chip data access latency

[Z. Li, HPCA 2009][A. Sharifi, MICRO 2012]

Wasted energy NoC accounts for 20-30% of total chip power

[J. D. Owens, IEEE Micro 2007][S. R. Vangal, IEEE JSSC 2008]


3


4

load request


5

data response

load request


6

data response

load request

Minimize wasted time Deliver data no later than needed


7


8

load request 0x2


9

data response

0 1 2 3 4 5 6 7

load request 0x2


10

data response

0 1 2 3 4 5 6 7

load request 0x2

Minimize wasted energy Deliver data no earlier than needed


11

Why store data in blocks of multiple words?

Exploit spatial locality in applications

Avoid large tag arrays in caches

Improve row buffer utilization in DRAM


12

Why store data in blocks of multiple words?

Exploit spatial locality in applications

Avoid large tag arrays in caches

Improve row buffer utilization in DRAM

Store data at a coarse granularity, but

move data at a fine granularity.


13

01:00

03:00

02:00

04:00


14

01:00

03:00

02:00

04:00


15

01:00

03:00

02:00

04:00


16

01:00

03:00

02:00

04:00

Arrive no later than needed. But expensive. Wasted money since some arrive too early.


17

01:00

03:00

02:00

04:00


18

01:00

03:00

02:00

04:00


19

01:00

03:00

02:00

04:00


20

01:00

03:00

02:00

04:00


21

01:00

03:00

02:00

04:00


22

01:00

03:00

02:00

04:00

Spend just enough money to arrive both no later and no earlier than needed.


23

01:00

03:00

02:00

04:00

Spend just enough money to arrive both no later and no earlier than needed.

Deliver data both no later and no earlier than needed Design for data criticality

Outline

24

Defining Criticality Data Criticality

Data Liveness

Measuring Criticality Energy Wasted

Addressing Criticality NoCNoC

Data Criticality

25

Data Criticality is the promptness with which an application uses a data word after fetching it from memory.

Critical: used immediately after being fetched.

Non-critical: used some time later after being fetched.

Defining Criticality

Data Criticality – blackscholes

26

for (i++) {

... = BlkSchlsEqEuroNoDiv(sptprice[i]);

}



27

for (i++) {


}


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

sptprice

i = 0


28

for (i++) {


}


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

sptprice

i = 15


29

for (i++) {


}


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

sptprice

criticality

Data Criticality – fluidanimate

30

for (iparNeigh++) {

if (borderNeigh) {

pthread_mutex_lock();

neigh->a[iparNeigh] -= ...;

pthread_mutex_unlock();

} else {

neigh->a[iparNeigh] -= ...;

}

}


Data Criticality

31

Data criticality is an inherent consequence of spatial locality and is exhibited by most (if not all) real-world applications.

Examples of non-criticality: Long-running code between accesses

Interference due to thread synchronization

Dependences from other cache misses

Preemption by the operating system


Data Criticality vs. Instruction Criticality

32

Instruction (or packet) criticality


load miss B

load miss A

load miss C

load miss D


33

Instruction (or packet) criticality


load miss B

load miss A

load miss C

load miss D


34

Data criticality


load miss B

load miss A

load miss C

load miss D

Data Liveness

35

Data Liveness describes whether or not an application uses a data word at all after fetching it from memory.

Live-on-arrival (live): used at least once during its cache lifetime.

Dead-on-arrival (dead): never used during its cache lifetime.


Data Liveness – fluidanimate

36

for (iz++)

for (iy++)

for (ix++)

for (j++) {

if (border(ix)) ... = cell->v[j].x;

if (border(iy)) ... = cell->v[j].y;

if (border(iz)) ... = cell->v[j].z;

}



37

for (iz++)

for (iy++)

for (ix++)

for (j++) {




}


x y z x y z x y z x y z x y z

cell->v


38

for (iz++)

for (iy++)

for (ix++)

for (j++) {




}


x y z x y z x y z x y z x y z

cell->v

Data Liveness

39

Data liveness measures the degree of spatial locality in an application.

Examples of dead words: Unused members of structs

Irregular or random access patterns

Heap fragmentation

Padding between data elements

Early evictions due to invalidations, cache pressure or poor replacement policies


Outline

40


Data Liveness



Measuring Criticality

41 Measuring Criticality

time

load miss A fetch A[i] use A[i]

fetch latency



time


fetch latency

access latency

𝑛𝑜𝑛−𝑐𝑟𝑖𝑡𝑖𝑐𝑎𝑙𝑖𝑡𝑦 = 𝑎𝑐𝑐𝑒𝑠𝑠 𝑙𝑎𝑡𝑒𝑛𝑐𝑦

𝑓𝑒𝑡𝑐ℎ 𝑙𝑎𝑡𝑒𝑛𝑐𝑦

1x for critical words, >1x for non-critical words



time


fetch latency

access latency


44

Full-system simulations: FeS2, BookSim, DSENT

16 2.0 GHz OoO cores

64 kB private L1 per core, 16-word cache blocks

16 MB shared distributed L2

Baseline NoC configuration: 4 x 4 mesh, 2.0 GHz, 128-bit channels

X-Y routing, 3-stage router pipeline, 6 4-flit VCs per port

Applications: PARSEC and SPLASH-2



45

Very low criticality


0%

20%

40%

60%

80%

100%

1x 2x 3x 4x 5x 6x 7x 8x 9x 10x

% a

cces

sed

wo

rds

(cu

mu

lati

ve)

access latency / fetch latency

blackscholes bodytrack fluidanimate streamcluster swaptions


46

Low criticality


0%

20%

40%

60%

80%

100%

1x 2x 3x 4x 5x 6x 7x 8x 9x 10x

% a

cces

sed

wo

rds

(cu

mu

lati

ve)


barnes lu_cb water_nsquared water_spatial


47

High criticality


0%

20%

40%

60%

80%

100%

1x 2x 3x 4x 5x 6x 7x 8x 9x 10x

% a

cces

sed

wo

rds

(cu

mu

lati

ve)


fft vips volrend x264


48

Very high criticality


0%

20%

40%

60%

80%

100%

1x 2x 3x 4x 5x 6x 7x 8x 9x 10x

% a

cces

sed

wo

rds

(cu

mu

lati

ve)


canneal cholesky radiosity radix

Measuring Criticality – Energy Wasted

49

Estimate energy wasted due to non-criticality

Model an ideal NoC where for each word: 𝑓𝑒𝑡𝑐ℎ 𝑙𝑎𝑡𝑒𝑛𝑐𝑦 ≈ 𝑎𝑐𝑐𝑒𝑠𝑠 𝑙𝑎𝑡𝑒𝑛𝑐𝑦




high criticality

low criticality




52

e.g., bodytrack:


0%

20%

40%

60%

80%

100%

1x 6x 11x 16x 21x 26x 31x

% a

cces

sed

wo

rds

(cu

mu

lati

ve)


subnet 1 subnet 2 subnet 3 subnet 4 subnet 5 subnet 6


53

e.g., bodytrack:


0%

20%

40%

60%

80%

100%

1x 6x 11x 16x 21x 26x 31x

% a

cces

sed

wo

rds

(cu

mu

lati

ve)



1x – 1.1x

1.1x – 2x

2x – 3.5x

3.5x – 7x

18x - ∞

7x – 18x


54

e.g., bodytrack:


0%

20%

40%

60%

80%

100%

1x 6x 11x 16x 21x 26x 31x

% a

cces

sed

wo

rds

(cu

mu

lati

ve)



1x – 1.1x, 2.0 GHz

1.1x – 2x, 1.8 GHz

2x – 3.5x , 1.0 GHz

3.5x – 7x, 571.4 MHz

18x - ∞, 111.0 MHz

7x – 18x, 285.7 MHz


55

Dynamic energy wasted due to non-criticality


0%

5%

10%

15%

20%

25%

30%

35%

40%b

arn

es

bla

cksc

ho

les

bo

dyt

rack

can

ne

al

cho

lesk

y fft

flu

idan

imat

e

lu_c

b

rad

iosi

ty

rad

ix

stre

amcl

ust

er

swap

tio

ns

vip

s

volr

end

wat

er_n

squ

ared

wat

er_s

pat

ial

x26

4

geo

mea

n

dyn

amic

en

ergy

was

ted


56



0%

5%

10%

15%

20%

25%

30%

35%

40%b

arn

es

bla

cksc

ho

les

bo

dyt

rack

can

ne

al

cho

lesk

y fft

flu

idan

imat

e

lu_c

b

rad

iosi

ty

rad

ix

stre

amcl

ust

er

swap

tio

ns

vip

s

volr

end

wat

er_n

squ

ared

wat

er_s

pat

ial

x26

4

geo

mea

n

dyn

amic

en

ergy

was

ted


57



0%

5%

10%

15%

20%

25%

30%

35%

40%b

arn

es

bla

cksc

ho

les

bo

dyt

rack

can

ne

al

cho

lesk

y fft

flu

idan

imat

e

lu_c

b

rad

iosi

ty

rad

ix

stre

amcl

ust

er

swap

tio

ns

vip

s

volr

end

wat

er_n

squ

ared

wat

er_s

pat

ial

x26

4

geo

mea

n

dyn

amic

en

ergy

was

ted


58

Dynamic energy wasted due to dead words


0%10%20%30%40%50%60%70%80%90%

100%b

arn

es

bla

cksc

ho

les

bo

dyt

rack

can

ne

al

cho

lesk

y fft

flu

idan

imat

e

lu_c

b

rad

iosi

ty

rad

ix

stre

amcl

ust

er

swap

tio

ns

vip

s

volr

end

wat

er_n

squ

ared

wat

er_s

pat

ial

x26

4

geo

mea

n

dyn

amic

en

ergy

was

ted


59

Dynamic energy wasted due to dead words


0%10%20%30%40%50%60%70%80%90%

100%b

arn

es

bla

cksc

ho

les

bo

dyt

rack

can

ne

al

cho

lesk

y fft

flu

idan

imat

e

lu_c

b

rad

iosi

ty

rad

ix

stre

amcl

ust

er

swap

tio

ns

vip

s

volr

end

wat

er_n

squ

ared

wat

er_s

pat

ial

x26

4

geo

mea

n

dyn

amic

en

ergy

was

ted

68.8% energy wasted due to non-critical and dead words.

Outline

60


Data Liveness



Addressing Criticality

61

A criticality-aware NoC design needs to:

1. Predict the criticality of a word prior to fetching it.

2. Separate the fetching of words based on their criticality.

3. Reduce energy consumption in fetching low-criticality words.

4. Eliminate the fetching of dead words.

NoCNoC (Non-Critical NoC)

A proof-of-concept, criticality-aware design


NoCNoC

62 Addressing Criticality


Binary predictor: either critical or non-critical.

NoCNoC



Binary predictor: either critical or non-critical. load miss (requested word 4)

NoCNoC


000000000000010X101000000000000

-15 … 0 … 15

instruction address

prediction table



NoCNoC


000000000000010X101000000000000

-15 … 0 … 15

instruction address

prediction table

0 … 15

prediction vector (requested word 4)

0010110100000000



NoCNoC


000000000000010X101000000000000

-15 … 0 … 15

instruction address

prediction table

0 … 15

prediction vector (requested word 4)

L1 request packet 0010110100000000



NoCNoC

70


Two physical subnetworks: critical and non-critical.


NoCNoC

71




NoCNoC


critical

non-critical



NoCNoC

73


DVFS in non-critical subnetwork.


NoCNoC

74




critical

if criticality very low

NoCNoC

75




critical

if criticality low

NoCNoC

76




critical

if criticality high

NoCNoC

77




critical

if criticality very high

NoCNoC

78


Binary predictor: either live or dead.


NoCNoC

79


Binary predictor: either live or dead.


000000000000010X101000000000000

-15 … 0 … 15

instruction address

prediction table

NoCNoC

80

More details and results in the paper:

DVFS scheme

Prediction tables

Comparison to instruction criticality (Aergia [R. Das, ISCA 2010])


NoCNoC – Prediction Accuracy


80%

85%

90%

95%

100%

criticality liveness

pre

dic

tio

n a

ccu

racy

correct over under

NoCNoC – Prediction Accuracy


80%

85%

90%

95%

100%

criticality liveness

pre

dic

tio

n a

ccu

racy

correct over under

Outperforms prior liveness predictor (70% accuracy) [H. Kim, NOCS 2011]

NoCNoC – Performance and Energy


0.6

0.7

0.8

0.9

1

1.1

dynamic energy runtime

no

rmal

ized

to

bas

elin

e

Conclusion

84

Define Data Criticality Deliver data both no later and no earlier than needed.

Measure Criticality 68.8% energy wasted due to non-critical and dead words.

Address Criticality NoCNoC, a proof-of-concept, criticality-aware design.

Thank you

Data Criticality in Network-On-Chip Design...Data Criticality in Network-On-Chip Design Joshua San...

Documents

Transcript of Data Criticality in Network-On-Chip Design...Data Criticality in Network-On-Chip Design Joshua San...