Platform-based Design

Platform-based Design

TU/e 5kk70Henk Corporaal

Bart Mesman

Exploiting ILPVLIW architectures (part a)

04/19/23 Platform Design H. Corporaal and B. Mesman 2

What are we talking about?

ILP = Instruction Level Parallelism =

ability to perform multiple operations (or instructions),from a single instruction stream,

in parallel

VLIW = Very Long Instruction Word architecture

operation 1 operation 2 operation 3 operation 4

Instruction format:

operation 5


VLIW: Topics Overview

• Enhance performance: architecture methods

• Instruction Level Parallelism– Limits on ILP

• VLIW– Examples

• Clustering

• Code generation

• Hands-on


Enhance performance: 3 architecture methods

• (Super)-pipelining

• Powerful instructions– MD-technique

• multiple data operands per operation

– MO-technique• multiple operations per instruction

• Multiple instruction issue


Architecture methods

Pipelined Execution of InstructionsIF: Instruction Fetch

DC: Instruction Decode

RF: Register Fetch

EX: Execute instruction

WB: Write Result Register

IF DC RF EX WBIF DC RF EX WB

IF DC RF EX WBIF DC RF EX WB

INS

TR

UC

TIO

N

CYCLE

1 2 43 5 6 7 8

12

3

4

Purpose of pipelining:• Reduce #gate_levels in critical path• Reduce CPI close to one• More efficient Hardware

Problems• Hazards: pipeline stalls

• Structural hazards: add more hardware• Control hazards, branch penalties: use branch prediction• Data hazards: by passing required

Simple 5-stage pipeline



Pipelined Execution of Instructions

Superpipelining:

• Split one or more of the critical pipeline stages

*



Powerful Instructions (1)

MD-technique• Multiple data operands per operation• SIMD: Single Instruction Multiple Data

Vector instruction:

for (i=0, i++, i<64) c[i] = a[i] + 5*b[i];

c = a + 5*b

Assembly:

set vl,64ldv v1,0(r2)mulvi v2,v1,5ldv v1,0(r1)addv v3,v1,v2stv v3,0(r3)




SIMD computing• Nodes used for independent

operations

• Mesh or hypercube connectivity

• Exploit data locality of e.g. image processing applications

• Dense encoding (few instruction bits needed)

SIMD Execution Method

tim

e

Instruction 1

Instruction 2

Instruction 3

Instruction n

node1 node2 node-K




• Sub-word parallelism– SIMD on restricted scale:

– Used for Multi-media instructions

• Examples– MMX, SUN-VIS, HP MAX-2, AMD-

K7/Athlon 3Dnow, Trimedia II

– Example: i=1..4|ai-bi|

* * * *




MO-technique: multiple operations per instruction

• CISC (Complex Instruction Set Computer)

• VLIW (Very Long Instruction Word)

sub r8, r5, 3 and r1, r5, 12 mul r6, r5, r2 ld r3, 0(r5)

FU 1 FU 2 FU 3 FU 4field

instruction bnez r5, 13

FU 5

VLIW instruction example


Execunit 1

Execunit 2

Execunit 3

Register file

Issue slot 1

Execunit 4

Execunit 5

Execunit 6

Execunit 7

Execunit 8

Execunit 9

Issue slot 2 Issue slot 3

VLIW architecture: central Register File


Execunit

Execunit

Execunit

Execunit

Execunit

Register file (128 regs, 32 bit, 15 ports)

Instruction register (5 issue slots)

Data cache

(16 kB)

PCInstruction

cache (32kB)

5 constant5 ALU2 memory2 shift2 DSP-ALU2 DSP-mul3 branch2 FP ALU2 Int/FP ALU1 FP compare1 FP div/sqrt

TM1000 DSPCPU


TriMedia TM32A processor

D-cache I-Cache

IFM

UL

1IF

MU

L1

IFM

UL

2IF

MU

L2

(FL

OA

T)

(FL

OA

T)

(FL

OA

T)

(FL

OA

T)

DS

PM

UL

1D

SP

MU

L1 D

SP

MU

L2

DS

PM

UL

2

FT

OU

GH

1F

TO

UG

H1

SH

IFT

ER

1S

HIF

TE

R1

AL

U1

AL

U1

FC

OM

P2

FC

OM

P2

DS

PA

LU

2D

SP

AL

U2

AL

U2

AL

U2

AL

U4

AL

U4

AL

U0

AL

U0

AL

U3

AL

U3

FA

LU

0F

AL

U0

FA

LU

3F

AL

U3

DS

PA

LU

0D

SP

AL

U0

SH

IFT

ER

0S

HIF

TE

R0

TA

G

TA

G

TAG

TAG

SEQUENCER / DECODE

I/OINTERFACE

0.18 micronarea : 16.9mm2

200 MHz (typ)1.4 W

7 mW/MHz

(MIPS=0.9 mW/MHz)


Architecture methods: Powerful Instructions (2)

VLIW Characteristics

• Only RISC like operation support Short cycle times

• Flexible: Can implement any FU mixture• Extensible• Tight inter FU connectivity required• Large instructions (up to 1000 bits)• Not binary compatible• But good compilers exist



Multiple instruction issue (per cycle)

Who guarantees semantic correctness?– can instructions be executed in parallel

• User specifies multiple instruction streams– MIMD (Multiple Instruction Multiple Data)

• Run-time detection of ready instructions– Superscalar

• Compile into dataflow representation– Dataflow processors


Multiple instruction issue

Three Approaches

a := b + 15;

c := 3.14 * d;

e := c / f;

Translation to DDG (Data Dependence Graph)

ld

+

st

&b

15

&a

ld *

/ st

ld

st

&f 3.14

&e

&d

&c

Example code


Generated Code

Instr. Sequential Code Dataflow Code I1 ld r1,M(&b) ld(M(&b) -> I2I2 addi r1,r1,15 addi 15 -> I3I3 st r1,M(&a) st M(&a)I4 ld r1,M(&d) ld M(&d) -> I5I5 muli r1,r1,3.14 muli 3.14 -> I6, I8I6 st r1,M(&c) st M(&c)I7 ld r2,M(&f) ld M(&f) -> I8I8 div r1,r1,r2 div -> I9I9 st r1,M(&e) st M(&e)

Notes:• An MIMD may execute two streams: (1) I1-I3 (2) I4-I9

– No dependencies between streams; in practice communication and synchronization required between streams

• A superscalar issues multiple instructions from sequential stream– Obey dependencies (True and name dependencies)

– Reverse engineering of DDG needed at run-time

• Dataflow code is direct representation of DDG


Multiple Instruction Issue: Data flow processor

Token Matching

TokenStore

InstructionGenerate

InstructionStore

FU-1 FU-2 FU-K

Reservation Stations

Re

sult

To

ken

s


Instruction Pipeline Overview

IF DC RF EX WB

IF DC/RF EX WB

CISC

RISC

IF1 DC1 RF1 EX1 ROBISSUE WB1



IFk DCk RFk EXk ROBISSUE WBk

Superscalar

IF1 IF2 IFs DC RF--- EX1 EX2 --- EX5 WBSuperpipelined

IF DC

RF1 EX1 WB1

RF2 EX2 WB2

RFk EXk WBk

VLIW

RF1 EX1 WB1

RF2 EX2 WB2

RFk EXk WBkD

AT

AF

LOW


Four dimensional representation of the architecture design space <I, O, D, S>

Instructions/cycle ‘I’

Superpipelining Degree ‘S’

Operations/instruction ‘O’

Data/operation ‘D’

Superscalar MIMD Dataflow

Superpipelined

RISC

VLIW

10 100

1010

0.1

Vector

10

SIMD100

CISC


Architecture design space

Architecture K I O D S MparCISC 1 0.2 1.2 1.1 1 0.26RISC 1 1 1 1 1.2 1.2VLIW 10 1 10 1 1.2 12Superscalar 3 3 1 1 1.2 3.6Superpipelined 1 1 1 1 3 3Vector 7 0.1 1 64 5 32SIMD 128 1 1 128 1.2 154MIMD 32 32 1 1 1.2 38Dataflow 10 10 1 1 1.2 12

Typical values of K (# of functional units or processor nodes), and

<I, O, D, S> for different architectures

Mpar = I*O*D*S

Op I_set

S(architecture) = f(Op) * lt (Op)


Overview

• Enhance performance: architecture methods

• Instruction Level Parallelism– limits on ILP

• VLIW– Examples

• Clustering

• Code generation

• Hands-on


General organization of an ILP architecture

Inst

ruct

ion

mem

ory

Inst

ruct

ion

fetc

h un

it

Inst

ruct

ion

deco

de u

nit

FU-1

FU-2

FU-3

FU-4

FU-5

Reg

iste

r fi

le

Dat

a m

emor

y

CPU

Byp

assi

ng n

etw

ork


Motivation for ILP• Increasing VLSI densities; decreasing feature size

• Increasing performance requirements

• New application areas, like– multi-media (image, audio, video, 3-D)– intelligent search and filtering engines– neural, fuzzy, genetic computing

• More functionality

• Use of existing Code (Compatibility)

• Low Power: P = fCVdd2


Low power through parallelism

• Sequential Processor– Switching capacitance C

– Frequency f

– Voltage V

– P = fCV2

• Parallel Processor (two times the number of units)– Switching capacitance 2C

– Frequency f/2

– Voltage V’ < V

– P = f/2 2C V’2 = fCV’2


Measuring and exploiting available ILP

• How much ILP is there in applications?• How to measure parallelism within applications?

– Using existing compiler

– Using trace analysis• Track all the real data dependencies (RaWs) of instructions from issue

window

– register dependence

– memory dependence

• Check for correct branch prediction

– if prediction correct continue

– if wrong, flush schedule and start in next cycle


Trace analysis

Program

For i := 0..2

A[i] := i;

S := X+3;

Compiled code

set r1,0

set r2,3

set r3,&A

Loop: st r1,0(r3)

add r1,r1,1

add r3,r3,4

brne r1,r2,Loop

add r1,r5,3

Trace

set r1,0

set r2,3

set r3,&A

st r1,0(r3)

add r1,r1,1

add r3,r3,4

brne r1,r2,Loop

st r1,0(r3)

add r1,r1,1

add r3,r3,4

brne r1,r2,Loop

st r1,0(r3)

add r1,r1,1

add r3,r3,4

brne r1,r2,Loop

add r1,r5,3How parallel can this code be executed?


Trace analysis

Parallel Trace

set r1,0 set r2,3 set r3,&A

st r1,0(r3) add r1,r1,1 add r3,r3,4

st r1,0(r3) add r1,r1,1 add r3,r3,4 brne r1,r2,Loop

st r1,0(r3) add r1,r1,1 add r3,r3,4 brne r1,r2,Loop

brne r1,r2,Loop

add r1,r5,3

Max ILP = Speedup = Lparallel / Lserial = 16 / 6 = 2.7


Ideal ProcessorAssumptions for ideal/perfect processor:

1. Register renaming – infinite number of virtual registers => all register WAW & WAR hazards avoided2. Branch and Jump prediction – Perfect => all program instructions available for execution3. Memory-address alias analysis – addresses are known. A store can be moved before a load provided addresses not equal

Also: – unlimited number of instructions issued/cycle (unlimited resources), and– unlimited instruction window– perfect caches– 1 cycle latency for all instructions (FP *,/)

Programs were compiled using MIPS compiler with maximum optimization level


Upper Limit to ILP: Ideal Processor

Programs

Inst

ruct

ion

Iss

ues

per

cycl

e

0

20

40

60

80

100

120

140

160

gcc espresso li fpppp doducd tomcatv

54.862.6

17.9

75.2

118.7

150.1

Integer: 18 - 60 FP: 75 - 150

IPC


35

41

16

6158

60

9

1210

48

15

67 6

46

13

45

6 6 7

45

14

45

2 2 2

29

4

19

46

0

10

20

30

40

50

60


Program

Inst

ruct

ion iss

ues

per

cyc

le

Perfect Selective predictor Standard 2-bit Static None

Window Size and Branch Impact• Change from infinite window to examine 2000

and issue at most 64 instructions per cycle FP: 15 - 45

Integer: 6 – 12

IPC

Perfect Tournament BHT(512) Profile No prediction


11

15

12

29

54

10

15

12

49

16

10

1312

35

15

44

910

11

20

11

28

5 56 5 5

74 4

54

5 5

59

45

0

10

20

30

40

50

60

70


Program

Inst

ruct

ion iss

ues

per

cyc

le

Infinite 256 128 64 32 None

Impact of Limited Renaming Registers• Changes: 2000 instr. window, 64 instr. issue, 8K 2-level

predictor (slightly better than tournament predictor)

Integer: 5 - 15 FP: 11 - 45

IP

C

Infinite 256 128 64 32


Program

Instr

ucti

on

issu

es p

er

cy

cle

0

5

10

15

20

25

30

35

40

45

50


10

15

12

49

16

45

7 79

49

16

45 4 4

6 53

53 3 4 4

45

Perfect Global/stack Perfect Inspection None

Memory Address Alias Impact• Changes: 2000 instr. window, 64 instr. issue, 8K

2-level predictor, 256 renaming registers

FP: 4 - 45(Fortran,no heap)

Integer: 4 - 9

IPC

Perfect Global/stack perfect Inspection None


Program

Instr

ucti

on

issu

es p

er

cy

cle

0

10

20

30

40

50

60

gcc expresso li fpppp doducd tomcatv

10

15

12

52

17

56

10

15

12

47

16

10

1311

35

15

34

910 11

22

12

8 8 9

14

9

14

6 6 68

79

4 4 4 5 46

3 2 3 3 3 3

45

22

Infinite 256 128 64 32 16 8 4

Window Size Impact• Assumptions: Perfect disambiguation, 1K Selective predictor, 16

entry return stack, 64 renaming registers, issue as many as window

Integer: 6 - 12

FP: 8 - 45

IPC

Infinite 256 128 64 32 16 8 4


How to Exceed ILP Limits of This Study?

• WAR and WAW hazards through memory: eliminated WAW and WAR hazards through register renaming, but not in memory

• Unnecessary dependences – compiler did not unroll loops so iteration variable

dependence• Overcoming the data flow limit: value prediction,

predicting values and speculating on prediction– Address value prediction and speculation predicts

addresses and speculates by reordering loads and stores. Could provide better aliasing analysis


Conclusions

• Amount of parallelism is limited– higher in Multi-Media– higher in kernels

• Trace analysis detects all types of parallelism– task, data and operation types

• Detected parallelism depends on– quality of compiler– hardware– source-code transformations


Overview• Enhance performance: architecture methods• Instruction Level Parallelism• VLIW

– Examples• C6

• TM

• IA-64: Itanium, ....

• TTA

• Clustering• Code generation• Hands-on


VLIW concept

A VLIW architecture with 7 FUs

Int Register File

Instruction Memory

Int FU

Data Memory

Int FU Int FU LD/ST LD/ST FP FU

Floating PointRegister File

FP FU

Instruction register

Functionunits


VLIW characteristics• Multiple operations per instruction• One instruction per cycle issued (at most)• Compiler is in control• Only RISC like operation support

– Short cycle times– Easier to compile for

• Flexible: Can implement any FU mixture• Extensible / Scalable

However: • tight inter FU connectivity required• not binary compatible !!

– (new long instruction format)


VelociTIC6x

datapath


VLIW example: TMS320C62

TMS320C62 VelociTI Processor

• 8 operations (of 32-bit) per instruction (256 bit)• Two clusters

– 8 Fus: 4 Fus / cluster : (2 Multipliers, 6 ALUs)– 2 x 16 registers– One bus available to write in register file of other cluster

• Flexible addressing modes (like circular addressing)

• Flexible instruction packing

• All instruction conditional

• 5 ns, 200 MHz, 0.25 um, 5-layer CMOS

• 128 KB on-chip RAM


Execunit

Execunit

Execunit

Execunit

Execunit

Register file (128 regs, 32 bit, 15 ports)

Instruction register (5 issue slots)

Data cache

(16 kB)

PCInstruction

cache (32kB)

5 constant5 ALU2 memory2 shift2 DSP-ALU2 DSP-mul3 branch2 FP ALU2 Int/FP ALU1 FP compare1 FP div/sqrt

VLIW example: Trimedia TM1000 DSPCPU


Intel Architecture IA-64

Explicit Parallel Instruction Computer (EPIC)• IA-64 architecture -> Itanium, first realization

Register model:• 128 64-bit int x bits, stack, rotating• 128 82-bit floating point, rotating• 64 1-bit boolean• 8 64-bit branch target address• system control registers


EPIC Architecture: IA-64

• Instructions grouped in 128-bit bundles– 3 * 41-bit instruction– 5 template bits, indicate type and stop location

• Each 41-bit instruction – starts with 4-bit opcode, and – ends with 6-bit guard (boolean) register-id

• Supports speculative loads


Itanium


Itanium 2: McKinley


EPIC Architecture: IA-64

• EPIC allows for more binary compatibility then a plain VLIW:– Function unit assignment performed at run-time– Lock when FU results not available

• See other website for more info on IA-64:– www.ics.ele.tue.nl/~heco/courses/ACA– (look at related material)

http://www.ics.ele.tue.nl/~heco/courses/ACA

Platform-based Design

Documents

Transcript of Platform-based Design