Dept. of Electrical & Computer Engineering

Self-Morphing Cores for Higher Power Efficiency and

Improved ResilienceNithesh Kurella, Sudarshan Srinivasan

Israel Koren, Sandip Kundu

Dept. Of Electrical & Computer Engineering

University of Massachusetts at Amherst

2

Outline

Asymmetric Multicores – benefits and limitations Challenge: Match the capabilities of the core to current computational needs of applications Approach: Self-morphing core that can adapt faster to application demands Main goal: Higher power efficiency with limited or no decrease in performance Secondary goal: Improve resilience to soft errors

3

Asymmetric Multicore Processors (AMPs)

Cores of different computing capabilities Different performance and power

characteristics

Typically consists of• Out-of-order (OOO) cores (High performance core)• In-Order (InO) cores (Low performance & Energy efficient core)

Core 1Core 1 Core 2

Core 2

Asymmetric multicore

OOO InOrder

4

Commercial ARM Big/Little Architecture

Use the right processor for the right task

5

Limitations of current AMP Architectures

• (1) Limited architectural flexibility• Limited choices of core capabilities (2)• Fixed number of large and small cores

• (2) Limited thread to core mapping flexibility• Applications have phases (time periods)

with different computational requirements • Swapping threads between cores can

reduce the power consumed, but• Task migration has a high overhead

(need to transfer thread state/data) Thread migration at granularity of

millions of instructions (coarse grain)

Core 1Core 1 Core 2

Core 2

Thread swapping

L1 cacheL1 cache

L1 cache

L2 cacheL2 cache

Thread 1Thread 2

6

Can we benefit from more than two core types?

Performance/Watt of 10 benchmarks

7

Applications can benefit from fine-grain assignment ~1000s of instructions [Lukefahr et al. Micro’12]

Temporal (fine-grain) variations for sjeng

8

Can we exploit Fine-Grain Changes in demands?• Develop a fine grain application to core assignment to

improve power efficiency without high migration overhead

A self-morphing core: can morph into two (or more) architecture types (core modes) with varying execution width and resource sizes Significantly lower thread migration overhead• Critical units (register file, caches and branch predictor)

are common to all core modes

9

Morphable Architectures

A Morphable architecture where OOO core turns into InO was proposed by [Lukefahr et al., Micro 2012] InO has much lower power consumption, but• Turning OOO core into InO in run time involves

significant micro-architecture changes• Higher design cost and verification

Questions to be investigated:1. Are two architecture modes (core types) sufficient

to match the large variance in application needs?2. Is an InO mode necessary, as its inclusion

complicates the design?

10

Designing a Self-Morphing Core

Goal: Design a core than can morph into various OOO modes with varying execution width and resource sizes

Questions:• How many core modes should we have?• What should be the architectural parameters of

these modes?• How fine-grained should mode switches be?• Under what conditions should we switch from one

mode to another?

11

Core Design Space Exploration

Find core types that would provide best performance/Watt at fine-grain instruction granularities

About 11,000 design combinations - pruned to 300 Pruning: group structures to be resized concurrently rather

than independent structure resizing

12

Number and Types of Cores (out of 300)

Objective: achieve the highest IPS2/Watt by allowing switching between core types at ~2K instruction granularity• IPS2/Watt is used instead of IPS/Watt to emphasize

performance Best core configuration selected from 300 candidates for

each 2K instruction interval based on IPS2/Watt Need to to determine IPS2/Watt improvement threshold (for

a mode switch) • A threshold of 20% yields 10 core types with only 15%

improvement• A higher threshold reduced the number to 4 with a higher

IPS2/Watt improvement Fixed number of core types to 4

13

Core Types obtained

128 1.6

Frequency and ROB size analysis for IPS2/watt for the Average (AC) core

Avg. Power consumed

14

How to decide on a mode switch?

Switching decision between modes is based on IPS2/Watt

To compute IPS2/Watt , we need to estimate performance and power• Hardware performance counters (PMCs) are used

to estimate performance and power at fine-grain granularity

Need to estimate power and performance on the currently active mode as well as 3 other core modes

15

Power & IPC Estimation based on PMCs

1. Identified 14 counters that are correlated with performance & power

2. Selected the smallest number of counters that have the highest possible correlation coefficient R2

4. Regression analysis power of a given core type =

f(chosen counters)IPC of a given core type =

f(chosen counters)

Explored HPCs

Stalls (S)

# Fetched instructions (F)

# Branch mispredictions (BMP)

L1 hit (L1h)

L1 miss (L1 miss)

L2 hit (L2h)

L2 miss (L2m)

TLB miss (TLB m)

# retired INT instructions (INT)

# retired FP instructions (FP)

# retired Ld instructions (Ld)

# retired St instructions (St)

# retired Branch instructions (Br)

IPC

specu

lati

veH

it/M

iss

Reti

red

Explored PMCs

16

Selection of PMCs

PMC AC => Power NC, denotes using the counters of the average core to estimate the power on the narrow core

17

Obtained Power and IPC expressions

18

Average Estimation Error using PMCs

PMC AC => Power/IPC denotes the average error in estimating power and IPC for the 3 other core types using the PMCs of the average core (AC)

Maximum average error of 16%; adequate accuracy

19

Frequency of core mode switching

Power and performance are monitored every “window” of instructions (size to be determined)

Want to identify stable phase and prevent frequent morphing during a transient behavior

Wait for several windows (“history_depth”) and follow the most frequent recommendation

Morphing decision based on behavior during the last n retired instructions, where

n=Window_Size x History_Depth

20

Frequency of morphing decisions Window_size=500 and history_depth=4 combination

yields the highest IPS2/Watt

Decision to reconfigure (or not) is taken every 2K instructions

21

Overhead for Morphing: DVFS

Morphing from one core mode to another involves frequency change

Traditional DVFS is applied at coarse grain granularity (millions of cycles) due to its high overhead

Use of On-Chip regulator reduces the time needed for scaling voltage to hundreds of cycles

The selected fine-grain DVFS technique (upon mode switch) has an overhead of 200 cycles

22

Overall morphing overhead

In-core morphing retains processor state and cache content• register file, caches and branch predictor are shared

Still additional cycles are needed upon reconfiguration• Powering off/on fetch, decode units• Power gating banks of ROB, RAT and LSQ units (10 clock

cycles to power off one bank)• Pipeline drain on every core mode switch• The fine grain DVFS overhead is 200 cycles

On Average morphing overhead takes 500 cycles. Exact overhead determined in run-time depending on mode we morph into

23

Results and Analysis

Gem5 simulator; McPAT to measure power

We evaluate our proposed scheme with SPEC2006 and SPEC2000 benchmark suites

Benchmarks ran for 2 billion with skipping the first 2 billion instructions

24

Time spent in core modes

• Each of the 4 modes is useful (>40%) for several benchmarks

25

Improvement in IPS2/Watt

Average IPS2/Watt improvement of 36% vs. AC baseline

26

The self-morphing scheme provides 33% energy savings compared to the AC core

Unconstrained

Energy savings of self-morphing

27

IPC of switching schemes (with AC as baseline)

Unconstrained

28

Impact on IPC of morphing scheme overhead

Morphing overhead increase from 500 to 1000 cycles reduces the average performance of FineGrain_PMC by only 3.5%

29

Comparison to Big-Little (OOO-InOrder): IPS2/Watt

30

Is adding an InOrder core necessary?

31

Impact of self-morphing on core reliability

Raw SER – Soft Error Rate independent of design details

Common reliability metric: Vulnerability to soft errors or Soft Error Rate (SER)

AVF – Architectural Vulnerability Factor

32

Performance/Watt vs SER tradeoff

Minimize RPE – Reliability Power_Efficiency

SER is the lowest for the SM core Tradeoff between SER and IPS2/Watt Use a Cobb-Douglas type production function

a+b=1

33

Decrease in SER, increase in IPS/Watt and IPC

34

Conclusions

Thread migration/core-hopping is expensive• Performed at coarse grain

In-core morphing preserves state & cache• Minimal overhead• Allowing frequent morphing - larger gains

Achieving average improvement in IPS2/Watt of 36%, power savings of 33% and performance increase of 16%

If reliability is also a design objective in-core morphing can achieve 11% SER reduction for a lower (24%) IPS/Watt improvement and a lower (6%) performance increase

35

Backup Slides

36

Error in estimation of AVF based on PMCs

PMCs AC PMCs NC PMCs LW PMCs SM

37

Counter selection heuristic

Input: PMCs & Power/IPC trace (of representative workloads) Objective: Minimum no. of PMCs to adequately fit power and

IPC Metric: Correlation coefficient, R2, of the fit (higher the better)

Approach:− Search counter space (14) iteratively− Each iteration:

• Choose a new counter that best fits IPC/Power trace along with previously selected counters

− Plot R2 coefficient− Best set of counters in region where R2 saturates

38

High Branch mispredicition rate

Execution of astar – SM preferred

39

Execution of mcf – NC preferred

Memory-bound application w/high L2 miss rate

40

CPU-bound applications w/high resource use

Execution of bzip2 – LW preferred

41

Power Constrained core designs

Core types for a 2W power constraint:

Core types for a 1.5W power constraint:

42

Comparing Power Constrained and Un-Constrained cores

• Considerable energy saving vs. AC core type

Unconstrained

43

Error distribution

Distribution of error in estimating IPC in various core types using PMCs of narrow core (NC)

Deviation of errors from mean is low with up to 80% between +/- 10% from the mean

44

Number of switches in self-morphing scheme

Average number of switches is 1250 in 100M instructions Every 2K instructions there is a 25% probability to perform a mode switch

45

IPC of switching schemes (with AC as baseline)

Unconstrained

46

Morphable architecture With/Without InO core

• 3-mode morphing scheme with the inclusion of InO provides only 2% extra IPS2/Watt benefit compared to the 2-mode morphing between OOO(AC)-OOO(SM)

47

Mediabench/MyBench

48

(2) Is InO mode necessary?• InO core has smaller cache and array structures with much

lower static power• Cache/Array leakage is no longer a problem as tri-gates

cut leakage by 10X at 22nm• Use instead a small OOO: Fetch, issue width of 1 and smaller

ROB, LSQ and IQ• For most benchmarks IPC/Watt of InO and small OOO are

comparable

Simulation with MCPAT 22nm tri-gate models

49

Effective Soft Error Rate (SER) Reduction

50

Microarchitecture of Morphable Core

IQ, ROB, LSQ are resized dynamically when morphing from one core type to another

ROB, LSQ and IQ are implemented as banked structures Resizing involves turning on/off banks Reduce/increase fetch width, Power-off/on half the decoders