Dept. of Electrical & Computer Engineering
Self-Morphing Cores for Higher Power Efficiency and
Improved ResilienceNithesh Kurella, Sudarshan Srinivasan
Israel Koren, Sandip Kundu
Dept. Of Electrical & Computer Engineering
University of Massachusetts at Amherst
2
Outline
Asymmetric Multicores – benefits and limitations Challenge: Match the capabilities of the core to current computational needs of applications Approach: Self-morphing core that can adapt faster to application demands Main goal: Higher power efficiency with limited or no decrease in performance Secondary goal: Improve resilience to soft errors
3
Asymmetric Multicore Processors (AMPs)
Cores of different computing capabilities Different performance and power
characteristics
Typically consists of• Out-of-order (OOO) cores (High performance core)• In-Order (InO) cores (Low performance & Energy efficient core)
Core 1Core 1 Core 2
Core 2
Asymmetric multicore
OOO InOrder
4
Commercial ARM Big/Little Architecture
Use the right processor for the right task
5
Limitations of current AMP Architectures
• (1) Limited architectural flexibility• Limited choices of core capabilities (2)• Fixed number of large and small cores
• (2) Limited thread to core mapping flexibility• Applications have phases (time periods)
with different computational requirements • Swapping threads between cores can
reduce the power consumed, but• Task migration has a high overhead
(need to transfer thread state/data) Thread migration at granularity of
millions of instructions (coarse grain)
Core 1Core 1 Core 2
Core 2
Thread swapping
L1 cacheL1 cache
L1 cache
L2 cacheL2 cache
Thread 1Thread 2
6
Can we benefit from more than two core types?
Performance/Watt of 10 benchmarks
7
Applications can benefit from fine-grain assignment ~1000s of instructions [Lukefahr et al. Micro’12]
Temporal (fine-grain) variations for sjeng
8
Can we exploit Fine-Grain Changes in demands?• Develop a fine grain application to core assignment to
improve power efficiency without high migration overhead
A self-morphing core: can morph into two (or more) architecture types (core modes) with varying execution width and resource sizes Significantly lower thread migration overhead• Critical units (register file, caches and branch predictor)
are common to all core modes
9
Morphable Architectures
A Morphable architecture where OOO core turns into InO was proposed by [Lukefahr et al., Micro 2012] InO has much lower power consumption, but• Turning OOO core into InO in run time involves
significant micro-architecture changes• Higher design cost and verification
Questions to be investigated:1. Are two architecture modes (core types) sufficient
to match the large variance in application needs?2. Is an InO mode necessary, as its inclusion
complicates the design?
10
Designing a Self-Morphing Core
Goal: Design a core than can morph into various OOO modes with varying execution width and resource sizes
Questions:• How many core modes should we have?• What should be the architectural parameters of
these modes?• How fine-grained should mode switches be?• Under what conditions should we switch from one
mode to another?
11
Core Design Space Exploration
Find core types that would provide best performance/Watt at fine-grain instruction granularities
About 11,000 design combinations - pruned to 300 Pruning: group structures to be resized concurrently rather
than independent structure resizing
12
Number and Types of Cores (out of 300)
Objective: achieve the highest IPS2/Watt by allowing switching between core types at ~2K instruction granularity• IPS2/Watt is used instead of IPS/Watt to emphasize
performance Best core configuration selected from 300 candidates for
each 2K instruction interval based on IPS2/Watt Need to to determine IPS2/Watt improvement threshold (for
a mode switch) • A threshold of 20% yields 10 core types with only 15%
improvement• A higher threshold reduced the number to 4 with a higher
IPS2/Watt improvement Fixed number of core types to 4
13
Core Types obtained
128 1.6
Frequency and ROB size analysis for IPS2/watt for the Average (AC) core
Avg. Power consumed
14
How to decide on a mode switch?
Switching decision between modes is based on IPS2/Watt
To compute IPS2/Watt , we need to estimate performance and power• Hardware performance counters (PMCs) are used
to estimate performance and power at fine-grain granularity
Need to estimate power and performance on the currently active mode as well as 3 other core modes
15
Power & IPC Estimation based on PMCs
1. Identified 14 counters that are correlated with performance & power
2. Selected the smallest number of counters that have the highest possible correlation coefficient R2
4. Regression analysis power of a given core type =
f(chosen counters)IPC of a given core type =
f(chosen counters)
Explored HPCs
Stalls (S)
# Fetched instructions (F)
# Branch mispredictions (BMP)
L1 hit (L1h)
L1 miss (L1 miss)
L2 hit (L2h)
L2 miss (L2m)
TLB miss (TLB m)
# retired INT instructions (INT)
# retired FP instructions (FP)
# retired Ld instructions (Ld)
# retired St instructions (St)
# retired Branch instructions (Br)
IPC
specu
lati
veH
it/M
iss
Reti
red
Explored PMCs
16
Selection of PMCs
PMC AC => Power NC, denotes using the counters of the average core to estimate the power on the narrow core
17
Obtained Power and IPC expressions
18
Average Estimation Error using PMCs
PMC AC => Power/IPC denotes the average error in estimating power and IPC for the 3 other core types using the PMCs of the average core (AC)
Maximum average error of 16%; adequate accuracy
19
Frequency of core mode switching
Power and performance are monitored every “window” of instructions (size to be determined)
Want to identify stable phase and prevent frequent morphing during a transient behavior
Wait for several windows (“history_depth”) and follow the most frequent recommendation
Morphing decision based on behavior during the last n retired instructions, where
n=Window_Size x History_Depth
20
Frequency of morphing decisions Window_size=500 and history_depth=4 combination
yields the highest IPS2/Watt
Decision to reconfigure (or not) is taken every 2K instructions
21
Overhead for Morphing: DVFS
Morphing from one core mode to another involves frequency change
Traditional DVFS is applied at coarse grain granularity (millions of cycles) due to its high overhead
Use of On-Chip regulator reduces the time needed for scaling voltage to hundreds of cycles
The selected fine-grain DVFS technique (upon mode switch) has an overhead of 200 cycles
22
Overall morphing overhead
In-core morphing retains processor state and cache content• register file, caches and branch predictor are shared
Still additional cycles are needed upon reconfiguration• Powering off/on fetch, decode units• Power gating banks of ROB, RAT and LSQ units (10 clock
cycles to power off one bank)• Pipeline drain on every core mode switch• The fine grain DVFS overhead is 200 cycles
On Average morphing overhead takes 500 cycles. Exact overhead determined in run-time depending on mode we morph into
23
Results and Analysis
Gem5 simulator; McPAT to measure power
We evaluate our proposed scheme with SPEC2006 and SPEC2000 benchmark suites
Benchmarks ran for 2 billion with skipping the first 2 billion instructions
24
Time spent in core modes
• Each of the 4 modes is useful (>40%) for several benchmarks
25
Improvement in IPS2/Watt
Average IPS2/Watt improvement of 36% vs. AC baseline
26
The self-morphing scheme provides 33% energy savings compared to the AC core
Unconstrained
Energy savings of self-morphing
27
IPC of switching schemes (with AC as baseline)
Unconstrained
28
Impact on IPC of morphing scheme overhead
Morphing overhead increase from 500 to 1000 cycles reduces the average performance of FineGrain_PMC by only 3.5%
29
Comparison to Big-Little (OOO-InOrder): IPS2/Watt
30
Is adding an InOrder core necessary?
31
Impact of self-morphing on core reliability
Raw SER – Soft Error Rate independent of design details
Common reliability metric: Vulnerability to soft errors or Soft Error Rate (SER)
AVF – Architectural Vulnerability Factor
32
Performance/Watt vs SER tradeoff
Minimize RPE – Reliability Power_Efficiency
SER is the lowest for the SM core Tradeoff between SER and IPS2/Watt Use a Cobb-Douglas type production function
a+b=1
33
Decrease in SER, increase in IPS/Watt and IPC
34
Conclusions
Thread migration/core-hopping is expensive• Performed at coarse grain
In-core morphing preserves state & cache• Minimal overhead• Allowing frequent morphing - larger gains
Achieving average improvement in IPS2/Watt of 36%, power savings of 33% and performance increase of 16%
If reliability is also a design objective in-core morphing can achieve 11% SER reduction for a lower (24%) IPS/Watt improvement and a lower (6%) performance increase
35
Backup Slides
36
Error in estimation of AVF based on PMCs
PMCs AC PMCs NC PMCs LW PMCs SM
37
Counter selection heuristic
Input: PMCs & Power/IPC trace (of representative workloads) Objective: Minimum no. of PMCs to adequately fit power and
IPC Metric: Correlation coefficient, R2, of the fit (higher the better)
Approach:− Search counter space (14) iteratively− Each iteration:
• Choose a new counter that best fits IPC/Power trace along with previously selected counters
− Plot R2 coefficient− Best set of counters in region where R2 saturates
38
High Branch mispredicition rate
Execution of astar – SM preferred
39
Execution of mcf – NC preferred
Memory-bound application w/high L2 miss rate
40
CPU-bound applications w/high resource use
Execution of bzip2 – LW preferred
41
Power Constrained core designs
Core types for a 2W power constraint:
Core types for a 1.5W power constraint:
42
Comparing Power Constrained and Un-Constrained cores
• Considerable energy saving vs. AC core type
Unconstrained
43
Error distribution
Distribution of error in estimating IPC in various core types using PMCs of narrow core (NC)
Deviation of errors from mean is low with up to 80% between +/- 10% from the mean
44
Number of switches in self-morphing scheme
Average number of switches is 1250 in 100M instructions Every 2K instructions there is a 25% probability to perform a mode switch
45
IPC of switching schemes (with AC as baseline)
Unconstrained
46
Morphable architecture With/Without InO core
• 3-mode morphing scheme with the inclusion of InO provides only 2% extra IPS2/Watt benefit compared to the 2-mode morphing between OOO(AC)-OOO(SM)
47
Mediabench/MyBench
48
(2) Is InO mode necessary?• InO core has smaller cache and array structures with much
lower static power• Cache/Array leakage is no longer a problem as tri-gates
cut leakage by 10X at 22nm• Use instead a small OOO: Fetch, issue width of 1 and smaller
ROB, LSQ and IQ• For most benchmarks IPC/Watt of InO and small OOO are
comparable
Simulation with MCPAT 22nm tri-gate models
49
Effective Soft Error Rate (SER) Reduction
50
Microarchitecture of Morphable Core
IQ, ROB, LSQ are resized dynamically when morphing from one core type to another
ROB, LSQ and IQ are implemented as banked structures Resizing involves turning on/off banks Reduce/increase fetch width, Power-off/on half the decoders
Top Related