How’s the Parallel Computing Revolution Going?

How’s the Parallel Revolution Going? 1

How’s the Parallel Computing Revolution

Going?

McKinley

Kathryn S. McKinleyThe University of Texas at Austin

20th Century Simplicity

McKinley

Hardware

software does not change

it just runs faster

How’s the Parallel Revolution Going? 3McKinley

hardware does not change

it just runs faster

Software

20th Century Simplicity

How could they pretend?

McKinley

Hardware Capabilities &

Complexity

sequential interface sequential interface

SoftwareCapabilities &

Complexity

Sequential interface hid explosion in capability & complexity

20th Century Virtuous Cycle

20th Century Languagesinsufficient for software complexity

NativeProgrammingLanguages

McKinley

21st Century Managed Language Revolution

McKinley

Complexity

sequential interface

Managed Languages

SoftwareCapabilities

Processor Evolution

Power 41.3 GHz130nm 174M Tr.267 mm2

2 Cores2001

i72.7 GHz45nm

731M Tr.263mm2

4 Cores x 2 SMT2008

i53.4 GHz32nm

382M Tr.81mm2

2C x 2T2010

Power 52.3 GHz90nm

276M Tr.389 mm2

2 Cores2005

Processor Evolutionwhy multicore?

on chip power constraints & wire delay slowed clock scaling

McKinley

Complexity

✗ sequential interface

Managed Languages

Parallel Hardware

Capabilities

Parallel interface Parallel interface

21st Century Virtuous Cycle?

? Managed Languages

21st Century Virtuous Cycle ?parallel interface

combines time and spacewicked to program

8MB L3

CPU CPU

8KB L1

512KB L2

Pentium 4w/ SMT

CPU CPU

4MB L2

Core 2 Quad

CPU CPU

4MB L2

system bus

32KB 32KB 32KB

Core i7

How is this new virtuous cycle going?

McKinley

What should we measure?

McKinley

performancepowerenergy

native languagesmanaged languages

sequential & parallel programs

McKinley

How do we measure power?

McKinley

Measured Power, Performance & Scaling

19Esmaeilzadeh et al

Looking Back on the Language & Hardware Revolutions:

Measured Power, Performance, and ScalingASPLOS 2011

McKinley

Stephen M. BlackburnAustralian National University

Kathryn S. McKinleyUniversity of Texas at Austin

Hadi EsmaeilzadehUniversity of Washington

Ting CaoAustralian National University

Xi YangAustralian National University

Workload4 groups weighed equally

61 benchmarks from 6 suites

Native Non-Scalable: SPECcpu 2006 Native Scalable: PARSEC 2008

Java Non-Scalable: SPECjvm98, JBB’05 DaCapo’06

Java Scalable: DaCapo’09

Intel Processors5 technology generations from

similar price points

Pentium 4130nm55M Tr.

131mm2

1C x 2T2003

Core 2 D65nm

291M Tr.143mm2

2C 2006

i745nm

731M Tr.263mm2

4C x 2T2008

Atom45nm

47M Tr.36mm2

1C x 2T2008

Core 2 D45nm

228M Tr.82mm2

2C2009

Atom D45nm

176M Tr.87mm2

2Cx2T+GPU2009

i532nm

382M Tr.81mm2

2C x 2T2010

TDP & Measured Power

Esmaeilzadeh et al

2 20 2000

P4 (130)C2D (65)C2Q (65)i7 (45)Atom (45)C2D (45)AtomD (45)i5 (32)

TDP (W) (log)

Measured Power vs Performance

Esmaeilzadeh et al

0.5 510

Performance / Reference Performance

1 2 3 4

2003Pentium 4 (130)

2008Core 2 Duo (45)

2006Core 2 Duo (65)

2008i7 (45)

2010i5 (32)

How is this new virtuous cycle going

for native non-scalable?

McKinley

Native Non-Scalable Performance

McKinley

470.lbm

465.to

437.les

435.gr

434.ze

462.lib

quantu

464.h2

445.go

458.sje

459.Gem

416.ga

444.na

436.ca

ctusADM

400.pe

454.ca

lculix

401.bz

447.de

483.xa

lancbm

482.sp

456.hm

471.om

453.po

473.as

403.gc

450.so

ilc0.00

2C1T 4C1T 4C2T

Native Non-Scalable Energy

McKinley

470.lbm

465.to

437.les

435.gr

434.ze

462.lib

quantu

464.h2

445.go

458.sje

459.Gem

416.ga

444.na

436.ca

ctusADM

400.pe

454.ca

lculix

401.bz

447.de

483.xa

lancbm

482.sp

456.hm

471.om

453.po

473.as

403.gc

450.so

ilc0.00

2C1T 4C1T 4C2T

for Java single threaded?

McKinley

Java Single Threaded Performance

McKinley

antlr fop

luindex

_209_d

_228_j

_213_j

_202_j

_222_m

_201_c

ss0.00

2C1T 4C1T 4C2T

Java Single Threaded Energy

McKinley

antlr fop

luindex

_209_d

_228_j

_213_j

_202_j

_222_m

_201_c

ss0.00

2C1T 4C1T 4C2T

for native scalable?

McKinley

Native Scalable Performance

McKinley

ferret

blacks

choles

raytra

fluida

nimate x26

facesi

bodytr

mcluste

al0.00

2C1T 4C1T 4C2T

Native Scalable Energy

McKinley

ferret

blacks

choles

raytra

fluida

nimate x26

facesi

bodytr

mcluste

al0.00

2C1T 4C1T 4C2T

for Java scalable?

McKinley

Java Multithreaded Performance

McKinley

sunflow

tomcat xal

lusear

_227_m

tradeb

ora pmd h2

2C1T 4C1T 4C2T

Java Multithreaded Energy

McKinley

sunflow

tomcat xal

lusear

_227_m

tradeb

ora pmd h2

2C1T 4C1T 4C2T

Is there hope?

McKinley

parallel interfacecombines time and space

wicked to program

8MB L3

CPU CPU

8KB L1

512KB L2

Pentium 4w/ SMT

CPU CPU

4MB L2

Core 2 Quad

CPU CPU

4MB L2

system bus

32KB 32KB 32KB

Core i7

Vision• Algorithms must be space and time efficient

• Scalable Runtimes– Runtime & application parallelism & concurrency– CMP aware runtime improves application scalability

• Communication– Cache coherency is expensive and performance sensitive– Memory bandwidth scaling is problematic

• Heterogeneity– Move non-critical path off power-hungry cores– Smarter, more aggressive analysis

• Specialization?– Tuned cores? Special purpose cores?

McKinley

Managed Languages

Challenges & Opportunities

McKinley

Must start with a

scalable managed runtime

McKinley

Sequential Managed Programs

McKinley

Application Managed Runtime

SingleCore

• Profiling• Dynamic Analysis• Compilation• Garbage Collection• Other Helper Threads• ……

Steps towards scalability

McKinley

Step 1. Parallel application

ApplicationThreads

Core 0Core 1Core 2Core 3Core 4Core 5Core 6Core 7

Unused cores

Each thread has different running time

McKinley

Step 2. Parallel runtime

Application

Threads

Runtime

Managed Application

Threads

Runtime waits for all application threads to pause

McKinley

Step 3. Parallel & concurrent runtime

Application

Threads

Runtime

Managed Application

Threads

Managed runtime on application’s critical pathmay perturb performance

Steps towards scalability Ideal model

McKinley

Step 4. Minimize perturbation

Application

Threads

Analysis

Application

Threads

Offload work to concurrent runtime threads

Whole runtime task taken off critical path

Steps towards scalability Ideal model

McKinley

Step 4. Minimize perturbation

Application

Threads

Analysis

Application

Threads

Worst case is parallel & concurrent

Scalable VM Services• Profiling (feedback directed optimization)

– Concurrent analysis– More invasive analysis on low-power cores– J. Ha et al. OOPSLA’09, Bond et al., PLDI’10, etc.

• GC– High performance parallel & concurrent GC– High performance mostly non-moving GC– Reduced synchronization overheads– Distributed & scratchpad GC– Blackburn et al. PLDI’10,CACM’08,PLDI’08,SIGMETRICS’04, etc.

• JIT– Concurrent, parallel JIT– Cost-benefit shift with low-power cores– Ha et al. PESPMA’09

• Architecture– Tuned and/or specialized cores for runtime services– Coherence tailored for restricted, common case of GC

McKinley

Today• Profiling (feedback directed optimization)

– Concurrent analysis– More invasive analysis on low-power cores– J. Ha et al. OOPSLA’09, Bond et al., PLDI’10, etc.

• GC– High performance parallel & concurrent GC– High performance mostly non-moving GC– Reduced synchronization overheads– Distributed & scratchpad GC– Blackburn et al. PLDI’10,CACM’08,PLDI’08,SIGMETRICS’04, etc.

• JIT– Concurrent, parallel JIT– Cost-benefit shift with low-power cores– Ha et al. PESPMA’09

• Architecture– Tuned and/or specialized cores for runtime services– Coherence tailed for restricted, common case of GC

McKinley

Garbage Collection

McKinley

Isn’t Garbage Collection retro?

McKinley

Mark-CompactStyger, 1967

Mark-SweepMcCarthy, 1960

Semi-SpaceCheney, 1970

canonical algorithms

Programmer Productivity

McKinley

Programmer Productivity

& Performance?

McKinley

GC FundamentalsAlgorithmic Components

Allocation Reclamation

McKinley

Identification

Bump Allocation

Free List

Tracing(implicit)

Reference Counting(explicit)

Sweep-to-Free

Compact

Evacuate3 1

Mark-Compact [Styger 1967]Bump allocation + trace + compact

GC FundamentalsCanonical Garbage Collectors

Sweep-to-Free

Compact

Evacuate

Mark-Sweep [McCarthy 1960]Free-list + trace + sweep-to-free

Semi-Space [Cheney 1970]Bump allocation + trace + evacuate

Garbage Collection

Total PerformanceSemiSpaceMarkCompactMarkSweep

Performance PathologiesMark-Sweep, Mark-Compact, Semi-Space

Mutator

Minimum Heap

Geometric mean of DaCapo’06, jvm98, and jbb2000 on 2.4GHz Core 2 Duo

Mark-SweepPoor locality

Semi-SpaceSpace

inefficient

Mark-Compact expensive multi-pass

McKinley

Can we have space and time efficiency?

McKinley

Mark-RegionPLDI 2008

McKinley

Kathryn S. McKinley Stephen M. BlackburnUniversity of Texas at Austin Australian National University

Mark-Regionwith Sweep-To-Region

McKinley

Sweep-to-Free

Compact

Evacuate

Reclamation

Sweep-to-Region

Mark-SweepFree-list + trace + sweep-to-free

Mark-CompactBump allocation + trace + compact

Semi-SpaceBump allocation + trace + evacuate

Mark-RegionBump + trace + sweep-to-region

Naïve Mark-Region

McKinley

• Contiguous allocation into regionsExcellent locality– Objects cannot span regions

• Simple mark phase– Mark objects and their region

• Free unmarked region

Region Size?Lines and Blocks

McKinley

Small Regions

Large Regions

✗ Fragmentation (can’t fill blocks)

✓ More contiguous allocation ✗ Fragmentation (false marking)

Lines & BlocksN pages approx 1 cache line

✓ Less fragmentation Objects span lines

✓ Fast common case Lines marked with objects

✗ Increased metadata o/h✗ Constrained object sizes

TLB locality, cache locality Block > 4 X max object sizeFree FreeRecyclable lines Recyclable lines

Allocation Policy(Recycling)

McKinley

• Recycle partially marked blocks first Minimizes fragmentation Maximizes sharing of freed blocks

Immix Mark-RegionParallel

Opportunistic defragmentation

Overflow allocation

Implicit marking

McKinley

Garbage Collection

Total Performance

MarkSweepMarkCompactSemiSpaceImmix

Immix Mark-RegionBump Allocation + Trace + Sweep-to-Region

Mutator

Minimum Heap

✓ Simple, very fast collection

✓Space

efficient✓Good

locality

✓Excellent

performance

Geometric mean of DaCapo’06, jvm98, and jbb2000 on 2.4GHz Core 2 DuoMcKinley

A Better Space-Time Tradeoff 65

Space & time efficiency Why now?

8MB L3

The PresentParallel interface

combines space & timewicked to program

McKinley

CPU CPU

8KB L1

512KB L2

Pentium 4w/ SMT

CPU CPU

4MB L2

Core 2 Quad

CPU CPU

4MB L2

system bus

32KB 32KB 32KB

Core i7

The FutureA parallel ecosystem?

space time efficiency

Parallel software stackruntime

applicationsalgorithms

McKinley

Software Challenges and Opportunities

Communication (efficient coherency)Analysis (off critical path, new analyses)GC (concurrent, parallel, high throughput)JIT (concurrent, parallel, more aggressive)Heterogeneity (exploit it)Memory (PCM, bandwidth limits)

McKinley

HardwareChallenges and Opportunities

Heterogeneity– Tune cores to specific workloads?– Specialize for workloads?

Coherence– SMT coherency does not scale– Software guarantees for simplified protocols?

Memory/Cache– Optimize access behavior of managed

languages

McKinley

The Future?

McKinley

Thank you

How’s the Parallel Computing Revolution Going?

Documents

Transcript of How’s the Parallel Computing Revolution Going?

So, How’s Your Self-Esteem?

The Why’s and How’s of Pension Reform

HOW’S THE WEATHER ours - Siemens STEM Day

When I say: Ina Yini? (How’s your Evening?)

HOW’S YOUR EVERYDAY MATH?

How’s Life? 2013 - OECD · 2018. 3. 1. · How’s Life? 2013 Measuring WeLL-being How’s Life? 2013 Measuring WeLL-being Every person aspires to a good life. But what does “a

So… How’s the Weather?

Welcome, How’s tricks?

Why’s & How’s of New Product Development

How’s it Working? Evaluating Your Program

How’s Your Spiritual Life?. How’s your spiritual life? Your answer: What do you mean? Your answer:

IFRS 17: How’s your biggest accounting standard ... · IFRS 17: How’s your biggest accounting standard implementation going? Thursday, December 6, 2018 | #KPMGIC

Web 3.0: How’s That Panning Out Then…

How’s Your Memory? - BioEd Onlinebioedonline.org › BioEd › cache › file › 6DC8E362-9DA8-1499-1155788… · How’s Your Memory? Can you remember what is in the drawing of

HOW’S THE WEATHER (1-2 HOURS) - Eco Energy for Schools · HOW’S THE WEATHER (1-2 HOURS) STEPS FOR HOW’S THE WEATHER 1. Warm-up Activity: Begin the class by asking students what

“So How’s My Child Doing ... - Bonnie Campbell Hill conference/Continuum... · “So How’s My Child Doing?” Developmental Continuums: Connecting Instruction, Standards, Assessment

How’s the Standard Model Doing at the Tevatron ?

How’S Your Memory

How’s the weather?

The what’s and how’s of eligibility verification.