Measuring HPC: Performance, Cost, & Value

Experts in numerical software andHigh Performance Computing

Measuring HPC:Performance, Cost & Value

IDC HPC User ForumAustin, TX6-8 September 2016

Andrew JonesVice-President Strategic HPC Services

WHY MEASURE?

3High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com

Instead, use your time tooptimize benefit from those decisions make sure the next decision is right

Weak ideaDon’t waste your valuable time justifying past decisions

“Justify the investments made”


Optimize usage to maximizescience/engineering impact and business value

Where possible, “fairness” innovation vs. production?

Understand how the current HPC capability is used


Use data on value delivered by current HPC capability

so future investments can be made with confidence

at the right scale, type and timing for maximum value


Make best use of current capability

Make best decisions for the future capability

MEASURING PERFORMANCE


Benchmarks

“How fast is my code?”

“How fast is this supercomputer?”

“Which system/processor

is faster?”

“How does my code scale?”


Drive value for money + technical confidence

Guide procurement

planning

Support procurement

evaluation

Acceptance testing


0

2

4

6

8

10

12

14

A B C D E F G

Perfo

rman

ce

0

2

4

6

8

10

12

14

A B C D E F G

Perfo

rman

ce


How well is the HPC facility/service run?

How well is it used?

How fair is it?

Cost, business impact, … [in next section]


How effectively do teams work together?

Productivity and happiness of users, service staff, etc.

Organizational structures

Governance

Etc.


… but also affects everyone

Performance becomes a discipline in its own right …

Lots of things to measure!Not just code speed

MEASURING COST


Computer Storage Network Staff Electricity

Cooling Facility costs Security Time (!) Resilience

Measurement & Reporting Financing effects Insurance Irrecoverable

taxesProcurement

project

Application porting Software licenses User training &

support Documentation Commissioning

Warranty, support,

maintenanceDecommissioning Availability /

throughput Data migration … and more …


?Which costs

can you control?

Which costs does your

budget bear?

How are you incentivized

against other costs?

How will costs change over

time?


Total Cost of Ownership (TCO)

• What are all the costs involved in owning and operating this HPC service?

• Most important to HPC owners/managers

• It costs money even if not used (always)

Total Cost of Solution (TCS)

• What are all the costs I will incur using this HPC service for a specific project?

• Most important to HPC users

• It costs nothing if not used (sometimes)


Much more complicated than just price of hardware …

Which costs can be controlled?

TCO vs. TCS

MEASURING VALUE


ROI is only part of the answer

Value to the business might be binary:project/product not possible without HPC

Determining value can be hard & sometimes subjective

SUMMARY


Measuring HPC can be very powerfulfor the business and the user when done well

Measuring performance, costs and valueare all complex and can be hard to get right

Measurement is only the start …effective reporting can be even more important!

NAG HPC consulting can help you


Experts in High Performance Computing, Algorithms and Numerical Software Engineering

www.nag.com | blog.nag.com | @NAGtalk

@hpcnotes

Please come to our tutorial at SC16 in Austin this November:Essential HPC Finance Practice – TCO, cost recovery models, etc.

Measuring HPC: Performance, Cost, & Value

Technology

Transcript of Measuring HPC: Performance, Cost, & Value