PRACEand the Future of Supercomputing in Europe

Thomas Lippert – Jülich Supercomputing Centre – PRACE Project Management

Computational Challenges in Fundamental Science

Geophysics

Solid State PhysicsStructure of Matter

Plasma PhysicsNuclear Physics

AstrophysikKosmologie

Astrophysics/Cosmology

Unknown Physics

Thomas Lippert - FZJ9.12.2010 2

Challenges inApplied Sciences

EnvironmentClimatology

Pollution

Aging SocietyMedicineBiology

EnergyPlasma Physics

Batteries

MaterialsSpintronics

Nano-TechnologyThomas Lippert - FZJ9.12.2010 3

Challenges inEngineering and Prototyping

•“ Virtual Prototyping”:

Faster to Market ! • Simulations comply with

ethical constraints

Imperativ ofSupercomputing !

Thomas Lippert - FZJ9.12.2010 4

Content

• Background• Commitment• Organization• Resources and Access• Peer Review• Development Projects

59.12.2010 Thomas Lippert - FZJ

Sum of Performance per Country (TOP500)

69.12.2010 Thomas Lippert - FZJ

• Europe must maintain its high standards in computational science and engineering

• Europe has to guarantee independent access to HPC-systems of the highest performance class for all computational scientists in its member states

• Scientific Excellence requires peer review on European scale to foster best ideas and groups

• User requirements as to variety of architectures requires coordinated procurement

• EU and national governments have to establish robust and persistent funding scheme

9.12.2010 7Thomas Lippert - FZJ

Rationale

BACKGROUNDPRACE

89.12.2010 Thomas Lippert - FZJ

PRACE Timeline

HPCEUR HET

2004 2005 2006 2007 2008

PRACE OperationPRACE Implementation Phase (1IP, 2IP)

2009 2010 2011 2012 2013

Phase

PRACE MoU PRACE Preparatory

EU-Grant: INFSO-RI-211528, 10 Mio. €

PRACE (AISBL), a legal entitywith (current) seat location in Brussels

23.4. 2010

99.12.2010 Thomas Lippert - FZJ

HPC on ESFRI Roadmap 2006First comprehensive definition of RIs at European level

RIs are major pillars of the European Research Area

A European HPC service strategic competitiveness attractiveness for researchers access based on excellence supporting industrial

development

109.12.2010 Thomas Lippert - FZJ

European HPC-facilities at the top of an HPC provisioning pyramid– Tier-0: 3-6 European Centres for Petaflop– Tier-0: ? European Centres for Exaflop– Tier-1: National Centres– Tier-2: Regional/University Centres

Creation of a European HPC ecosystem– Scientific and industrial user communities– HPC service providers on all tiers– Grid Infrastructures– The European HPC hard- and software industry

9.12.2010 11Thomas Lippert - FZJ

The ESFRI Vision for a European HPC service

Tier-0

Tier-1

Tier-2

PRACE

DEISA/PRACE

capa

bilit

y

# of systems

9.12.2010 12Thomas Lippert - FZJ

April, 23rd 2010 creation of the legal entity (AISBL) PRACEwith seat location in Brussels, Belgium

16 founding members (today: 20 European members, +1 potential observer)

13

Bulgaria: A Member of PRACE• National Center for Supercomputing Applications

– Joined the PRACE Initiative (MoU) in September 8, 2009– Is a founding member of the PRACE AISBL, established on April

23, 2010 in Brussels• NCSA is actively participating in the PRACE 1st

Implementation Phase project PRACE-1IP:– Further development of the PRACE structure and processes– HPC promotion through user training, support and community

building for academic and industrial users– Enabling applications for petascale systems

NCSA is instrumental for building a strong PRACE user community in Bulgaria !

14

Bulgaria is Increasingly Active in PRACE• NCSA is playing an important role in the planned 2nd

Implementation Phase project PRACE-2IP:– Continuation and extension of the tasks in PRACE-1IP– Participation in a Tier-1 exchange programme – to integrate the

national Supercomputer in the PRACE infrastructure and provide a seamless upgrade path for Bulgarian users from Tier-1 to the Top-level PRACE Tier-0 systems

– Prototyping of technologies for future multi-petascaleSupercomputers and their programming

In PRACE-2IP NCSA is extending its commitment and integration in PRACE !

COMMITMENTMember States

159.12.2010 Thomas Lippert - FZJ

• Binding commitments by Germany, France, Italy, Spain – 100 Mio € over 5 years in terms of cycles– Contribution accounted as TCO

• Access strictly by peer review at a European level– Early access call: May 2010– Start of provision: 1.8.2010

1. Test / evaluation access2. Project access – for a specific project, grant period ~ 1 year3. Programme access – resources managed by a community

– Free-of-charge for European scientific communities16

Provision of Capacity and Access

9.12.2010 Thomas Lippert - FZJ

• PRACE goes for a set of machines with successively increasing capability– 1 PF (2010) + 1.5 PF (2011) + 3 PF (2012) + 3 PF (2012)

+ 5 PF (2013) and will add upgrade steps

– Accumulated capability of more than12 PF in 2013

– PRACE will Include Tier-1 sites (continuing DEISA)

• PRACE will provide support competence centres over several sites

17

Provision of Capability and Support

9.12.2010 Thomas Lippert - FZJ

Tier-0 Ramp Up Until 2012

Country 2010 2011 2012Germany /GCS FZJ 1 PF peak HLRS LRZFrance GENCIItaly CIN ECASpain BSC

9.12.2010 18Thomas Lippert - FZJ

19

High-End HPC Resources granted

Year TFlop/s years granted

Provider

2005 6 DEISA2006 6 DEISA2007 15 DEISA2008 24 DEISA2009 30 DEISA2010 201 PRACE

199.12.2010 Thomas Lippert - FZJ

ORGANIZATIONPRACE

209.12.2010 Thomas Lippert - FZJ

Organization

Director (CEO)

STRATOS

Scientific Steering

CommitteeCouncilExecutive Committee

Financial Advisory

Committee

TechnicalSteering

Committee

OperationCommittee

UserForum

AccessCommittee

Council Chair:Achim Bachem

219.12.2010 Thomas Lippert - FZJ

Scientific Steering CommitteeScope defined in the Statutes of the AISBL• The SSC is responsible for giving opinions on all

matters of scientific and technical nature• 21 members• Members appointed by Council based on a list of

candidates prepared by the SSC• Two year term (renewable twice)• Propose the members of the Access Committee

229.12.2010 Thomas Lippert - FZJ

SSC Peers• Richard Kenway (UK, Part. Phys., Chair)• E.J.Baerends (Netherland, Chemistry)• Kurt Binder (Germany, Cond. Matter)• Miquel Coll (Spain, Biology)• Filippo Giorgi (Italy, Climate)• Olivier Pironneau (France, Math)

239.12.2010 Thomas Lippert - FZJ

THE FIRST PRACE PETAFLOP SYSTEM

PRACE

249.12.2010 Thomas Lippert - FZJ

JSC – The First European Supercomputing Centre

Thomas Lippert 25

PEER REVIEW PRACE

269.12.2010 Thomas Lippert - FZJ

PRINCIPLES

Transparency

Expert assessment

Confidentiality

Right to reply

Prioritisation

Managing interests

No parallel assessment

Ensure fairness to the

science proposed

279.12.2010 Thomas Lippert - FZJ

Types of Access

Preparatory access

− only technical peer review

Project access

− both technical and scientific peer review

Programme access

− both technical and scientific peer review

289.12.2010 Thomas Lippert - FZJ

Council’s Role• Council will be informed on resource distribution

• In case of significant imbalance between total

allocation of resources made to particular member in

comparison with the contributions of such members

• Council will ensure a fair distribution of the

Infrastructure's resources

299.12.2010 Thomas Lippert - FZJ

Procedure1. Technical peer review (system and code suitability)

by hosting centre representatives2. Scientific peer review by 3 external reviewers3. Applicants have the right to reply; these remarks

are sent together with the reviewers comments to the panel

4. Applicants have the right to appeal to a decision of the Council

309.12.2010 Thomas Lippert - FZJ

HPC FOR INDUSTRYPRACE

319.12.2010 Thomas Lippert - FZJ

Numbers of Systems in Industry

54%

46%

TOP500 Europe (145)Industry Rest

63%

37%

TOP500 worldwideIndustry Rest

32

Numbers of Systems in Industry from TOP100

12%

88%

TOP100 EU (34)Industry Rest

11%

89%

TOP100 worldwideIndustry Rest

33

34

Industry Seminars

• Goals– Understanding industrial needs and expectations– Options for industry involvement in PRACE

• Achim Bachem (Council Chair)– HPC is a key technology in the 21st century and we need

independend access to HPC in Europe– PRACE: single legal entity in 2010– PRACE will foster industrial application on top-end– PRACE pushes HPC technology development in EU

35

We learnt:• Jean-Francois Hamelin, EDF

– Why Petaflop/s to generate TeraWatts of Energy?– Planning 100 years ahead– Predictive simulations to narrow down design space– Industry needs well trained academics in HPC

• Jesús Garcia San Luis, Repsol– Oil exploration and Seismic Imaging are important examples: Marriage

of HPC and data intensive computation– Limitation: computer resources and efficient application

• Sharan Kalwani, GM– Worthwhile to share experience with DoE INCITE program– Council of Competitiveness glues academia and industry together

36

DEVELOPMENT PROJECTSPRACE

379.12.2010 Thomas Lippert - FZJ

• 16 Partners (14 countries), funding 10 Million €

• Coordination: FZJ/JSC

• 1.1.2008 – 30.6.2010, successfully finished

• Review October 1st in Brussels

9.12.2010 38Thomas Lippert - FZJ

Preparatory Project

• WP1 Management• WP2 Organizational concept Statutes• WP3 Dissemination, outreach and training• WP4 Distributed computing• WP5 Deployment of prototype systems• WP6 Software enabling for prototype systems • WP7 Petaflop/s systems for 2009/2010• WP8 Future Peta to Exaflop/s technologies

9.12.2010 39Thomas Lippert - FZJ

Work Packages

Particle Physics 23.5

Computational Chemistry 22.1

Condensed Matter Physics 14.2

CFD 8.6

Earth & Climate 7.8

Astronomy & Cosmology 5.8

Life Sciences 5.3

Computational Engineering 3.7

Plasma Physics 3.3

Other 5.8

• Surveys of PRACE partners’ HPC systems and major applications

• 24 systems and 69 applications

• Quantitative basis for selecting representative applications

9.12.2010 40Thomas Lippert - FZJ

Example: Categorisation of Applications (2009)

• 12 core applications, plus 8 additional applications

– NAMD, VASP, QCD, CPMD, GADGET, Code Saturne, TORB,

ECHAM5, NEMO, CP2K, GROMACS, N3D

– Additional: AVBP, HELIUM, TRIPOLI_4, PEPC, GPAW,

ALYA, SIESTA, BSIT

– Synthetic benchmarks for architecture evaluation

• Integrated into JuBE (Juelich Benchmark Environment)9.12.2010 41Thomas Lippert - FZJ

PRACE Benchmark Suite

Installed Petaflop Prototypes (WP5/7/8)

IBM BlueGene/P (FZJ)01-2008 (MPP)

IBM Power6 (SARA)07-2008

Cray XT5 (CSC)11-2008

IBM Cell/Power (BSC)12-2008

IBM Cell/Power (BSC)12-2008

Intel Nehalem/Xeon (CEA/FZJ): installation date April 2009

42

1st Implementation Project

• 21 partners (20 countries), funding 20 Million €

• Coordination: FZJ/JSC

• 1.7.2010 – 30.6.2012, started

• Stronger focus on application scaling in cooperation

with user groups

439.12.2010 Thomas Lippert - FZJ

2nd Implementation Project

• 21 partners (20 countries), funding 20 Million €

• Preparation: FZJ/JSC/PRACE PMO

• Mid 2012 – Mid 2014, proposal submitted

• Scaling of important user codes

• Tier-1 Integration

449.12.2010 Thomas Lippert - FZJ

SUMMARY and OUTLOOK

• PRACE is established• Operation started mid of 2010• Scientific steering group to be approved

on Oct. 5, 2010• Expect > 10 Petaflop/s in 2013• Exascale development projects planned

459.12.2010 Thomas Lippert - FZJ