EU Astrophysics Network Overview

www.eu-network.orgSources of Gravitational Radiation

EU Astrophysics Network Overview

Ed Seidel Albert Einstein Institute

Principal Network Coordinator

Ed SeidelAlbert Einstein Institute


Scientific and technological reasons for carrying out

research in our field Exploring Einstein’s General RelativityWant to develop theoretical lab to probe this fundamental

theory Fundamental theory of Physics (Gravity) Among most complex equations of physics Predict black holes, gravity waves, want much more

Exciting new field: Gravitational Wave Astronomy LIGO, VIRGO, GEO, LISA, … ~ $1 Billion worldwide! Fundamentally new information about Universe A last major test of Einstein’s theory: do GWs exist?

A century later, both of these developments happening at the same time: very exciting coincidence!

But, the community needed to carry out theoretical work is lacking…Not enough people or training



This EU Network Astrophysics

10 EU Institutions, 3 years, €1.5MContinue these problemsEntire Community becoming Grid enabled

NSF Black Hole Grand Challenge

8 US Institutions, 5 years, $4M

Solve problem of colliding black holes (try…)

NASA Neutron Star Grand Challenge

5 US Institutions, 3years, $1.4MSolve problem of colliding neutron stars (try…)

QuickTime™ and a decompressor

are needed to see this picture.

NSF ASC Project

5 US Institutions, 3 years, $2.2MDevelop “Collaboratory”

EU GridLab10 EU Institutions, 3 years, €5MDevelop Grid TechFor these projects

Grand Challenge Collaboratories

Building Communities to Solve These Problems



Socio-Economic Aspects of our Project for the EU…

Actually came up with an answer to this one! We work hard together,

but… We have fun, too.

– Actively building our Network community

– Probably first Network to support socio-economic tourism with new €

Having Fun Explaining BHSpin

Listening to a talk



Network research objectives

The formation of a close alliance among the different expert groups to solve urgent problems required for GWA, too large and complex for any single group. Such a Network was needed, but did not exist.

The development and training of a young community of researchers to populate this emerging research area of GWA. Strong, but somewhat isolated European groups already in

relevant areas Need coordinated effort to build up the broad-based expertise

required for a systematic attack on these problems. The development of a community simulation code for

relativistic astrophysics Simulation tools are essential for the success of this research

area. Cactus is an important part of the training program for this

Network.



The development of appropriate approximation tools to provide better physical understanding and confirmation of numerical simulations. Perturbation theory Post-Newtonian Theory

The application of these numerical and approximation tools to a set of core astrophysics problems GW signals from NS, BS, ??? Most promising sources for LIGO/GEO/VIRGO, … Good focus problems for training a community



Theoretical ToolsEinstein Eqs

Pert. theory

Post-NewtonianTheoryEnabler

s

Focus Physics Areas

Outreach /Collab

GridLab

ZIB/GarchingLBL/NCSA/ANL NCSA/WashU

CactusCommunity

NSF KDI

Computational Tools

Cactus

Grid

Viz

KDI

3D BH Collisions Other (strange stars,Core collapse, etc)

3D NS processes andmergers

World

General View of Network



Our Team7 Focus areas, links created, strengthened by

Network

Valencia

Ports

SOTON

Meudon

Jena

Palma

Trieste

AUTHRome

WashU

Potsdam

Jena

Cactus Dev/Training

CS EffortsWorldwide

PotsdamPorts

SOTON

Valencia

Rome

Trieste

AUTH

BH Data/Evolutions

Potsdam

Nonlinear GR HydroAnd Applications

Valencia

Trieste

Meudon

Potsdam

Characteristic Codes

SOTON

Meudon

Palma

RomeAUTH

JenaSOTON

Ports

Palma

Stellar Pert Theory

Valencia

Jena

AUTH

Palma

Perturbative Time Evolutions

Valencia

JenaJena

Post-Newtonian Schemes

Ports

Palma

Rome

Valencia



Project View: Code Leverage

NS Studies

Meudon BBH Meudon BNS

Hydra

Projects extremely CollaborativeBuild on each otherShare Modules or Build them togetherCactus an important collaborative technology

EE’s

MOL

Horizon Finders

Wave Extractors

Lazarus

Elliptics

Parallel I/O

Gauges

AMR

EE’s

MOL

Horizon Finders

Wave Extractors

Lazarus

Elliptics

Parallel I/O

Gauges

AMREE’s Community

Workshop this spring/fallimproves EE’s, all simulations

SuperEEs Super

EEs

Vacuum BH Studies



Developing a Community:People Leverage

Actual Network funding rather small 10-12 positions (some positions converted to 2 for shorter

times)– 6-7 postdocs, 4-5 predocs

Very Strong Leverage! 6:1 ratio of paid/volunteer effort 10 coordinators 65 others officially working on the project at least part-

time Other sites in Europe joining in

– Is it possible to get additional funding to help them? Other projects in US and EU adding value

Attending this meeting ~65!!

This project has seeded a much larger effort Now Even more later



Community Code: Advantages

Sharing of Expertise: No single group or community can do these problems

Sharing of Code among projects Infrastructure BH routines apply directly to NS work, etc… But free to keep routines in group as long as desired

Better Code Open source encourages people to be more careful in coding! Encourages documentation Encourages deeper thinking about how it interfaces to another

code More trusted code, results

When code becomes open, and people can run it for themselves, they will begin to believe the results

Improvements propagate quickly though community Not well accepted yet, but we are starting a good

trend…



Scientific Highlights of Work Thus Far

The Network is working Community developing Exciting meetings Tools being used

Extraordinary new BH evolution results between AEI/Meudon groups

New Hydro Effort between AEI/SISSA/Thessaloniki/Valencia

…



Pre-Network 3D BH Results

NSF Grand Challenge 1994 Equal mass, no spin, no J Head on Collision in 3D Code crashes by t=40M

See New Diener Result!!

Grazing Collision 1999 Big Advance: 3D,

unequal mass, spin, J Physics! Code crash by 40M



New 3D Black Hole Evolutions

New Techniques Excision Gauges, formulations

Baumgarte initial data Estimated Innermost

Stable Circular Orbit (ISCO)

Other circular orbits, too (“Pre-ISCO sequence: PI-1-10)

OP Group has generated most astrophysically relevant binary BH data to date: now in Cactus!



High Profile These simulations will break

ground in various ways We feel we are about to crack (not solve…) this

problem Stumped community for 4 decades $4.5M NSF Grand Challenge project helped lay

groundwork No other group close to doing a BBH orbit

US Dept of Energy Special allocation for us to do work on 4.5TB machine Very unusual: DOE HQ approval given!

NCSA Special allocation

Public Outreach Discovery Channel Movie for June Scientific American Cover in April



EU Hydro Code EffortAEI/Valencia/Thessaloniki/Triest

e Originally a small collaboration between AEI and

WashU NASA GC project Several papers published jointly, code developed/tested jointly

– Hydro code itself primarily written by Mark Miller at WashU Decided in Dec, 2001 to develop independent EU code,

based on original lessons learned from old code Visit of Baiotti and Montero to AEI, with Ian Hawke,

many others contributing, critical to Network Hydro code

From start to finish, about 3 months Initial Applications

Mergers: NS-NS, strange-strange, accretion Later Applications

NS-BH, etc… Stellar Core Collapse (w/Dimmelmeier)



The methodological approach and work plan

Scientifically organized into Fully Numerical, Post-N, and Perturbation Theory

Management: Project and Institutional tracks Each subarea has person clearly assigned to

shepard activities– Facilitating Connections– Phone calls, conference calls, Emails, etc

Each Institute has Local Coordinators assigned to manage Institutional matters

Project Coordinator tries to have global view of things

Advisory Board gives feedback Students feedback solicited, incorporated

– Peter Diener assigned to be student Ombudsperson…



Advisory Board

Bernard Schutz AEI Director Heavily involved in GEO, other GW Detector Efforts

Tsvi Piran, Hebrew University Pioneer in numerical relativity Very familiar with both relativity and astrophysics problems we

face Karsten Danzmann

AEI Director Leads GEO Project

Jean-Pierre Lasota Astrophysicist/Relativist A Leader of Meudon group, expert in many problems of interest to

us Wai-Mo Suen

Close colleague, involved in many similar projects Leader of some major US-based projects (ASC, NASA, others)



My Favorite Advisory Board Member

“Sathya” Expert in

many things Very polite A true

gentleman



Work Plan

Cactus

Nonlinear BHsimulations

Hydrodynamics

Post-Newtoniantheory

Stellarperturbation

theory

Perturbative time evolutions

CharacteristicMethods

Training complete,AMR tested

12 months 24 months 36 months

Remote/distributed Simulations, input from projects below

Addressing EfficacyOf Characteristic

Hydro

Full communityCode release, with

documentation

Merger simulations with studies of waves

Continually refined as techniques developed

Completed binaryNS Initial

Data Module

Simulations refined as techniques developed

AccretionCode

PN GW, reaction expressions

Post-N.Hydro Code,Post-N Initial Data

Module

Simulationscontinue

Pert Eqs. for rotating NS, nonlinear results

Modes computed, valuableinput to num. simulations

Close limit module, time evolution for

rotating NSs

Perturbative evolutions,Input to num. simulations

Non

line

ar N

umer

ical

Cod

es,

Sim

ulat

ion s

Pert

. Tec

hniq

ues,

co

des,

Time



How the network functions and how the partners

collaborate Our Website for communications:

http://www.eu-network.org/ Notes Codes Workplans Results Newsletter

Visits Exchanges AEI Visitor Program

Conference calls All Pis every 2 months Various subgroups at regular intervals

Video Conferencing: (Apply for a supplement??)



Electronic Collaborations

CVS for common source code, reports, etc Central repository at AEI All partners in Network (and worldwide) can update

local installations via simple commands (cvs update)

Email Cactus

Collaborative Modular Testsuites! Very advanced computational tools, developed

worldwide, used, tested by our group Leveraging Technologies of ASC, TiKSL, GridLab

projects



Leveraging Other Projects

Cactus NSF KDI-ASC Project

Suen (WashU), Norman, Shalf (LBL),Seidel (AEI/NCSA), Foster (Argonne), Parashar (Rutgers)

Norman’s Zeus Astrophysics Suite GrACE: Parallel AMR for Many Applications Portal: Global Resource Management, building

generic GUI interfaces for resource selection Viz Tools

German DFN TiKSL/GriKSL projects GridLab



Cactus Computational Toolkit

Science, Autopilot, AMR, Petsc, HDF, MPI, GrACE, MDS, Remote Steering...

The Astrophysical Simulation Collaboratory (ASC)$2.2M NSF project (Suen, Seidel, Shalf, Foster, Norman, Parashar,

…)

1. User has scienceidea...

3. Selects Appropriate Resources...

5. Collaborators log in to monitor...

4. Steers simulation, monitors performance...

2. Composes/Builds Code Components w/Interface...

Want to integrate and migrate this technology to the generic user…



CactusOur collaborative

Infrastructure

Core “Flesh”

Plug-In “Thorns”(modules)

driverdriver

input/outputinput/output

interpolationinterpolation

SOR solverSOR solver

coordinatescoordinates

boundaryboundary conditionsconditions

black holesblack holes

equations of stateequations of state

remote steeringremote steering

wave evolverswave evolvers multigridmultigrid

parametersparameters

gridgrid variablesvariables

errorerror handlinghandling

schedulingscheduling

extensibleextensible APIsAPIs

makemake systemsystem

ANSI CANSI CFortran/C/C++Fortran/C/C++



Cactus User CommunityUsing and Developing Physics

Thorns

Goddard Penn State

Wash UAEI

TACTuebingen

Southampton

SISSA

Thessaloniki

Meeting Next Year:

Theoretical Foundations of Einstein’s Equations for Numerical Relativity

(Using Cactus to investigate and compare different implementations)

Climate Modeling(NASA,+)

Chemical Engineering (U.Kansas)

Bio-Informatics(Canada)

Geophysics(Stanford)

Astrophysics(Zeus)

Plasma(Princeton)

EU AstrophysicsNetwork

NASA Neutron StarGrand Challenge

Early Universe(LBL)

Numerical Relativity Other Applications

Portsmouth

RIKEN

Impact on otherProjects!



Cactus ASC Portal www.ascportal.org (prototype/works

now)

Part of NSF KDI Project (Astrophysics Simulation

Collaboratory) Use any Web Browser !! Portal (will) provides:

Single access to all resources

Locate/build executables Central/collaborative

parameter files, thorn lists etc

Job submission/tracking Access to new Grid

Technologies



GridLab: www.GridLab.org Enabling Dynamic Grid Applications

EU Project ~€5M AEI, ZIB, PSNC, Lecce,

Athens, Cardiff, Amsterdam, SZTAKI, Brno, ISI, Argonne, Wisconsin, Sun, Compaq

Grid Application Toolkit (APIs/Tools)

Develop new grid scenarios for 2 main apps:Numerical relativityGrav wave data analysis



Physicist has new idea !

S1 S2

P1

P2

S1S2

P2P1

SBrill WaveFound a horizon,

try out excision

Look forhorizon

Calculate/OutputGrav. Waves

Calculate/OutputInvariants

Find bestresources

Free CPUs!!

NCSA

SDSC

RZG

LRZ

Archive data

SDSC

Add more resources

Clone job with steered

parameter

Queue time over, find new machine

Archive to LIGOpublic database

Dynamic Grid Computing



Connections to industry

We are connected to industry through computational science projects Microsoft Sun Compaq Intel GridLab Grid Systems

No direct coupling through our science (that I know of)



The training programme

Feedback on talks, practice talks, etc Review Talk topics selected by YR’s School planned for 2003 Training sessions (twice), extensive online tutorials

High performance computing, Cactus (our code framework) Visualization tools, Code maintenance systems (CVS), etc

Want to give Video courses on scientific topics! Not able so far Access Grid funding with NSF?



Use of the Budget

Midterm report shows basically on track Most of the money (1.5M€) spent on YR About 1/3 of funding spent to date

Roughly proportionate for Networking and salary costs

Will accelerate spending as more YRs hired– Portsmouth, Valencia, AUTH made late hires– UIB just now making hires

Any proposed revision to the contract? No major changes forseen at this time, but

questions



Young Researcher Issues

Special attention paid to their training and well being…

Any difficulties encountered? Peter will report!

Responsibilities given to YRs Peter Diener’s role as YR coordinator Helping to prepare reports! Developing Web sites, documentation Building Central Codes

Developing “soft skills” Collaboration/Communication built in to this

project



Seek Input Actively seek YR input into all activities of the

Network,– Owe several beers: Dorota, Johannes, Reinhard– 1.5 cases to Denis, Werner…– topics for review talks, suggestions for improving their

training Anonymous FAQ on web site

Give Input/Advice Comment constructively on the talks Last time had rule that senior researchers could not

ask questions until after YRs did: wait and prod until they do!



General Problems Encountered

Recruitment difficult Restrictive rules No existing EU community

Travel budgets not enough Restrictions on overseas travel! Insufficient EU travel/subsistence funding for visits No real budget to support extended visits we are supposed

to have Bureaucracy

Reports, regulations far more than any NSF, NASA, or DFN grant I know!

Computing No central EU computing facilities We are quite lucky and unique due to US connections



Specific Questions from/for EC

Staffing issues/Collaborations Students finishing after Network ends

– Must care for them!– No-Cost Extension?– New Project/Continuation

Status of UIB– Strong historical connection to AEI– Postdocs hired beginning next month and month after

Generally plan to fulfill person months Could use some supplement for extended visits

Role of WashU group



Strengthening Ties with Other Projects US groups

– WashU– NSF projects, comoputing facilities, etc– New NSF, NASA projects? Joint funding?

GridLab, other EU projects– GridLab officially connected through Pis, technologies

Enabling other EU groups to join as “Associate Members”

– Various other groups want to join– A small amount of funding could go a long way

Various issues to discuss later…

EU Astrophysics Network Overview

Documents

Transcript of EU Astrophysics Network Overview