Summary of Accounting Discussion at the GDB in Bologna

Dave Kant

CCLRC, e-Science Centre

Operations Workshop, Sept 2005 - 3

Accounting in LCG World

• Accounting is an Active Area EGEE and NorduGrid have developed implementations

• APEL deployed in LCG/EGEE production grid for 1 Year. AFTER the event accounting 4 Million Job Records published, 127 Sites, 100K records per week

• DGAS – a banking system – developed by Italy Fully Deployed in INFNGrid (latest version available in gLite 1.4)

• SGAS – a banking system – developed for SweGrid

• OSG are interested in APEL and DGAS

Operations Workshop, Sept 2005 - 4

Accounting Summary

… But

• Accounting MUST address the requirements of the project as a whole: Funding Bodies, Experiments, Users CRRB wants a table showing CPU consumed for each LHC VO Anonymous level: How much resource has been provided to each VO APEL and DGAS provide this

• Experiments want this as well but at finer granularity. User Level (Both APEL and DGAS can do this) Group Level within a VO Work needed by Both to interface into VOMS ATLAS and CMS provided input at the meeting.

Operations Workshop, Sept 2005 - 5

What Obstacles Are in the Way …

• … to report Usage at the user Level Reporting? If 10,000 CPU hours were consumed by Atlas VO, who are the users

that submitted the work? Data privacy laws are an issue here because you have to harvest job

records with “DN” from all the sites i.e. you have send data across international borders.

APEL: collects DN, but by default this does not leave the site DGAS: collects DN, signs usage record and send off site. Is it sufficient to encrypt the data and provide restricted access to the data? Recognised that this is a general problem (affects other services e.g. job

monitoring) and has been escalated to the security group for discussion (Dave Kelsey)

What has been discussed?• Can CA policies change to support anonymous DNs and reverse DN mappings? • Get all Users to sign a declaration when they register with the VO

This is an issue for everyone (MOU?)

Operations Workshop, Sept 2005 - 6

Accounting for Other Things?

• Recognise that accounting isn’t just about “job usage” CPU Usage Also Storage & Network Usage

• How do we describe this data? Luckily there is a GGF Usage record which provides a generic description of

resource usage Are these descriptors stable? Are they sufficient to describe the data? Can we get Network and Storage people to use the same schema? CPU is consumed; Storage is Occupied and can be recycled

• For LCG it is important that everyone buys into the same usage schema Can do this for accounting job records

Operations Workshop, Sept 2005 - 7

World Wide Accounting Service for LCG

• Project involves combining results from all three peer infrastructures and presenting an aggregated view of resource usage for LHC VOs to the RRB

Peer Infrastructures in LCG• Open Science Grid + Others (Ruth Pordes, Philippe Canal, Matteo

Melani)• Nordugrid (Per Oster, Thomas Sandholm)• LCG/EGEE (Kors Bos, Dave Kant)

GRID-ACCOUNTING@LISTSERV.RL.AC.UK

Arena for continued discussion

Operations Workshop, Sept 2005 - 8

Resource Usage Service

• If we can agree upon a common schema to describe Usage records then as far as LCG in concerned, we can satisfy requirements using an RUS.

A user invokes the query service through a web browser, using SSL for client authentication, to ensure that usage information at user level belongs to the user. Servlet sends query to RUS web service and gets user data.

Service

InterfaceRUS WS Application

ACL

DB

Web Service Container

Operations Workshop, Sept 2005 - 9

Possible Roadmap

Stage 1: Lets try to get some data from each of Tier-1ssummary records describing VO usage over a finite period of time• Before end 2005• SweGrid, Fermilab and DGAS ARE providing Data!

Stage 2: Centralised database with a web service interface (RUS) to publish/query accounting data

• Anonymous Summary records overcome the data privacy issue • Sometime in 2006

Stage 3: Distributed databases with a complete RUS implementation including permission model.

• Sometime early 2007