Hybrid Event Store Integration with Athena/StoreGate

David Adams

ATLAS

Hybrid Event Store Integration with Athena/StoreGate

David Adams

BNL

March 5, 2002

ATLAS Software Week

Event Data Model and Detector Description Session

March 5, 2002HES in Athena SW week – EDM session 2

David Adams

ATLAS

Contents• Introduction• Athena/HES components

–HES

–StoreView

–StoreSvc

–Proxy

–DataHandle

–StoreGateSvc

–DataLink

–AHesView

• Tasks and schedule


David Adams

ATLAS

Athena/HES componentsFigure

• The following page shows selected components of Athena and HES and their physical dependencies.

• The Athena/StoreGate components (DataLink,…) shaded in blue are new or will have to be modified to integrate HES into Athena/StoreGate.

– These are discussed in the following sections.


David Adams

ATLAS

Athena/HES components (cont)XA lg o rithm

D a taL ink H e sV iew

AHe sV iew

S to reG a te S vc

D a taHa ndle

C o nve rs io nS vc

P ro xy

A lg o rithm

S to re S vc

S vc

S to reV iew


David Adams

ATLAS

HESDefinition

• Define a subsystem called HES which has no dependencies on StoreGate, Athena or Gaudi.

• HesView is the top of this subsystem.• This subsystem will be discussed in the DB

meeting on Thursday.• Links to this talk and that one can be found on

the hybrid web page– http://www.usatlas.bnl.gov/~dladams/hybrid


David Adams

ATLAS

StoreViewPurpose

• StoreView is a new abstract interface which allows Athena to interact with different persistent stores in a generic manner.

• Each persistent storage mechanism to be used in Athena will provide an implementation of this interface.

– HES

– ZEBRA

– Objectivity?


David Adams

ATLAS

StoreView (cont)Interface

• Select the next event ID.• Read an input event

– Identify the included type-keys and create proxies in StoreGate

– Locate the data (on demand from proxy) for any of these EDO’s

• Write an output event– Take ownership of selected new EDO’s

> (add EDO ID to proxy)

– Write EDO’s to selected output streams


David Adams

ATLAS

StoreView (cont)Configuring a job

• The choice of store view (or views) and their configuration is made at run time using Gaudi JobOptions

– StoreView is a Gaudi service

– No physical dependence on concrete implementation except that library must be provided at run time


David Adams

ATLAS

StoreSvcPurpose

• StoreSvc provides the interface between the store views and the rest of Athena/StoreGate

• StoreSvc holds one or more store views– Requests are fulfilled by forwarding to a selected

view or looping over all views.

– Each view is assigned an ID called store ID.


David Adams

ATLAS

StoreSvc (cont)Interface for reading

• Assign the next event ID– Pass request to a designated store

– Or assign in some other manner (e.g. user input)

• Define input event– Loop over store views

• Fetch data– Given a proxy for an EDO that has not be located,

> identify the appropriate view and

> ask the view to locate the EDO


David Adams

ATLAS

StoreSvc (cont)Interface for writing

• Stream selection– Run selection algorithms and notify views which

streams should be written.

• EDO ownership– Give each view the opportunity to take ownership of

EDO’s in StoreGate.

• Writing data– Give each view the option to to write EDO’s from

StoreGate (or from that view).


David Adams

ATLAS

StoreSvc (cont)Configuring a job

• Configure in job options– StoreSvc is a Gaudi service

– No physical dependency on Athena/StoreGate


David Adams

ATLAS

ProxyPurpose

• The transient data store (StoreGateSvc) “holds” EDO’s by proxy.

• This allows us to– Defer output conversion until all data is produced

and selections are made

– Defer input conversion until data is accessed

HES environment• It is sensible to also defer location on input• Some data may not be accessible


David Adams

ATLAS

Proxy (cont)Proxies will be modified for HES

• From persistent store, there will be four states:– No data; proxy has type, key, store ID and EDO ID

– Above plus IOpaqueAddress (EDO located)

– Above plus transient address (EDO converted)

– Marked inaccessible (EDO cannot be located)

• For proxies from transient store, it will be possible to assign ownership (EDO ID).

• Proxy uses StoreSvc to locate EDO when desired.


David Adams

ATLAS

DataHandleCurrent usage

• A data handle is returned when a user asks StoreGate for an EDO.

• Conversion may be deferred.

Changes (implied by change in proxy)• Location may also be deferred.• DataHandle may be invalid because the EDO

cannot be located.


David Adams

ATLAS

StoreGateSvcPurpose and current implementation

• StoreGateSvc is the transient data store.• Single instance holds data (by proxy) for

current event.– Pileup adds more

• Given a type and key, StoreGate returns a DataHandle for the corresponding EDO

– Any type-key may not be duplicated in the store


David Adams

ATLAS

StoreGateSvc (cont)Changes

• One instance will hold proxies for visible data– Directly accessed from algorithms

• Other(s) will hold proxies for data which is accessible (by reference) but not visible.

• Request for data from DataLink will include EDO ID in addition to type-key.

– Only EDO with matching ID is returned.

– Someone has to search the other transient stores if match is not found in the visible store

> Visible store knows the reference stores?


David Adams

ATLAS

DataLinkPurpose and current implementation

• DataLink provides a persistent mechanism to reference objects in other EDO’s.

• Persistent data includes:– EDO specifier and

– index specifying the location of object in EDO.

• At present, EDO specifier is the type and key and an event ID implied by context.

• Any StoreGate EDO with matching type and key can serve as the reference EDO


David Adams

ATLAS

DataLink (cont)Changes:

• In the new model, we explicitly recognize that persistent store may hold multiple EDO’s with the same event ID, type and key.

• Persistent data for the DataLink will also include the ID of the referenced EDO.

• May also need to add (transient) store ID to resolve EDO ID ambiguities between different store types.


David Adams

ATLAS

AHesViewImplementation of StoreView for HES

• Depends on both Athena/StoreGate and the HES core software.

– Creates proxies

– Puts proxies in StoreGate

– Calls converters to write data

• See talk at DB session for information on the HES core.


David Adams

ATLAS

Tasks and scheduleRequirements

• Sufficient functionality to use HES as the primary storage mechanism for DC1-2.

• We would like a first implementation by June 1.

• Estimate the Athena/HES piece will require 7 FTE-weeks for this implementation.

– 2X contingency

• We need to adjust priorities to meet this schedule.

Hybrid Event Store Integration with Athena/StoreGate

Documents

Transcript of Hybrid Event Store Integration with Athena/StoreGate