ATLASRucioDBcharacteristics(basedontheOracleRDBMS)

Ruciocommunityworkshop,Oslo,Feb-March2019GanchoDimitrov(CERN)

Aboutme

1

• MemberoftheATLASdatabasegroupsince2006

• Acquired(some)DBrelatedknowledgethroughoutthelast15+years

• Mainfocuson:Ø datamanagementØ databaseschemadesignØ databaseperformancetuning

• CertifiedinOracleRDBMS

• RegularspeakeratOracleUserGroupcommunityconferencesinUK,BulgariaandGermany

RucioCommunityWorkshop,Oslo,Feb2019

Rucioarchitecture

2RucioCommunityWorkshop,Oslo,Feb2019

Ruciodatabaseschema

4

Designedin2013

RucioCommunityWorkshop,Oslo,Feb2019

Ruciodatabaseschema

5

61tables

RucioCommunityWorkshop,Oslo,Feb2019

Ruciodatabaseschema

6

74indices

RucioCommunityWorkshop,Oslo,Feb2019

Ruciodatabaseschema

7

5triggers

RucioCommunityWorkshop,Oslo,Feb2019

Ruciodatabaseschema

8

3functions

RucioCommunityWorkshop,Oslo,Feb2019

Ruciodatabaseschema

9

12procedures

RucioCommunityWorkshop,Oslo,Feb2019

Ruciodatabaseschema

10

17jobs

RucioCommunityWorkshop,Oslo,Feb2019

Tablesegments

11

Physicallyorganised basedonRucio’s scopeunit

scopedata=tablepartition**separatesegment(providesflexibilityinsetofoperations)

RucioCommunityWorkshop,Oslo,Feb2019

Rucioscopegeneration

12

In2014:5140scopes(systemstartup)

2015-2018=~400peryear

RucioCommunityWorkshop,Oslo,Feb2019

Currently

13

About7000scopesaremappedto34500table

partitions

RucioCommunityWorkshop,Oslo,Feb2019

DIDS(DataIDentifierS)*

14

1014million(AVGrowsize:220bytes)

*Theprimaryaddressableunitofdata(files,datasets,containers)

RucioCommunityWorkshop,Oslo,Feb2019

DIDSreplicasontheGrid

15

1106million(AVGrowsize:150bytes)

RucioCommunityWorkshop,Oslo,Feb2019

Rucio<-->DBinteraction

16

SQLcommandsgeneratedby

SQLAlchemy ORM*

*Object-RelationalMapperRucioCommunityWorkshop,Oslo,Feb2019

DBsessions

17

1100-1400DBsessions15-25activesessions

RucioCommunityWorkshop,Oslo,Feb2019

Rucioinsert/update/deleterate

18

~2-4Kexec/sec

RucioCommunityWorkshop,Oslo,Feb2019

RucioDBqueriesrate

19

~4-8Kexec/sec2Mblockreads(15GB)/sec

RucioCommunityWorkshop,Oslo,Feb2019

DBserverCPUusage

20

~30-40%

RucioCommunityWorkshop,Oslo,Feb2019

RucioDBvolumegrowth2016-2019

21RucioCommunityWorkshop,Oslo,Feb2019

April-June2019

22

Excitingperiod:

TestandvalidationonOracle18c

DBserverupgradefrom11gto18c

RucioCommunityWorkshop,Oslo,Feb2019

Spring2020

23

ExpectedtohavenewDBhardware

RucioCommunityWorkshop,Oslo,Feb2019

RucioDBschemaextension

24

EnrichRucioDIDSwithmetadatastoredinJSON*

*CERNcomputingseminaronDBbuilt-inJSONfeaturesplannedfor15th May2019

RucioCommunityWorkshop,Oslo,Feb2019

Take-homemessages• RucioDBsetuponOracleRDBMSkeepsonperformingwellseveralconsecutiveyears.•WorkforsmoothDBserverupgradeisnecessary• PlannedextensionofthecurrentDBschematosupportDIDSmetadatainJSON

SpecialthankstotheRucioteamforthecollaborativeworkintheyears.

WelltunedDB=goodnight’ssleepforall!25RucioCommunityWorkshop,Oslo,Feb2019