Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway...
-
Upload
janel-stephens -
Category
Documents
-
view
215 -
download
2
Transcript of Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway...
![Page 1: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/1.jpg)
Policy Based Data Management
Reagan W. MooreArcot Rajasekar
Mike WanWayne Schroeder
Mike ConwayJason Coposky
{moore,sekar,mwan, schroeder}@[email protected]
http://irods.diceresearch.org
1
![Page 2: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/2.jpg)
22
Policy-based Data Environments• Purpose - reason a collection is assembled
• Properties - attributes needed to ensure the purpose
• Policies - controls for enforcing desired properties,• mapped to computer actionable rules
• Procedures - functions that implement the policies
• mapped to computer actionable workflows
• State information - results of applying the procedures
• mapped to system metadata
• Assessment criteria - validation that state information conforms to the desired purpose
• mapped to periodically executed policies
2
![Page 3: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/3.jpg)
3
User w/ClientCan Search, Access, Add and
Manage Data& Metadata
Access distributed data with Web-based Browser or iRODS GUI or Command Line clients.
Overview of iRODS Architecture
iRODS Data Server
Disk, Tape, etc.
iRODS Metadata
CatalogTrack information
iRODS Middleware
iRODS Rule Engine
Tracks Policies
3
![Page 4: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/4.jpg)
4
Applications• Data grids
– Astronomy – NOAO, CyberSKA, LSST
– High Energy Physics – BaBar. KEK
– Earth Systems – NASA MODIS data set
• Institutional repositories– Carolina Digital Repository
• Libraries– Texas Digital Libraries
– Seismology - Southern California Earthquake Center
• Archives– Ocean Observatories Initiative
![Page 5: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/5.jpg)
Data VirtualizationData Virtualization
Storage SystemStorage System
Storage ProtocolStorage Protocol
Access InterfaceAccess Interface
Policy Enforcement PointsPolicy Enforcement Points
Standard Micro-servicesStandard Micro-services
Map from the actions requested by the client to multiple policy enforcement points.
Map from policy to standard micro-services.
Map from micro-services to standard Posix I/O operations.
Map standard I/O operations to the protocol supported by the storage system
Standard I/O OperationsStandard I/O OperationsDat
a G
rid
5
![Page 6: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/6.jpg)
Data Grid Clients
(48)
API Client Developer Language
Browser
DCAPE UNC iExplore RENCI-Oleg C++ JUX IN2P3 Jargon Peta Web browser PetaShare iDrop web browser Mike Conway Java Davis web interface ARCS Rich web client Lisa Stillwell - RENCI Digital Library Akubra/iRODS DICE Jargon Dspace MIT Fedora on Fuse IN2P3 FUSE Fedora/iRODS module DICE Jargon Islandora DICE Jargon Curators Workbench CDR-UNC-CH JargonFile System Davis - Webdav ARCS Jargon Dropbox / iDrop DICE-Mike Conway Jargon FUSE IN2P3, DICE, FUSE FUSE optimization PetaShare FUSE OpenDAP ARCS PetaFS (Fuse) Petashare - LSU Petashell (Parrot) PetaShare
6
![Page 7: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/7.jpg)
iRODS Clients (Cont.)
Grid GridFTP - Griffin ARCS Jsaga IN2P3 Jargon Parrot UND - Doug Thain SRM Academia Sinica Saga KEK I/O Libraries PRODS - PHP Renci - Lisa Stillwell C API DICE-Mike Wan C C I/O library DICE-Wayne Schroeder C Fortran Schroeder C Eclipse file system CDR - UNC-CH Jargon Jargon DICE-Mike Conway Jargon Pyrods - Python SHAMAN-Jerome Fusillier PythonPortal EnginFrame NICE / RENCI Jargon Petashare Portal LSU JargonTools Archive tools-NOAO NOAO Big Board visualization RENCI iFile GA Tech i-commands DICE Pcommands PetaShare Resource Monitoring IN2P3 Sync-package Academica Sinica URSpace Teldap - Academica Sinica Web Service VOSpace IVOA Shibboleth King's College Workflows Kepler - actor DICE Jargon Stork - interoperability LSU Workflow Virtualization LSU Taverna - actor RENCI 7
![Page 8: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/8.jpg)
Policy Enforcement Points
• Currently have 71 locations within iRODS framework where policies are checked.– Each action may involve multiple policy
enforcements points
• Policy enforcement points– Pre-action policy (selection of storage
location)– Policy execution (file deletion control)– Post-action policy (derived data products)
8
![Page 9: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/9.jpg)
Policy Enforcement Points (71)ACTIONacCreateUseracDeleteUseracGetUserbyDN acTrashPolicyacAclPolicy acSetCreateConditionsacDataDeletePolicy acRenameLocalZoneacSetRescSchemeForCreateacRescQuotaPolicyacSetMultiReplPerRescacSetNumThreadsacVacuumacSetResourceListacSetCopyNumberacVerifyChecksum acCreateUserZoneCollectionsacDeleteUserZoneCollections acPurgeFilesacRegisterData acGetIcatResults acSetPublicUserPolicyacCreateDefaultCollectionsacDeleteDefaultCollections
POST-ACTION POLICYacPostProcForCreateUseracPostProcForDeleteUseracPostProcForModifyUseracPostProcForModifyUserGroupacPostProcForDeleteacPostProcForCollCreate acPostProcForRmCollacPostProcForModifyAVUMetadataacPostProcForModifyCollMetaacPostProcForModifyDataObjMetaacPostProcForModifyAccessControlacPostProcForOpen acPostProcForObjRenameacPostProcForCreateResourceacPostProcForDeleteResourceacPostProcForModifyResourceacPostProcForModifyResourceGroupacPostProcForCreateTokenacPostProcForDeleteTokenacPostProcForFilePathRegacPostProcForGenQueryacPostProcForPutacPostProcForCopyacPostProcForCreate
PRE-ACTION POLICYacPreProcForCreateUseracPreProcForDeleteUseracPreProcForModifyUser acPreProcForModifyUserGroupacChkHostAccessControl acPreProcForCollCreateacPreProcForRmCollacPreProcForModifyAVUMetadataacPreProcForModifyCollMetaacPreProcForModifyDataObjMetaacPreProcForModifyAccessControlacPreprocForDataObjOpenacPreProcForObjRenameacPreProcForCreateResourceacPreProcForDeleteResourceacPreProcForModifyResourceacPreProcForModifyResourceGroupacPreProcForCreateTokenacPreProcForDeleteTokenacNoChkFilePathPerm acPreProcForGenQueryacSetReServerNumProcacSetVaultPathPolicy
9
![Page 10: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/10.jpg)
iput ../src/irm.c checks 10 policy hooks
srbbrick14:10900:ApplyRule#116:: acChkHostAccessControlsrbbrick14:10900:GotRule#117:: acChkHostAccessControlsrbbrick14:10900:ApplyRule#118:: acSetPublicUserPolicysrbbrick14:10900:GotRule#119:: acSetPublicUserPolicysrbbrick14:10900:ApplyRule#120:: acAclPolicysrbbrick14:10900:GotRule#121:: acAclPolicysrbbrick14:10900:ApplyRule#122:: acSetRescSchemeForCreatesrbbrick14:10900:GotRule#123:: acSetRescSchemeForCreatesrbbrick14:10900:execMicroSrvc#124:: msiSetDefaultResc(demoResc,null)srbbrick14:10900:ApplyRule#125:: acRescQuotaPolicysrbbrick14:10900:GotRule#126:: acRescQuotaPolicysrbbrick14:10900:execMicroSrvc#127:: msiSetRescQuotaPolicy(off)srbbrick14:10900:ApplyRule#128:: acSetVaultPathPolicysrbbrick14:10900:GotRule#129:: acSetVaultPathPolicysrbbrick14:10900:execMicroSrvc#130:: msiSetGraftPathScheme(no,1)srbbrick14:10900:ApplyRule#131:: acPreProcForModifyDataObjMetasrbbrick14:10900:GotRule#132:: acPreProcForModifyDataObjMetasrbbrick14:10900:ApplyRule#133:: acPostProcForModifyDataObjMetasrbbrick14:10900:GotRule#134:: acPostProcForModifyDataObjMetasrbbrick14:10900:ApplyRule#135:: acPostProcForCreatesrbbrick14:10900:GotRule#136:: acPostProcForCreatesrbbrick14:10900:ApplyRule#137:: acPostProcForPutsrbbrick14:10900:GotRule#138:: acPostProcForPutsrbbrick14:10900:GotRule#139:: acPostProcForPutsrbbrick14:10900:GotRule#140:: acPostProcForPut 10
![Page 11: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/11.jpg)
Policies• Retention, disposition, distribution, arrangement• Authenticity, provenance, description• Integrity, replication, synchronization• Deletion, trash cans, versioning• Archiving, staging, caching• Authentication, authorization, redaction• Access, approval, IRB, audit trails, report generation• Assessment criteria, validation• Derived data product generation, format parsing• Federation
11
![Page 12: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/12.jpg)
12
KEK Paper
IRODS in an Neutrino Experiment
Adil Hasan
for
Francesca Di Lodovico (QMUL), Yoshimi Iida (KEK), Takashi Sasaki (KEK)
https://www.irods.org/index.php/iRODS_User_Group_Meeting_2011
![Page 13: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/13.jpg)
13
iRODS in an Neutrino Experiment
• Tokai to Kamioka data grid in Japan– Provide access to global collaborators– Must aggregate files for storage in HPSS in 1-
GB containers– File sizes ranged from kiloBytes to MegaBytes
• Created policies to:– Automate bundling of files– Replicate containers into HPSS– Purge cache and backup resources
![Page 14: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/14.jpg)
14
Rule to Bundle Files
acKEKBundle(*collPath, *bundlePath, *cacheRes, *compRes, *archive, *threshold)||
msiCheckCollSize(*collPath, *cacheRes, *threshold,
*aboveThreshold, *status)##
ifExec(*aboveThreshold == 1,
msiWriteRodsLog("Creating bundle", *status)##
msiPhyBundleColl(*collPath, *compRes,*status)##
msiWriteRodsLog("Finished bundling, starting to replicate",
*status)##
msiCollRepl(*bundlePath, verifyChksum++++backupRescName =*archive, *status)##
msiWriteRodsLog("Finished replicating bundle", *status),
nop##nop##nop##nop##nop, nop, nop) |nop##nop
![Page 15: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/15.jpg)
15
iRODS Rule to Replicate Files
acKEKReplicate(*collPath, *cacheRes, *archive, *threshold)||
msiCheckCollSize(*collPath, *cacheRes, *threshold, *aboveThreshold, *status)##
ifExec(*aboveThreshold == 1, nop, nop,
msiWriteRodsLog("Starting to backup files", *status)##
acGetIcatResults(list, COLL_NAME LIKE '*collPath', *List)##
forEachExec(*List, msiGetValByKey(*List, DATA_NAME, *Data)##
msiGetValByKey(*List, COLL_NAME, *Coll)##
msiGetValByKey(*List, DATA_RESC_NAME, *dataRes)##
ifExec(*dataRes == *cacheRes,
msiWriteRodsLog("Replicating file *Coll/*Data", *status)##
msiDataObjRepl(*Coll/*Data, verifyChksum++++backupRescName=
*archive, *status)##
msiWriteRodsLog("Completed replicating file *Coll/*Data",
*status),
nop##nop##nop, nop, nop), nop##nop##nop), nop##nop##nop)|nop##nop
![Page 16: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/16.jpg)
16
iRODS Rule to Trim Replicas
acKEKTrimData(*collPath, *cacheRes)||
acGetIcatResults(list, COLL_NAME LIKE '*collPath', *List)##
forEachExec(*List, msiGetValByKey(*List, DATA_NAME, *Data)##
msiGetValByKey(*List, COLL_NAME, *Coll)##
msiGetValByKey(*List, DATA_RESC_NAME, *DataResc)##
msiGetValByKey(*List, DATA_REPL_NUM, *DataRepl)##
ifExec(*DataResc == *cacheRes,
msiWriteRodsLog("About to trim file *Coll/*Data", *status)##
msiDataObjTrim(*Coll/*Data, *cacheRes, *DataRepl, 1,
IRODS_ADMIN_KW=irodsAdmin, *status)##
msiWriteRodsLog("Completed trimming replicas of *Coll/*Data",
*status),
nop##nop##nop, nop, nop), nop##nop##nop##nop##nop) |nop##nop
![Page 17: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/17.jpg)
Data Distribution Thought Experiment
Data Handling Platform
( r )
Supercomputer( R )
Bandwidths linking systems are Bd > Bs
Operations per byte for analysis is s
Operations per byte for data transfer is t
Reduce size of data from S bytes to s bytes and then analyze
Should the data reduction be done before transmission?
DataStorage Bd Bs
Execution rates are r < R
Storage System
17
![Page 18: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/18.jpg)
Complexity Analysis
Moving all of the data is faster, T(Super) < T(Archive)if the complexity is sufficiently high!
s > t (1-s/S) [1 + r/R + r/(t Bs)] / (1-r/R)
Note, as the execution ratio approaches 1, the required complexity becomes infinite
Also, as the amount of data reduction goes to zero,the required complexity goes to zero.
For sufficiently low complexity, it is faster to do the computation at the storage location
18
![Page 19: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/19.jpg)
Micro-Services
• Functions written in C• Provided with the iRODS server code• Provide:
– Standard operations– Queries on metadata catalog– Interaction with web services– Invocation of external applications– Workflow constructs (loops, conditionals, exit)– Remote and delayed execution control
19
![Page 20: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/20.jpg)
Micro-services - How many are needed?print_hello_argmsiVacuummsiQuotamsiGoodFailuremsiSetResourcemsiCheckPermissionmsiCheckOwnermsiCreateUsermsiCreateCollByAdminmsiSendMailrecover_print_hellomsiCommitmsiRollbackmsiDeleteCollByAdminmsiDeleteUsermsiAddUserToGroupmsiSetDefaultRescmsiSetRescSortSchememsiSysReplDataObjmsiStageDataObjmsiSetDataObjPreferredRescmsiSetDataObjAvoidRescmsiSortDataObjmsiSysChksumDataObjmsiSetDataTypeFromExtmsiSetNoDirectRescInpmsiSetNumThreadsmsiDeleteDisallowedmsiOprDisallowed
msiDataObjCreatemsiDataObjOpenmsiDataObjClosemsiDataObjLseekmsiDataObjReadmsiDataObjWritemsiDataObjUnlinkmsiDataObjReplmsiDataObjCopymsiExtractNaraMetadatamsiSetMultiReplPerRescmsiAdmChangeCoreIRBmsiAdmShowIRBmsiAdmShowDVMmsiAdmShowFNMmsiAdmAppendToTopOfCoreIRBmsiAdmClearAppRuleStructmsiAdmAddAppRuleStructmsiGetObjTypemsiAssociateKeyValuePairsToObjmsiExtractTemplateMDFromBufmsiReadMDTemplateIntoTagStructmsiDataObjPutmsiDataObjGetmsiDataObjChksummsiDataObjPhymvmsiDataObjRenamemsiDataObjTrimmsiCollCreate
msiRmCollmsiReplCollmsiCollReplmsiPhyPathRegmsiObjStatmsiDataObjRsyncmsiFreeBuffermsiNoChkFilePathPermmsiNoTrashCanmsiSetPublicUserOprwhileExecforExecdelayExecremoteExecforEachExecmsiSleepwriteStringwriteLinewriteBytesBufwritePosIntwriteKeyValPairsmsiGetDiffTimemsiGetSystemTimemsiHumanToSystemTimemsiStrToBytesBufmsiApplyDCMetadataTemplatemsiListEnabledMSmsiSendStdoutAsEmailmsiPrintKeyValPair
msiGetValByKeymsiAddKeyValassignifExecbreakapplyAllRulesmsiExecStrCondQuerymsiExecStrCondQueryWithOptionsmsiExecGenQuerymsiMakeQuerymsiMakeGenQuerymsiGetMoreRowsmsiAddSelectFieldToGenQuerymsiAddConditionToGenQuerymsiPrintGenQueryOutToBuffermsiExecCmdmsiSetGraftPathSchememsiSetRandomSchememsiCheckHostAccessControlmsiGetIcatTimemsiGetTaggedValueFromStringmsiXmsgServerConnectmsiXmsgCreateStreammsiCreateXmsgInpmsiSendXmsgmsiRcvXmsgmsiXmsgServerDisConnectmsiString2KeyValPairmsiStrArray2StringmsiRdaToStdout
20
![Page 21: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/21.jpg)
Micro-services (229)msiRdaToDataObjmsiRdaNoResultsmsiRdaCommitmsiAW1msiRdaRollbackmsiRenameLocalZonemsiRenameCollectionmsiAclPolicymsiRemoveKeyValuePairsFromObjmsiDataObjPutWithOptionsmsiDataObjReplWithOptionsmsiDataObjChksumWithOptionsmsiDataObjGetWithOptionsmsiSetReServerNumProcmsiGetStdoutInExecCmdOutmsiGetStderrInExecCmdOutmsiAddKeyValToMspStrmsiPrintGenQueryInpmsiTarFileExtractmsiTarFileCreatemsiPhyBundleCollmsiWriteRodsLogmsiServerMonPerfmsiFlushMonStatmsiDigestMonStatmsiSplitPathmsiGetSessionVarValuemsiAutoReplicateService
msiDataObjAutoMovemsiGetContInxFromGenQueryOutmsiSetACLmsiSetRescQuotaPolicymsiPropertiesNewmsiPropertiesClearmsiPropertiesClonemsiPropertiesAddmsiPropertiesRemovemsiPropertiesGetmsiPropertiesSetmsiPropertiesExistsmsiPropertiesToStringmsiPropertiesFromStringmsiRecursiveCollCopymsiGetDataObjACLmsiGetCollectionACLmsiGetDataObjAVUsmsiGetDataObjPSmetamsiGetCollectionPSmetamsiGetDataObjAIPmsiLoadMetadataFromDataObjmsiExportRecursiveCollMetamsiCopyAVUMetadatamsiGetUserInfomsiGetUserACLmsiCreateUserAccountsFromDataObjmsiLoadUserModsFromDataObj
msiDeleteUsersFromDataObjmsiLoadACLFromDataObjmsiGetAuditTrailInfoByUserIDmsiGetAuditTrailInfoByObjectIDmsiGetAuditTrailInfoByActionIDmsiGetAuditTrailInfoByKeywordsmsiGetAuditTrailInfoByTimeStampmsiSetDataTypemsiGuessDataTypemsiMergeDataCopiesmsiIsCollmsiIsDatamsiGetCollectionContentsReportmsiGetCollectionSizemsiStructFileBundlemsiCollectionSpidermsiFlagDataObjwithAVUmsiFlagInfectedObjs
21
![Page 22: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/22.jpg)
State Information - How Many?ZONE_IDZONE_NAMEZONE_TYPEZONE_CONNECTIONZONE_COMMENTZONE_CREATE_TIMEZONE_MODIFY_TIMEUSER_IDUSER_NAMEUSER_TYPEUSER_ZONEUSER_DNUSER_INFOUSER_COMMENTUSER_CREATE_TIMEUSER_MODIFY_TIMERESC_IDRESC_NAMERESC_ZONE_NAMERESC_TYPE_NAMERESC_CLASS_NAMERESC_LOCRESC_VAULT_PATHRESC_FREE_SPACERESC_FREE_SPACE_TIME
RESC_INFORESC_COMMENTRESC_CREATE_TIMERESC_MODIFY_TIMERESC_STATUSDATA_IDDATA_COLL_IDDATA_NAMEDATA_REPL_NUMDATA_VERSIONDATA_TYPE_NAMEDATA_SIZEDATA_RESC_GROUP_NAMEDATA_RESC_NAMEDATA_PATHDATA_OWNER_NAMEDATA_OWNER_ZONEDATA_REPL_STATUSDATA_STATUSDATA_CHECKSUMDATA_EXPIRYDATA_MAP_IDDATA_COMMENTSDATA_CREATE_TIMEDATA_MODIFY_TIME
DATA_ACCESS_TYPEDATA_ACCESS_NAMEDATA_TOKEN_NAMESPACEDATA_ACCESS_USER_IDDATA_ACCESS_DATA_IDCOLL_IDCOLL_NAMECOLL_PARENT_NAMECOLL_OWNER_NAMECOLL_OWNER_ZONECOLL_MAP_IDCOLL_INHERITANCECOLL_COMMENTSCOLL_CREATE_TIMECOLL_MODIFY_TIMECOLL_ACCESS_TYPECOLL_ACCESS_NAMECOLL_TOKEN_NAMESPACECOLL_ACCESS_USER_IDCOLL_ACCESS_COLL_IDMETA_DATA_ATTR_NAMEMETA_DATA_ATTR_VALUEMETA_DATA_ATTR_UNITSMETA_DATA_ATTR_IDMETA_DATA_CREATE_TIME
22
![Page 23: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/23.jpg)
State Information (112)META_DATA_MODIFY_TIMEMETA_COLL_ATTR_NAMEMETA_COLL_ATTR_VALUEMETA_COLL_ATTR_UNITSMETA_COLL_ATTR_IDMETA_NAMESPACE_COLLMETA_NAMESPACE_DATAMETA_NAMESPACE_RESCMETA_NAMESPACE_USERMETA_RESC_ATTR_NAMEMETA_RESC_ATTR_VALUEMETA_RESC_ATTR_UNITSMETA_RESC_ATTR_IDMETA_USER_ATTR_NAMEMETA_USER_ATTR_VALUEMETA_USER_ATTR_UNITSMETA_USER_ATTR_IDRESC_GROUP_RESC_IDRESC_GROUP_NAMEUSER_GROUP_IDUSER_GROUP_NAMERULE_EXEC_IDRULE_EXEC_NAME
RULE_EXEC_REI_FILE_PATHRULE_EXEC_USER_NAMERULE_EXEC_ADDRESSRULE_EXEC_TIMERULE_EXEC_FREQUENCYRULE_EXEC_PRIORITYRULE_EXEC_ESTIMATED_EXE_TIMERULE_EXEC_NOTIFICATION_ADDRRULE_EXEC_LAST_EXE_TIMERULE_EXEC_STATUSTOKEN_NAMESPACETOKEN_IDTOKEN_NAMETOKEN_VALUETOKEN_VALUE2TOKEN_VALUE3TOKEN_COMMENTAUDIT_OBJ_IDAUDIT_USER_IDAUDIT_ACTION_IDAUDIT_COMMENTAUDIT_CREATE_TIMEAUDIT_MODIFY_TIME
SL_HOST_NAMESL_RESC_NAMESL_CPU_USEDSL_MEM_USEDSL_SWAP_USEDSL_RUNQ_LOADSL_DISK_SPACESL_NET_INPUTSL_NET_OUTPUTSL_CREATE_TIMESLD_RESC_NAMESLD_LOAD_FACTORSLD_CREATE_TIME
23
![Page 24: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/24.jpg)
Open Source Software
• Community driven software development– Focus on features required by user communities– Focus on bug-free software– Focus on highly reliable software– Focus on highly extensible software– Approximately 3-4 software releases per year
• Distributed under a BSD license– International collaborations on software development– IN2P3 (France), SHAMAN (UK), ARCS (Australia), Academia
Sinica (Taiwan)
24
![Page 25: Policy Based Data Management Reagan W. Moore Arcot Rajasekar Mike Wan Wayne Schroeder Mike Conway Jason Coposky {moore,sekar,mwan, schroeder}@diceresearch.org.](https://reader030.fdocuments.in/reader030/viewer/2022032703/56649cf85503460f949c92a2/html5/thumbnails/25.jpg)
25
iRODS is a "coordinated NSF/OCI-Nat'l Archives research activity" under the auspices of the President's NITRD Program and is identified as among the priorities underlying the President's 2009 Budget Supplement in the area of Human and Computer Interaction Information Management technology research.
Reagan W. [email protected]
http://irods.diceresearch.org
NSF OCI-0848296 “NARA Transcontinental Persistent Archives Prototype”NSF SDCI-0721400 “Data Grids for Community Driven Applications”
25