fcdfdata016

fcdfdata016 Disk/Cache

Stationcentral-analysis

smaster

Stager

stagerng

FSScentral-analysis

fss

Stager

stagerng

Disk/Cache

Disk/Cache

Disk/CacheStager

stagerng

Stager

stagerng

Stager

stagerng

Node1

Cache

Node2

Cache

Node3

Cache

Node4

Cache

Node5

Cache

Station

smaster

Stager

stagerng

Stager

stagerng

Stager

stagerng

Stager

stagerng

Stager

stagerng

fcdfdata016<fcdfdata016>

Disks/Cache



smaster

Disks/Cache



smaster

Disks/Cache

Stager

stagerng

<nglas08> setenv SAM_STATION chris

<nglas08> sam dump station --disks

*** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 53 minutes 25 seconds, admins: jozwiak terekhov

Known batch systems: lsf

Default batch system: lsf

No replica selection criteria

There are 0 authorized transfer groups

Minimum delivery is 1KB; external deliveries are unconstrained

STATION DISKS:

disk 7844 nglas08.fnal.gov:/sam/test9/jozwiak/dev/chris, 29947KB/20GB free

disk 8064 nglas08.fnal.gov:/sam/test10/jozwiak/dev/chris, 93110KB/20GB free

*** END OF STATION DUMP ***

sam dump station --disks

<nglas08> sam dump station --groups

*** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 57 minutes 3 seconds, admins: jozwiak terekhov






AUTHORIZED GROUPS:

group test: admins: jozwiak , swap policy: LRU, fair share: 0, quotas (cur/max): projects = 0/150, disk: 13054803KB/40GB, locks:0B/0KB

group test1: admins: jozwiak , swap policy: LRU, fair share: 0, quotas (cur/max): projects = 0/40, disk: 1714466KB/30GB, locks:0B/0KB

group test2: admins: jozwiak , swap policy: LRU, fair share: 0, quotas (cur/max): projects = 0/50, disk: 7170234KB/40GB, locks:0B/0KB


sam dump station --groups

<nglas08> sam dump station --projects

*** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 1 hours 4 minutes 49 seconds, admins: jozwiak terekhov






PROJECT MANAGER: fileReleaseTO = 1 days, max files given to project: Unlimited

NO PROJECTS


sam dump station --projects

fcdfdata016<fcdfdata016>sam submit--script=userscript--group=groupname--cpu-per-event=--defname=


smaster

Disks/Cache

Stager

stagerng

fcdfdata016<fcdfdata016>>>>>>> Starting project with the Station MasterStation Master contacted, result: Started project 49008(49008_sam_) for group testWaiting for the project to initialize...


smaster

Disks/Cache

Stager

stagerng

fcdfdata016<fcdfdata016>Callback from server: 'OK|Project is ready'


smaster

Disks/Cache

Stager

stagerng

Project

pmaster

fcdfdata016<fcdfdata016>>>>>>> Submitting the job to the batch system.


smaster

Disks/Cache

Stager

stagerng

Project

pmaster

fcdfdata016<fcdfdata016>Job <52554> is submitted to queue <sam_lo>.


smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> PSUSP

<nglas08> sam dump station --projects

*** BEGIN DUMP STATION chris version v3_2_2 running at nglas08 1 hours 12 minutes 44 seconds, admins: jozwiak terekhov






PROJECT MANAGER: fileReleaseTO = 1 days, max files given to project: Unlimited

STATION PROJECTS:

project 49205_sam_(49205) user jozwiak.test started 01 Nov 14:08:45 UNIX pid 158400787 still wants/currently uses 5/0 files


Sam dump station --projects

<nglas08> sam dump project --project=49205_sam_

*** BEGIN GPM DUMP ***

Input files: 1003853..1011900

1003853: sim.ztautau.1000evts.017-1442-c5.01, size=0K, unbuffered yet

1003854: sim.ztautau.1000evts.017-1442-c5.02, size=0K, unbuffered yet

1011651: sim.pmc02_01.pythia.ztautau_mb1.1av_200evts.267_1553, size=0K, unbuffered

1011900: sim.pmc02_01.pythia.ztautau_mb1.1av_200evts.276_1152, size=0K, unbuffered

Cached (not buffered) files: (none)

Buffered files: (none)

External files with delivery problems: (none)

Umer contexts (name, state, join time, nSeen): (no umers)

Proc contexts (ID: name, state, join time [, current|last]): (no procs)

Processes waiting for call back:(none)

*** END GPM DUMP ***

sam dump project –project=<project name>



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> PSUSP

Optimizer



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> PSUSP

eworker

eworkereworker

eworker



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> PSUSP

eworker

eworkereworker

eworker

encp

encp encp encp



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> PSUSP

eworker

eworkereworker

eworker

encp

encp encp encp

Enstore



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> PSUSP

eworker

eworker

encp encp

Enstore



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

eworker

eworker

encp encp

Enstore



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

eworker

eworker

encp encp

Enstore

samscript.sh

userscript



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

eworker

eworker

encp encp

Enstore

samscript.sh

userscript

consumer

SAMManager:sam Getting next input file...

SAMManager:sam Project master will call back.

<nglas08> sam dump project --project=49225_sam_

*** BEGIN GPM DUMP ***

Input files: 1099393..1099756

1099417: d0g.test_file_1G_a_dev.0001_001, size=0K, unbuffered yet

1099418: d0g.test_file_1G_a_dev.0002_001, size=0K, unbuffered yet

Cached (not buffered) files: (none)

Buffered files: (none)

External files with delivery problems: (none)

Umer contexts (name, state, join time, nSeen):

36422: jozwiak(test-harness:1), active, 05 Nov 13:59:09, 31

Proc contexts (ID: name, state, join time [, current|last]):

144663: jozwiak(test-harness:1)@nglas08, wait, 05 Nov 13:59:10, 1099415

Processes waiting for call back:

CID=36422: [email protected]:11872 (05 Nov 20:53:43)

*** END GPM DUMP ***

Sam dump project –project=49225_sam_



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

eworker

eworker

encp encp

Enstore

samscript.sh

userscript

consumer



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

samscript.sh

userscript

consumer



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

samscript.sh

userscript

consumer

rm

rm



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

samscript.sh

userscript

consumer



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

samscript.sh

userscript

consumer

Optimizer



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

samscript.sh

userscript

consumer

eworkereworker



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

samscript.sh

userscript

consumer

eworkereworker

rcp rcp



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

samscript.sh

userscript

consumer

eworkereworker

rcp rcp

Other Cache



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

samscript.sh

userscript

consumer

eworkereworker

rcp rcp



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)

52554 <user> RUN

samscript.sh

userscript

consumer



smaster

Disks/Cache

Stager

stagerng

Project

pmaster

Batch (LSF)



smaster

Disks/Cache

Stager

stagerngBatch (LSF)

fcdfdata016<fcdfdata016>sam submit….

<fcdfdata016>sam submit….

<fcdfdata016>sam run project…


smaster

Disks/Cache

Stager

stagerngBatch (LSF)



smaster

Disks/Cache

Stager

stagerngBatch (LSF)

52668 <user1> RUN52675 <user2> RUN

52756 <user3> PSUSP

Project

pmaster

Project

pmaster

Project

pmaster

samscript.shsamscript.sh

userscriptuserscript

consumer consumer

eworker

eworker

rcp

encp

Other CacheEnstore


Disks

FSSCentral-analysis

fss

Stager

stagerng

<fcdfdata016> sam dump fss

FSS version v3_2_2 at station central-analysis running on fcdfdata016.fnal.gov 6 hours 57 minutes 34 seconds

No routing (all transfers are direct)

Configuration for operation retrial (count, interval/timeout)

DBS contact: 3, 1 hours

Opter contact: 1, 1 hours

Authorization receipt:1, 1 hours

Stager contact: 1, 1 hours

Transfer (retrials upon timeout and upon failure): 3, 6 hours

Relay (multi-stage routing only): 3, 1 hours

File Storage Server Dump:

Stagers are known at nodes: fcdfdata016.fnal.gov

No requests ever submitted

Sam dump fss

fcdfdata016<fcdfdata016>sam store descrip.py --source=<file loc>[--dest=/pnfs…..]

Disks

FSSCentral-analysis

fss

Stager

stagerng


Disks

FSSCentral-analysis

fss

Stager

stagerngDescrip.pyMetadataInfo about file

Sam checks info,checks location,


Disks

FSSCentral-analysis

fss

Stager

stagerng

eworker

encp, rcp,bbftp

Node from ReallyFar Away

Disk

FssFrom Really

Far Away

Stager

fcdfdata016

Fsscentral-analysis

Stager

Tmp Disk

Enstore


Disk

FssRouting:

fcdfdata016

Stager

fcdfdata016

Fsscentral-analysis

Stager

Tmp Disk

Enstore

sam store enstore


Disk

FssRouting:

fcdfdata016

Stager

fcdfdata016

Fsscentral-analysis

Stager

Tmp Disk

Enstore

eworker

bbftp fcdfdata016


Disk

FssFrom really

Far away

Stager

fcdfdata016

Fsscentral-analysis

Stager

Tmp Disk

Enstore

eworker

bbftp fcdfdata016


Disk

FssFrom really

Far away

Stager

fcdfdata016

Fsscentral-analysis

Stager

Tmp Disk

Enstore


Disk

FssFrom really

Far away

Stager

fcdfdata016

Fsscentral-analysis

Stager

Tmp Disk

Enstore

eworker

encp


Disk

FssFrom really

Far away

Stager

fcdfdata016

Fsscentral-analysis

Stager

Tmp Disk

Enstore

rm


Disk

FssFrom really

Far away

Stager

fcdfdata016

Fsscentral-analysis

Stager

Tmp Disk

Enstore

fcdfdata016

Documents

Transcript of fcdfdata016