PTP 565 Fundamental Tests and Measures Thomas Ruediger, PT, DSc, OCS, ECS Statistics Overview.
INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich /...
-
Upload
meagan-cummings -
Category
Documents
-
view
214 -
download
0
Transcript of INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich /...
![Page 1: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/1.jpg)
INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
GILDA WMS Praticals
Ruediger Berlich / Forschungszentrum KarlsruheMike Mineter / National e-Science Centre
slides contributed by INFN Catania
Melbourne, 30.01.2006
![Page 2: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/2.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 2
Enabling Grids for E-sciencE
INFSO-RI-508833
User Interface Access
Host: glite-tutor.ct.infn.it
Username: melbourneXX (XX=01…30)
Password: GridMELXX (XX=01…30)
PassPhrase: MELBOURNE
GENIUS: https://glite-tutor.ct.infn.it
![Page 3: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/3.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 3
Enabling Grids for E-sciencE
INFSO-RI-508833
VOMS/Proxy/MyProxy Management
![Page 4: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/4.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 4
Enabling Grids for E-sciencE
INFSO-RI-508833
acknowledgements
• With acknowledgements to:Emidio GiorgioINFNCatania
![Page 5: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/5.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 5
Enabling Grids for E-sciencE
INFSO-RI-508833
Outline
• VOMS proxy creation– Create VOMS proxy certificate using voms-proxy-init
• MyProxy – Store a proxy certificate– Retrieve a new proxy from MyProxy server
![Page 6: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/6.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 6
Enabling Grids for E-sciencE
INFSO-RI-508833
Please logon to the GILDA
User Interface
Login : [email protected] where
XX=01,..30Passwd : GridMELXX XX=01,..,30
![Page 7: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/7.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 7
Enabling Grids for E-sciencE
INFSO-RI-508833
Preliminary : .globus directory
• .globus directory contains your personal public / private keys
Pay attention to permissions !In future when you use the EGEE grid, you must obtain
and download these to this directory ... Details according to your Certificate Authority. On the GILDA testbed this is done for you.
[glite-tutor] /home/giorgio > ls -l .globustotal 8-rw-r----- 1 giorgio users 1613 Oct 4 19:30 usercert.pem-r-------- 1 giorgio users 1914 Oct 4 19:30 userkey.pem
![Page 8: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/8.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 8
Enabling Grids for E-sciencE
INFSO-RI-508833
voms-proxy-init : options
• Main options -voms <vo-name:[command]>
command syntax is :/<voname>/group for group specify (default none) command syntax is :/<voname>/Role=<role name> for Role choice
(default none)
-valid x:y, create a proxy valid for x hours and y minutes-vomslife x, create a proxy with AC valid for x hours (max 24 h)
voms-proxy-init –-voms gildav:/gildav/Role=VO-Adminvoms-proxy-init --voms gildav:/gildav/tutors
PEM PASSPHRASE : MELBOURNE
![Page 9: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/9.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 9
Enabling Grids for E-sciencE
INFSO-RI-508833
VOMS proxy creation
voms-proxy-init --voms gildavYour identity: /C=IT/O=GILDA/OU=Personal
Certificate/L=INFN/CN=Emidio Giorgio/[email protected]
Enter GRID pass phrase for this identity:[insert your certificate passphrase]Creating temporary
proxy ............................................. Done
/C=IT/O=INFN/OU=Host/L=CNAF/CN=cert-voms-01.cnaf.infn.it
/C=IT/O=INFN/CN=INFN Certification AuthorityCreating proxy ............................................. DoneYour proxy is valid until Mon Jun 13 09:06:00 2005
![Page 10: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/10.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 10
Enabling Grids for E-sciencE
INFSO-RI-508833
Verify your credentials
• voms-proxy-info
• Principal options : --all prints all proxy options --file specifies a different location of proxy file
![Page 11: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/11.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 11
Enabling Grids for E-sciencE
INFSO-RI-508833
Verify obtained credentials
[giorgio@glite-tutor:~]$ voms-proxy-info --all subject : /C=IT/O=GILDA/OU=Personal
Certificate/L=INFN/CN=Emidio Giorgio/[email protected]/CN=proxy
issuer : /C=IT/O=GILDA/OU=Personal Certificate/L=INFN/CN=Emidio Giorgio/[email protected]
identity : /C=IT/O=GILDA/OU=Personal Certificate/L=INFN/CN=Emidio Giorgio/[email protected]
type : proxystrength : 512 bitspath : /tmp/x509up_u513timeleft : 20:59:53VO : gildavsubject : /C=IT/O=GILDA/OU=Personal
Certificate/L=INFN/CN=Emidio Giorgio/[email protected]
issuer : /C=IT/O=INFN/OU=Host/L=CNAF/CN=cert-voms-01.cnaf.infn.it
attribute : /gildav/Role=NULL/Capability=NULLtimeleft : 20:58:28
![Page 12: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/12.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 12
Enabling Grids for E-sciencE
INFSO-RI-508833
Long term proxy : MyProxy
• myproxy server:– myproxy-init
Allows to create and store a long term proxy certificate:
– myproxy-info Get information about stored long living proxy
– myproxy-get-delegation Get a new proxy from the MyProxy server
– myproxy-destroy– Check out them with myproxy-xxx --help option
![Page 13: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/13.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 13
Enabling Grids for E-sciencE
INFSO-RI-508833
myproxy-init[giorgio@glite-tutor:~]$ myproxy-init -s grid001.ct.infn.it Your identity: /C=IT/O=GILDA/OU=Personal
Certificate/L=INFN/CN=Emidio Giorgio/[email protected]
Enter GRID pass phrase for this identity:Creating proxy .....................................................
DoneProxy Verify OKYour proxy is valid until: Sun Jun 19 21:18:27 2005Enter MyProxy pass phrase:Verifying password - Enter MyProxy pass phrase:A proxy valid for 168 hours (7.0 days) for user giorgio now exists
on grid001.ct.infn.it.
•Principal options • -c hours specifies lifetime of stored credentials • -t hours specifies the maximum lifetime of credentials when retrieved• -s <hostname> specifies the myproxy server where to store credentials• -d stores credential with the distinguished name in proxy, instead of user name (mandatory for some data management services and proxy renewal)• For proxy renewal it’s also mandatory –n (no passphrase). You’ve to specify also subject of principals that can renew a delegation (-R subject, or -A for any principal)
![Page 14: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/14.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 14
Enabling Grids for E-sciencE
INFSO-RI-508833
myproxy-info
• Useful to retrieve info on stored credentials• Need local credentials to be performed• If credentials have been initialized with –d switch, you have
also to specify it there
[giorgio@glite-tutor:~]$ myproxy-info -s grid001.ct.infn.it username: giorgioowner: /C=IT/O=GILDA/OU=Personal Certificate/L=INFN/CN=Emidio Giorgio/[email protected] timeleft: 167:55:34 (7.0 days)
![Page 15: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/15.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 15
Enabling Grids for E-sciencE
INFSO-RI-508833
myproxy-get-delegation
• This command is used to retrieve a delegation from a long lived proxy stored on myproxy server
• It is independent by the machine ! You don’t need to have your certificate on board
• If credentials have been initialized with –d switch, you have to specify it also in myproxy-get-delegation request
[giorgio@glite-tutor:~]$ myproxy-get-delegation \-s grid001.ct.infn.itEnter MyProxy pass phrase:A proxy has been received for user giorgio in /tmp/x509up_u513
![Page 16: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/16.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 16
Enabling Grids for E-sciencE
INFSO-RI-508833
myproxy-destroy
• Delete, if existing, the long lived credentials on the specified myproxy server
[glite-tutor] /home/giorgio > myproxy-destroy \ -s grid001.ct.infn.it Default MyProxy credential for user giorgio was successfully removed.
![Page 17: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/17.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 17
Enabling Grids for E-sciencE
INFSO-RI-508833
Workload Management System
Giuseppe La RoccaValeria Ardizzone
![Page 18: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/18.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 18
Enabling Grids for E-sciencE
INFSO-RI-508833
• In the glite middleware a user can submit and cancel jobs, query their status, and retrieve their output. These tasks go under the name of Workload Management.
• There are two different User Interfaces to accomplish these tasks. One is the Command Line Interface and the other is the Graphical User Interface.
Workload Management System
![Page 19: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/19.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 19
Enabling Grids for E-sciencE
INFSO-RI-508833
It is possible to see which CEs are eligible to run a job specified by a given JDL file using the command
glite-job-list-match hello.jdl
where <jdl file> is the file containing the job description
Connecting to host lxshare0380.cern.ch, port 7772
Selected Virtual Organisation name (from UI conf file): dteam
***************************************************************************
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
adc0015.cern.ch:2119/jobmanager-lcgpbs-infinite
adc0015.cern.ch:2119/jobmanager-lcgpbs-long
adc0015.cern.ch:2119/jobmanager-lcgpbs-short
***************************************************************************
Command Line Interface (cont.)
![Page 20: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/20.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 20
Enabling Grids for E-sciencE
INFSO-RI-508833
Command Line Interface
• Job Submission– Perform the job submission to the Grid.
$ glite-job-submit [options] <jdl_file>
--vo <vo name> : perform submission with a different VO than the UI default one.
--output, -o <output file> save jobId on a file.--resource, -r <resource value> specify the
resource for execution. --nomsgi neither message nor errors on the stdout
will be displayed.
![Page 21: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/21.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 21
Enabling Grids for E-sciencE
INFSO-RI-508833
If the submission is successful, the output is similar to:
glite-job-submit test.jdl
======================glite-job-submit Success =======================
The job has been successfully submitted to the Network Server.
Use glite-job-status command to check job current status.
Your job identifier (edg_jobId) is:
- https://lxshare0234.cern.ch:9000/rIBubkFFKhnSQ6CjiLUY8Q
===================================================================
In case of failure, an error message will be displayed instead, and an exit status different form zero will be retured.
Command Line Interface (cont.)
![Page 22: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/22.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 22
Enabling Grids for E-sciencE
INFSO-RI-508833
If the command returns the following error message:
**** Error: API_NATIVE_ERROR ****
Error while calling the "NSClient::multi" native api
AuthenticationException: Failed to establish security context...
**** Error: UI_NO_NS_CONTACT ****
Unable to contact any Network Server
it means that there are authentication problems between the UI and the Network Server (check your
proxy or have the site administrator check the certificate of the server).
(applies to UIPnP)
Command Line Interface (cont.)
![Page 23: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/23.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 23
Enabling Grids for E-sciencE
INFSO-RI-508833
After a job is submitted, it is possible to see its status using the glite-job-status command.
glite-job-status https://lxshare0234.cern.ch:9000/X-ehTxfdlXxSoIdVLS0L0w
*************************************************************
BOOKKEEPING INFORMATION:
Printing status info for the Job:
https://lxshare0234.cern.ch:9000/X-ehTxfdlXxSoIdVLS0L0w
Current Status: Ready
Status Reason: unavailable
Destination: lxshare0277.cern.ch:2119/jobmanager-pbs-infinite
reached on: Fri Aug 1 12:21:35 2003
*************************************************************
Command Line Interface (cont.)
![Page 24: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/24.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 24
Enabling Grids for E-sciencE
INFSO-RI-508833
The option -i <file path> can be used to specify a file with a list of job identifiers (saved previously with the -o option of glite-job-submit).
glite-job-status -i jobs.list---------------------------------------------------------------------------------------------------1 : https://lxshare0234.cern.ch:9000/UPBqN2s2ycxt1TnuU3kzEw2 : https://lxshare0234.cern.ch:9000/8S6IwPW33AhyxhkSv8Nt9A3 : https://lxshare0234.cern.ch:9000/E9R0Yl4J7qgsq7FYTnhmsA4 : https://lxshare0234.cern.ch:9000/Tt80pBn17AFPJyUSN9Qb7Qa : allq : quit---------------------------------------------------------------------------------------------------Choose one or more edg_jobId(s) in the list - [1-4]all:
If the - -all option is used instead, the status of all the jobs owned by the user submitting the command is retrieved.
Command Line Interface (cont.)
![Page 25: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/25.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 25
Enabling Grids for E-sciencE
INFSO-RI-508833
The --status <state> (-s) option makes the command retrieve only the jobs that are in the specified state, and the --exclude
<state> (-e) option makes it retrieve jobs that are not in the specified state.
This two lasts options are mutually exclusive, although they can be used with --from and --to.
Example: All jobs of the user that are in the state DONE or RUNNING are retrieved.
glite-job-status --all -s Done -s Running
Example: All jobs that were submitted before the 17:35 of the current day, and that were not in the Cleared state are retrieved.
glite-job-status --all -e Cleared --to 17:00
Command Line Interface (cont.)
![Page 26: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/26.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 26
Enabling Grids for E-sciencE
INFSO-RI-508833
A job can be canceled before it ends using the command glite-job-cancel.
glite-job-cancel https://lxshare0234.cern.ch:9000/dAE162is6EStca0VqhVkog
Are you sure you want to remove specified job(s)? [y/n]n :y
=================== glite-job-cancel Success====================
The cancellation request has been successfully submitted for the following job(s)
- https://lxshare0234.cern.ch:9000/dAE162is6EStca0VqhVkog
===========================================================
Command Line Interface (cont.)
![Page 27: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/27.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 27
Enabling Grids for E-sciencE
INFSO-RI-508833
After the job has finished (it reaches the DONE status), its output can be copied to the UI
glite-job-output https://lxshare0234.cern.ch:9000/snPegp1YMJcnS22yF5pFlg
Retrieving files from host lxshare0234.cern.ch*****************************************************************JOB GET OUTPUT OUTCOMEOutput sandbox files for the job:- https://lxshare0234.cern.ch:9000/snPegp1YMJcnS22yF5pFlghave been successfully retrieved and stored in the directory:/tmp/jobOutput/snPegp1YMJcnS22yF5pFlg*****************************************************************
By default, the output is stored under /tmp, but it is possible to specify in which directory to save the
output using the - -dir <path name> option.
Command Line Interface (cont.)
![Page 28: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/28.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 28
Enabling Grids for E-sciencE
INFSO-RI-508833
Exercises
![Page 29: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/29.jpg)
EGEE Tutorial, Melbourne, 30.01..2006
Enabling Grids for E-sciencE
INFSO-RI-508833
hello.jdl
Type = "Job";JobType = "Normal";Executable = "/bin/echo";StdOutput = "hello.out";StdError = "hello.err";OutputSandbox = {"hello.err","hello.out"};Arguments = "Hello World";RetryCount = 7;
Create JDL file similar to:
![Page 30: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/30.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 30
Enabling Grids for E-sciencE
INFSO-RI-508833
Exercises
• Check where hello.jdl can be executed• Run the hello world example• Submit multiple times• Cancel a job• Check output
![Page 31: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/31.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 31
Enabling Grids for E-sciencE
INFSO-RI-508833
Exercises
• Simple job submission
– cp hello.jdl exercise1.jdl
– Modify exercise1.jdl file Instead of running hostname command, run a bash
script you have just created (yourscript.sh).
– Submit the job, check its status and when done retrieve the output
Create a bash script which displays hostname and current date
Test script locally
Save the script as yourscript.sh
![Page 32: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/32.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 32
Enabling Grids for E-sciencE
INFSO-RI-508833
JDL – Examples
Type = "Job";JobType = "Normal";Executable = "/bin/bash";StdOutput = “std.out";StdError = “std.err";InputSandbox = {“yourscript.sh"};
OutputSandbox = {“std.err",“std.out"};Arguments = "yourscript.sh";
#!/bin/bash/bin/hostname/bin/date
exercise1.jdl
yourscript.sh
![Page 33: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/33.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 33
Enabling Grids for E-sciencE
INFSO-RI-508833
JDL – Requirements
#Insert a requirement to parse only the short queues.Requirements = (other.GlueCEPolicyMaxWallClockTime >
720);
#Insert a requirement to parse only the long queues.Requirements = (other.GlueCEPolicyMaxWallClockTime >
1440);
#Insert a requirement to parse only the infinite queues.
Requirements = (other.GlueCEPolicyMaxWallClockTime > 2880);
#Insert a requirement to stear the execution on a particular CE Queue.
Requirements = other.GlueCEUniqueID == "grid010.ct.infn.it:2119/jobmanager-lcgpbs-long";
![Page 34: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/34.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 34
Enabling Grids for E-sciencE
INFSO-RI-508833
Exercises
• Simple job using Requirements– Modify exercise1.jdl file so that user with a even
workstation number will submit their job on a “long” queue, and the other to an “infinite” one
– Verify the list of CE suitable for this job execution– Submit the job, check its status and retrieve the output
![Page 35: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/35.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 35
Enabling Grids for E-sciencE
INFSO-RI-508833
JDL with Requirements
Type = "Job";JobType = "Normal";Executable = "/bin/sh";Arguments = "start_povray_cubo.sh";StdOutput = "povray_cubo.out";StdError = "povray_cubo.err";InputSandbox =
{"start_povray_cubo.sh","cubo.pov"};OutputSandbox =
{"povray_cubo.out","povray_cubo.err","cubo.png"};
Requirements = Member("POVRAY-3.5",other.GlueHostApplicationSoftwareRunTimeEnvironment);
![Page 36: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/36.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 36
Enabling Grids for E-sciencE
INFSO-RI-508833
start_povray_cubo.sh
#!/bin/bashmv cubo.pov OBJECT.POV #rename input file/usr/bin/povray
/usr/share/povray-3.5/ini/res800.ini #run povray
mv OBJECT.png cubo.png #rename output file
![Page 37: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/37.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 37
Enabling Grids for E-sciencE
INFSO-RI-508833
Exercises
• Modify povray_cubo.jdl, specifying the resource for execution into the jdl file
• Check job status and when done retrieve the output
• Display .png file obtained as output, using ImageMagick (if possible – transfer file on Windows machines or leave out this step)
![Page 38: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/38.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 38
Enabling Grids for E-sciencE
INFSO-RI-508833
Run an ls command on the selected resource.[ Executable = "ls.sh"; Arguments = "-alt"; StdError = "stderr.log"; StdOutput = "stdout.log"; InputSandbox = "ls.sh"; OutputSandbox = {"stderr.log", "stdout.log"}; ]
ls.sh#!/bin/bash /bin/ls $1
Exercises
![Page 39: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/39.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 39
Enabling Grids for E-sciencE
INFSO-RI-508833
Type = "Job"; JobType = "MPICH"; Executable = "MPItest.sh"; Arguments = "cpi 2"; NodeNumber = 2; StdOutput = "test.out"; StdError = "test.err"; InputSandbox = {"MPItest.sh","cpi"}; OutputSandbox = {"test.err","test.out","executable.out"};Requirements = other.GlueCEInfoLRMSType == "PBS" ||
other.GlueCEInfoLRMSType == "LSF";
mpi.jdl
![Page 40: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/40.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 40
Enabling Grids for E-sciencE
INFSO-RI-508833
MPItest.sh
#!/bin/sh#
# this parameter is the binary to be executedEXE=$1# this parameter is the number of CPU's to be reserved
for parallel executionCPU_NEEDED=$2
# prints the name of the master nodeecho "Running on: $HOSTNAME"
if [ -f "$PWD/.BrokerInfo" ] ; then TEST_LSF=`edg-brokerinfo getCE | cut -d/ -f2 | grep
lsf`else TEST_LSF=`ps -ef | grep sbatchd | grep -v grep`fi
![Page 41: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/41.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 41
Enabling Grids for E-sciencE
INFSO-RI-508833
MPItest.sh (cont.)
if [ "x$TEST_LSF" = "x" ] ; then # prints the name of the file containing the nodes
allocated for parallel execution echo "PBS Nodefile: $PBS_NODEFILE" # print the names of the nodes allocated for parallel
execution cat $PBS_NODEFILE HOST_NODEFILE=$PBS_NODEFILEelse # print the names of the nodes allocated for parallel
execution echo "LSF Hosts: $LSB_HOSTS" # loops over the nodes allocated for parallel
execution HOST_NODEFILE=`pwd`/lsf_nodefile.$$
for host in ${LSB_HOSTS} do host=`host $host | awk '{ print $1 } '` echo $host >> ${HOST_NODEFILE} donefi
![Page 42: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/42.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 42
Enabling Grids for E-sciencE
INFSO-RI-508833
# prints the working directory on the master nodeecho "Current dir: $PWD"
for i in `cat $HOST_NODEFILE` ; do echo "Mirroring via SSH to $i" # creates the working directories on all the nodes
allocated for parallel execution ssh $i mkdir -p `pwd` # copies the needed files on all the nodes allocated
for parallel execution /usr/bin/scp -rp ./* $i:`pwd` # checks that all files are present on all the nodes
allocated for parallel execution echo `pwd` ssh $i ls `pwd` # sets the permissions of the files ssh $i chmod 755 `pwd`/$EXE ssh $i ls -alR `pwd`done
MPItest.sh (cont.)
![Page 43: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/43.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 43
Enabling Grids for E-sciencE
INFSO-RI-508833
# execute the parallel job with mpirunecho "*********************************"echo "Executing $EXE"chmod 755 $EXEmpirun -np $CPU_NEEDED -machinefile $HOST_NODEFILE
`pwd`/$EXE > executable.outecho "*********************************"
MPItest.sh (cont.)
![Page 44: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/44.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 49
Enabling Grids for E-sciencE
INFSO-RI-508833
Exercises
Modify c_sample.c file as follow:
#include <stdio.h>int main(int argc, char **argv){
printf("\n\n\n");printf("Hello !\n");
printf("Welcome to EGEE Tutorial, Melbourne\n\n\n");
exit(0);}
This exercise allows user to submit a C program.
![Page 45: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/45.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 50
Enabling Grids for E-sciencE
INFSO-RI-508833
Compile your script using make.
Submit the c_sample.jdl job to the grid using the glite-job-submit c_sample.jdl command.
Inspect the status and retrieve its output when the job is finished.
Exercises (cont.)
![Page 46: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/46.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 51
Enabling Grids for E-sciencE
INFSO-RI-508833
Exercises
Modify c_sample.c file as follow:
#include <stdio.h>int main(int argc, char **argv){ char *name = argv[1]; printf("\n\n\n"); printf("Hello !\n"); printf("Welcome to EGEE Tutorial\n\n\n"); exit(0);}
![Page 47: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/47.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 52
Enabling Grids for E-sciencE
INFSO-RI-508833
Exercises (cont.)
Compile your script with make.
Modify the start_c_sample.sh script as follow:
#!/bin/shchmod 777 c_sample./c_sample $1
Modify c_sample.jdl’s Arguments as follow:Arguments = "start_c_sample.sh <Your Name>";
Submit, inspect the status and retrieve its output when the job is finished.
![Page 48: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/48.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 53
Enabling Grids for E-sciencE
INFSO-RI-508833
Questions…
![Page 49: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/49.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 54
Enabling Grids for E-sciencE
INFSO-RI-508833
DataGrid Accounting System
Giuseppe La Rocca
![Page 50: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/50.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 55
Enabling Grids for E-sciencE
INFSO-RI-508833
Exercise
View user Credits
![Page 51: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/51.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 56
Enabling Grids for E-sciencE
INFSO-RI-508833
$ dgas-check-balance
User: Giuseppe La RoccaE-mail:: [email protected]: /C=IT/O=GILDA/OU=Personal Certificate/L=INFN
Catania/CN=Giuseppe La Rocca/[email protected]
Assigned credits (0=infinite): 0Booked credits: 0Used credits: 451Used wall clock time (sec): 1187Used CPU time (sec): 264Accounted jobs: 22
![Page 52: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/52.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 57
Enabling Grids for E-sciencE
INFSO-RI-508833
View CE Price
Exercise
![Page 53: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/53.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 58
Enabling Grids for E-sciencE
INFSO-RI-508833
Usage: dgas-check-ce-price <CE name>:2119/jobmanager-lcgpbs-<queue>
Example: dgas-check-ce-price grid010.ct.infn.it:2119/jobmanager-lcgpbs-short
Price Authority queried at: Thu Oct 20 18:43:39 CEST 2005
Computing Element: grid010.ct.infn.it:2119/jobmanager-lcgpbs-short
Price (credits for 100 CPU secs): 170
![Page 54: INFSO-RI-508833 Enabling Grids for E-sciencE GILDA WMS Praticals Ruediger Berlich / Forschungszentrum Karlsruhe Mike Mineter / National.](https://reader035.fdocuments.in/reader035/viewer/2022062520/56649f125503460f94c25571/html5/thumbnails/54.jpg)
EGEE Tutorial, Melbourne, 30.01..2006 59
Enabling Grids for E-sciencE
INFSO-RI-508833
Questions…