Technology on the NGS Pete Oliver NGS Operations Manager.

download Technology on the NGS Pete Oliver NGS Operations Manager.

If you can't read please download the document

  • date post

    19-Dec-2015
  • Category

    Documents

  • view

    223
  • download

    3

Embed Size (px)

Transcript of Technology on the NGS Pete Oliver NGS Operations Manager.

  • Slide 1
  • Technology on the NGS Pete Oliver NGS Operations Manager
  • Slide 2
  • Technology NGS Offers a diverse set of Technology Very acronym rich! Answer Questions such as: Who am I and what can I do? What resources do I have? How do I run Jobs How do I get access to my data? What does my Institution need to get connected? Whats the next new thing?
  • Slide 3
  • Who and What Who am I and What can I do? Authentication UK Certification Authority https://ca.grid-support.ac.uk/ https://ca.grid-support.ac.uk/ Recognised World Wide Access Management Federation (SARoNGS) UK Authorisation Virtual Organisations (VO) ie communities of users Partners Support All VOs Affiliates Choose who they support NGS offers VOM to manage the VO membership https://voms.ngs.ac.uk/ Supports established groups AND Adhoc communities that may be short lived
  • Slide 4
  • Tools Who am I and What can I do? Tools Certificate management tool JAVA based Sets up authentication and authorisation Certificate Wizard Uploads credentials the NGS Myproxy Service VOMS aware SSH terminals MEG Allows users to use any ssh terminal tool if MEG installed on site GSISSH-TERM NGS Myproxy Service Acts on your behalf for authentication to resources
  • Slide 5
  • Resources How much CPU time have I got? Resource providers upload to NGS RUS system Keeps track of CPU time used on a user, site and VO level Web based interface for Users, VOs and Institutions https://uas.ngs.ac.uk/ https://uas.ngs.ac.uk/ Institutions can then manage there own collaborations Adhoc communities Formal arrangements Support each other during downtime etc
  • Slide 6
  • Applications Finding applications and running jobs Resource Providers publish applications they have via to the NGS BDII using Universal Execution Environment Scheme (UEE) All Application launch scripts are in /usr/ngs/app_name Can be links to existing software Launch script is customised by the Resource provider to make sure the app runs on their site Integration with GridPP Resource Providers Publish which VOs can use their resource
  • Slide 7
  • UEE and BDII NGS BDII NAMD CASTEP User Specifies UEE name NAMD in WMS JDL and submits HEI publish using has CASTEP using UEE to NGS BDII HEI Compute HEI publish using has NAMD using UEE to NGS BDII HEI Compute NGS WMS NGS WMS Query NGS WMS Launch How UEE and BDII are used HEI publish using has NAMD using UEE to NGS BDII HEI Compute free Busy Free busy
  • Slide 8
  • Portals Web Based Portal Dont need to login to UI Login to portal instead http://portal.ngs.ac.uk Configure jobs using Portal interfaces Upload download files Track jobs Integration with WMS (to do)
  • Slide 9
  • Information Whats out there? All NGS Partners and Affiliates publish Information to NGS BDII CPUs available Applications available Plus a lot of other info used by WMS NGS Website uses this info to construct Usage meters Applications lists
  • Slide 10
  • Monitoring How do I know my job will work Monitoring Tests infrastructure as a user to ensure it works! All Partners and Affiliates are monitored Site alerted when tests fail Reviewed by NGS Operations INCA http://inca2.ngs.ac.uk http://inca2.ngs.ac.uk Downtime in GOCDB (EGEE activity) https://goc.gridops.org/ https://goc.gridops.org/ When will it run? WMS makes decision on load of sites Very difficult to give exact timings
  • Slide 11
  • Data and Storage Storing Data NGS does not provide storage Except to the life time of running jobs Allows Resource Providers to make available resources Gridftp servers, SRB Vaults Moving data Small amounts can be done with the job The WMS manages the transfers SRB Infrastructure (MCAT databases) Partners and Affiliates can supply Vaults which use the central infrastructure Guis to help HERMES
  • Slide 12
  • Institutional Resources How to setup your Institutions resources Access to NGS experts and guides Tested installations Various OSs RedHat, SL, SUSE etc Software Stacks VDT, Glite etc Knowledge will increase the more we do together
  • Slide 13
  • Resources out there The NGS itself does not provide any resources it allows HEIs to make their resources available to who they want to! Eg STFC-RAL 256 Core Cluster to all NGS users 30+ applications Oracle 1800 Core Cluster to STFC staff, Facilities and Facility users (ISIS, CFL, DLS) 30+ applications
  • Slide 14
  • Research & Development NGS R&D Advanced Reservation Novel environments Virtualisation Clouds Agile Deployments Very fast Networking OPN
  • Slide 15
  • Summary I TechnologyUser needsHEI need to runRun by NGS For all UI** Myproxyuploadtool** Portal** MEG/GSISSH-TERM** Hermes** UAS** INCA Monitoring** Compute and Data Resources** End User Applications** RUS Clients** UEE** RA** Middle ware stack Globus/gLite/other** BDII Server* Myproxy Server* RUS Server* SARoNGS* VOM Server* WMS* CA and RA Network* INCA Server*
  • Slide 16
  • Questions? http://www.ngs.ac.uk support@grid-support.ac.uk