XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... ·...
Transcript of XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... ·...
![Page 1: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/1.jpg)
February 23, 2015
XSEDE New User/Allocation Mini-Tutorial
Vincent C. Betro, Ph.D.
University of Tennessee NICS/ORNL
XSEDE Training Manager
![Page 2: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/2.jpg)
Outline
• What’s XSEDE? • How do I get an alloca7on? • What is the User Portal?
– We’ll create an account today
• What is available there? – Globus File Transfer – Common job running interface – Gateways – Training
![Page 3: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/3.jpg)
Don’t sweat taking notes!
• This presenta7on is located at: – h"p://hpcuniversity.org/trainingMaterials/192
![Page 4: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/4.jpg)
• Extreme Science and Engineering Discovery Environment
• Follow on NSF project to TeraGrid in 2012
• Centers operate machines, and XSEDE provides seamless infrastructure for allocaKons, access, and training
• Researchers propose resource use through XRAS
• Supports thousands of scienKsts in fields such as: – Chemistry – BioinformaKcs – Materials Science – Data Sciences
![Page 5: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/5.jpg)
XSEDE Allocations
• Want to use XSEDE resources to teach a class? – hIps://portal.xsede.org/alloca7ons-‐overview#types-‐educa7on
• Just looking to try out a larger resource or a special resource your campus doesn’t have? – hIps://portal.xsede.org/alloca7ons-‐overview#types-‐startup
![Page 6: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/6.jpg)
XSEDE Allocations
• See a Campus Champion (Morgan State needs someone…you?)
• Ready to scale up your research? – hIps://portal.xsede.org/alloca7ons-‐overview#types-‐research
![Page 7: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/7.jpg)
XSEDE Allocations • WriKng and SubmiRng a Successful XSEDE AllocaKon Proposal Recorded: April 9, 2014 Topics covered in detail include:
• research objecKves • computaKonal methodologies • jusKficaKon for SUs needed • applicaKon efficiencies
• View: Example requests from several domains • Read AddiKonal: AllocaKon Request Guidelines and Tips
![Page 8: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/8.jpg)
XSEDE Allocations
When wriKng your research requests consider the following objecKves and guidelines: • Research must be clear and concise. • Outline all the proposed computa7onal algorithms and simula7ons
• Include any input parameters to be inves7gated and explain the choice of simula7on parameters.
• Describe any previous results and progress. • Jus7fy the choice of resources. • Jus7fy the amount of 7me requested.
![Page 9: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/9.jpg)
Now what?
Yeah! I got an XSEDE
allocaKon!
![Page 10: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/10.jpg)
XSEDE User Portal (XUP)
• URL: portal.xsede.org • Single point-‐of-‐entry to informaKon about XSEDE services and uKliKes for using them
• Anyone can create an XUP user account and access non-‐project features
• Only XSEDE allocaKon project members can access project features
![Page 11: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/11.jpg)
Using the XUP
• Create and login to your XUP Account • Use XSEDE resources responsibly • Get added to your XSEDE project • Navigate your personal My XSEDE webpage • Navigate the informaKon in the XUP
![Page 12: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/12.jpg)
Create and login to your XUP account
1. From the XUP homepage, click CREATE ACCOUNT
2. Complete the User Account Form
3. Verify your account request
4. Select your username and password
5. Login to the XUP
portal.xsede.org
Click the CREATE ACCOUNT link to access the XUP User Account Form
![Page 13: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/13.jpg)
XSEDE Acceptable Use Policy
• Must accept the User ResponsibiliKes Form acer creaKng your XUP account and again at the beginning of each allocaKon you receive.
• Choose a strong password and protect it. • Close SSH terminals and log out of the User Portal when you
are finished with your session. • Report Suspicious AcKvity : email [email protected] or call
1-‐866-‐907-‐2383 immediately, regardless of the Kme of day.
XSEDE Cybersecurity Tutorial h"p://www.citutor.org
![Page 14: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/14.jpg)
Get Added to Your XSEDE project
• PIs automaKcally have full access to their project’s account.
• The PI is responsible for managing users on their account.
• Ask the PI, or their allocaKon manager, to add your XUP username to the project.
![Page 15: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/15.jpg)
Your My XSEDE webpage (1) WELCOME TO XUP • Quick access to
commonly used features.
(2) LATEST UPDATES • Latest information
specific to your user account.
(3) MY ACTIVE ALLOCATIONS • Summary of the active
projects for which you are either a PI or member.
![Page 16: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/16.jpg)
Update your XUP User Profile
MY XSEDEàProfile View and or change your user information (organization, address). Make sure your email address is correct. XSEDE staff will use it to
communicate with you regarding your allocation.
![Page 17: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/17.jpg)
Navigating the XUP
• My XSEDE • Resources • DocumentaKon • AllocaKons
• Training • User Forums • Help • About
![Page 18: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/18.jpg)
View the XSEDE Systems Monitor • Resources -‐> Systems Monitor Provides technical and status informaKon for all of XSEDE's resources. The STATUS column indicates whether the system is up or down. If down, can click on status to find when the machine is expected to come back up.
![Page 19: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/19.jpg)
Accessing XSEDE Resources Authentication Methods 1. Password
• XUP credentials • Site-password • One-time password
2. Key-based
Single Sign-On • Enables logging in once to
access all of your allocated resources
Connection Methods 1. GSI-OpenSSH 2. OpenSSH
![Page 20: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/20.jpg)
XSEDE SSO Login Hub An SSO enabled connec7on point to XSEDE resources
Ø SSH to login.xsede.org using your XUP credenKals
Ø Move among resources using gsissh command
![Page 21: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/21.jpg)
Moving Files - Globus Online • A fast, reliable, and secure file transfer service geared to the big data needs of the research community.
• Moves terabytes of data in thousands of files • AutomaKc fault recovery • Easy to use • No client socware installaKon • Consolidated support and troubleshooKng • Supports file transfer to any machine • Accounts are free -‐ www.globusonline.org
![Page 22: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/22.jpg)
Globus Online Dashboard
![Page 23: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/23.jpg)
Sign in with XSEDE login
![Page 24: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/24.jpg)
Logging in with your XSEDE credentials
![Page 25: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/25.jpg)
Connecting globus account
![Page 26: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/26.jpg)
Creating a login on globus.org
![Page 27: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/27.jpg)
Globus Online File Transfer
![Page 28: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/28.jpg)
Choosing a file to move…
![Page 29: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/29.jpg)
Running Jobs Overview
Compute Nodes
Home Scratch
File System
Job Data
Data
Researcher
Login
Login Nodes
Command Line
Use for tasks such as file editing, code compilation,
data backup, and job submission.
Read/write data from compute nodes to Scratch directory.
Store project files Such as source code, scripts, and input data sets to Home directory.
Run jobs by submitting your batch script to the compute nodes using the "qsub" command.
When you connect to a resource, you are on a login node shared by many users.
Your job is submitted to a queue and will wait in line until nodes are available. Queues are managed by a job scheduler that allows jobs to run efficiently.
Batch Script
Commands for code execution, copy input files to scratch,… Specify number/type of nodes, length of run, output directory, …
![Page 30: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/30.jpg)
Login Nodes
• When you login to an XSEDE resource, you connect to a login node.
• Use login nodes for basic tasks such as file ediKng, code compilaKon, data backup, and job submission.
• Do not run compute jobs on the login nodes.
Where do I run compute jobs?
![Page 31: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/31.jpg)
Running Compute Jobs
• Jobs are run on the compute nodes by submiRng a batch script on a login node
• All jobs are placed in a batch queue acer they are submi"ed.
• All XSEDE compute resources use a batch scheduler for running jobs.
• Resource User Guides on the XUP have details on your system’s scheduler.
![Page 32: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/32.jpg)
• portal.xsede.org à Help Submit Kcket
• portal.xsede.org à My XSEDE à Tickets – Submit Kcket – View past Kckets (both open and closed)
• Can also email [email protected] or call 1-‐866-‐907-‐2383, at any hour (24/7)
Need help? Reporting and Tracking Issues
![Page 33: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/33.jpg)
Discussing your problems… User Portal: User Forums • The User Forums are a great place to ask quesKons, get help, or discuss ideas about XSEDE.
![Page 34: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/34.jpg)
More “helpful” resources xsede.orgàUser Services • Resources available at each Service Provider
• User Guides describing memory, number of CPUs, file systems, etc.
• Storage faciliKes • Socware (Comprehensive Search)
• Training: portal.xsede.org à Training • Course Calendar • On-‐line training • CerKficaKons
• Get face-‐to-‐face help from XSEDE experts at your insKtuKon; contact your local Campus Champions.
• Extended CollaboraKve Support (formerly known as Advanced User Support (AUSS))
![Page 35: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/35.jpg)
XSEDE Moodle Site If you are interested in receiving a Mozilla badge for taking the full
version of this New User Training, please visit: h"ps://www.hpc-‐training.org/xsede/moodle/login/index.php starKng May 1 to take a quiz about XSEDE and earn a badge! Also, be
on the lookout for other courses and badge opportuniKes!
![Page 36: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/36.jpg)
ACM SIGHPC Education Chapter
Check out the following link to get more involved with contribuKng to HPC training efforts across the country and world!
h"p://sighpceducaKon.acm.org/
![Page 37: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/37.jpg)
Contact Info Dr. Vincent Charles Betro Training Manager, XSEDE University of Tennessee NaKonal InsKtute for ComputaKonal Sciences Oak Ridge NaKonal Laboratory Phone: 865-‐576-‐8905 email: [email protected]
![Page 38: XSEDE New User/Allocation Mini-Tutorialhpcuniversity.org/media/TrainingMaterials/19/XSEDE_New... · 2015-02-24 · February 23, 2015 XSEDE New User/Allocation Mini-Tutorial Vincent](https://reader034.fdocuments.in/reader034/viewer/2022042909/5f3a22306ded9f7979275235/html5/thumbnails/38.jpg)
February 23, 2015
Thanks for listening and welcome to XSEDE!