Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director,...
-
Upload
silas-lamb -
Category
Documents
-
view
215 -
download
0
Transcript of Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director,...
![Page 1: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/1.jpg)
globus online
Software-as-a-Service for Research Data ManagementSteve Tuecke
Deputy Director, Computation Institute
University of Chicago & Argonne National Laboratory
![Page 2: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/2.jpg)
www.globusonline.org
Big Science built on Globus Toolkit
Earth System Grid
Cancer BiologiyInformatics Grid
LIGO data grid
LHC
![Page 3: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/3.jpg)
www.globusonline.org
Thinking about “small and medium labs”
• Big projects like LHC, LIGO, ESG, etc., can run resource-level services reliably—and build and operate effective collective services
• Small labs and collaborations have problems with both
• They need solutions, not toolkits—ideally outsourced solutions
Can we harness the power of the cloud to scale access to the grid? 3
![Page 4: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/4.jpg)
www.globusonline.org
Time-consuming tasks in science
• Run experiments• Collect data• Manage data• Move data• Acquire computers• Analyze data• Run simulations• Compare experiment
with simulation• Search the literature
• Communicate with colleagues
• Publish papers• Find, configure, install
relevant software• Find, access, analyze
relevant data• Order supplies• Write proposals• Write reports• …
4
![Page 5: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/5.jpg)
www.globusonline.org5
• Most research performed in small laboratories• Researchers are trained in their field, not in IT
– They are not experts in collecting, moving, storing, indexing, analyzing, mining, sharing, updating, publishing, and archiving massive amounts of data
• Only limited capital is available for them to spend on data and IT support
• Investment is spent on traditional research tools (e.g., microscopes)—but the world is changing– Now need substantial and sophisticated IT to perform
research, data manipulation, data mining, collaboration
Researchers lack advanced IT infrastructure
![Page 6: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/6.jpg)
www.globusonline.org6
Globus Toolkit Globus OnlineUse the Grid
Reliable file transfer Software-as-a-Service
globusonline.org
Build the Grid
Components for building custom grid solutions
globustoolkit.org
![Page 7: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/7.jpg)
www.globusonline.org
Time-consuming tasks in science
• Run experiments• Collect data• Manage data• Move data• Acquire computers• Analyze data• Run simulations• Compare experiment
with simulation• Search the literature
• Communicate with colleagues
• Publish papers• Find, configure, install
relevant software• Find, access, analyze
relevant data• Order supplies• Write proposals• Write reports• …
7
![Page 8: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/8.jpg)
www.globusonline.org
A B
Discover endpoints, determine available protocols, negotiate firewalls, configure software,
manage space, determine required credentials, configure protocols, detect and respond to failures, determine expected performance, determine actual performance, identify diagnose and correct network misconfigurations, integrate with file systems, …
Starting with data movement
8
![Page 9: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/9.jpg)
www.globusonline.org9
Globus Online In Action
28.6 Terabytes31,000 files56h 44mNo human involvement
Astrophysics simulation datagenerated in Tennessee, moved to Illinois for visualization(Enzo, UCSD; Futures Lab, Argonne)
![Page 10: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/10.jpg)
www.globusonline.org
Globus Online highlights
10
Fire-and-forget data movementMany files and lots of dataThird-party transfers Performance optimizationAcross multiple security domainsExpert operations and support
Web interface
Command line interfacels alcf#dtn:~scp alcf#dtn:~/myfile \ nersc#dtn:~/myfile
HTTP REST interfacePOST https://transfer.api.globusonline.org/ v0.10/transfer <transfer-doc>
GridFTP serversFTP servers
High-performancedata transfer nodes
Globus Connecton local computers
![Page 11: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/11.jpg)
www.globusonline.org
Globus Online architecture
11
![Page 12: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/12.jpg)
www.globusonline.org
Globus Connect to/from your laptop
12
![Page 13: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/13.jpg)
www.globusonline.org
What’s next?
• Run experiments• Collect data• Manage data• Move data• Acquire computers• Analyze data• Run simulations• Compare experiment
with simulation• Search the literature
• Communicate with colleagues
• Publish papers• Find, configure, install
relevant software• Find, access, analyze
relevant data• Order supplies• Write proposals• Write reports• …
13
![Page 14: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/14.jpg)
www.globusonline.org14
Our goal: To leverage software-as-a-service (SaaS) to accelerate the pace of discovery and innovation worldwide, by providing millions of researchers with unprecedented access to powerful research tools
“Civilization advances by extending the number of important operations which we can perform without thinking of them”
Alfred North Whitehead , 1911
Looking to the future
![Page 15: Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c00a1a28abf838cc7cb8/html5/thumbnails/15.jpg)
www.globusonline.org15
www.globusoline.org