Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil...
Transcript of Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil...
![Page 1: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/1.jpg)
BuildingaSLURMBankingSystemAuthorsSahilHasan*,HarrisonKuo*,CassieZhang*,QuinnDombrowski*,SteveMasover*,PatrickSchmitz*,KrishnaMuriki(*,**)YongQin(*,**)
*BerkeleyResearchComputing,UniversityofCalifornia,Berkeley*HighPerformanceComputingServices,LawrenceBerkeleyNationalLaboratory
![Page 2: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/2.jpg)
myBRCDashboardUIDatabaseSystemRESTAPISLURMPlugins
- Overview-
BuildingaSLURMBankingSystem
![Page 3: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/3.jpg)
• BerkeleyResearchComputing
• Deliverreliable,sustainableresourcesandservicestomeetcomputationaldemandsofResearchgroupsinUCBerkeley.
• Savioisa400nodeinstitutionalHPCLinuxclusterwhichformsthefoundationoftheBRCprogram.
• NumberofresearchgroupswithCondocontributions-22
• Numberofresearchgroupsusingfreecomputeallowances- 195
• Numberofuniqueusers- 1120
WhatisBRC?
![Page 4: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/4.jpg)
• Managedthroughsacct,sacctmgr
• CurrentBRCaccountinginfrastructureislimitedandnoteasilyintegratedintomyBRC
• HardtocreatewebapplicationstoprovideuserinterfacesforaccountingusingSLURMAPI
StateofSLURMAccounting
![Page 5: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/5.jpg)
Why?
• Facultygetcomputeallowanceof300KServiceUnits(SU=1core-hour)
• FacultyComputingAllowances(FCA)areusuallyformedforfacultyresearch•FCA~10softrustedusersinagroup
• InstructionComputingAllowances(ICA)areusuallygivenforinstructors•ICA~10-100sofpotentiallyuntrustedusers
• Instructors/Facultywanttocontrolquotasandourcurrentinfrastructurecannotsupportthis
![Page 6: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/6.jpg)
Goals
• Createanopensource,plugandplayaccountingsystem
• IndependentofexistingSLURMdatabase
• Easilyallowforwebapplicationstobebuiltontopofstack
![Page 7: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/7.jpg)
Overview
SLURMplugins
API+Logic
SLURMplugins myBRCDashboard
Database
![Page 8: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/8.jpg)
• AgraphicalDjango-baseddashboardthataidsaccountownersinchangingallocations,generatingvisualizations,etc.
• AdatabasesystemseparatefromexistingSLURMinfrastructure
• RESTAPItoprotectintegrityofdatabase
• Asystemofpluginsthatenableseasysub-allocationofTRES-minutesandtracksburndownofsub-allocationswithinanaccount
Components
![Page 9: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/9.jpg)
DatabaseSystemRESTAPISLURMPlugins
Overview
- myBRCDashboardUI-
BuildingaSLURMBankingSystem
![Page 10: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/10.jpg)
DashboardIntegratedwithBRCWebsite
![Page 11: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/11.jpg)
Login
![Page 12: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/12.jpg)
NotificationCenter
![Page 13: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/13.jpg)
StatisticsVisualization
![Page 14: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/14.jpg)
AllocationManager
![Page 15: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/15.jpg)
ServiceUnitCalculator
![Page 16: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/16.jpg)
RESTAPISLURMPlugins
OverviewmyBRC DashboardUI
- DatabaseSystem-
BuildingaSLURMBankingSystem
![Page 17: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/17.jpg)
DatabaseSystem
• NeededtoreflectwhatwasusedinSLURMaccounting
• Neededtohavegoodassociationsbetweenusers,accountsandjobs
• Customizablepartitions,QOS,jobstatus,etc.
• UsesmySQL
![Page 18: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/18.jpg)
![Page 19: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/19.jpg)
SLURMPlugin
OverviewmyBRC DashboardUIDatabaseSystem
- RESTAPI-
BuildingaSLURMBankingSystem
![Page 20: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/20.jpg)
RestAPI
• Neededtosupportviewingquotas
• Neededtoallowprivilegeduserstoeditotherusers’quotas
• Threeendpoints:/users,/accounts,and/jobs
• Allowsalayerofconvenienceandsecuritybetweentheuser/applicationandthedatabase/logic/SLURM
• UsesDjangoWebFramework
![Page 21: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/21.jpg)
- SLURMPlugin-
OverviewmyBRC DashboardUIDatabaseSystemRESTAPI
BuildingaSLURMBankingSystem
![Page 22: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/22.jpg)
SLURMPlugin
• SLURMpluginsperformwebAPIcallstointeractwithdatabase
• Allowsforjobsrunfromtheterminaltoberegisteredindatabase
• Splitintotwofunctions:asubmit-timeandanepilogue
![Page 23: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/23.jpg)
LifecycleofaSuccessfulJob
Job submittedBy user
Job created with hold status
Submit-Time
Job updatedwith finish
status Check SUs used
Epilogue
Check if job is less than allowance
Deducts user balance
Job run
![Page 24: Building a SLURM Banking System - Research IT€¦ · Building a SLURM Banking System Authors Sahil Hasan*, Harrison Kuo*, Cassie Zhang*, Quinn Dombrowski* ... • Savio is a 400](https://reader030.fdocuments.in/reader030/viewer/2022040215/5ed51624a1ec216f0014fd73/html5/thumbnails/24.jpg)
Thankyou