DanSellars Manager,SoftwareDevelopment CANARIE€¦ · support research, education, and scientific...
Transcript of DanSellars Manager,SoftwareDevelopment CANARIE€¦ · support research, education, and scientific...
Reusable Platform InvestigationDan Sellars | Manager, Software Development | CANARIE
27
CANARIEisinvestigatingtheadoptionofareusableplatform:
>Whyarewegoinginthisdirection?
>Whocanbestbenefitfromthisplatform?
>Howareweapproachingtheinvestigation?
>Whichoptionsareweconsidering?
Introduction
28
ReusablePlatform
OurmandateistosupportCanadianresearchersby:
> Fundingsoftwaredevelopment
> Providingsoftwaretoacceleratetimetodiscovery
> EnablingeffectiveuseofCanadianDigitalInfrastructure
> Promotingre-useofexistingsoftware— sowecanfundmore!
> SupportingthecommunityofCanadianresearchsoftwaredevelopers
ResearchSoftwareatCANARIE
29
> IdentifyareusableplatformwhereCANARIEcandevelopexpertise
>Promoteitforre-useinthecommunity
>Determineifapermanentdevelopmentteamcouldassistresearchsoftwaredevelopers
>Usetheresultingplatformasaninteroperabilitytestbed
>Wewanttocomplement— notcompete
WhatisCANARIE’sgoal?
30
>Wouldsuchanapproachbeacceptedbythecommunity?• Whatabout“NotInventedHere”?
>CanCANARIEaddenoughvalue?
>Canwesupportenoughteamstobeofvalue?
>Woulditbeadaptableenough?
PossiblePitfalls
31
>Thosewhodonotyethaveaplatform
>Thosewhowanttofocusonresearch-facingsoftware• Avoidingframeworkdevelopment• Donotwanttorecreatewhathasbeendonebefore
>Non-software-expertsinneedofawayintodigitalinfrastructure• Theso-called,“long-tail”ofbigdataresearchers
Whocanbestbenefitfromthisplatform?
32
Whatisanidealreusableplatform?
33
>First,whatisaplatform?• “asoftwaresystemthatsupportsmostorallofthefunctionalityaresearcherusesintheirworkflow”
>Areusableplatformwouldmeetthefollowingneeds:• Community-friendly• Computeintegrations• Datamanagementcapability• Extendable
DataManagement
Compute
Community
CANARIE
ResearchSpecific
Extensions
>Wedonotexpectonesolutiontofitall• Multipleoptionscouldbeviable• Acombinationmayalsoprovideamorecompletesolution
>Researchsoftwareisanotonesizefitsall• Butwethinkagoodbasewouldbeofwidebenefit
>Reusableplatformsarenotforeverybody• Therewillalwaysbeaneedforcustomdevelopmenttobringnewtechnologiestothecommunity
Morethoughtsonreusableplatforms
34
Aphasedapproachtotheinvestigation:
> Phase1:Highlevelreviewof~6options• Assessbyinspectionagainstdefinedcriteria• Selectashortlistforthenextphase
> Phase2:Closeevaluationof2-3platforms• Narrowdowntoonecandidateforthenextphase• Refinephase3goals
> Phase3:Deploymentofaprototype• Deployandprovidesampleextensionstothisplatform
Solicitfeedbackfromthecommunity
WhatapproachisCANARIEtaking?
35
• Ease ofdeployment• Flexibility• Supportformultipleprojects• Easeofuse• Cost• Stability
Whatarethedefinedcriteria?
36
• Reliabilityandsupport• Researchdatamanagementsupport• Securityandprivacy• Sharingandcommunication• Suitability• Existingfeatureset
JustconcludedPhase1• Startedbroadinvestigationfirst
Projectsthatweinvestigated• HubZero https://hubzero.org/• Open ScienceFrameworkhttps://osf.io• JupyterNotebook/Jupyter Hubhttp://jupyter.org/• CBRAIN http://mcin-cnim.ca/neuroimagingtechnologies/cbrain/• Alces Flighthttp://alces-flight.com/• Airavatahttp://airavata.apache.org• GenApphttp://genapp.rocks/
Wherearewenow?
37
HUBzero:asoftwareplatformusedtocreatedynamicwebsitesthatsupportresearch,education,andscientificcollaboration.> 60+HUBzero-basedsitesservingdifferentcommunities• Interactivesimulationtoolsdeliveredthroughwebbrowser• Usergroupsforprivatecollaboration• Mechanismforuploadingandsharingresources• Ratingandcitations• Wikisandblogs• Usersupportarea• Statisticsaboutusersandusagepatterns
HUBzero
38
OpenScienceFramework:providesfreeandopensourceprojectmanagementsupportforresearchersacrosstheentireresearchlifecycle.
• Structuredprojects• Controlledaccess• Enhancedworkflow• Trackprojectchanges• Viewprojectanalytics
>HostedintheUS
OpenScienceFramework
39
JupyterNotebook:awebapplicationthatenablesuserstocreateandsharedocumentsthatincludelivecode,visualizations,equations,andtext.
> Jupyter Notebookcombinesthreecomponents:• Thewebapplication• Kernels• Notebookdocuments
>WeknowComputeCanadaareinvestigating• Sowillnotproceedwiththisasaprimaryoption
Jupyter Notebook
40
CBRAIN:aweb-enabledplatformtoaccessandprocessdistributeddatawithhigh-performancecomputingcenters(HPCs).
> ConsistsoftwoRuby-on-RailsApplications:
• BrainPortal:aweb-basedgraphicaluserinterface• Userscanupload,tag,andsearchtheirfiles• Launchcompute-intensiveprocessingjobsonremoteHPC
• Bourreau: thebackendoftheCBRAINarchitecture– Managesthebackendtasks
CBRAIN
41
AlcesFlight:afully-featured,scalableHighPerformanceComputing(HPC)environmentforresearchandscientificcomputing.
>Mainfeatures:• CanbeconfiguredwithAlces Gridware
• provides750+applicationsandlibraries• ChoiceofHPCjobscheduler• SSHandgraphical-desktopconnectivityforusers• Datamanagementforfile-basedstorageandobjectstorage
Alces Flight
42
ApacheAiravata:asoftwareframeworkforsupportingexecutionandmanagementofcomputationalscientificapplicationsandworkflows.• Mainfocusisingridbasedsystems
> Severalcomponents:• Airavata API• Orchestrator• WorkflowInterpreter• ApplicationFactory(Gfac)• Registry• Messaging
Airavata
43
GenApp:amodularframeworkformulti-scalesciencecomputations.
> BuildsGUI-basedandweb-basedapplicationsonacollectionofJSONwrappedmodules• Extendableandadaptabletonewtargetlanguagesandenvironments
>CanbeintegratedwithAiravata• Providesthecapabilitytoharnessdistributedcomputingresources
*NottobeconfusedwithGenAP!
GenApp*
44
> JustabouttostartPhase2>Gaincommunityfeedback>RefineactivitiesforinclusioninPhase3e.g.• DeployatestinstanceontheDAIRCloud?• Willrepackagingmakeiteasiertoinstall?• Adding/integratingservicesofusetoallusers i.e.fromscience.canarie.ca?• Bettertrainingmaterials?• Wouldconsultingservicesbebeneficial?• PossibleintegrationwithResearchDataManagement(RDM)tools
>Offerasareusableplatformsupportedbyusforuseinafuturecall
NextSteps
45
>Wouldthishavebeenahelpwhenyoustarteddevelopment?>Doyouhaveanyadvice?>WhatdoyouseeasthebestwaythatCANARIEcouldoffersupport?• Provideexpertiseonintegration?• Extendinginfrastructurefunctionality?• IntegratewithComputeCanadaservices?
>Wouldyoubeinterestedingettinginvolved?• Advisorycapacity?• Usecaseprovider?• Offeringresearchfacingsoftwareforintegration?
Whatcommunityinputwouldwelike?
46
>TalkwiththeCANARIESoftwareteamduringtheevent> Email:[email protected]>Chooseitasadiscussiontopicfortomorrow>WewillaskagainintheclosingQ&Asession>Now,ifyouhavesome!Wehavesomequestions…
HowtoGiveFeedback
47
Thank you!Questions?
48