Dr Kim Sweeny Dr Masha Fridman Professor Bruce Rasmussen · Nectar Virtual Laboratories Report to...
Transcript of Dr Kim Sweeny Dr Masha Fridman Professor Bruce Rasmussen · Nectar Virtual Laboratories Report to...
Estimatingthevalueandimpactof
NectarVirtualLaboratories
ReporttoNectar
DrKimSweeny
DrMashaFridman
ProfessorBruceRasmussen
VictoriaInstituteofStrategicEconomicStudies
VictoriaUniversity
September2017
1
Contents
1.Introduction 2
2.TheNectarproject 3
3.Methodology 7
4.TheVirtualLaboratories 13
4.1BiodiversityandClimateChangeVirtualLaboratory 13
4.2.CharacterisationVirtualLaboratory 15
4.3GenomicsVirtualLaboratory 16
4.4.HumanitiesNetworkedInfrastructure 18
5.DataonusageofNectarVirtualLaboratories 23
6.EstimatingthevalueandimpactofNectarVirtualLaboratories 26
7.FurtherassessmentofthevalueandimpactoftheCharacterisationVirtualLaboratory 35
8.QualitativevaluationofNectarVirtualLaboratories 39
9.Summaryandconclusions 47
References 49
Appendices 53
1. VirtualLaboratoriesfundedbyNectar 532. CVLimagingfacilitypartners 54
2
1.Introduction
In2016theVictoriaInstituteofStrategicEconomicStudies(VISES)atVictoriaUniversitywascommissionedbytheNationaleResearchCollaborationToolsandResourcesproject(Nectar)toestimatethevalueandimpactofNectarVirtualLaboratories(VLs).
NectarprovidesanonlineinfrastructurethatsupportsresearcherstoconnectwithcolleaguesinAustraliaandaroundtheworld,allowingthemtocollaborateandshareideasandresearchoutcomes,whichwillultimatelycontributetoourcollectiveknowledgeandmakeasignificantimpactonoursociety.
Itprovidessupportforanetworkof14VLswhicharedomain-orientedonlineenvironmentsthatdrawtogetherresearchdata,models,analysistoolsandworkflowstosupportcollaborativeresearchacrossinstitutionalanddisciplineboundaries.
GiventheresourcesavailableandthecentralroleofVLsintheNectarproject,itwasagreedthatthemostappropriateapproachtoestimatingthevalueandimpactofNectarwastoconcentrateonestimatingthevalueandimpactof4VLs,namely
• BiodiversityandClimateChangeVirtualLaboratory(BCCVL)• CharacterisationVirtualLaboratory(CVL)• GenomicsVirtualLaboratory(GVL),and• HumanitiesNetworkedInfrastructure(HuNI)
MostVLsfundedbyNectarhaveonlybeenactiveforafewyearsandarestillintheirgrowthstages.AnevaluationoftheiroverallimpactandvaluemightbestbedonefromtheperspectiveofsomeyearsinthefuturewhentheVLsareinamorematuregrowthphase.Thereforetheanalysisandconclusionsdrawninthisstudyshouldbetreatedaspreliminaryanddependsignificantlyontheassumptionsmadeaboutfuturegrowthpaths.
ItbecameclearduringthecourseofthisstudythatwhileVLssharemanyfeaturesincommon,theydiffersignificantlyfromeachotherintermsoftheservicestheyprovidedtotheirtargetcommunities.Furthermoretheydifferinhoweasyitistoexpresstheirvalueandimpactintraditionaleconomicterms.Consequently,wehaveadoptedanumberofquantitativeandqualitativeapproachestorevealingvalueandimpactandhaveusedtheCharacterisationVirtualLaboratoryasacasestudytoanalysethisinmoredepth.OurapproachtoHumanitiesNetworkedInfrastructure(HuNI)hasbeenexploratoryinnaturewiththeaimofidentifyingdifferentapproachesthatmightbemorefullydevelopedinafuturestudy.
3
2.TheNectarproject
NectarhasitsoriginsinCommonwealthgovernmentinitiativestosupportnationalresearchinfrastructure.TheNationalCollaborativeResearchInfrastructureStrategy(NCRIS)wasestablishedbytheCommonwealthGovernmentin2004andcurrentlyprovidesfundingfor27activeprojectsacross222institutionsemployingover1700technicalexperts,researchersandfacilitymanagers.NCRISfacilitiesareusedbyover35,000researchers,bothdomesticallyandinternationallyandalsoattractfundingfromorganisationsinthehighereducation,government,non-profitandindustrysectors(DepartmentofEducation2017).
SinceitsestablishmenttherehavebeenanumberofreviewsandroadmapsoftheNCRISprogram.
TheStrategicRoadmapforAustralianResearchInfrastructure,releasedinAugust2008,updatedasimilarroadmapin2006butplacedincreasedemphasisoneResearch
“inrecognitionofthepervasiveandunderpinningrelevanceofICTtoresearch.Ascollaborativeresearchincreases,eResearchisprovidingthemostinfluentialandeffectivewayofenablinginstitutionstoworktogether,usingsharedinfrastructure,resourcesandpolicies.”1
Inits2009/10BudgettheAustralianGovernmentannouncedaSuperScienceInitiativetofurthersupportresearchinfrastructurebyaddressingtheprioritiesidentifiedinthe2008StrategicRoadmap.TheSuperScienceInitiativewasfinancedthroughtheEducationInvestmentFund.Nectarwasestablishedin2009bytheAustralianGovernmentfollowinga2009/10Budgetannouncementof$47milliontosupportNectaraspartoftheSuperScienceinitiativefinancedbytheEducationInvestmentFund(EIF),andsubsequentlyreceivedNCRISfunding.Nectarhasreceived$61millioningovernmentfunding,matchedbyco-investmentof$54millionfromAustralianuniversitiesandresearchorganisations
TheUniversityofMelbourneistheleadagentfortheadministrationofNectar.GovernanceofNectarisprovidedbytheNectarProjectBoard,chairedbyRussellYardley,with9othermembersdrawnfromparticipatingresearchinstitutions.
TheNectardirectoratecomprisestheequivalentof6.5full-timestaffandisledbytheDirector,AssociateProfessorGlennMoloney.
1Theroadmapfurtherstatesthat“anewcapabilityintheHumanities,ArtsandSocialSciences(HASS)hasbeenidentifiedinrecognitionofthewiderangingcontributionsthesedisciplinesmaketothenationalinterest.InvestmentinthisareawouldrelatetoaHASSeResearchinfrastructureincludingdatacreationanddigitisationofresearchmaterials”.TheHumanitiesNetworkedInfrastructure(HuNI)canbeviewedasaresponsetothis.
4
NectarisanexampleofasciencegatewaywhichShahand(2015)definesasa“web-basedenterpriseinformationsystemsthatprovidescientistswithcustomizedandeasyaccesstocommunity-specificdatacollections,computationaltoolsandcollaborativeservicesone-Infrastructures.”
Arecentreviewofsciencegateways(Barkeretal2017)describestheirbenefitsasfollows.
Sciencegatewaysareakeycomponentofthefuturedigitalresearchenvironment,enablingresearcherstoutilizeaglobalnetworkofinteractingdigitalplatformstoaccessandsharetheleading-edgedataandtoolsthatarecriticalfortheirresearch.Theybothfacilitate,andaresupportedby,broadermovementssuchasopenresearch,openscience,opensourcesoftwareandopendata.Consequently,sciencegatewaysarevaluabletoarangeofstakeholders:individualresearchers,researchcommunities,researchorganizationsandinstitutions(includingindustryandgovernment)andfundingagencies.
Definingsciencegatewaysintermsofcommoncharacteristicsandfunctionalityassistsinidentifyingtheirvaluetotheirstakeholders.Sciencegatewayslowerbarriersbyhidingthecomplexityoftheunderlyingdigitalresearchinfrastructureandsimplifyingaccesstobest-practicetools,dataandresources,therebydemocratizingtheirusage.Theycanenablecollaborationandbuildcommunities,sharingdataandanalysesamongmultidisciplinaryandgeographicallydispersedresearchgroups,leadingtoincreasedopenness.Theycanenableresearchtobeundertakenmoreefficientlythroughtheprovisionofmodellingandothersoftwareandhardwareresourcesthroughasingleportal,andenableresearchtobeundertakenthatwouldnototherwisebeconducted.Researchersnolongerneedtobephysicallyco-locatedbecauseresourcescanbegloballydistributed,andthisalsoenablesinclusionoflessadvantagedresearchers/institutions.Bysharingresourcesacrossmultipleinstitutions,thecostsofsettingupandsupportingresearchinfrastructureislowered,aseachinstitutionisnolongerrequiredtosupportareplicaofdata,computeandtoolsattheirsite.Forgatewaysthatareopensource,theirverybuildingandevolutioncanbedemocratized.Anycommunitymembercandownloadandusecodeandalsocontributefeatures,forexample,viagitpullrequests.
Nectarprovidesthemajorityofitsfundingthroughtwomaininvestmentprogramsine-researchinfrastructure:
• VirtualLaboratories• ResearchCloud
5
VirtualLaboratories
NectarVLsarerichdomain-orientedonlineenvironmentsthatdrawtogetherresearchdata,models,analysistoolsandworkflowstosupportcollaborativeresearchacrossinstitutionalanddisciplineboundaries.TheyarebuiltandledbytheAustralianresearchsectorandareusednationallyandinternationallybytheresearchcommunityandotherstakeholders,includingindustry.
Dataanalysisisincreasinglyanessentialaspectofresearchinmostacademicdisciplines,withlargeamountsofdatarequiringmorestorageandcomputingpowerthanmostdesktopenvironmentscanprovide.VirtualLaboratoriescanprovideHighPerformanceComputing(HPC)resourcestoanalyseandstoresuchdataandprovidetraininginadvancedanalyticalmethodsneededtoanalyselargedatasets.
NectarVLsarehostedonremoteserversandareaccessibleremotely,viatheinternet.Researchersnolongerneedtobephysicallyco-located:anyresources,includingpeople,canbegloballydistributed.Allthatisneededisaninternetconnectionforaccesstocollaborators,data,computationalandanalyticaltools.
Remoteaccesstoresearchresourcesincreasestheefficiencyofresearchandexpandsitsimpact.ThecostofHPCinfrastructurecanbeshared–itnolongerneedsnotbereplicatedineveryresearchfacilitythatrequiresit.ThesavingsincludecostlyupgradesthatareneededeveryfewyearstomaintainHPC.TheVLsplatformiswellsuitedfordeliveringtraininginadvancedanalyticalmethodstoupdateanalyticalskillsandtotrainanewgenerationofresearchers.Furthermore,VLsprovideaplatformforcollaboration,enablingresearcherstoshareideasandresearchoutcomeswithcolleaguesinAustraliaandaroundtheworld,acrossinstitutionalanddisciplineboundaries,ultimatelyexpandingtoourcollectiveknowledge.
OnceNectarwasestablisheditinitiatedtwoRequestsforProposalsforVLsinSeptember2010andApril2012.ContractswiththeinstitutionsthatproposedthesuccessfulVLsweresignedfromMay2012toJanuary2013.
VirtualLaboratoriesfundedunderNectarrecordedover19,000usersin2017across12VLs,including,onaverage,usersfromover20internationalorganizationsand30Australianorganizations(Barkeretal2017).
AcompletelistofVLsisgiveninAppendix1.
Sincetheirinception,accesstotheVLsandotherservicesfundedbyNectarhasbeenfreetousers.
ThefourVLsconsideredinthisstudyhavereceivedfundingfromNectarasshowninTable2.1.
6
Table2.1 FundingforfourVLsto2016-17,$
BCCVL CVL GVL HUNINectarcash to2013-14 1,236,063 1,618,108 2,334,121 1,329,0002014-15 220,000 320,000 445,000 193,0002015-16 230,000 225,000 230,000 192,5002016-17 177,000 177,000 177,000 120,000Total 1,863,063 2,340,108 3,186,121 1,834,500Partners Cash 600,000 In-kind 1,315,971 1,836,740 2,917,707 2,500,000 Total 3,779,034 4,176,848 6,103,828 4,334,500NectarCloud
NectarCloudprovidescomputinginfrastructure,softwareandservicesthatallowAustralia’sresearchcommunitytostore,access,andrundata,remotely,rapidlyandautonomously.NectarCloud’sself-servicestructureallowsuserstoaccesstheirowndataatanytimeandcollaboratewithothersfromtheirdesktopinafastandefficientway.
TheNectarResearchCloudisasinglenationalcloudcomputinginfrastructure,comprisedofsevencollaborating“nodes”.Nodesprocurehardwarenecessaryforrunningthecloudcompute,storageandnetworkservices.
Nectarisaworldleaderindeployinghighlyinnovativecloudcomputingtechnologyforthebenefitofresearch–providingopportunitiesforfederationwithemerginginternationalresearchclouds.TheNectarResearchCloudsupportstheNCRISmissiontodeliverworldclassresearchfacilitiessothatAustralianresearcherscansolvecomplexproblemsinAustraliaandinternationally.
7
3.Methodology
3.1Approachestoestimatingvalueandimpact
TheassessmentofprogramssuchasNectarVLs,whichprovidefreeaccesstoarangeofweb-basedinformationtechnologydatabases,analysissoftware,trainingandotherservices,isarelativelynewfieldandinvestigatorshaveusedavarietyofapproachestoestimatethevalueandimpactsoftheseprograms.
InarecentstudyofthevalueandimpactoftheEuropeanBioinformaticsInstitute(EBI),acentreforresearchandservicesinbioinformatics,whichispartofEuropeanMolecularBiologyLaboratory(EMBL).,BeagrieandHoughton(2016)reviewtheapproachestakenbystudiesofsciencefacilities.Theynotethatmostoftheseareoflargescaleinfrastructurefacilitiessuchassynchrotrons,andonlyafewofdatarepositoriesandrelatedinfrastructureandservices.TheyquoteareviewofstudiesbythegroupEvaRIO(EvaluationofResearchInfrastructuresinOpeninnovationandresearchsystems)(2013)whichfoundthatcost-benefitanalysisandtechniquesforestimatingthereturnonR&Dexpenditurewererelativelyrareintheestimationofthevalueandimpactofsciencefacilities.
BeagrieandHoughton(2016)foundthatthreemaintypesofanalysiswereusedinthesestudies:variousformsofinput-outputanalysis,casestudiesandexamples,andformsofcost-benefitanalysisusingactivitycostingand/orcontingentvaluationasthebasisoftheanalysis.
Input-outputanalysisisbestsuitedtoanalysisofsinglesitefacilitiessuchasasynchrotronwhilecasesstudiesarelimitedintheextenttowhichtheresultscanbescaleduporgeneralised.
BeagrieandHoughtonthereforeuseamixedmethodapproachtocost-benefitanalysisdrawingupontheirexperiencevaluingarangeofdataservicessuchastheNationalCrystallographyServiceatSouthamptonUniversityandtheUKDataArchiveattheUniversityofEssex(Beagrieetal.2010),theUKEconomicandSocialDataService(Beagrie,Houghton,Palaiologk,andWilliams2012),theUKArchaeologyDataServices(BeagrieandHoughton2013a),andtheBritishAtmosphericDataCentre(BeagrieandHoughton2013b).
Again,inmanyrespects,theservicesanalysedinthesestudiesaresimilartoNectarVLs.
InstudyingtheEBIandtheseotherservices,BeagrieandHoughton(2016)notethat,becauseoftheiropenaccessnature,theydonotrecordmuchinformationabouttheusersoftheservices,beyondwhatisavailablefromIPaddressesorwebdownloadswhenusersaccesstheseservices.Theynotefurtherthatmorecomprehensiveandreliabledatawouldimproveeconomicanalysisoftheseservices.
8
FollowingtheseauthorswehaveapproachedthetaskofmeasuringthevalueandimpactofNectarVLsinanumberofways.
TheseareshowninFigure3.1.Aswithmostlargesciencefacilities,theVLsprovidesvaluebothtotheuserandtothewidersociety.TheVLsprovideaservicetoacommunityofresearcherswhichcanbevaluedinaseveralways.Thesearediscussedinmoredetailbelowbutoneisgivenbythevalueofthetimespentusingtheserviceandanother,acontingentvaluation,isanestimateofitsmarketvaluei.e.thevalueauserwouldplaceontheserviceifitwasofferedinthemarketplace.InmeasuringthecontributiontoGDP,serviceswithnomarketvalue,areoftensimplyincludedatcost.Howeverevenwherethereisnomonetarytransactionitispossibletoestimatea‘market’valueofaservicetoaconsumer.Thismarketvalueis,theamounttheconsumerwouldeitherbewillingtopay(WTP)touseitoraccept(WTA)nottouseit.
Figure3.1 Impactvaluationmethodologies
Source:Beagrie,N.&Houghton,J.W.(2016)TheValueandImpactoftheEuropeanBioinformaticsInstitute(www.beagrie.com/publications).
ThiswillingnesstopayoracceptderivesfromtheefficiencygainseitherfromusingtheVL,orforgonefromnotusingit,respectively.TheseefficiencygainscanbeseparatelyestimatedfromdatacollectedfromVLusersandprovidesacredibilitycheckontheWTPandWTAestimates.
Beyondthesesourcesofvalue,liethereturnstoR&DwhicharederivedfromthevalueoftheresearchoutputsoftheusersoftheVLs.Thesearemoredifficulttoestimatebutincludesignificantsocialreturnsarisingfromspillovers,andotherbroadercommunitybenefits,
9
fromnewinnovativeproductssourcedinpartatleastfromthebenefitsgainedfromusingtheVLs.
1.Investmentandusevalue
Theinvestmentvalueofaproductistheamountofresourcesspentonitsproductionanddelivery.Theusevalueontheotherhandistheamountofresourcesspentbyusersonobtainingandusingtheproduct.MeasuresoftheamountoftimeandmoneyusedbyNectarVLusersinaccessingandusingtheVLservicesisanindicatoroftheminimumamountthattheserviceisworthtothem.
Informationabouttheresourcesinvolvedinthediscovery,access,anduseofNectarVLservicescanbeobtainedbyusersurveysandinterviews.
2.Contingentvaluation
Contingentvaluationinvolvestheassignmentofmonetaryvaluestonon-marketgoodsandservicesbasedonpreferences(i.e.,PreferenceTheory).Itisoftenusedinthevaluationofenvironmentalandotherassets,theservicesofwhich,incontrasttogoodsandservicestradedinthemarketplace,areprovidedfreeoratnominalpricestousers.
TheusualmeasureofeconomicactivityisGrossDomesticProduct(GDP).Thisprovidesanaggregatevalueofgoodsandservicestradedinthemarketplaceincurrencyunits.Thevalueofthesetransactionsislargelyestablishedthroughapricesettledbydemandandsupplyoutcomes.However,therearewidelyacknowledgedshortcomingswithGDPasameasureofeconomicvalue(StiglitzCommission2008).Oneismeasuringthevalueofservices.
Services,particularlythoseprovidedbythepublicsector,areoftenincludedatcostbecausethereisnodirectdemandandsupplyinteractionbetweenbuyersandsellers.Ineconomicterms,thereisnomechanismwherebytheconsumerisabletorevealherpreferencesofhowmuchshewouldbewillingtobuyataparticularprice.ThisoftenmeansthatservicesareundervaluedinestimatingGDP,becausetheuserswouldbewillingtopaysignificantlymorethanthecostofproductioniftheyweregiventhatopportunity.ThisislikelytobethecasefortheNectarvirtuallabs.
ThevalueoftheservicesprovidedbytheVLswillbeincludedinGDPlargelybasedonthesalarycostsofoperatingthelabs,wheninfact;thevalueplacedontheservicesbyusersmaybesignificantlygreater.Thecontingentvaluationmethodologyprovidesonewayofestimatingthisvalue.
Ifagoodorservicecontributestohumanwelfare,ithaseconomicvalue,andwhethersomethingcontributestoanindividual’swelfareisdeterminedbywhetherornotitsatisfiesthatindividual’spreferences.Preferencesarerevealedbywhatanindividualiswillingtopayforagoodorserviceand/orbytheamountoftimeandotherresourcesspentobtainingthe
10
preferredgoodorservice.Wherepreferencesarenotrevealedinthemarket,individualscanbeaskedwhattheywouldbewillingtopayfor,ortoacceptinreturnforbeingwithout,thegoodorserviceinahypotheticalmarketsituation(i.e.,statedpreference).Forapublicgood,thevalueisthesumof“willingnesses”,asconsumptionisnon-rivalrous(i.e.,thesameinformationcanbeconsumedmanytimes).Thekeydifferencebetweenwillingnesstopayandwillingnesstoacceptisthattheformerisconstrainedbyabilitytopay(typicallybydisposableincome),whereasthelatterisnot.Hence,willingnesstopaydirectlymeasuresthedemandcurvewithabudgetaryconstraint,whilewillingnesstoacceptmeasuresthedemandcurvewithoutabudgetaryconstraint(BritishLibrary2004).
Thecontingentvaluationapproachinvolvesaskingwhatpeoplewouldbewillingtopay(toreceiveabenefitorremoveacost)oraccept(toforgoabenefitorincompensationforacost)ifamarketexistedfortheexternaleffect.(CommonwealthofAustralia2006)
3.Efficiencyimpacts
Widerbenefitsandimpactscanbeexploredbylookingattheefficiencygainsenjoyedbyusersandassigninganeconomicvaluetothem,suchasthevalueoftimesavings(productivity),andtheavoidanceofcostsforusersthatwouldotherwisebeinvolvedinthecreation/collectionofthedataforthemselvesorobtainingitelsewhere.Forthiswecombineusersurveyquestionsaboutperceivedefficiencyimpactswithactivitycosting.
4.Returnoninvestment
TherehavebeenanumberofstudiesbothinAustraliaandoverseasthathaveattemptedtomeasurethereturnstobothpublicandprivateinvestmentinresearchanddevelopment,andthecontributionthatR&Dandtechnologydevelopmentmorebroadlyhavemadetoeconomicgrowth.
TheProductivityCommissionhasundertakenanumberoflargestudiesontheroleofR&DinAustralia(e.g.IndustryCommission1995,ShanksandZheng2006).
AnanalysisofthereturnstohealthR&DbyAccessEconomics(2008)foundthathealthR&DprovidesreturnstoAustraliaof117%perannum,withabenefit-costratioof2.17.Inanextensionofthisstudyin2014.DeloitteAccessEconomics(2014)foundthateverydollarinvestedbytheMedicalResearchFutureFundwouldgeneratereturnsof$3.39infuturehealthandproductivitygains.
InexaminingtheseandotherAustralianandinternationalstudies,BeagrieandHoughton(2016)concludedthatthereturnstoresearchanddevelopmentweretypicallyintherangeof20%to60%perannum.IntheiranalysisbasedonamethodologydevelopedbyHoughtonandSheehan(2009)andHoughtonetal(2009)basedonamodifiedSolow-Swanmodelofeconomicgrowth,theyusedaconservativevalueof40%.
11
Theseandotherstudiesofhaveconsistentlyfoundlargereturnstoinvestmentinbothpublicandprivateresearchanddevelopment.
BalancedValueImpactModel
Inourdiscussionswiththe4VLsthatarethefocusofthisstudybothHuNIandtheGVLdrewourattentiontotheBalancedValueImpactModelwhichisanevaluationschemadevelopedbyProfessorSimonTannerofKing’sCollegeLondon(Tanner2012).Thispublicationisamanualforuserstoundertaketheseevaluationsandthemethodologyiscomplex.Fromabriefreviewoftheliteratureitdoesnotappeartohavebeenimplementedfullyinanyevaluationtodate.
WediscusstheBalancedValueImpactModelinmoredetailinSection7below.
3.2Surveydata
MostoftheapproachesoutlinedabovedependonhowusersuseVLservicesandthevaluethatcanbeputonthisuse.
AmajorlimitationexperiencedwithstudiesofthevalueandimpactofresearchinfrastructuresuchasNectarVLsisthepaucityofdatarelevanttotheapproachesoutlinedintheprevioussection.WhencommentingonthesestudiesBeagrieandHoughton(2016)notedthevariabilityofresearchinfrastructuremetricsandtheimplicationsforeconomicanalysisandquotedfromtheirreview(BeagrieandHoughton2014)asfollows
Itisalsoclearfromthesestudiesthatdifferentdatacentrescollectfinancialandoperationaldata,suchasuserstatistics,datadeposit,accessanddownloadstatistics,tovaryinglevelsofdetailandusingdifferentdefinitions.Moreguidanceisneededonthecollectionofsuchdata.Doingsowouldhelptoensureagreaterdegreeofstandardisationofoperationalrecordsacrossdatacentres.Thiswouldbeofgreatestbenefittofundersinvestinginarangeofdatacentres,andwouldprovidemorecomprehensiveandreliabledataforeconomicanalysis.Therewouldbeconsiderableadvantagetoprovidingguidanceregardingthecollectionofsuchdataasitisfundamentaltotheeconomicanalysisandinmakingthebusinessorfundingcase.
Funderswillneedtoensureallowanceismadeinbudgetstoenablesuchcentrestocollectadequateinformationsothatproperevaluationscanbeundertaken.
ToaddressthepotentiallimitationsofusagedatacollectedbyNectarVLs,andtoprovidedatarelevanttotheapproachesoutlinedabove,anonlinesurveyquestionnairewasdevelopedtomeasureaspectsoftheuserexperiencewithVLsusingtheQualtricson-linesoftwaretool.(https://www.qualtrics.com/homepage/)
ThestartingpointforthequestionnairewasthatusedbyBeagrieandHoughton(2016)butmodifiedtoaddressthesituationofeachVL.Draftinitialsurveyquestionnaireswere
12
developedbyVISESanddiscussedwithkeyNectarandVLpersonnel.Theywerethenrefinediterativelyoveranumberofdiscussioncyclesandpilottested.
Thequestionnaireusedarangeofstandardsurveyapproaches,includingtheuseof“criticalinstances”,suchasthelastdataaccessed/downloaded(forusers).Anumberofquestionssoughtspecificinformationon:thetimeandcostofaccessforusers;thebenefitsandefficiencyimpactsofaccess;andcontingentvaluation(i.e.,willingnesstopayoraccept)usingstatedpreferencetechniques.Answerstothesequestionswereinterpretedcarefully,inthecontextofopen-endedtextcommentsinthesurvey.ThesequantitativequestionsweresupplementedbyqualitativequestionsaskingforviewsontheimportanceandimpactoftheVLforusers,toensurethatthequantitativeandqualitativefindingswereinaccord.
ThesurveyswereprogressivelyimplementedfromApriltoMay2017.
13
4.TheVirtualLaboratories
4.1BiodiversityandClimateChangeVirtualLaboratory(BCCVL)
TheBiodiversityandClimateChangeVirtualLaboratorywasformedunderanagreementinNovember2012betweenNectarandGriffithUniversity.TheothermainpartnersinBCCVLare
• JamesCookUniversity• UniversityofNewSouthWales• MacquarieUniversity• UniversityofCanberra• AtlasofLivingAustralia(ALA),ane-infrastructurefundedbyNCRIS• TerrestrialEcosystemResearchNetwork(TERN),anationalobservatoryfor
Australianecosystems,deliveringdatastreamsthatenableenvironmentalresearchandmanagement,withfundingfromNCRIS
• QueenslandCyberInfrastructureFoundation(QCIF)aHPCconsortiumofQueenslanduniversities
Undertheagreement,Nectarhasprovidedfundingof$1,863,063to2016-17andpartnershavecommittedto$600,000incashand$1,315,971in-kind.TheBCCVLwebsitewaslaunchedinAugust2014.
TheBCCVLprojectteamof8peopleislocatedatGriffithUniversityandheadedbyMrHamishHolewa.IthasaSteeringCommitteeof17peopledrawnfromparticipatinginstitutionsandNectar,aswellasanEcologicalModellingScientificAdvisoryGroupof8people.
TheBiodiversityandClimateChangeVirtualLaboratory(BCCVL)isa“onestopmodellingshop”thatsimplifiestheprocessofbiodiversity-climatechangemodelling.ItsmissionistoconnecttheresearchcommunitytoAustralia’snationalcomputationinfrastructurebyintegratingasuiteoftoolsinacoherentonlineenvironmentwhereresearcherscanaccessdataandperformdataanalysisandmodelling.
Previously,thelinesofinquiryintobiodiversityandclimatechangeimpactswerestymiedduetoresearchers’inabilitytoaccessastandardisedsetoftoolsforanalysisandrequisitedatasources,andcomputationallimitations.
ThegoaloftheBCCVListointegratethesetoolsanddatasetswithhigh-performancecomputersandmajordatastoragefacilities,therebyenablingmoreefficientinvestigationofbiologicalsystemsandfacilitatingthedevelopmentofadditionalresearchtrajectoriescurrentlynotpossibleduetocomputationallimitations.
14
CentraltothemodellingfacilitatedbyBCCVLisspeciesdistributionmodelling,alternativelyknownasenvironmentalnichemodelling,(ecological)nichemodelling,predictivehabitatdistributionmodelling,orclimateenvelopemodelling.Itreferstotheprocessofusingcomputeralgorithmstopredictthedistributionofspeciesingeographicspaceonthebasisofamathematicalrepresentationoftheirknowndistributioninenvironmentalspace(=realisedecologicalniche).Theenvironmentisinmostcasesrepresentedbyclimatedata(suchastemperature,andprecipitation),butothervariablessuchassoiltype,waterdepth,andlandcovercanalsobeused.Thesemodelsallowforinterpolatingbetweenalimitednumberofspeciesoccurrenceandtheyareusedinseveralresearchareasinconservationbiology,ecologyandevolution.
Developingaspeciesdistributionmodelbeginswithobservationsofspeciesoccurrences:theseareplaceswhereweknowaspecieshasbeenfound.Theseoccurrencesaremostlypoint-basedandcomefromsourcessuchasmuseumrecordsandobservationsofexpertsinthefield.BCCVLuserscanuploadtheirownspeciesdistributiondataoraccessoneormoreofthemanydatasetsprovidedbyBCCVL.
Tocalibrateacorrelativespeciesdistributionmodeltwotypesofinputdataareneeded:speciesoccurrences,andmeasurementsofasuiteofenvironmentalvariables,suchastemperatureandrainfall.Thesetwotypesofdataarethenputintoanalgorithmtofindassociationsbetweentheknownoccurrencesofaspeciesandtheenvironmentalconditionsatthosesites,toidentifytheenvironmentalconditionsthataresuitableforaspeciestosurvive.Thisprovidesinformationaboutwherespeciesoccurandsomethingabouttheenvironmentalconditionsofthoseplaces.Thealgorithmusesthesetwotypesofinformationtoestimatetheprobabilityofaspeciesoccurringinaplaceassomefunctionoftheenvironmentalconditionsofthatplace.
BCCVLcurrentlysupports17differenttypesofspeciesdistributionalgorithms.
BCCVLalsooffersarangeofclimatechangemodelsthatcanbeusedinconjunctionwithspeciesdistributionmodellingtoestimatetheimpactofclimatechangeonspeciesdistribution.
TheBCCVLwebsiteoffersaKnowledgeBasewhichcanguideusersthroughthemodellingprocessbyprovidinginformationaboutthestepsinvolvedinmodellingandthedatabasesandotherresourcesavailable.
Thewebsitealsooffersatrainingcourseof10modulesexplainingSDM,thechoiceofalgorithmsandhowtointerpretmodellingresults.
BCCVLoffersworkshopsaimedatacademicsandindustryprofessionals,forinstanceHDRstudents,environmental/climatescientistsandresearchers,ecologists,decision-makers,membersofgovernmentandindustrygroups.Italsoprovidesworkshopsforundergraduatestudentsstudyinginareassuchasecology,environmentalscience,sustainability,climate
15
changeimpacts,biology,floraandfauna,animalbehaviour,planninganddevelopment,andconservation.
4.2CharacterisationVirtualLaboratory(CVL)
TheCharacterisationVirtualLaboratory(CVL)wasformedunderanagreementinMay2012betweenNectarandMonashUniversity.FoundingpartnersinCVLare
• AustralianMicroscopyandMicroanalysisResearchFacility• AustralianNuclearScienceandTechnologyOrganisation• AustralianSynchrotron• NationalImagingFacility• AustralianNationalUniversity• UniversityofSydney• UniversityofQueensland
Throughitslife,anumberofprojectpartnershavejoinedtheCVLandengageddirectlythroughprojectfunding:
• CSIRO• DeakinUniversity• Intersect,anditspartners• QCIF,anditspartners• RMIT• ThePawseySupercomputingCentre• TheTerrestrialEcosystemResearchNetwork• UniversityofMelbourne• UniversityofNewSouthWales• UniversityofWesternAustralia• VicNode,anditspartners
Undertheagreement,Nectarhasprovidedfundingof$2,340,108to2016-17andpartnershavecommittedto$1,836,740in-kind.
TheCVLwebsitewaslaunchedinMarch2013.
ThepredominantpractiseoftheCVLprojectistoworkwithinstrumentfacilitiestoprovidedatacapture,analysisandvisualisationservicesandtherebyunderpinthefacilityusercommunity.
CVLhasworkedwithorisworkingwith26facilitiestointegrateover100instrumentswithatotalvalueofaround$250million..AcompletelistofthesefacilitiesisgiveninAppendix2.
16
TheCharacterisationVirtualLaboratory(CVL)aimstointegrateAustralia’simagingequipmentwithspecialisedHighPerformanceComputingcapabilitiesandwithdatacollectionnodesandprovidescientistswithacommonenvironmentforanalysisandcollaboration.Itinitiallydevelopedfourresearchapplication(‘drivers’)inmulti-modalorlarge-scaleimaginginneuroscience,structuralbiology,atomprobeandX-rayscience,andthroughpartnership,thishasbeenextendedtocytometry,neutron-beamimaging,lightmicroscopy,andbioinformatics.
Eachdriverisledbyaworld-classresearchgroup,issupportedbyanAustralianresearchconsortiumandisinanationalresearchpriorityarea.TheresultsfromthisdevelopmentaredistributedtothecommunitythroughCVL“Workbenches”.
CVLusershavetheoptionofusingthecomputingresourcesontheNectarresearchcloudoraccessingMASSIVE(Multi-modalAustralianScienceSImagingandVisualisationEnvironment)theHPCfacilityjointventurebetweenMonashUniversity,CSIROandtheAustralianSynchrotron
AnimportantroleforCVLhasbeentoestablishthecapabilitywithinimagingsitesforautomaticcaptureandstorageofimagingdatatothecloud.ThisenablesuserstobeabletoaccesstheirimagingdatathroughtheCVLwebsiteandtoperformanalysisusingthesoftwaretoolsprovidedbyCVLbasedaroundtheMyTardisandStore.Monashtools(Ceguerraetal2013).
CVLhasdevelopedanumberofapplicationsthatarereusedacrossarangeofAustralianfacilities.Theseinclude:extensionstotheMyTardisdatamanagementplatformtosupportinstrumentfacilities;aninstrumentintegrationapp,calledMyDatatomakeintegrationquicker,simplerandlessreliantonspecialistITsupport;andStrudel,atoolthatmakesaccessingremoteanalysisenvironmentseasier.
4.3GenomicsVirtualLaboratory(GVL)
TheGenomicsVirtualLaboratorywasformedunderanagreementinMay2012betweenNectarandtheUniversityofQueensland.TheothermainpartnerinGVListheUniversityofMelbourne.ThefollowingorganisationsarepartnersorcollaboratorswithGVL
• TheUniversityofQueensland• QFABBioinformatics• QueenslandCyberInfrastructureFoundation• MelbourneBioinformatics(formerlyVictorianLifeSciencesComputationInitiative)
VictorianeResearchStrategicInitiative• TheUniversityofMelbourne• BakerIDIHeartandDiabetesInstitute• PeterMacCallumCancerCentre
17
• TheGarvanInstituteofMedicalResearch• VictorChangCardiacResearchInstitute• CSIRO• BioplatformsAustralia• MonashUniversity• TheUniversityofSydney• TheUniversityofWesternAustralia• AustralianBioinformaticsNetwork• EMBLAustralia• AustralianGenomeResearchFacility• AustralianNationalDataService
Undertheagreement,Nectarhasprovidedfundingof$3,186,121to2016-17andpartnershavecommittedto$2,917,707in-kind.
TheGVLisadministeredbyMelbourneBioinformatics.Ithasastaffof8ledbyAssociateProfAndrewLonie,asDirectorofMelbourneBioinformatics.
TheGenomicsVirtualLaboratoryprovidesacloud-basedsuiteofgenomicsanalysistoolsforlifescienceresearchandtraining.
Biologistswithoutcomputersciencetrainingcangostraighttoauser-friendlyplatformwhichhostsasuiteoftestedbioinformaticstoolsandpipelinesforfast,consistent,dataanalysis.Theplatformisconstantlyupdatedtohavethelatestfeaturesinusebyexpertbioinformaticians.Adoptedbothlocallyandoverseas,theGVLhasalreadybeenrecognisedasaqualityplatformtohelpaddresstheshortageofbioinformaticsexpertisearoundtheworldandmanagethecomplex,multiple-layereddataanalysistasksconfrontinglifescientiststoday.
Nationallyandinternationallyitisbeingusedbothbylifescientistsworkingwithgenomicdataandacademicsteachingbioinformaticsatundergraduateandpost-graduatelevels.
LifescientistswithoutaccesstobioinformaticsexpertisearetheprimaryusersoftheGVL.Practisingbioinformaticianswhoknowhowtofindtherighttoolforthedataanalysisjob,findthattheGVLisworkingfortheminotherways.SmallbioinformaticsgroupsorlonepractitionersusetheGVLtotraintheirlocalteamstodotheirownsimplebioinformaticstasksontheGVL,freeingupcapacitytoworkonmorecomplexresearchproblemsortocollaboratemorebroadly.TheGVLwebsiteisbuiltaroundGalaxy,anopen,web-basedplatformfordataintensivebiomedicalresearch.GalaxyisaninternationalcollaborativeprojectmanagedbytheCenterforComparativeGenomicsandBioinformaticsatPennStateUniversity,andtheDepartmentofBiologyatJohnsHopkinsUniversity.
18
UsersofGVLhavetheoptionoflaunchingtheirownprivateserveronapublicorprivatecloud,withtheGVLpre-installed,oraccessingitthroughoneofthepublicGVLservicesontheGVLwebsitemanagedbyGVL.MostuserswithoutrelevantITskillsuseoneofthreemainpublicservicesnamely
• GalaxyMelbourne• GalaxyQueensland• GalaxyTut
GalaxyTutisusedfortrainingpurposes.Aswell,theGVLprovideson-linebasicandadvancedGalaxytutorialsandcommandlinetutorialswhichgivestepbystepinstructionsforanexampleanalysis.Protocolsoutlinetheanalysismethodsandsuggestandcomparetoolsforeachstepratherthanprescribethem.
GVLuserstypicallyloadgenomicsequencesthatarethenmanipulatedandanalysedusingacomprehensivesuiteoftools.
AsidefromthepublicservicesmanagedbytheGVL,anumberofotherinstitutionssuchasCSIROandLaTrobeUniversityhavedownloadedtheGVLtotheirserverstolocallymanageaccessfortheirinstitutionalusers.
4.4HumanitiesNetworkedInfrastructureVirtualLaboratory(HuNI)
TheHumanitiesNetworkedInfrastructureVirtualLaboratory(HuNI)wasformedunderanagreementwithDeakinUniversityinMay2012.Nectarhascontributedfundingof$1,834,500to2016-17andpartnershavecommittedto$2,500,000in-kind.
ThepartnersinHuNIhaveincluded
• AustralianInstituteofAboriginalandTorresStraitIslanderStudies(AIATSIS)• AustralianNationalUniversity• DeakinUniversity• FlindersUniversity• IntersectAustralia• MacquarieUniversity• RMITUniversity• UniversityofMelbourne• UniversityofNewSouthWales• UniversityofQueensland• UniversityofSydney• UniversityofWesternAustralia• V3Alliance• NationalLibraryofAustralia
19
• AustralianCentrefortheMovingImage(ACMI)
TheProjectDirectorofHuNIisProfessorDebVerhoevenatDeakinUniversity.
GoogleAnalyticsdataprovidedbyHuNIindicatesaround4,600usersofHuNIduring2016.
HuNI(HumanitiesNetworkedInfrastructure)combinesdatafrommanyAustralianculturalwebsitesintoalargehumanitiesandcreativeartsdatabase.HuNIdatacoversalldisciplinesandbringstogetherinformationaboutthepeople,works,events,organisationsandplacesthatmakeupthecountry'srichculturallandscape.
HuNIcombinesinformationfrom32ofAustralia’smostsignificantculturaldatasets.Thesedatasetscomprisemorethan17millionauthoritativerecordsrelatingtothepeople,organisations,objectsandeventsthatmakeupAustralia'srichculturalheritage.HuNIalsoenablesresearcherstoworkwithandsharethislarge-scaleaggregationofculturalinformation.HuNIhasbeendevelopedasapartnershipbetween16publicinstitutions,ledbyDeakinUniversity.
AccesstotheHuNIwebsitewasfirstavailableinJuly2012andwasofficiallylaunchedinOctober2014.
AsVerhoevenandBurrows2015suggeststhisplacesHuNI
‘somewherebetweena“datawarehouse”inwhichtheincomingdataarefirstcleanedandorganisedintoaconsistentschemaanda“datalake”inwhichtheincomingdataareingestedintheirrawformandtheresponsibilityormakingsenseofthedataliesentirelywiththeenduser’p418
Theconstituentdatabases,havebeenreconfiguredbyHuNIsothattherecordsaremappedtosixcoreentities:Person,Organization,Event,Work,Place,andConcept.(VerhoevenandBurrows2015).
Theuserbeginstheirsearchforanitemofinterestinoneoftheseentities.Thatcouldbeanameofapersonoraparticularorganisationoraconcept(topic)ofinterest.ThepowerofHuNIistoretainasdatawithinHuNItheconnectionsdiscoveredbetweenitemsintheentities,suchasaconnectionlinkingapersontoothersand/oranorganisationandaconceptetc.madebytheuser.Thesearesavedbytheuserandmaybeplacedinthepublicdomainandthusmadeavailableforotherusers.Subsequentusersmaydisputethelinksmadeorsupplementthelinks,creatingarichersetofnetworksconnectedtooneormoreoftheentities.
Anexampleofsuchanetwork(aHuNIKnowledgeGraph)isshowninFigure4.1.
Thisgraphshowslinksrelatedtothecollection:AustralianFashion1850-1950,includingrelatedpersonsandworks.InparticularitlinkstheAustralianFashionCollectionviaa1938
20
filmwithafashiontheme,DadandDaveCometoTowntoacollectiongeneratedbyadifferentHuNIuserinvestigatingworksdirectedbyKenHallandproducedbyCinesoundProductions.TherearefurtherlinksmadetootherfilmsdirectedbyKenHall.BytracingthepathwayofthesevisuallinksaHuNIuserisabletoexpandtheirdiscoveryopportunitiesthroughtheserendipitousconnectionsprovidedbyamultitudeofHuNIusers.
Figure4.1 AnexampleofaHuNIKnowledgeGraph
SourceVerhoeven2016,p23
Asthefigurebelowillustrates,thisprocessconvertsisolateddataandinformationtoknowledgebycapturingtherelationshipsarisingfromtheexpertiseofHuNIusers.The
21
knowledgecreatedbytheseconnectionsisfurtherenrichedthroughthecollaborationofotherhumanitiesusersofthewebsite.
Figure4.2 ConnectionsbetweendataandinformationusingHuNI
SourceVerhoeven2016,p13
Thisfacilitationofanunstructuredapproachtoknowledgecreationwithinthedatabaseencouragesaserendipitousrelationshipsearchprocess.ThisisseenasamajoraddedvalueoftheVL,onethatisparticularlyapplicabletothehumanitiesstyleofresearchandonethatisnotcateredforbyconventionalarchivaldatabases.
‘HuNImovesbeyondthinkingofserendipityasonlyatechnicalproblemandinsteadtreatsitasamatterofsocial,philosophical,andpoliticalsignificance’(p23Verhoeven2016).
HuNIhasbeenestablishedwiththeideathatvaluableknowledgeandnewperspectivesareburiedintraditionalarchivaldatabases,whichcanbereleasedbythepartlyunanticipatedsearchprocessessupportedandusedbyHuNIusers(Verhoeven2016).PerhapsthiscreationofnewconnectionsisHuNI’shighestvalue.Itnotonlyfacilitatesthecreationofnewnetworksthroughlinkswithmultipledatabases,butalsohasthesoftwarethatallowstheconnectionsmadeinthesearchprocesstoberecorded,storedandmadeavailabletootherresearchers.OvertimethatarchiveoftheseHuNIKnowledgeGraphswilladdsignificantvaluetotheHuNIproject.
Atthisstagetheresourcesavailabletotheprojecthavebeenlargelydevotedtothedevelopmentofthesoftwarethatfacilitatesthecharacterisationoftheunderlyingarchivaldatabasesintermsofsearchableentities.Thefacilitytostoreuserdevelopedconnectionsisstillinitsinfancy.Anewsystemupgrade,includinganewuserinterface,isabouttobeinstalled,whichwillbetterfacilitatethecreation,storageandsearchingofvaluableHuNIKnowledgeGraphs.
22
HuNI’svalueliesintheincreasingparticipationofitsusers.ThemorelinksandcollectionsthataremadeinHuNI,themorevaluableatoolitbecomesforeveryone.ThismeansthatearlyonthebenefitversuseffortforHuNIusersisrelativelylow,thisrisesexponentiallyasmoreusersaddtheirexpertisetotheplatform.InvestmentinthepromotionofHuNIandbuildingofthecommunityisrequiredtoensureHuNImovespastthetippingpointofusers“gettingmoreoutofHuNIthantheyputin”.
23
5.DataonusageofNectarVirtualLaboratories
ThemethodologiesformeasuringthevalueandimpactofNectar’sVLsdependonbeingabletodescribetheextenttowhichtheVLshavebeenabletoattractusersoftheirservices.IntheevaluationswerelyondataprovidedbytheVLsonusageandontheresponsestotheon-linesurveysofusers.
AlthoughtheVLsdifferintheextentandtypeofdatatheycaptureaboutusageofVLservices,theyreadilyprovidedusagedatabasedontheregularcollectionstheymakeformonitoringpurposesandinresponsetofurtherrequestsfordifferentormoredetaileddata.Theirabilitytoprovidethisdatawaslimitedforsomebecauseitwasnotdatathatwasusuallycollectedinthecourseoftheiroperations.
ThetablesandfiguresbelowsummarisetheusageofVLservicesgenerallyfromwhentheybecameactivetothepresent.Forsome,veryearlydatawasnotavailableandthemostrecentdatawasforearly2017.
ItisclearthatthenumberofusersforaparticularVLincreasesrapidlyinthefirstfewyearsandhascontinuedtogrowoverthepastyear.
ThisstudyonthevalueandimpactofVLsisthereforeoccurringrelativelyearlyintheirlifetimes,presentingchallengesinassessingthelikelynumberofusersoverthenextfewyearsandthepotentialmaximumnumberofusers.ItisdifficulttoassesswhatpercentofthecurrentresearchbaserepresentsthemaximumachievablebyeachVLandwhenthiswilloccur.Itwouldbeexpectedthatoncethisoccurred,thefuturerateofgrowthwouldbelowerandjustreflectthenumbersofnewresearchersenteringtheresearchfieldsofrelevancetotheVL.
BCCVL
BCCVLprovidedmonthlyusagedatafromJune2015toMarch2017.FromthisdataweshowinTable5.1thenumbersofnewusersforeachcalendaryearandthecumulativenumberofusersatDecemberofeachyear.Tothisweaddestimatesofthenewusersandcumulativeuserscalculatedasfollows.
Overthe11monthstoMarch2017,thenumberofusersincreasedby3.75timescomparedtotheprevious11months.Ifitisassumedthatthenumberofnewusersin2017willbe2.0timesthatin2016,andthevaluein2018willbe1.5times2017,andwiththesamenumberofnewusersin2019and2020asin2018,thenthenumberofnewusersandthecumulativenumberofusersfor2017to2020willbeasinTable5.1.
ThesegrowthassumptionssuggestthatthecumulativenumberofusersbyDecember2020willbe11,246.
24
Table5.1 BCCVLactivitymeasures
Newusers
Cumulativeat
December2015 254 2542016 916 1,1702017 1,832 3,0022018 2,748 5,7502019 2,748 8,4982020 2,748 11,246CVL
CVLconsultedwitharangeofimagingfacilitiespartnersandprovidedestimatesofthenumberofusers.ToAugust2017thisamountedto2558,soweassumethatthenumberatDecember2016was2400.Basedonaninitialcohortof101atDecember2012,weinterpolatedthenumberofnewusersfrom2012to2016andestimatedthatthiswouldgrowby10%intheyearsto2020.Thisresultedinanestimateofthecumulativetotalof6,485bytheendofDecember2020(Table5.2).
Table5.2 CVLactivitymeasures
Newusers
Cumulativeat
December2012 101 1012013 300 4012014 500 9012015 700 1,6012016 800 2,4012017 880 3,2812018 968 4,2492019 1,065 5,3142020 1,171 6,485GVL
GVLhasprovidedusagedataforthethreemajorpublicserviceportals.Ithasnotbeenpossibletoobtainandactivitydataforthoseorganisationsandindividualsthathavedownloadedthedataandlaunchedtheirownprivateserveronapublicorprivatecloud,withtheGVLpre-installed.
GVLprovidedmonthlyusagedataforGalaxy–QueenslandfortheperiodDecember2013toFebruary2017.ThedataforGalaxy–MelbournecoveredSeptember2015toFebruary2017andforGalaxy–TutitwasDecember2012toFebruary2017.
25
ThenumbersofnewusersandthecumulativenumbersofusersatDecemberforthethreeportalsisshowninTable5.3.InasimilarmannertothatusedforBCCVL,weestimatethenumberornewusersandcumulativeusersto2020butrecognisethatthe3portalshavebeenoperatingfordifferentlengthsoftime.
ForGalaxy–QueenslandandGalaxy–Tutweassumea10%growthinnewusersforeachoftheyears2017to2020.ForGalaxy–Melbournethenumberofnewusersinthe9monthstoFebruary2017was2.6timesthatoftheprevious9months.Weassumethatthenumberofnewusersin2017willbe2.5timesthatof2016,thenumbersin2018willbe2.0timesthatof2017,thenumbersin2019willbe1.5timesthatof2018andthenumbersin2020willbethesameas2019.TheestimatednewusersandcumulativeusersatDecembereachyearareshowninTable5.3.
Table5.3 GVLactivitymeasures
Galaxy-Queensland Galaxy-Melbourne Galaxy-Tut Newusers Cumulative
atDecember
Newusers Cumulativeat
December
Newusers Cumulativeat
December2012 40 402013 27 27 117 1572014 285 312 1,060 1,2172015 586 898 65 65 510 1,7272016 631 1,529 408 473 587 2,3142017 694 2,223 1,020 1,493 646 2,9602018 764 2,987 2,040 3,533 710 3,6702019 840 3,826 3,060 6,593 781 4,4512020 924 4,750 3,060 9,653 859 5,311
26
6.EstimatingthevalueandimpactofNectarVirtualLaboratories
MostofthecalculationsreportedinthissectionrelyonestimatesofthetimespentbyusersaccessingVLservicesandthesalariesoftheseusers.
6.1Costdata
ForacademicuserswehavecalculatedthehourlyratefordifferentacademicpositionsbyobtainingsalarydatafromMonashUniversityandcalculatingthevaluefor2017.TheannualsalariesshowninTable6.1areaveragesofthelevelswithineachacademicgradeforthesepositions.WehavecheckedtheselevelswiththoseinanumberofotheruniversitiesandothersourcesofinformationandregardtheMonashUniversityvaluesasrepresentativeofsalarieswithinAustralianuniversities.Forcomparisonpurposes,thesalariesarearound5%higherthanthoseforVictoriaUniversity.Toobtainanhourlycostsequivalentwedividetheannualsalariesbytheaveragenumberofworkingdaysinayear(225)andbythenumberofhoursworkedperday(7.35).
FollowingBeagrieandHoughton(2016)weadd30%tothistoaccountfornon-wagelabourcosts.ThisisalsoinlinewithcurrentAustralianuniversityguidelines.TheindicativehourlycostscalculatedinthiswayareshowninTable6.1.
Forstudentandotherusersthestartingpointisthemedianstartingsalariesofbachelordegreegraduatesin2015publishedbyGraduateCareersAustraliaLtd.Theirmostrecentdataisfor2015.Toestimatethevaluesfor2017,weaddafurther4.4%,beingtheestimatedtwoyearincreaseinaverageweeklyearningsforfull-timeadults(ABS2017).
HourlycostsarecalculatedinthesamewayasforacademichourlycostsandtheseareshowninTable6.2.Fromthisweassumethatthehourlycostforstudentsisthehourlygraduatecostof$44.52
Table6.1 Academicsalaries,andhourlycosts
Level Salary,2017 Hourlysalary,2017
Hourlycosts,2017
$,000 $ $AcademicA 77,819 47.06 61.17AcademicB 102,469 61.96 80.55AcademicC 123,546 74.71 97.12AcademicD 145,211 87.81 114.15AcademicE 178,005 107.64 139.93
27
Table6.2 Annualgraduatesalariesandhourlycosts
Qualification Salary,2015 Salary,2017,est.
Hourlysalary,2017,est.
Hourlycosts,2017,est.
$,000 $,000 $ $AgriculturalScience 50.0 52.2 31.56 41.03BiologicalSciences 50.0 52.2 31.56 41.03ComputerScience 54.0 56.4 34.09 44.32EarthSciences 60.0 62.6 37.88 49.24Engineering 60.0 62.6 37.88 49.24Mathematics 60.0 62.6 37.88 49.24PhysicalSciences 50.0 52.2 31.56 41.03VeterinaryScience 50.0 52.2 31.56 41.03Average 54.3 56.6 34.25 44.52 TOTAL(all
qualifications)54.0 56.4 34.09 44.32
Source:Table1:Medianstartingsalariesofbachelordegreegraduatesinfirstfull-timeemploymentandagedlessthan25,byfieldofeducationandsectorofemployment,2015($,000,n)¤†,GraduateCareersAustraliaLtd,GraduateSalariesReport2015,GraduateCareersAustraliaLtd,2016athttp://www.graduatecareers.com.au/research/researchreports/graduatesalaries/
Intheon-linesurveyofusers,participantswereaskedabouttheirpositionwithintheirorganisation.Allocatingbothacademicandgraduateleveltothesepositionsandusingthedistributionofanswersfromrespondents,wecalculatedameanhourlycostofusersbasedonthehourlycostsinTables6.1and6.2.
ForBCCVL,CVLandGVLthehourlycostofuserswere$74.82(or$63,127annually),$66.39($56,015annually)and$71.02($59,925annually)respectively.
Investmentvalue
IntheirstudyofEBI,BeagrieandHoughtonwereunabletoestimatethetotalvalueofallthecomponentsofEBI’sinvestmentvalue,suchasdataacquisition,depositingdata,collaborationcostandaddingvaluetothedata.Theyreliedthereforeonanestimateoftheaverageannualoperatingexpenditureasaproxy.
ThesetupandoperatingcostsofeachVLtotheyear2016-17weredescribedinTable2.2inSectionearlier.
IfweassumethattheannualoperatingexpensesofeachVLisequaltotheaverageoftheyears2014-15to2016-17asshowninTable2.2andweapplythattoeachyearoutto2020,wecancalculatetheestimatedaccumulatedcostofeachVLto2020.Thisisshownintable6.3andistheestimatedinvestmentvalueofeachVL.
28
DividingthisbythenumberofyearsfromVLinceptionto2020,wecancalculatetheannualvalueoftheoverallinvestmentmadeineachVL.AscanbeseenfromTable6.3thisisintherange$550,000to$600,000forBCCVL,CVLandHuNIandissomewhathigheratabout$870,000forGVL.
Table6.3 EstimatedtotalandannualcostforVirtualLaboratories
BCCVL CVL GVL HUNITotalcoststo2016-17 3,779,034 4,176,848 6,103,828 4,334,500Averageannualoperatingcost 209,000 240,667 284,000 168,500Totalcoststo2020 4,406,034 4,898,848 6,955,828 4,840,000Annualvalueoftotalcoststo2020 550,754 612,356 869,479 605,000
Contingentvaluation
Thecontingentvalueofanon-marketgoodorserviceistheamountusersarewillingtopayforitand/orarewillingtoacceptinreturnforgivingitup.Forapublicgoodthevalueisthesumofwillingnesses,asconsumptionisnon-rivalrous(e.g.,thesameinformationcanbeconsumedmanytimes).Thekeydifferenceisthattheamountthatusersarewillingtoacceptinreturnforgivingupaccessistypicallyhigherthantheamounttheywouldbewillingtopay,primarilybecausethelatterisconstrainedbywhattheycanafford(e.g.,bydisposableincome,limitedresearchgrants,etc.).
BCCVL
ForBCCVL,themeanamountthatuserswerewillingtopayforaccesstoBCCVLserviceswas$1,154(medianvalue$250).OntheotherhandthemeanamountuserswouldbewillingtoaccepttoforegoBCCVLserviceswas$10,005(medianvalue$5,000).UsersnaturallyputahighervalueonBCCVLserviceswhentheyarenotsubjecttoanincomeconstraint.Asacomparison,BeagrieandHoughtonfoundthemeanwillingnesstopayforEBIwasaround$2,800(GBP1628).
Usingthemeanwillingnesstopayvalueof$1,154,thecontingentvaluationofBCCVLbyuserswouldthereforehavebeen$1,350,180intheperiodtoDecember2016,oranannualequivalentof$852,745,risingto$13.0millionbyDecember2020,or$2.3millionperyear.(Table6.4).TheannualequivalentforeachyeariscalculatedbydividingthecumulativevaluetothatpointbythenumberofmonthselapsedsinceinceptionoftheVLandmultiplyingby12.AsusageofBCCVLbeganinJune2015,theannualvalueestimatedfor2015ishigherthanthecumulativevalue.
Usingthehighervaluemeasuredbyusers’willingnesstoaccept,thecontingentvaluationofBCCVLwouldbe$112.5millionbyDecember2020or$20.1millionperyear.
29
Table6.4 BCCVLcontingentvaluation
Cumulativeusers
WTP WTA
Cumulative Annual Cumulative Annual2015 254 293,116 502,485 2,540,000 4,354,2862016 1,170 1,350,180 852,745 11,700,000 7,389,4742017 3,002 3,464,308 1,341,022 30,020,000 11,620,6452018 5,750 6,635,500 1,851,767 57,500,000 16,046,5122019 8,498 9,806,692 2,139,642 84,980,000 18,541,0912020 11,246 12,977,884 2,324,397 112,460,000 20,142,090CVL
ThemeanamountthatCVLuserswerewillingtopaywas$1,524(median$900),whiletheirmeanwillingnesstoacceptwas$14,130(median$20,000).
Usingthemeanvalueof$1,524andtheestimatesofCVLusersabove,suggeststhatbyDecember2020thecontingentvalueofCVLwouldbe$9.9millionoranaverageannualvalueof$1.2million.Usingthewillingnesstoacceptvaluethiswouldbe$91.6millionbyDecember2020,or$11.5millionannually(Table6.5).
Table6.5 CVLcontingentvaluation
Cumulativeusers
WTP WTA
Cumulative Annual Cumulative Annual2013 301 611,124 611,124 5,666,130 5,666,1302014 601 1,373,124 686,562 12,731,130 6,365,5652015 1,051 2,439,924 813,308 22,622,130 7,540,7102016 1,550 3,659,124 914,781 33,926,130 8,481,5342017 2,100 5,000,244 1,000,050 46,360,530 9,272,1062018 2,705 6,475,476 1,079,246 60,038,370 10,006,3952019 3,371 8,098,231 1,156,890 75,083,994 10,726,2862020 4,103 9,883,262 1,235,410 91,634,180 11,454,271GVL
ThemeanamountthatGalaxy-Melbourneuserswerewillingtopaywas$606(median$100),whiletheirmeanwillingnesstoacceptwas$1285(median$800).
Usingthemeanvalueof$606andtheestimatesofallGVLusersabove,suggeststhatbyDecember2020thecontingentvalueofGVLwouldbe$11.9millionoranaverageannualvalueof$1.9million.Usingthewillingnesstoacceptvaluethiswouldbe$25.3millionbyDecember2020,or$4.0millionannually(Table6.6).
30
Table6.6 GVLcontingentvaluation
Cumulativeusers
WTP WTA
Cumulative Annual Cumulative Annual2013 184 135,744 284,167 287,840 602,5662014 1,529 926,574 528,529 1,964,765 1,120,7262015 2,690 1,630,140 718,806 3,456,650 1,524,2002016 4,316 2,615,496 858,905 5,546,060 1,821,2762017 6,676 4,045,535 1,070,515 8,578,403 2,269,9862018 10,190 6,174,885 1,363,932 13,093,610 2,892,1662019 14,871 9,011,667 1,684,004 19,108,898 3,570,8672020 19,714 11,946,691 1,901,364 25,332,505 4,031,769
Efficiencyimpacts
Intheon-linesurveys,userswereaskedthefollowingquestionsabouttheamountoftimetheyspentdoingresearch,theshareofthattimeworkingwithdataandtheirestimatesofthetimesavingsfromusingtheVL.
Overthelasttwelvemonths,onaveragehowmanyhoursperweekdidyouspendonresearch?
Canyouestimatetheapproximateshareofyourtotalresearchworkingtimespentwithdataduringthelasttwelvemonths(e.g.creating,manipulatingandanalysingdata)?
Alldata:
approximatepercentofmytotalresearchworkingtime
DatafromGVL:
approximatepercentofmytotalresearchworkingtime
Towhatextent,ifany,hasyouruseofGVLservicesandresourceschangedyourresearchefficiency(i.e.thetimesavedcomparedtothesituationifGVLdidnotexist?
ThetimespentonresearchexpressedinhoursperweekisshowninTable6.7foreachVL.
UsingtheaverageannualcostperusercalculatedinSection6.1,thevaluesinTable6.7canbeusedtocalculatetheannualcostperuserof(i)thetimespentonresearch,(ii)thetimespentworkingwithdata,(iii)thetimespentworkingwithVL,and(iv)theresultingincreaseinresearchefficiency.TheresultsofthesecalculationscanbeseeninTable6.7.TheaverageannualvalueperuseroftheefficiencyimpactofBCCVL,CVLandGVLare$11,898,$13,240and$23,431respectively.Asacomparison,BeagrieandHoughtonfoundthatEBIwouldbeworthGBP26,000or$44,828perpersonperannum.
31
ApplyingthesevaluestothecumulativenumberofusersforeachVLweestimatethattheefficiencyimprovementduetoeachVLisasshowninTable6.9.Moreconservatively,ifweapplythesevaluesjusttothenumberofnewuserseachyearforeachVL,thecalculatedvaluesareasshowninTable6.10.WhichevermethodisuseddemonstratesthattherearesubstantialbenefitsinresearchefficiencyarisingfromtheVLs.
Table6.7 Surveyresponsesonresearchandefficiency
BCCVL CVL GVLTimespentonresearch,hoursperweek mean 22.8 28.5 37.9 median 20.0 30.0 40.0Shareofresearchtimeworkingwithdata,% mean 65.1 59.9 66.4 median 70.0 50.0 75.0ShareofresearchtimeworkingwithVL,% mean 11.8 28.0 11.4 median 5.0 23.0 7.0Increaseinresearchefficiency,% mean 31.0 31.1 39.1 median 30.0 30.0 50.0
Table6.8 Averageannualvalueofefficiencyimpactperuser
BCCVL CVL GVLAnnualcosts 63,127 56,015 59,925 Annualcostoftimespentonresearch mean 38,381 42,572 59,925 median 33,668 44,812 59,925Annualcostofresearchtimeworkingwith
datamean 24,986 25,500 39,790
median 23,567 22,406 44,944AnnualcostofresearchtimeworkingwithVL mean 4,529 11,920 6,831 median 1,683 10,307 4,195Increaseinresearchefficiency mean 11,898 13,240 23,431 median 10,100 13,444 29,963
Table6.9 ValueofVirtualLaboratoriesonresearchefficiency,usingcumulativeusers,$
Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 5,309,154 4,311,244 0 5,309,154 4,311,2342014 0 11,929,046 35,825,502 0 5,964,523 17,912,7072015 3,022,137 21,196,896 63,028,516 1,007,386 7,065,632 21,009,4542016 13,920,867 31,788,724 101,126,793 3,480,241 7,947,181 25,281,6362017 35,718,327 43,439,735 156,423,186 7,143,715 8,687,947 31,284,5602018 68,414,518 56,255,846 238,758,578 11,402,498 9,375,974 39,792,9992019 101,110,708 70,353,569 348,437,568 14,444,486 10,050,510 49,776,673
32
2020 133,806,898 85,861,065 461,912,327 16,725,977 10,732,633 57,738,899
Table6.10 ValueofVirtualLaboratoriesonresearchefficiency,usingnewusers,$
Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 3,971,935 3,374,009 0 3,971,935 3,374,0092014 0 6,619,892 31,514,180 0 3,309,946 15,757,0902015 3,022,158 9,267,849 27,202,947 1,007,386 3,089,283 9,067,6492016 10,898,805 10,591,828 38,098,184 2,724,701 2,647,957 9,524,5462017 21,797,610 11,651,011 55,296,257 4,359,522 2,330,202 11,059,2512018 32,696,415 12,816,112 82,335,190 5,449,402 2,136,019 13,722,5322019 32,696,415 14,097,723 109,678,720 4,670,916 2,013,960 15,668,3892020 32,696,415 15,507,495 113,474,480 4,087,052 1,938,437 14,184,310
AdditionalresearchmadepossiblebyVirtualLaboratories
Theon-linesurveyaskedusersthefollowingquestion(oranequivalentvariant)abouthowimportanttheVLwastotheirresearch
IfGVLhadnotexisted,wouldyouhavebeenabletoobtaintheresourcesyoulastusedfromanothersource?
ForBCCVL,CVLandGVLtheproportionofuserswhoanswered“No”tothisquestionwas55.26%,29.03%and34.3%respectively.ItisclearthereforethattheVLsareimportantagentsenablingresearchintheirfields.Againbywayofcomparison,about45%ofEBIusersalsoindicatedthattheycouldnothaveproceededintheirresearchwithoutEBI.IfweassumethatthesepercentagestranslateintoresearchenabledbytheVLswhichwouldnototherwiseoccur,thisimpliesthatwecancalculatethevalueoftheresearchundertakenbyVLusersbymultiplyingthenumberofusersbytheirannualaveragecostandapplyingthepercentagesquotedabove.Aswiththeprevioussectionwepresenttworesults.Thefirstusersthecumulativenumberofusersforthecalculationandthesecondusesthenumberofnewusers.Tables6.11and6.12showtheresults.Aswiththevalueofresearchefficiency,thisdemonstratesthattherearesubstantialbenefitsarisingfromtheVLsinenablingresearchwhichwouldotherwisenothaveoccurred,whichevermethodisused.
33
Table6.11 ValueofVirtualLaboratoriesinenablingadditionalresearch,usingcumulativeusers,$
Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 6,520,762 3,781,977 0 6,520,762 3,781,9772014 0 14,651,388 31,427,409 0 7,325,694 15,713,7052015 8,860,592 26,034,265 55,290,864 2,953,531 8,678,088 18,430,2882016 40,814,537 39,043,267 88,712,033 10,203,634 9,760,817 22,178,0082017 104,722,428 53,353,169 137,220,003 20,944,486 10,670,634 27,444,0012018 200,584,264 69,094,061 209,447,547 33,430,711 11,515,677 34,907,9252019 296,446,101 86,409,042 305,661,872 42,349,443 12,344,149 43,665,9822020 392,307,937 105,455,522 405,205,981 49,038,492 13,181,940 50,650,748
Table6.12 ValueofVirtualLaboratoriesinenablingadditionalresearch,usingnewusers,$
Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 4,878,376 2,959,808 0 4,878,376 2,959,8082014 0 8,130,626 27,645,432 0 4,065,313 13,822,7162015 8,860,592 11,382,877 23,863,455 2,953,531 3,794,292 7,954,4852016 31,953,945 13,009,002 33,421,169 7,988,486 3,252,250 8,355,2922017 63,907,891 14,309,902 48,507,970 12,781,578 2,861,980 9,701,5942018 95,861,836 15,740,892 72,227,545 15,976,973 2,623,482 12,037,9242019 95,861,836 17,314,981 96,214,325 13,694,548 2,473,569 13,744,9042020 95,861,836 19,046,480 99,544,109 11,982,730 2,380,810 12,443,014
ReturnstoresearchactivitiesmadepossiblebyVirtualLaboratories
Becausetechnologyisimportantinthecontinuingdevelopmentofnewproductsandimprovedproductivity,theroleofresearchanddevelopmentanditscontributiontotheeconomyhasbeenstudiedwidely.AreviewofthesestudieshasbeenprovidedinSection3abovewhichconcludedthata40%annualreturnoninvestmentisaconservativeestimateofthevalueofresearch.TakingthevalueoftheadditionalresearchduetotheVLsfromTable6.12wecalculatethevalueofthisresearchfollowingtheproceduresetoutbyBeagrieandHoughton.Weassumethattheusefullifeofanyresearchis30yearsandwedepreciatethevalueat3.4%ayearsothatthevalueinyear30iszero.Foreachoftheyears2013to2020asshowninTable6.12,wecalculatetheresultingbenefitsstreambyapplyingthismethodology.Thiswillgenerate8benefitsstreamswhichwesumforeachoftheyears2017to2047.Wethendiscountfuturebenefitstreamsusinga3%discountrateandcalculateanetpresentvalue(NPV)ofthesummedbenefitsstreams.FinallywecanexpresstheNPVofthesebenefitsasanannualised
34
valuebydividingby30.BoththetotalNPVandtheannualisedvalueareshownforeachVLinTable6.13.
Table6.13 ReturnstoadditionalresearchmadepossiblebyVirtualLaboratories$m.
30yearbenefit AnnualisedbenefitBCCVL 2,277.8 75.9CVL 609.5 20.3GVL 2,356.0 78.5
35
7.FurtherassessmentoftheimpactandvalueoftheCharacterisationVirtualLaboratory
DifferentVirtualLaboratories(VLs)sharethesamepurpose,whichistofacilitatedataanalysis,provideresearcherswithITresourcesandpromotecollaborativeresearch.WheretheVLsdifferisintheirscopeandimplementation.AmajorpointofdifferenceforCVLisitsmethodofoperationbecauseitworksprimarilywithinstrumentfacilitiesratherthanendusers.Bydevelopingdatacapture,analysisandvisualisationworkflowsdirectlywithinstrumentfacilities,CVLaimstounderpintheentirefacilityusercommunity.Thisintegrationenablesseamlesstransferofdatafromtheinstrumentsforstorageandanalysis.
TheCVLmethodofoperationandeaseoftransfermeansthatusersarenotalwaysawarethattheyareusingCVL,andmanyconsiderthecapabilitiesprovidedbyCVLasacomponentoftheinstruments,ratherthanasaseparateservice.LowawarenessofCVLamongitsusersraisesconcernsabouttherepresentativenatureofresponsesfromtheCVLusersurvey.
IncollaborationwithDrWojtekGoscinski,themanagerofCVL,thecurrentstudyusedanadditionalapproachtoevaluateCVLfromtheperspectiveofmanagersfromanumberofimagingfacilities.Imagingfacilitieshouseandmaintainadvancedimaginginstruments,andtheyrepresentconsiderablenationalinvestmentinAustralia’sresearchinfrastructure.AlistofimagingfacilitiesthatutiliseCVLservicesisgiveninAppendix2.Todate,CVLhasbeenintegratedwith69instrumentsin26imagingfacilities.
TheinterviewsconductedwithfacilitymanagersprovidedaglimpseintodiverseaspectsofCVLvalue.MuchofthatvalueisadditionaltothedirectbenefitofCVLtoindividualusers.Theadditionalbenefitsincludedeconomiesofscale,thepotentialforscalingupCVLapplicationandtheintrinsicvalueofCVLasadigitalresource.Thesedifferenttypesofvalueareoutlinedbelow.
(i) AcentralisedHPCcapabilitycutscostsbyreducingHPCreplicationacrossresearchorganisations
ImaginginstrumentscangeneratevastamountsofdatathatrequiresextensiveHPCcapabilitiesforanalysis.DrGeorgRamm(CliveandVeraRamaciottiCentreforStructuralCryo-ElectronMicroscopy,MonashUniversity)managesanelectronmicroscopyfacilitythatcontainsmanysuchinstruments,mostnotablytheTitanKrioscryo-electronmicroscope,whichisusedtypicallyfor2Dand3Dcharacterizationofcellularormolecularstructuresanditisparticularlydataintensive.Therearearound100usersatthecentre,ofwhomapproximatelyhalfrelyontheHPCcapabilityofCVLonMASSIVEfordatastorage,whiletheremainderrequiretheanalyticalcapacityofCVLfordataprocessing.
CentralisingHPCatthelevelofeachimagingfacilitywouldbeacheaperalternativetouser-fundedHPC,thoughstillconsiderablymoreexpensivethanthecostofmaintainingCVL:
36
• Asmallcomputingclusterwouldcostaround$200,000anditwouldneedtobeupdatedwithin5years.Salarycostsforpart-timePhD-levelstafftomanagetheclusterwouldaddaround$50,000peryeartotheoutlay,bringingthetotaltoaround$90,000peryear.
• DrAndrewJanke(CentreforAdvancedImaging,UniversityofQueensland)estimatedCVLreplacementcostat$250,000.Theamountwasbasedonthecostofsettingupanin-houseplatformforthecentre.
• ProfSampson(CentreforMicroscopy,CharacterisationandAnalysis,UWA)estimatedthatwhenimplementedatfullcapacity,thevalueofCVLservicestothecentrecouldwellbeworth$50,000peryear.Atthisstage,flowandmasscytometeristheonlyinstrumentintegratedwithCVL,althoughintegrationwithotherinstrumentsisbeingrolledout.
• DrIanHarperfromMonashMicroImaging(MMI)estimatedthecostofreplacingtheaccesstoHPCthroughCVLat$20,000peryear.
UsingDrHarper’smostconservativeestimate,itwouldcostover$500,000peryeartoreplaceHPCservicescurrentlyprovidedbyCVLto26facilities.ThisfigureiswellabovetheCVLrunningcostof$177,000-$320,000peryear(Table2.2).TheestimatedreplacementcostisforHPCalone;itdoesnottakeintoaccountthevalueofallotherservicesprovidedbyCVL,whichareoutlinedbelow.
(ii) PotentialforscalinguptheCVLplatform
CVLhasbeenadoptedwidelyinarelativelyshorttime.Inlessthanfouryearssinceitsinception,ithasbeenintegratedwith69instrumentsacross26imagingfacilities.TheexpansionofCVLisongoing.Inaddition,CVLintegrationwithcommonlyusedtechnologyhasthepotentialforscalingup.Forexample,theintegrationoftheflowcytometerattheCentreforMicroscopy,CharacterisationandAnalysisinUWA,isexpectedtobereplicatedwidely.Flowcytometryisincreasinglyusedforcellcountingandsortingnotonlyinresearch,butalsoinmajorhospitals.Thereareover20machinesandaround100usersofflowcytometryinWA,over100machinesinAustraliaandaround5,000worldwide.IntegrationwithCVLhasthepotentialtobereplicatedinothercentres,attractingmoreuserstotheplatform.
Lightmicroscopyisanotherwellestablishedandcommontechnology.DrIanHarperfromMonashMicroImaging(MMI),outlinedplanstointegrateMMIinstrumentswithCVL.MMIisalightmicroscopyfacilitythatcontainsinstrumentsvaluedataround$50million,andwhichattractapproximately500users.MMIislocatedonthreecampuses:MonashUniversity(Clayton),AlfredHospitalandMonashMedicalCentre.AtthisstageonlytheinstrumentsattheClaytoncampusareintegratedwithCVL.Withinthecomingyear,40-48MMImicroscopesatallthreecampusesareexpectedtobeintegratedwithCVL.
37
(iii) Increasingtheefficiencyofresearchandtheimpactofresearchinvestment
Inprovidingaccesstosecurestorageforexperimentaldata,CVLcontributestoimprovedresearchoutcomes.DrIanHarperfromMonashMicroImaging(MMI)describedproblemswithdatalosspriortointegrationwithCVL,whenUSBdatatransferwasused.DrRamm(CentreforStructuralCryo-ElectronMicroscopy)alsorecalledprolongedperiodsofdowntimewhenusersbroughttheirownharddrivestodownloaddata,makinginstrumentsvulnerabletoattackbycomputerviruses.
Theabilitytostoreallexperimentaldatasecurelyallowsresearcherstointerrogateolddatainsubsequentyearstoanswernewresearchquestions.TheabilitytosharedatawithcollaboratorsthroughtheCVLplatformalsomeansthatthesamedatacanbeinterrogatedbyotherresearchers.Reusingexperimentaldatainthiswayandimprovestheefficiencyandimpactofeachexperiment;italsoreducestheneedtorepeatexperiments
Increasingreproducibilityofresearchfindingsisanotherwaytoimprovetheefficiencyandimpactofexperimentalresearch.DrAndrewJanke(CentreforAdvancedImaging,UQ)emphasisedthatoneofthemainbenefitsofCVLtohisfacilityisthatitfacilitatesreproducibility.DrJankeusedtheexampleof“quarantining”supersededversionsofanalyticalsoftware,suchasFSL,apackageofimageanalysistoolsusedforMRIbrainimagingdata.AfterareleaseofupdatedFSLolderversionsarenolongeravailable.Thisoftenmeansthatresultsofearlierexperimentscannotbereplicated.Arecentstudyestimatedthatfewerthan16%ofneuroimagingresultsarereproducible(Russelletal2017).CVLpreservesallreleasesofthesoftware.
DrCeguerra(AustralianCentreforMicroscopy&Microanalysis,UniversityofSydney)notedthatintheabsenceofCVL,thereturnonthe$5millioninvestmentintheAtomProbewouldbegreatlydiminishedintermsofresearchoutcomes.TheAtomProbeenables3Dimagingandchemicalcompositionmeasurement.Becauseofthenoveltyofthistechnology,therearefewanalyticaltoolsavailableandusersarerequiredtodeveloptheirownmethodsandsoftware.TheAtomProbeworkbenchdevelopedbyCVLactsasarepositoryofanalyticalsoftwarethatisdevelopedbyusers.Thisrepositoryrepresentsasignificantinvestmentinresearchtime–over14yearsofeffortfrommultiplepeople.AccordingtoDrCeguerra,withoutCVLsomeprojectswouldbesetbackbyasmuchasfiveyears,withtheresearchershavingtodevelopthetoolstheyneedbythemselves.CVLisalsocriticalfordisseminatingresearchoutcomesintheAtomProberesearchcommunity.
(iv) ValueofCVLasadigitalresource
AnotherimportantaspectofthevalueofCVListhedevelopmentofitsintrinsicworthasadigitalresourceis.DrCeguerraemphasisedtheimportanceofCVLasaresourceforthesmallcommunityofcommunityofresearchersworkingwiththeAtomProbe.Thereare
38
closetoonehundredsuchinstrumentsworldwide2witharound500-600researcherstrainedtousethem,34ofthematthefacilityinSydney.Despiteitssmallsize,thiscommunityrepresentanimportantfieldofresearchthathasproducedfar-reachingdiscoveriesinmaterialsengineering.
Asalreadymentioned,theatomprobeisnoveltechnologywithoutacanonofwell-establishedanalyticalmethods.Ithasfallentouserstodevelopthemethodstheyneed.Therepositoryofuser-generatedsoftwareontheCVLAtomProbeworkbenchhasnoequivalentandhasbecomethekeyplatformforAtomProberesearchworldwide.CVLisalsocriticalfordisseminatingresearchoutcomesintheAtomProberesearchcommunity.ThevaluationoftheintrinsicvalueofVLsasadigitalresourceisoutlinedinmoredetailinSection8ofthisreport.
DrAndreJankesaidthatCVLprovidednoviceuserswithaneasytonavigatemechanismfortorapidlygetuptospeedwiththeiranalysiswithouttheneedtoinstallorconfiguresoftware,somethingthatpreviouslytookalargeinvestmentoftime.DrJankeestimatedthesavingintimeasapproximatelythreemonths.Thattimeisvaluedatover$19,000juniorforapostdoctoralfellow(commencementlevelAcademiclevelAsalaryis$77,819,Table6.1)or$6,670foraPhDstudent(MonashUniversityPhDannualstipendsare$26,682)
Interviewswithimagingfacilitiesmanagersprovidedaninsightintofar-reachingbenefitsofCVL,whichwouldnotbeapparentfromsurveyresponsesofindividualusers.ThecategoriesofvalueidentifiedinthiswaywouldbeequallyapplicabletotheotherVLs.Inasimilarway,BCCVL,GVLandHuNIoffereconomiesofscalethroughcentralisingITresources;andaimtodevelopvalueasdigitalresources,toimprovetheefficiencyandimpactofresearchandtoexpandtheirservicesoutsidethehostorganisationsandoutside
2AtomProbefacilitiesaroundtheworld(http://www.atomprobe.com/2ndLinks/apt-internet-sites.aspx
39
8.QualitativevaluationofNectarVirtualLaboratories
TheimpactandvalueofNectarVLscalculatedintheSection6arenecessarilyrestrictedtothoseforwhichthekeyinputstothecalculationcanbereadilyquantifiedandexpressedineconomicterms.
ItisclearhoweverthattheactivitiesofVLsextendwellbeyondthisintoareaswhicharemoredifficulttoquantify.WeattempttocapturesomeofthequalitativebenefitsofNectarVLsusingtheBalancedValueImpactModelframeworksuggestedbyTanner(2012).Table8.1providesabriefdescriptionofthesetypesofvalue.Table8.2attheendofthissectionsummarisesimportantaspectsofthevaluesdescribedinTable8.1forBCCVL,CVLandGVL.
Table8.1 TypesofvalueinBalancedValueImpactframework
TypeofValue Definition
UtilityValue Peoplevaluetheutilityaffordedthroughuseofthedigitalresourcesnoworsometimeinthefuture.
Existenceand/orPrestigeValue
Peoplederivevalueandbenefitfromknowingthatadigitalresourceischerishedbypersonslivinginsideandoutsidetheircommunity.Thisvalueexistswhethertheresourceispersonallyusedornot.
EducationValue Peopleareawarethatdigitalresourcescontributetotheirownortootherpeople’ssenseofculture,education,knowledgeandheritageandthereforevalueit.
CommunityValue Peoplebenefitfromtheexperienceofbeingpartofacommunitythatisaffordedbythedigitalresource.
Inheritance/BequestValue Peoplederivebenefitfromtheinheritancepasseddowntothemandsatisfactionfromthefactthattheirdescendantsandothermembersofthecommunitywillinthefuturebeabletoenjoyadigitalresourceiftheysochoose.
BasedonTanner(2012)p37
Utilityvalue
TheprecedingsectionhasshownthevaluethatusersputonVLservicestotheextentthatthiscanbeexpressedineconomicterms.
40
Intheon-linesurveysuserswereaskedthefollowingquestion
TowhatextentdoyoubenefitfromusingVLinanyofthefollowingways?
Haven'tused,Nobenefit,Lowbenefit,Mediumbenefit,Highbenefit,Veryhighbenefit
Table8.2 ExtentofbenefitsfromVirtualLaboratories,%ofrespondents
Services BCCVL CVL GVLMediumandhigher
Highandveryhigh
Mediumandhigher
Highandveryhigh
Mediumandhigher
Highandveryhigh
Data&tools 73.8 57.4 72.5 70.0 60.6 44.8
Collaborations 23.0 11.5 50.0 30.0 48.7 25.0
Training 37.7 27.9 27.5 15.0 51.2 32.9
Usersupport 27.9 19.7 --- --- 36.8 29.0Other 27.9 19.7 7.5% 2.5% 36.8 29.0
Intheon-linesurveysuserswereaskedthefollowingquestion
WhatimpactwouldithaveonyourworkorstudyifyoucouldnotaccessVLservicesandresources?
Noimpact,Slightimpact,Moderateimpact,Majorimpact,Severeimpact
ForBCCVLusers18.2%answeredthatiftheycouldnotaccesstheservicesandresourcesprovidedbytheVLthiswouldhaveamajororsevereimpact.ForCVLandGVLthepercentageswerehigherat52.8%and35.7%respectively(Table8.3).
Howevertakingmoderate,majorandsevereimpactassignificant,thepercentagesofrespondentswere41.8%,75.0%and72.9%.
Table8.3 ImpactifcouldnotaccessVirtualLaboratoryservices,%
BCCVL CVL GVLNoimpact 23.6 5.6 11.4Slightimpact 34.5 19.4 15.7Moderateimpact 23.6 22.2 37.1Majorimpact 12.7 27.8 27.1Severeimpact 5.5 25.0 8.6 Majorandsevereimpact 18.2 52.8 35.7Moderate,majorandsevereimpact 41.8 75.0 72.9
41
CVL
WeconductedphoneinterviewswiththemanagersofanumberofimagingfacilitiesthathaveworkedcloselywithCVLonintegratingtheirfacilitieswiththeon-linestorageandcomputingfacilitiesofferedthroughCVL.
TheyprovidedthefollowingdescriptionsofthevalueofCVLtotheirfacilities.
DrKeithSchulze,ImageAnalyst,MonashMicroImaging,MonashUniversity
MyDataandStore.MonashrepresentasignificantadvanceindatahandlingatMonashMicroImaging.Transferandstorageofuserdataisnowreliable,seamlessandsecure.Store.Monashprovidesuserswiththeabilitytosharedatawithcolleaguesandcollaboratorsinaconvenientandsecuremanner.Italsohasmechanismstocaptureandexposeimportantmetadatathatallowforbetterreuseandreproducibilityofdata.
Next-generationimagingtechnologies,liketheLatticelight-sheetmicroscope,arecapableofproducingdataatarateofTerabytesperhour.ThetoolsforautomatedhandlingandstorageofdataprovidedbyCVLareakeyenablerforresearcherstoderivethemostbenefitfromtheselargedatasetsi.e.,theyspendlesstimestrugglingwithdatatransfersandmoretimeextractinginterestinginformationfromtheirdata.Moreover,CVLandNectarprovideacrucialplatformonwhichtoolstoanalyseandvisualthisdatacanbedevelopedanddeployed.
DrAndrewMehnert,GroupLeader–DataManagement,AnalysisandVisualisation,CentreforMicroscopyCharacterisationandAnalysis,TheUniversityofWesternAustralia
TheCentreforMicroscopy,CharacterisationandAnalysis(CMCA)isaUniversityfacilitythatcollaboratesinmicroscopyandcharacterisation,supportingresearchexcellencelocally,nationally,andinternationally.Itcomprises~40staff,morethan500usersandhosts~50instrumentplatformsacrosscytometry,opticalmicroscopy,micro-magneticresonanceimaging(microMRI),preclinicalbioimaging,electronmicroscopy(EM),X-raymicro-computedtomography(microCT),secondaryionmassspectrometry,bio-organicmassspectrometry,X-raydiffractionandnuclearmagneticresonance(NMR)spectroscopy.TheCMCAcollaboratesinandsupportsresearchacrossbiologicalscience,biomedicalscience,earthscienceandphysicalscience.
CMCAusersareacquiringever-largermulti-dimensionaldatasetsandareincreasinglyusingmultipleinstrumentplatforms;e.g.microCTtogetherwithEMandRamanspectroscopy.ThispresentsaBigDatachallenge,notonlyintermsofmanaging/curatingthisdataovertheresearchlifecycle,butalsointermsof
42
analysingthisdatatofacilitatenewdiscoveries.TosolvethischallengetheCMCAisleveragingtheCharacterisationVirtualLaboratory(CVL)andCVL-supportedservicesincludingMyTardis.Overthelast12monthstheCMCAhasbeenworkingwithMonashUniversitytodeveloptheCytometryworkbench(collectionofsoftwaretools)fortheCVLtosupporttheanalysisofdatafromimage-,mass-andflow-cytometryinstruments.TheCMCAhasalsointegratedseveralEMinstrumentswithMyTardisandisintheprocessofintegratingitsNIFflagshipMRIscanner.TheCMCAplanstointegrateallofitsinstrumentswithMyTardisandtodevelopadditionalworkbenchesandworkflowsintheCVLinsupportofitsusers/researchers.TheCVLandsupportedservicesrepresenttheonlyviablesolutionacrossmultipleinstrumentplatformsandmodalities.Moreover,lookingtothefuture,theCVLoffersasolutiontotheanalysisandvisualisationofverylargedatasetsusingtoolsandcomputeresourcesnototherwiseavailablefromadesktopworkstation.
DrAndrewJanke,InformaticsFellow,NationalImagingFacility,UniversityofQueensland
ThevalueofCVLandmanagedworkbenchesinneuroimagingandpreclinicalimagingtotheNationalImagingFacility(NIF)ispredominatelyaroundencouragingouruserstoperformreproduciblescience.CVLprovidesatoolinwhichresearcherscanprocessandanalysetheirimagingdatasafeintheknowledgethatiftheyneedtoreproducetheanalysisin5years’timethesameversionsofsoftwarewillstillbeavailable.
Traditionallythishasnotbeenthecaseinimagingresearchandsoftwarepackagesandoperatingsystemsareamovingtarget.CVLprovidesamechanismtofreezeintimeaparticularanalysispipeline.InadditionCVLprovidesNIFwithaneasytonavigatemechanismfornoviceuserstorapidlygetuptospeedwiththeiranalysiswithouttheneedtoinstallorconfiguresoftware,somethingthatpreviouslytookalargeinvestmentofNIFpersonneltime.
Educationvalue
Intheon-linesurveysBCCVLandGVLuserswereaskedthefollowingquestion
HowdidyoulearntousetheservicesprovidedbytheVL?
Asshowninthetablebelow,53.8%ofBCCVLusers,50.0%ofGVLusersrelyontherespectiveVLstolearntheanalyticaltoolsrequiredfordataanalysisintheirrespectivefields.
AsisevidentfromTable8.4,animportantaspectofbothBCCVLandGVListhetrainingtheyprovidetoresearchersandstudents.
43
Table8.4 TrainingprovidedbyVirtualLaboratories,%ofrespondents
BCCVL GVLAlreadyproficient 11.5 21.1TutorialthroughtheVLwebsite 21.3 18.4Face-to-facecourseprovidedbyVLpersonnel 32.8 9.2Face-to-facecourseprovidedbysomeoneelse 0.0 22.4Anotherway 16.4 15.8Missing 18.0 13.2Total 100.0 100.0
OneoftheimportantbenefitsofGVListoprovidetouserstheresourcestheyneedwhichotherwisewouldrequireadvicefromexperiencedbio-informaticians.Thesupplyoftrainedbio-informaticiansisacriticalbottleneckinexpandingresearchwithinthefieldofgenomics,soenablingresearcherstobypassthisbottleneckisanimportantcontributionthatGVLmakes.
OvertheperiodtoFebruary2017,GVLhasundertakentrainingcoursesfor313users,whogiveanaverageratingof4.5outof5forthesecourses.
Benefitstousers
BCCVL
InitsSnapshotfor2016,BCCVL(2016)providesthefollowingquotesfromusers.
...becauseitprovidesanenvironmentforexperimentingwithmodels,andalsokeepscarefultrackoftheseexperiments,BCCVLalsoencouragesbestpracticeandtransparencyinmodelling.
Withitseasytouseinterface,accessiblefromanywhere,itopensthefieldtoawholenewarrayofresearcherswhounderstandthesystemstheyareworkingon,butdonothavethetechnicalskill-setsorhardwaretoproperlyanswertheirquestions.
…essentiallytheBCCVLhasenabledustoaskquestionsthatwecouldn’taskbefore–questionswemayhavewantedtoaskbutcouldn’tlogisticallyhopetoanswer,soit’sopenedupawholenewfieldofenquiry.
CVL
Initsannualreportfor2015-16,theMassiveprojectatMonashUniversitynotedthat
CVLhasunderpinnedworkflowdeploymentattwoAustralianSynchrotronbeamlinesforaccesstotheMASSIVEDesktop.ResearchersvisitingboththeImagingandMedicalBeamline(IMBL)andtheX-rayFluorescenceMicroscopy(XFM)beamlineareautomaticallycreatedabeamlinevisitproject,anduseraccounts.Authenticationis
44
integratedwithAustralianSynchrotronsothatuserscanusetheirAScredentialstologintothesystem.Asetofbeamlinechangeoverscripts,developedincollaborationwithAustralianSynchrotron,controltheflowofdata.Theimpactofthisworkissignificant.Researchersareprovidedwithaccesstoaremotedesktopenvironmentforthedurationoftheirbeamlinevisitthatprovidesaccesstothedatacaptured,andtoolsfordataprocessingandvisualisation
Additionally,CVLsoftwareisnowbeingadoptedbytwomajorinternationalsupercomputingcentres:
>JulichSupercomputingCentre,isintheprocessofdeployingStrudeltosupportvisualisationusersandhigh-endcommercialengineeringapplications;
>EdinburghParallelComputingCentreforindustryaccesstoHPCsystemsundertheEUFP7project,Fortissimo.
WithCVLNectarfunding,MonashUniversityisdevelopinganinstrumentintegrationapp,calledMyDatatomakeintegrationquicker,simplerandlessreliantonspecialistITsupport.AsaresultofthisprojectMyDataisnowusedat35instruments,across10facilities,at4institutions,inaddition17furtherinstrumentsareplannedat4institutions.Inaddition,thisprojecthasmadeMyTardiseasiertouseforFacilityManagers.(p23)
HuNI
DescriptionsbytwoHuNIusersofitsvalue,whichincludesnewapproachestomulti-disciplinaryresearchandconnectingwithotherresearchers.‘Userscancapturerelationshipsbetweencontent,buildpathwaysandstructures,andshareanddistributedata.Itprovidesaplatformwhereserendipitouscollaborativeopportunitiescanquicklyemerge’(User2).
User1(literarystudies)
Iworkintheareaofliterarystudies,particularlyliteraryandculturalhistoryinthenineteenthcentury,andmyresearchcutsacrossarangeaphysicalarchivesandlibrariesaroundtheworld.WhileIuseanumberofdigitaltoolsinmyresearch(particularlyasdigitisationhasopenedupaswatheofhistoricprintedtexts),HuNIopenedupmythinkingandapproachestoresearchinwaysthattheseindividualtoolshadneverencouraged.Inparticular,HuNIinnatelyencouragesuserstoidentifytheinterdisciplinaryconnectionswithintheirresearchfield.Asourresearchmethodshavelargelybeenshapedwithinourdisciplinarysilos,IwasnotevenawareofsomeofthedatasetsthatHuNIdrawsonbutwhichprovedtobesupremelyusefultomyresearchproject.Moreover,HuNIrepresentsarealstepforwardinefficiencyinthatsimpleawarenessoftheseotherdatasetsontheirownwouldstillmeanincreasedlabourin
45
individuallylocatingandsearchingthem.PerhapsmostexcitingandunexpectedofallwastheabilityofHuNItoconnectmyideasandresearchwiththeworkofotherresearchersthroughitsvisualisationcapabilities.Thesekindsofassociationsinrelationtospecificresearchsites(particularplays,books,orpublicfigures,forexample)aresimplynotpossibleinanyotherforumapartfromtherandomnetworksweformasacademics.Inthisrespect,HuNIclearlyhasthepotentialtoradicallytransformhowweconductourresearchinrelationto—andpotentiallyintandemwith—otherscholarsandmembersofthepublic.
User2(Archivist)
HuNI(HumanitiesNetworkedInfrastructure)makesasignificantandvaluablecontributiontoAustralia’sinformationinfrastructure.Inbringingtogether30curated,authoritativeandscholarlydatasetsinawell-designedonlineresource,HuNIsupportsdiverseresearchandprovidesresearcherswithdiversecontentwithoutthenoiseoflargeaggregators.Throughthis,HuNIalsohelpsincreasethevisibilityanduseofthese30foundationalhumanitiesresourceswhichtogetherconstituteasubstantialinvestmentofresourcesandscholarshipspanningmorethantwodecades.
Moreimportantly,unlikeaggregatorsand‘portals’,HuNIbringsthiscontentintoaresearchplatformwhichsupportsthetypeofrelational,non-hierarchical,iterativeengagementwithcollectionsanddatathatiscentraltocontemporaryhumanitiespractice.Userscancapturerelationshipsbetweencontent,buildpathwaysandstructures,andshareanddistributedata.Throughthis,HuNInotonlyfostersserendipitousdiscovery,itprovidesaplatformwhereserendipitouscollaborativeopportunitiescanquicklyemergeinanonlineenvironment.Astheuserbasegrows,thisaspectisincreasinglylikelytosparkbroadercollaborativeworkbyhumanitiesresearchers,contributingtothestrengthanddiversityofthesectorasawhole.
46
Table8.5 SummaryofbenefitsprovidedbyBCCVL,CVLandGVLusingBalancedValueImpactframework
Valuedriversandindicators BCCVL CVL GVL
1.UtilityValue• Reportedbenefitfromaccesstodataandtools1
ofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)
45(73.8%)35(57.4%)
29(72.5%)28(70.0%)
46(60.6%)34(44.8%)
2.Existenceand/orPrestigeValue• reachacrossdifferentorganisations[e-mailsuffixes]2
NumberofUniversitiesNumberofResearch/Not-for-profitorganisationsNumberofGovernmentagencies/departmentsNumberofCompanies
• ReachoutsideAustralia2Numberofcountries(includingAustralia)Australia(numberofusers)Othercountries(numberofusers)
• recognitionbyprovidersofotherdigitalplatforms
70(46.7%)20(13.3%)38(25.3%)22(14.7%)201005(94.8%)55(5.2%)Seetext
46(49.5%)33(35.5%)7(7.5%)7(7.5%)152,388(93.4%)170(6.6%)Seetext
24(64.9%)8(21.6%)--4(10.8%)1665NoneSeetext
3.EducationValue• trainingtroughcoursesbyVLpersonnel/VLwebsite
• VLtraining
ofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)
• satisfactionsurveysforGVLcourses(meanscoreoutof5)
33(54.1%)23(37.7%)17(27.9%)--
--
11(27.5%)6(15.0%)--
21(27.6%)39(51.2%)25(32.9%)4.5
4.CommunityValue• useofVLplatformsforcollaboration
ofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)
• usersupportofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)
14(23.0%)7(11.5%)17(27.9%)12(19.7%)
20(50.0%)12(30.0%)
----
37(48.7%)19(25.0%)28(36.8%)22(29.0%)
5.Inheritance/BequestValue• VLusersuploadnewdatasetstoaVL(numberofrespondents)• NumberofdatasetsuploadedbyDec2016• Repositoryofuser-writtenalgorithms
14(22.9%)
----
--
35,787yes5
22(29.0%)
--yes
1SurveyofVLusers2Informationderivedfromusere-mailsuffixes3Studentfeedbackfromparticipantsin31GVLtrainingcourse(informationprovidedbyVL
47
9.Summaryandconclusions
TheNectarfundingforthreeVLsexaminedinthisstudyhashaddemonstrablebenefits,whenmeasuredinbothquantitativeandqualitativeterms.ForanexpectedexpenditurebyNectarandpartnersofaround$4.4to$7.0milliontotheyear2020(oraround$550,000to$870,000inannualterms)thethreeVLshavegeneratedeconomicvalueexceedingthis.
Table9.1summarisesthecalculationofeconomicbenefitsexpressedinannualtermsandcomparesthistotheannualisedcostofeachVL.
Table9.1 SummaryofeconomicbenefitsandcostsforVirtualLaboratories
Annualisedbenefit,$
Annualisedcost,$
Ratio
Contingentvaluation Willingnesstopay
BCCVL 2,324,397 550,754 4.2CVL 1,235,410 612,356 2.0GVL 1,901,364 869,479 2.2
Willingnesstoaccept BCCVL 20,142,090 550,754 36.6CVL 11,454,271 612,356 18.7GVL 4,031,769 869,479 4.6
Efficiencyimpacts BCCVL 4,087,052 550,754 7.4CVL 1,938,437 612,356 3.2GVL 14,184,310 869,479 16.3
Additionalresearchimpact BCCVL 11,982,730 550,754 21.8CVL 2,380,810 612,356 3.9GVL 12,443,014 869,479 14.3
Returnstoadditionalresearch BCCVL 75,925,851 550,754 137.9CVL 20,318,154 612,356 33.2GVL 78,532,855 869,479 90.3
TheratiosforallVLsarehigherthanoneindicatingthattheyallgeneratebenefitsinexcessoftheircosts.ForBCCVLandGVLtheratiosareconsistentlyhighacrossallmeasuresofvalueandarehigherthanthoseoftenachievedbyconventionalphysicalinfrastructure.ThestudyindicatesthattheVLscontributevalueindifferentways.Forinstance,theefficiencyimpactratioforGVLishigherthanthatforBCCVL,thecontingentvaluationratioandtheadditionalresearchratioarehigherforBCCVLthanforGVL.ThesevariationsreflectdifferencesinthewayinwhichtheVLsdelivervaluetotheirusers.
48
Takingawiderperspectiveonbothquantitativeandqualitativebenefits,itisclearfromthecasestudyofCVLthatithasgeneratedconsiderablebenefitsfortheimagingfacilitieswithwhichitworks.Thesefacilitieswouldotherwiseincurconsiderablecosts(discussedinSection7butnotfullycapturedbythestudy)inreplicatingtheservicestheyprovide.ThefacilitymanagershavealsoprovidedevidenceofthewiderbenefitsofworkingwithCVL.WehaveusedtheBalancedValueImpactframeworktodescribecharacteristicsofVLsthatprovidedarangeofbenefitstousersandcommunities.ThisapproachcouldbeadaptedfurtherforprovidinginsightintothevalueandimpactofotherVLs.
WenotedintheIntroductionthatmostVLsfundedbyNectarhaveonlybeenactiveforafewyearsandarestillintheirgrowthstages.AnevaluationoftheiroverallimpactandvaluemightbestbedonefromtheperspectiveofsomeyearsinthefuturewhentheVLsareinamorematuregrowthphase.Thereforetheanalysisandconclusionsdrawninthisstudyshouldbetreatedaspreliminaryanddependsignificantlyontheassumptionsmadeaboutfuturegrowthpaths.
AlthoughVLssharemanyfeaturesincommon,theydiffersignificantlyfromeachotherintermsoftheservicestheyprovidedtotheirtargetcommunities.ThismeansthattheapproachtotheevaluationofvalueandimpactmustbetailoredtothecircumstancesofeachoneandthatconsiderableeffortshouldbemadeinunderstandingtheservicesprovidedbyeachVLandhowtheseservicesaredeliveredtoandbenefittheirtargetcommunities.
49
References
ABS2017,6302.0AverageWeeklyEarnings,Australia,athttp://www.abs.gov.au/AUSSTATS/[email protected]/DetailsPage/6302.0Nov%202016?OpenDocument
AccessEconomics2008,Exceptionalreturns:thevalueofinvestinginhealthR&DinAustraliaII.Canberra(Australia):AustralianSocietyforMedicalResearch;2008.
AfganE,SloggettC,GoonasekeraN,MakuninI,BensonD,CroweM,GladmanS,KowsarY,PheasantM,HorstR,LonieA.,GenomicsVirtualLaboratory:APracticalBioinformaticsWorkbenchfortheCloud.,PLoSOne.2015Oct26;10(10):e0140829.
BarkerMichelle,NancyWilkins-Diehr,SandraGesing,DanielS.Katz,SilviaDelgadoOlabarriaga,ShayanShahand,ScottHenwood,TristanGlatard,KeithJeffery,BrianCorrie,AndrewTreloar,HelenGlaves,LesleyWyborn,NeilChueHong,AlessandroCosta2017,TheGlobalImpactofScienceGateways,VirtualResearchEnvironmentsandVLs,
BarkerM2016,TheAustraliansciencegatewaysecosystem.PeerJPreprints;2016Oct6.
BCCVL2016,Our2016Snapshot,Jan5,2017,availableathttp://www.bccvl.org.au/our-2016-snapshot/
Beagrie,N.,Lavoie,B.,andWoollard,M.(2010)KeepingResearchDataSafe2FinalReportLondon:Jisc.http://www.jisc.ac.uk/publications/reports/2010/keepingresearchdatasafe2.aspx#downloads.
Beagrie,N.,Houghton,J.W.,Palaiologk,A.,andWilliams,P.(2012)EconomicImpactEvaluationoftheEconomicandSocialDataService(ESDS),EconomicandSocialResearchCouncil,London.http://www.esrc.ac.uk/files/research/evaluation-and-impact/economic-impact-evaluation-of-theeconomic-and-social-data-service/
Beagrie,N.andHoughton,J.W.(2013a)TheValueandImpactoftheArchaeologyDataServices:AStudyandMethodsforEnhancingSustainability,JointInformationSystemsCommittee,BristolandLondon.http://www.jisc.ac.uk/whatwedo/programmes/preservation/ADSImpact.aspx
Beagrie,N.andHoughton,J.W.(2013b)TheValueandImpactoftheBritishAtmosphericDataCentre,JointInformationSystemsCommitteeandtheNaturalEnvironmentResearchCouncilUK,BristolandLondon.http://www.jisc.ac.uk/whatwedo/programmes/di_directions/strategicdirections/badc.aspx
50
Beagrie,N.andHoughton,J.W.(2014)TheValueandImpactofDataSharingandCuration:ASynthesisofThreeRecentStudiesofUKResearchDataCentres,JointInformationSystemsCommittee(Jisc),BristolandLondon.http://repository.jisc.ac.uk/5568/DepartmentofInnovation,Industry,ScienceandResearch2008,StrategicRoadmapforAustralianResearchInfrastructure,August2008,availableathttps://docs.education.gov.au/system/files/doc/other/national_collaborative_research_infrastructure_strategic_roadmap_2008.pdf
Beagrie,NeilandHoughton,John2016.TheValueandImpactoftheEuropeanBioinformaticsInstitute,Fullreport,January2016,EMBL-EBI
BritishLibrary.(2004)MeasuringtheEconomicImpactoftheBritishLibrary,Powerpointpresentationonunpublishedreport.http://www.bl.uk/aboutus/stratpolprog/increasingvalue/publicvalue/confpres/pungelwesmarks.pdf
CeguerraAV,LiddicoatPV,RingerSP,GoscinskiWJ,AndroulakisS.Atoolforscientificprovenanceofdataandsoftware.InComputationalScienceandEngineering(CSE),2013IEEE16thInternationalConferenceon2013Dec3(pp.561-565).IEEE.
CommonwealthofAustralia,HandbookofCostBenefitAnalysis,January2006,FINANCIALMANAGEMENTREFERENCEMATERIALNO.6athttps://www.finance.gov.au/sites/default/files/Handbook_of_CB_analysis.pdf
DeloitteAccessEconomics2014,Extrapolatedreturnsfrominvestmentinmedicalresearchfuturefund(MRFF)AustralianSocietyforMedicalResearch17October2014
DeloitteAccessEconomics2015,TheeconomiccontributionsofAustralia’sresearchuniversities–theUNSWexampleFinalreportUNSWAustraliaOctober2015
DepartmentofEducation2017,NationalCollaborativeResearchInfrastructureStrategy(NCRIS),availableathttps://www.education.gov.au/national-collaborative-research-infrastructure-strategy-ncris
EvaRIO2013b,Corestudy:AdaptingtheBETAmethodtothecaseoftheevaluationoftheimpactofResearchInfrastructures,Finalreport:ResultsandPolicyRecommendations-PARTI.
Goecks,J.;Nekrutenko,A.;Taylor,J.;GalaxyTeam,T.(2010)."Galaxy:Acomprehensiveapproachforsupportingaccessible,reproducible,andtransparentcomputationalresearchinthelifesciences".GenomeBiology.11(8):R86.doi:10.1186/gb-2010-11-8-r86.PMC2945788Freelyaccessible.PMID20738864.
Goscinski,W.J.,P.McIntosh,U.C.Felzmann,A.Maksimenko,C.J.Hall,T.Gureyev,D.Thompson,A.Janke,G.Galloway,N.E.B.Killeen,P.Raniga,O.Kaluza,A.Ng,G.Poudel,
51
D.Barnes,T.Nguyen,P.BonningtonandG.F.Egan(2014)."TheMulti-modalAustralianScienceSImagingandVisualisationEnvironment(MASSIVE)highperformancecomputinginfrastructure:applicationsinneuroscienceandneuroinformaticsresearch."FrontiersinNeuroinformatics8.
GoscinskiWJ,BamberyK,FelzmannC,HallC,HinesC,MaksimenkoA,McIntoshP,PanjikarS,PatersonD,RyanC,ThompsonD.MASSIVE:anHPCCollaborationtoUnderpinSynchrotronScience.In15thInt.Conf.onAcceleratorandLargeExperimentalPhysicsControlSystems(ICALEPCS'15),Melbourne,Australia,17-23October20152015Dec(pp.640-644).JACOW,Geneva,Switzerland.
GraduateCareersAustraliaLtd,GraduateSalariesReport2015,GraduateCareersAustraliaLtd,2016athttp://www.graduatecareers.com.au/research/researchreports/graduatesalaries/
HallgrenW,BeaumontL,BownessA,ChambersL,GrahamE,HolewaH,LaffanS,MackeyB,NixH,PriceJandVanderwalJ.2016.TheBiodiversityandClimateChangeVirtualLaboratory:Whereecologymeetsbigdata.EnvironmentalModelling&Software,76,pp.182-186.
Houghton,J.W.andSheehan,P.(2009)‘Estimatingthepotentialimpactsofopenaccesstoresearchfindings,’EconomicAnalysisandPolicy39(1).http://www.eap-journal.com/vol_39_iss_1.php
Houghton,J.W.,Rasmussen,B.,Sheehan,P.J.,Oppenheim,C.,Morris,A.,Creaser,C.,Greenwood,H.,Summers,M.,andGourlay,A.(2009)EconomicImplicationsofAlternativeScholarlyPublishingModels:ExploringtheCostsandBenefits,ReporttoTheJointInformationSystemsCommittee(JISC).http://www.jisc.ac.uk/publications/reports/2009/economicpublishingmodelsfinalreport.aspx
IndustryCommission(1995)ResearchandDevelopment,ReportNo44,15May1995,AustralianGovernmentPublishingService,Canberra.
Massive.2016,MassiveAnnualreport2015-2016,availableathttps://www.massive.org.au/images/stories/MASSIVE_AR_15-16_small.pdf
PoldrackRussellA.,ChrisI.Baker,JokeDurnez,KrzysztofJ.Gorgolewski,PaulM.Matthews,MarcusR.Munafò,ThomasE.Nichols,Jean-BaptistePoline,EdwardVul&TalYarkoni,Scanningthehorizon:towardstransparentandreproducibleneuroimagingresearch,NatureReviewsNeuroscience18,115–126(2017)doi:10.1038/nrn.2016.167.
Shahand,S.Sciencegatewaysforbiomedicalbigdataanalysis[dissertation].Amsterdam:UniversityofAmsterdam;2015.178p.http://hdl.handle.net/11245/1.490613.
52
Shanks,S.andZheng,S.2006,EconometricModellingofR&DandAustralia’sProductivity,ProductivityCommissionStaffWorkingPaper,Canberra,April.
Tanner,S.,2012.MeasuringtheImpactofDigitalResources:TheBalancedValueImpactModel.KingsCollegeLondon,2012athttps://kclpure.kcl.ac.uk/portal/files/5675881/BalancedValueImpactModel_SimonTanner_October2012.pdf
VerhoevenD.2014.DoingtheSheepGood.InKBodeandAPLongley,AdvancingDigitalHumanities:Research,Methods,Theories.Palgrave,NewYork.
VerhoevenD,BurrowsT.2015.AggregatingCulturalHeritageDataforResearchUse:TheHumanitiesNetworkedInfrastructure(HuNI).InEGaroufallouetal.(eds),MetadataandSemanticsResearch:9thResearchConference,MTSR2015.BruceManchester,UK,September9-11,Proceedings,Springer,pp.417–423,2015.DOI:10.1007/978-3-319-24129-6_36.
VerhoevenD.2016.Asluckwouldhaveit.FeministMediaHistories,2:7-28.\
53
Appendix1VirtualLaboratoriesfundedbyNectar
Name
All-SkyVirtualObservatory
Alveo
BiodiversityandClimateChangeVirtualLaboratory
CharacterisationVirtualLaboratory
ClimateandWeatherScienceLaboratory
EndocrineGenomicsVirtualLaboratory
MicrobialGenomicsVirtualLaboratory
GenomicsVirtualLaboratory
HumanitiesNetworkedInfrastructure
IndustrialEcologyVirtualLaboratory
MarineVirtualLaboratory
VirtualGeoChemistryLaboratory
VirtualGeophysicsLaboratory
VirtualHazardImpactandRiskLaboratory
54
Appendix2CVLimagingfacilitypartners
Imagingfacility
AnimalMRIFacility,FloreyNeuroscienceInstitutes
AustralianCentreforMicroscopy&Microanalysis,USydney
AustralianSynchrotron
BiologicalOpticalMicroscopePlatform(MDHS),UoM
BiologicalResourcesImagingLaboratory,UNSW
AustralianCentreforNeutronScattering,ANSTO
CentreforAdvancedImaging,UQ
CentreforMicroscopyandMicroanalysis,UQ
CentreforMicroscopy,CharacterisationandAnalysis,UWA
Florey,MelbourneBrainCentre
FlowCore,MonashUniversity
MelbourneBrainCentreImagingUnit,UoM
MicroNanoResearchFacility,RMIT
MonashBiomedicalImaging
MonashBiomedicalProteomicsFacility
MonashInjuryResearchInstitute
MonashMicroImaging
MonashMicroImaging(AMREP)
UniversityofNewcastle,LightSheetMicroscopy
QueenslandBrainInstitute
SingleMoleculeScience,UNSW
RoyalChildren'sHospital
RoyalMelbourneHospital
55
StVincentsHospital
TheCliveandVeraRamaciottiCentreforStructuralCryo-ElectronMicroscopy
X-rayMicroscopyFacilityforImagingGeo-materials(XMFIG),Monash