nci.org.au @NCInews

   

HPC  vs  Cloud:  the  Return  on  Investment    in  eResearch      

 Allan  Williams  

Associate  Director,  NCI    

31  July  2017  

nci.org.au

Our Mission: To provide world-class, high-end computing services for Australian research and innovation Through: Enabling a comprehensive, integrated research environment Backed by Nationally and Internationally renowned, expert support team

 is  High  Performance  Compute  and  Big  Data  

nci.org.au

   

   

Disclaimer    The  view  expressed  in  this  talk  are  my  personal  experiences  and  not  necessarily  

those  of  the  NCI  management.  

nci.org.au

NaLonal  Research  Infrastructure  Roadmap    

2016  Na(onal  Research  Infrastructure  Roadmap  Terms  of  Reference  

 The  2016  Roadmap  will  develop  a  prioriLsed  plan  for  the  coming  decade  for  investment  in  naLonal  research  infrastructure  capability  that  will  advance  science  and  research  for  a  healthy,  sustainable  and  prosperous  Australia  and  posiLon  the  naLon  to  respond  to  the  world’s  big  research  challenges    

Accordingly,  the  2016  Roadmap  will:    •  idenLfy  Australia’s  naLonal  research  infrastructure  needs  to  underpin  future  

research  and  innovaLon  capability  •  determine  areas  where  capacity  building  of  the  naLonal  research  infrastructure  

system  or  decommissioning  of  exisLng  capacity  will  be  of  strategic  benefit  to  Australia’s  research  effort    

 

nci.org.au

Key  NaLonal  Research  Infrastructure      

 •  Digital  Data  and  eResearch  Pla>orms–  All  areas  of  research  are  increasingly  

dependent  on  data  and  eResearch  infrastructure.  Through  naLonal,  state  and  insLtuLonal  investments  over  the  past  decade,  Australia  has  built  an  internaLonally  compeLLve  eResearch  system.  ConsolidaLng  the  gains  of  the  past  decade  through  the  creaLon  of  an  Australian  Research  Data  Cloud  will  deliver  a  more  integrated,  coherent  and  reliable  system  to  meet  the  needs  of  data-­‐intensive,  cross-­‐disciplinary  and  global  collaboraLve  research.  

 

2016 National Research Infrastructure Roadmap 26

Table 2: Alignment of National Science and Research Priorities and Focus Areas

National Research Infrastructure

Focus Areas

National Science and Research Priorities

Food

Soil

and

Wat

er

Tran

spor

t

Cybe

r Sec

urity

Ener

gy

Reso

urce

s

Adva

nced

M

anuf

actu

ring

Envi

ronm

enta

l Ch

ange

Hea

lth

0110

Digital Data and eResearch Platforms 3 3 3 3 3 3 3 3 3

Platforms for Humanities, Arts and Social Sciences 3 3 3 3 3

Characterisation 3 3 3 3 3 3 3

Advanced Fabrication and Manufacturing 3 3 3 3 3

Advanced Physics and Astronomy 3 3 3 3 3 3

Earth and Environmental Systems 3 3 3 3 3 3

Biosecurity 3 3 3 3

Complex Biology 3 3 3 3 3 3 3

Therapeutic Development 3 3 3

nci.org.au

Focus  Area  

2016 National Research Infrastructure Roadmap 32

Consideration should be given to extending the network into regional areas where commercial services are not available and not likely to expand into these areas. Facilities and people in regional and remote areas are generating increasing amounts of data of potential interest to researchers working in areas such as precision agriculture and resource management. This will further enhance state and territory based monitoring. Without appropriate network access the value of this data may not be fully exploited.

Access and Authentication

Australia’s access and authentication infrastructure should be extended further to provide additional access to international researchers, where possible. Connecting the AAF to the rest of the world is the next step for Australia’s national authentication service for research and education. Implementation will connect Australian researchers with their counterparts across the globe, and allow international collaboration partners to access Australia’s national research infrastructure.

Australia’s ongoing participation in the global initiative eduGAIN22 will progress international access for researchers and make international collaborations much easier. This should include consideration of both authentication and authorisation.

Integrated Data-Intensive Infrastructure

Australia has the opportunity to consolidate the gains of the past decade and create a more integrated, coherent and reliable system to deal with the various needs of data-intensive, cross-disciplinary and global collaborative research. An Australian Research Data Cloud would build on existing eResearch infrastructure to create a cohesive, seamless experience for researchers that provides a fully integrated system.

The Australian Research Data Cloud should broadly align with the European Open Science Cloud and other global initiatives. It should support research data management from creation and discovery, through description and provenance, integration and storage, manipulation and analysis, and preservation. This improves the quality, reliability, durability, and accessibility of data, ensuring the outputs of research are more transparent. It should provide digital platforms that meet specific research requirements and integrate other data rich research infrastructure. It should support the sharing of informatics and software techniques to enable the deployment and wide use by researchers.

The underpinning Australian eResearch infrastructure should include cloud computing, HPC, networks, access, authentication and trusted data repositories. Data, collaboration and software services, skills and knowledge provided by the Australian Research Data Cloud will be an essential part of the new system.

Table 3: Priority Areas for National Research Infrastructure - Digital Data and eResearch Platforms

Elements National Research Infrastructure response

Tier 1 HPC Enhance existing national HPC – NCI and Pawsey.

Explore governance integration of these Tier 1 HPC facilities.Create Australian Research Data Cloud

Enhance existing capability through the integration of existing capability – ANDS, NeCTAR and RDS to establish an integrated data-intensive infrastructure system, incorporating physical infrastructure, policies, data, software, tools and support for researchers.

Research networks Enhance the capability and capacity of the AREN.Access and authentication Enhance capability and international relationships in access,

authentication and authorisation services.

22. eduGAIN simplifies access to content, services and resources for the global research community. [online] Available at: http://www.geant.org/Services/Trust_identity_and_security/eduGAIN/.

nci.org.au

With  limited  funds  where  do  you  spend  the  MONEY?    

TEAM  HPC   TEAM  Cloud  

nci.org.au

Random  comparisons….  

2013-2015 Nectar NCICapital 9,715,657.00$ 26,000,000.00$Co-investment 12,247,152.00$ 30,000,000.00$

56% 54%21,962,809.00$ 56,000,000.00$

Numberofcores 30372 57,000Costpercore 723.13$ 982.46$Percoreperyear 241.04$ 327.49$

Numbervcpus 30372 114000Costpervcpu 241.04$ 163.74$

Numberofresearchers 10,000 4,000Active 3935 1500

Costperactiveresearcher/yr 1,860.47$ 12,444.44$

numbervCPuperresearcher 8 761/2dualcorePC 4.75

@1500perPC 750.00$ 7,125.00$

nci.org.au

Investment  Decision    -­‐  Bang  for  Buck  

 According  to  the  July  2012  ISACA  White  Paper:  Calcula8ng  Cloud  ROI:  From  the  Customer  Perspec8ve      Return  on  Investment  (ROI)  is  just  one  of  several  financial  metrics  available  to  esLmate  the  financial  outcome  of  business  investments    Others  include  –    TCO  (Total  cost  of  Ownership)    

       NPV  (Net  Present  Value)            IRR  (Internal  rate  of  return)    

 

© 2 0 1 2 I S A C A . A L L R I G H T S R E S E R V E D .

CALCULATING CLOUD ROI: FROM THE CUSTOMER PERSPECTIVE

7

ROI =(Gain From Investment – Cost of Investment)

Cost of Investment

Figure 2—Formula to Calculate Simple ROI

For example, the ROI for a new cloud-based application (SaaS) that is expected to have an investment of US $600,000 over a period of five years and provide benefits (cost savings and new revenue) of $900,000 over the same period of time will yield a return of 50 percent.

ROI = = 50%$900,000 - $600,000

$600,000

ROI calculations used as the only financial measurement for decision making do not help predict the likelihood of realizing the return or the risk involved with a particular investment. Ideally, the enterprise will use multiple financial metrics (e.g., TCO, net present value [NPV], internal rate of return [IRR], payback period) in considering whether to adopt cloud computing services.

TCO is different from ROI because it accounts only for the cost associated with an acquisition for its entire life span or a period of time determined for the calculation. NPV compares anticipated benefits and costs over a predetermined time period using a rate that helps calculate the present value of future cash flow transactions. IRR is a variant of NPV used to find the discount rate that would make the NPV of the investment equal to zero. TCO, NPV and IRR are more significant and complex calculations; therefore, they require additional data and variables for their calculation. ROI’s simplicity makes it a more popular term to use in marketing materials and project analysis. Additional information about these financial terms and their formulas is found in Appendix A.

For investments that have clear and quantifiable benefits and costs that are easily known, the ROI calculation is simple. However, for more complex investments such as cloud computing services, the ROI calculation can be complex and misleading. Generating a meaningful result is dependent on accounting for all quantifiable variables and defining a clear and consistent time period. Intangible benefits and risk may not be included in the calculation unless the business is able to assign a value based on historical or statistical data. Investments based solely on business objectives may be better justified using a business case supported by multiple financial metrics.

Cloud BenefitsThe cloud promises a range of benefits that include the ability to shift cost from capital to operational expenses, lower overall cost, greater agility and standardization, the ability to shift IT resources to higher-value-added activities, improve employee satisfaction and competitive advantage. Some of these benefits are quite subjective and, therefore, are difficult to include in financial (mathematical) calculations.

Generating a meaningful result is dependent on accounting for all

quantifiable variables and defining a clear and consistent time period.

nci.org.au

What  is  the  Gain?  

To  calculate  the  ROI  we  need  to  know  two  things:    1)  The  investment  cost        =  IniLal  capital  costs  +  co-­‐investment      

 2)  The  Gain  from  the  Investment      =  value  of  research  outcomes    

 How  do  you  measure  the  value  of  research  outcomes?    Unfortunately….  “Since  lifle  is  known  about  the  period  of  Lme  necessary  for  economic  benefit  to  be  derived,  any  pafern  uLlized  becomes  a  best  guess.”    

Micheal  Preuss  Research  Management  Review,  Volume  21,  Number  1  (2016)      

nci.org.au

Derived  Benefits    

 He  goes  on  to  say…      “ConsideraLon  of  return  on  investment  has  a  place  in  the  grant  world,  even  though  fiscal  ROI  should  be  viewed  as  a  tool  that  is  limited  in  accuracy,  scope,  and  applicability…...  Toward  this  end,  the  benefits  derived  perspecLve  on  ROI  should  become  the  preferred  perspecLve  in  a  grants  context.”    

So  what  are  some  of  the  derived  benefits  of  the  Cloud  and  HPC?  

nci.org.au

IDC  Report  

In  2013  IDC  released  a  Special  Case  Study:  “CreaLng  Economic  Models  Showing  the  RelaLonship  Between  Investments  in  HPC  and  the  ResulLng  Financial  ROI  and  InnovaLon  —  and  How  It  Can  Impact  a  NaLon's  CompeLLveness  and  InnovaLon”      It  used  a  couple  of  macroeconomic  models  and  the  noLon  of  an  innova8on  index  to  try  and  answer  the  quesLon  –    “How  high-­‐performance  compu8ng  (HPC)  investments  can  improve  economic  success  and  increase  scien8fic  innova8on?”  The  results  showed  substan(al  return  for  investments  •  $356.5  on  average  in  revenue  per  dollar  of  HPC  invested  •  $38.7  on  average  of  profits  (or  cost  savings)  per  dollar  of  HPC  invested    

BUT..  “One  must  be  very  careful  in  using  the  data  for  making  key  decisions.  The  pilot  study  data  is  large  enough  to  show  the  major  trends  but  isn't  large  enough  to  support  detailed  economic  comparisons”      

nci.org.au

Cloud  ROI??  

What  about  the  ROI  of  the    Cloud?    Most  of  the  literature  was  aimed  at  the  CIO…  •  Cost  reducLon    •  Enhanced  producLvity    •  Scalability  •  Agility  •  Performance  •  Improved  security/compliance    •  CollaboraLon    •  Green  

Within  the  research  arena  much  of  the  literature  was  case  studies  and  followed    similar  themes  to  those  above  with  the  excepLon  of  cost  reducLon.    

nci.org.au

Head  to  Head…  

Cost  reduc(on    The  cloud  is  cheaper  …  just  like  a  car  and  a  bike.  •  Cloud  CPUs  generally  cheaper  BUT  do  less  work    

–  Run  ~15%  of  the  Lme  vs  ~85%  –  Less  powerful  =>  need  to  run  longer  on  cloud  to  get  same  output  

•  Interconnect  between  nodes  cheaper  BUT  slower  –  Takes  longer  to  load  large  data  sets  =>  Longer  wait  

Enhanced  produc(vity    •  Both  Cloud  and  HPC  accessible  from  the  network  •  With  batch  processing  researchers  using  HPC  can  “set  and  forget”    •  Admin  tasks  looked  aser  by  specialist  staff  in  PC  but  Cloud  is  osen  DYI    •   Cloud  can  provide  an  “instant”  start  as  no  queue  but  comes  at  a  price  with  need  

to  have  idle  machines  

 

nci.org.au

Head  to  Head…  cont    

Scalability    •  Both  the  Cloud  and  HPC  scale  but  in  different  ways  

–  Large  numbers  of  small  jobs  vs  Large  job  over  mulLple  machines    

Agility    •  Cloud  can  provide  an  “instant”  start  as  no  queue  but  comes  at  a  price  with  need  

to  have  idle  machines    •  Cloud  can  provide  rapid  deployment  of  different  OS  BUT  now  with  Singularity  

HPC  may  also  provide  this  

Performance  -­‐  HPC  wins  hands  down  BUT  high  spec  clouds  can  also  deliver  the  performance  for  

some  jobs        

nci.org.au

Head  to  Head…  

Improved  security  /  compliance    •  Seen  as  a  posiLve  when  compared  to  on  site  systems  BUT  a  negaLve  if  

researchers  don’t  keep  their  machines  up  to  date  in  the  cloud.    

Green  •  As  with  any  shared  system  –  HPC  and  Cloud  can  be  seen  as  green  as  researchers  

only  use  what  they  need    

Collabora(on    •  With  systems  on  the  network  –  sharing  of  data  and  applicaLons  has  become  key  

 

nci.org.au

BEST  ROI  

So  who  wins  ?    What  has  the  best  ROI?      

HPC  or  Cloud    ROI  depends  on  where  you  sit  and  who  is  asking  the  quesLon.    Actually  it’s  the  wrong  quesLon…    •   Both  are  needed  and  acknowledged  in  the  roadmap    •   Data  is  a  key  ingredient  in  delivering  collaboraLon  value      How  can  we  conLnue  to  reduce  barriers  for  researchers  to  allow  “fricLon  free”  research  in  a  cost  effecLve  way?        

nci.org.au

From  the  Roadmap  again..  

The  current  HPC  environment  has  evolved  to  encompass  the  needs  of  big  data  (processing,  analysis,  data  mining,  machine  learning),  in  addiLon  to  its  tradiLonal  role  of  computaLonal  modelling  and  simulaLon.  The  contemporary  environment  comprises  Lghtly-­‐integrated,  high-­‐performance  infrastructure  able  to  handle  the  computaLonal  and  data-­‐intensive  workfows  of  research,  together  with  experLse  in  computaLonal  science,  data  science  and  data  management.    

nci.org.au

Focus  Area  

2016 National Research Infrastructure Roadmap 32

Consideration should be given to extending the network into regional areas where commercial services are not available and not likely to expand into these areas. Facilities and people in regional and remote areas are generating increasing amounts of data of potential interest to researchers working in areas such as precision agriculture and resource management. This will further enhance state and territory based monitoring. Without appropriate network access the value of this data may not be fully exploited.

Access and Authentication

Australia’s access and authentication infrastructure should be extended further to provide additional access to international researchers, where possible. Connecting the AAF to the rest of the world is the next step for Australia’s national authentication service for research and education. Implementation will connect Australian researchers with their counterparts across the globe, and allow international collaboration partners to access Australia’s national research infrastructure.

Australia’s ongoing participation in the global initiative eduGAIN22 will progress international access for researchers and make international collaborations much easier. This should include consideration of both authentication and authorisation.

Integrated Data-Intensive Infrastructure

Australia has the opportunity to consolidate the gains of the past decade and create a more integrated, coherent and reliable system to deal with the various needs of data-intensive, cross-disciplinary and global collaborative research. An Australian Research Data Cloud would build on existing eResearch infrastructure to create a cohesive, seamless experience for researchers that provides a fully integrated system.

The Australian Research Data Cloud should broadly align with the European Open Science Cloud and other global initiatives. It should support research data management from creation and discovery, through description and provenance, integration and storage, manipulation and analysis, and preservation. This improves the quality, reliability, durability, and accessibility of data, ensuring the outputs of research are more transparent. It should provide digital platforms that meet specific research requirements and integrate other data rich research infrastructure. It should support the sharing of informatics and software techniques to enable the deployment and wide use by researchers.

The underpinning Australian eResearch infrastructure should include cloud computing, HPC, networks, access, authentication and trusted data repositories. Data, collaboration and software services, skills and knowledge provided by the Australian Research Data Cloud will be an essential part of the new system.

Table 3: Priority Areas for National Research Infrastructure - Digital Data and eResearch Platforms

Elements National Research Infrastructure response

Tier 1 HPC Enhance existing national HPC – NCI and Pawsey.

Explore governance integration of these Tier 1 HPC facilities.Create Australian Research Data Cloud

Enhance existing capability through the integration of existing capability – ANDS, NeCTAR and RDS to establish an integrated data-intensive infrastructure system, incorporating physical infrastructure, policies, data, software, tools and support for researchers.

Research networks Enhance the capability and capacity of the AREN.Access and authentication Enhance capability and international relationships in access,

authentication and authorisation services.

22. eduGAIN simplifies access to content, services and resources for the global research community. [online] Available at: http://www.geant.org/Services/Trust_identity_and_security/eduGAIN/.

nci.org.au

 Best  PracLce?  

The  answer  has  to  be  that  our  Cloud  and  HPC  systems  need  to  research  driven….      We  need  to  be  focused  on  reducing  costs  and  delivering  the  best  bang  for  the  buck.      This  implies  aggregaLon  of  services,  reduced  overheads….    This  can  only  happen  when  we  have  HPC  +  Cloud  +  Data      Just  like  hospitals  –  we  need  aggregaLon  of  services  to  make  it  easier  for  users  but  equally  we  can’t  have  just  a  single  site.    Perhaps  we  can  take  the  lead  from  others  in  this  space…  The  Canadian  Academy  of  Health  Sciences  undertakes  an  evaluaLon  of  research  for  three  main  purposes:  accountability  purposes,  advocacy  purposes,  and  learning  purposes.      More  work  is  needed  to  agree  how  to  measure  these….      

nci.org.au nci.org.au

@NCInews

QuesLons?