Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre...
-
Upload
laura-hopkins -
Category
Documents
-
view
217 -
download
2
Transcript of Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre...
![Page 1: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/1.jpg)
Climate models and open source software
Andy Pitman and Steven PhippsClimate Change Research Centre
University of New South Wales
![Page 2: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/2.jpg)
Background
• The Earth’s climate is changing
![Page 3: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/3.jpg)
Background
260-280 ppmv over10,000 years
385 ppmv – increasedfrom 280 in ~100 years
![Page 4: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/4.jpg)
Temperature
• Continued greenhouse gas emissions at or above current rates would cause further warming - very likely larger than those observed during the 20th century.
20052006
20072008
![Page 5: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/5.jpg)
How do we know ?
• Observations– radiation changes– temperature changes– pressure, humidity, acidity etc
• Basic theory
• Backs of envelopes … and
• Climate models
![Page 6: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/6.jpg)
Climate models
• Mathematical representations of– Atmosphere– Ocean– Terrestrial surface– Cryosphere– Fluxes of energy, mass and carbon between
these systems
![Page 7: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/7.jpg)
Climate ModellingGoverning equations
Forcing conditions
Initial conditions Model
output
![Page 8: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/8.jpg)
Three-DimensionalClimate Models(GCM)
![Page 9: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/9.jpg)
Climate models
• ~300 x 300 km grid
• ~20 vertical levels in atmosphere, 30 in ocean– 300,000 grid points
• Solves basic dynamics equations, parameterizes physics
• Solve all equations with a ~15 minute timestep for 200 year integrations
• Something like 2 x 1018 calculations per run
![Page 10: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/10.jpg)
Climate models work
Rainfall [annual]
Which is observed and which is modelled ?
![Page 11: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/11.jpg)
Climate models
![Page 12: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/12.jpg)
Climate models
• Written in Fortran – Mainly f95 or f90, some f77– best language [efficiency and safety]– python, c, other wraps– Mostly written by scientists not software engineers
• Cannot [usefully] re-write in other languages
• Run on NCI, or similarly large systems – main rationale for a Pflop system in Australia
• Generate ~5 Tb per run. Data sets now > 1 Pbyte
![Page 13: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/13.jpg)
Model Nationality Open source ?BCC China
BCCR Norway
CCSM USA
CGCM Canada
CNRM France
CSIRO Aus
ECHAM Germany
ECHO-G Germany
FGOALS China
GFDL USA
GISS USA
INM Russia
MIROC Japan
MRI Japan
PCM USA
UKMO UK
![Page 14: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/14.jpg)
Model Nationality Open source ?BCC China
BCCR Norway
CCSM USA
CGCM Canada
CNRM France
CSIRO Aus
ECHAM Germany
ECHO-G Germany
FGOALS China
GFDL USA
GISS USA
INM Russia
MIROC Japan
MRI Japan
PCM USA
UKMO UK
![Page 15: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/15.jpg)
Model Nationality Open source ?BCC China
BCCR Norway
CCSM USA
CGCM Canada
CNRM France
CSIRO Aus Via license, never latest versionECHAM Germany
ECHO-G Germany
FGOALS China
GFDL USA
GISS USA
INM Russia
MIROC Japan
MRI Japan
PCM USA
UKMO UK Via license, never latest version
![Page 16: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/16.jpg)
Model Nationality Open source ?BCC China
BCCR Norway
CCSM USA
CGCM Canada
CNRM France
CSIRO Aus Via license, never latest versionECHAM Germany
ECHO-G Germany A variant may be availableFGOALS China
GFDL USA A variant may be availableGISS USA
INM Russia
MIROC Japan
MRI Japan
PCM USA
UKMO UK Via license, never latest version
![Page 17: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/17.jpg)
Model Nationality Open source ?BCC China
BCCR Norway
CCSM USA Yes – fully accessibleCGCM Canada
CNRM France
CSIRO Aus Via license, never latest versionECHAM Germany
ECHO-G Germany A variant may be availableFGOALS China
GFDL USA A variant may be availableGISS USA Yes – fully accessibleINM Russia
MIROC Japan
MRI Japan
PCM USA
UKMO UK Via license, never latest version
![Page 18: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/18.jpg)
Why ?• Not resourced or requested to provide this code in suitable standard
• Costs of supporting open source– hardware compatibility – complier variability, – user support problems– Code maintenance– Dispute resolution/managing expectation
– It’s a myth that climate modeling groups are well resourced …
• Many climate models are variants of forecasting systems – commercial value
![Page 19: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/19.jpg)
Why ?• Groups sometimes associated with defense departments
• Culture
• Fear of inappropriate use
• IP concerns
• Unwillingness to be exposed to independent evaluation
• Strategic issues with best models
• Why would you provide open source ?
![Page 20: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/20.jpg)
So why open source?
• Bugs per 1000 lines of code:– Commercial software:
• ~1-7 [Jones, 2000]• ~20-30 [CyLab, 2004]
– Open source software:• 0.17 [Linux kernel, Coverity, 2004]• 0.434 [open source projects, Coverity, 2006]
• So open source more reliable; better
![Page 21: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/21.jpg)
So why open source?
• A state of the art climate model has:
• ~1 million lines of code
• a software development lifecycle that lacks:– external testing, – comprehensive internal testing – error checking. – Analagous to commercial software?
• contains ~1000-30,000 bugs?
![Page 22: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/22.jpg)
So why open source?
• National Institute of Standards & Technology, USA, 2002
– software errors cost US economy US$59.5 billion/yr
– Despite industry practice of spending 80% of development costs on identifying and correcting faults
– Most modeling groups would not have one person dedicated to identifying and correcting faults
• An open source climate model has a community dedicated to identifying and correcting faults
![Page 23: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/23.jpg)
The climate model challenge
• More components
• Longer simulations
• Finer spatial and temporal resolution
• Serious computational challenges
• 6 months to port to new computer system
• Super computer time valuable
![Page 24: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/24.jpg)
What I want … • model test plans with pre-defined performance
– Software developers know how …
• Code developed in a F95 via subversion– You are welcome to try
• Simple and elegant scripting– Suggestions welcomed
• Tools to slice and dice netCDF– Talk to the modelers – embed in R
• Enhancements to openDAP …
• Implementation of agreed coding standards
![Page 25: Climate models and open source software Andy Pitman and Steven Phipps Climate Change Research Centre University of New South Wales.](https://reader035.fdocuments.in/reader035/viewer/2022062422/56649f385503460f94c54f38/html5/thumbnails/25.jpg)
Summary
• Climate models have been developed over 20 years – with IT systems engineers in some cases
• Few are open source – those that are were commonly developed with IT engineers
• Hard to contribute – cannot ad hoc – little low hanging fruit
• But a commitment in this area can be an invaluable contribution over time. Needs to be based on a dialog with the modelers