Tuning GENIE Earth System Model Components using a Grid Enabled

Data Management SystemAndrew Price, Gang Xue, Andrew Yool, Dan Lunt, Tim Lenton,

Jasmin Wason, Graeme Pound, Simon Cox and the GENIE team.

http://www.genie.ac.uk/

UK e-Science – All Hands Meeting

3rd September 2004

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 2/22

Outline

• Introduction• Scientific aims of GENIE• e-Science tools

– Data Management System– Geodise Toolboxes– OPTIONS Design Search and Optimisation

• Results• Future work• Conclusions

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 3/22

Introduction

The GENIE project is developing a Grid-based system to:

• Flexibly couple together state-of-the-art components to form a unified Earth system model

• Execute the resulting model on the Grid

• Share the distributed data produced in simulations

• Provide high-level open access to the system, creating and supporting virtual organisations of Earth system modellers

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 4/22

Scientific Aims

ice-age ice-ageice-ageice-age

• Orbital parameters affect incident radiation and climate• Biological and geological processes interact with, and

feedback upon, the climate (via, for instance, CO2)

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 5/22

The target GENIE Model

3D atmosphere

3D ocean

2D sea ice

AtmosphericCO2

2D land surface

Land vegetation

Ocean biogeochemistry

Ocean sediments

3D ice sheets

Atmosphere – Bristol’s IGCM3Ocean – SOC’s GOLDSTEIN

Land – Met. Office’s TRIFFIDLand ice – Bristol’s GLIMMER

Ocean biogeochemistry andsediments – UEA’s BioGEM

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 6/22

Initial GENIE experiments

• Initial studies in GENIE performed parameter sweeps to investigate the properties of the model

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 7/22

e-Science Tools

• Data Management System (augmented version of the Geodise Database System)

• Matlab scripting environment• Geodise Toolboxes• XML Toolbox• OPTIONS Design Search and Optimisation

package• Template and Example scripts

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 8/22

Data Management System

JavaClient Code

Apache Axis

CoG

JythonFunctions

Globus Server

Geodise Database Toolbox

Metadata Database

Client Grid

SOAP

MatlabFunctions

DatabaseWeb Services

AuthorisationService

LocationService

Metadata Archive & Query

Services

Jython

XML Schema

GridFTP

Portal

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 9/22

Grid Computation

National Grid Service (GT2)Oxford Leeds

RAL Manchester

Jython

Local Resources (GT2)

Java CoG

Imperial Condor PoolSouthampton Condor Pool

Flocked Condor Pools

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 10/22

Geodise Toolboxes

Geodise Compute Toolbox

gd_createproxy.m Creates a Globus proxy certificate for the user's credentials

gd_destroyproxy.m Destroys the local copy of the user's Globus proxy certificate

gd_jobsubmit.m Submits a compute job to a Globus GRAM job manager

gd_jobstatus.m Gets the status of a Globus GRAM job

gd_putfile.m Puts a remote file using GridFTP

gd_getfile.m Retrieves a remote file using GridFTP

gd_rmfile.m Deletes a remote file using GridFTP

gd_makedir.m Creates a remote directory using GridFTP

gd_rmdir.m Deletes a remote directory using GridFTP

Geodise Database Toolbox

gd_archive.m Archives a file or data structure to the database

gd_query.m Query the database for data matching specified criteria.

gd_retrieve.m Retrieves a file or data structure from the database

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 11/22

Scripting a Tuning Study

GENIE Database

Grid Resource

MATLAB

function RMS_Error = cgoldstein(params)

config file

results file

CG binary

return RMS_Error

optimum = fminsearch( … @cgoldstein, params, … )

gd_query(results)

gd_putfile(CG binary)

gd_putfile(config file)

gd_jobsubmit(RSL)

gd_getfile(results file)

gd_archive(results)

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 12/22

Matlab Optimisation Toolbox% ************************% Specify a starting point% ************************parameters = [ 0.5 ];

% ************************% Perform the minimisation% ************************optimum = fminsearch( @cgoldstein_1D, parameters, optimisation_parameters )

% ************************% Specify a starting point% ************************parameters = [ 420 5000000 ];

% ************************% Perform the minimisation% ************************optimum = fminsearch( @cgoldstein_2D, parameters, optimisation_parameters )

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 13/22

OPTIONS

• Matlab interface to the Options design exploration system– http://www.soton.ac.uk/~ajk/options/welcome.html

• State of the art design search and optimisation algorithms– Design of Experiment methods– Response Surface Modelling– Over 30 search methods including:

• Adaptive Random Search (ADRANS), Powell's Direct Search (PDS),• Simplex Method (SIMP), Genetic Algorithm (GA),• Simulated Annealing (SA), Evolutionary Programming (EP)

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 14/22

Grid Computation

OptionsMatlab

National Grid Service (GT2)

Oxford Leeds RAL Manchester

Local Resource (GT2)

GENIE Database

objfun.m objfun_parse.m

optjobparallel.m

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 15/22

OptionsMatlab

>> OptionsInput = createOptionsStructure(4.0)

DNULL: -777

OLEVEL: 2

MAXJOBS: 100

NVRS: 12

VNAM: {'SCLTAU' 'INVDRAG' 'OCNHORZDF' ... }

LVARS: [1.3000 2.0000 2500 ... ]

UVARS: [2.1000 4.8000 5700 ... ]

VARS: [1.7000 3.4000 4100 ... ]

ONAM: 'RMSERROR'

OMETHD: 4.0000

DIRCTN: -1

NITERS: 1000

OPTFUN: 'cgoldstein_12D'

OPTJOB: 'optjobparallel'

GA_NPOP: 100

>> OptionsOutput = OptionsMatlab(OptionsInput);

Available Optimisation Methods:

1.1 for OPTIVAR routine ADRANS 1.2 for OPTIVAR routine DAVID 1.3 for OPTIVAR routine FLETCH 1.4 for OPTIVAR routine JO 1.5 for OPTIVAR routine PDS 1.6 for OPTIVAR routine SEEK 1.7 for OPTIVAR routine SIMPLX 1.8 for OPTIVAR routine APPROX 1.9 for OPTIVAR routine RANDOM 2.1 for user specified routine OPTUM1 2.2 for user specified routine OPTUM2 2.3 for NAG routine E04UCF 2.4 for bit climbing 2.5 for dynamic hill climbing 2.6 for population based incremental learning 2.7 for numerical recipes routines 2.8 for design of experiment based routines 3.11 for Schwefel library Fibonacci search 3.12 for Schwefel library Golden section search 3.13 for Schwefel library Lagrange interval search 3.2 for Schwefel library Hooke and Jeeves search 3.3 for Schwefel library Rosenbrock search 3.41 for Schwefel library DSCG search 3.42 for Schwefel library DSCP search 3.5 for Schwefel library Powell search 3.6 for Schwefel library DFPS search 3.7 for Schwefel library Simplexsearch 3.8 for Schwefel library Complexsearch 3.91 for Schwefel library two membered evolution strategy 3.92 for Schwefel library multi membered evolution strategy 4 for genetic algorithm search 5 for simulated annealing 6 for evolutionary programming 7 for evolution strategy

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 16/22

Twin-Test Experiment

Attempt to recover a known state of the model using a Genetic Algorithm.

Performed 10 generations of a 100 member population. Then applied a local Simplex search of the best candidate.

Population too small to find optimal solution – suitable for finding local minima

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 17/22

Tuning using Observational Data

Model Sea Surface Temperatures

NCEP Sea Surface Temperatures

Model Air Temperatures

NCEP Air Temperatures

• Apply the same method but calculate the RMS error statistic by comparing the model state with NCEP observational data.

• The lack of a land surface in the model means tuning cannot match the observational data.

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 18/22

IGCM3 Atmosphere Model

iii

cellscells

ncepi

cellscells

ncepi

obsi

i

xfwxobjfun

N

Areax

N

Areaxx

xf2

2

• The objective function is a weighted sum of the RMS differences between the model state and NCEP data.• Winter and Summer averages for a number model fields.

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 19/22

IGCM Results

• 25% reduction in error statistic compared to default parameters

• Similar result to a parallel study performed using the Ensemble Kalman Filter

• Model physics insufficient to perfectly match observational data.

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 20/22

e-Science Summary

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 21/22

Conclusions

• Provided the environmental scientist with a toolset for tuning GENIE models:– Scripting environment– Database repository– Computational Grid interface– Suite of generic optimisation algorithms

• A Global minimum can reliably be found in low dimensional problem space.

• For higher dimensional problems, the tools are appropriate for locating local minima in the state space.

03/09/04 UK e-Science - All Hands Meeting, Nottingham, 2004 22/22

The GENIE Team

Coordinator: Tim Lenton – CEH Edinburgh

Principal investigator:Paul Valdes – Bristol

Research Team and Collaborators:James Annan – FRSGC, Japan Chris Brockwell – UEA Norwich

David Cameron – CEH Edinburgh

Peter Cox – Hadley Centre (UKMO)

Neil Edwards – Bern, Switzerland

Murtaza Gulamali – London e-Science Centre Julia Hargreaves – FRSGC, Japan Phil Harris – CEH Wallingford

Dan Lunt – Bristol

Bob Marsh – SOC

Andrew Price – Southampton e-Science Centre

Andy Ridgwell – UBC, Canada

Ian Rutt – Bristol

Gang Xue – Southampton e-Science Centre

Andrew Yool – SOC

Management Team:Melvin Cannell – CEH Edinburgh

Trevor Cooper-Chadwick – Southampton e-Sci. Centre

Simon Cox – Southampton e-Sci. Centre

John Darlington – London e-Science Centre

Richard Harding – CEH Wallingford

Tony Payne – Bristol

John Shepherd – SOC

Andrew Watson – UEA Norwich

Thanks toSteven Newhouse