Contracting out Modelling Projects to Consultants Setting a Framework for Success Dr S A Joynes,...

Contracting out Modelling Projects to Consultants

Setting a Framework for Success

Dr S A Joynes, Golovin, NZ

a business dedicated to best practise in water resource modelling

Introduction

Making the intangible, tangible

Simplifying the complicated


Modelling Education

There is no formal academic teaching It is learned on the job Training only by software vendors Not recognised (yet) by IPENZ Guidelines, handbooks and software

manuals


Modellers Expertise

Based on who they worked for What software packages they have

the opportunity to use What training courses attended

There is NO formal qualification or training programme


Water Modelling Today Majority of infrastructure developments

require water modelling input $Millions spent of water modelling Graduates thrown straight into

“sophisticated” modelling Production driven NOT quality driven Most consultancies have modellers Most Council's have modellers


Standards and Guidelines The New Zealand Standard for Managing Flood Risk – A

Process Standard (NZS 9401:2008) (Appendix 2). Surface Water, Document E1, Building Industry Authority Roughness characteristics of NZ rivers, NIWA,

September 1998, ISBN 0-477-02608-7 Guidelines for stormwater runoff modelling in the

Auckland Region, Auckland Regional Council, 1999, TP108

WaPUG Integrated Urban Drainage Modelling Guide STOWA Good Modelling Practise Handbook Water New Zealand Water Supply Modelling Guidelines Water New Zealand Wastewater Modelling Guidelines


Questionnaire – NZ Asset Managers

Does the Council have a standard methodology when reviewing the results of models within reports?

92% said No

Could the standard of modelling outputs provided by 3rd parties be improved to assist you in your interpretation for decision-making?

100% said Yes

What percentage of applications needed requests for further information due to the lack of clarity of modelling inputs/outputs?

Average score of 50%

Do you undertake due diligence on software used in hydraulic analysis to ensure its fit-for-purpose whether used in house or by 3rd party applicants?

66% Never 25% Sometimes8% said “??”


Example

Two different hydrological methods TOPNET & HEC-HMS used for change in land-use

TOPNET produced flows that caused flood levels to rise by 30mm

HEC-HMS produced flows that caused flood levels to rise by 270mm

Which one do you believe?


From a Senior Consultant in a Major Company

“It’s predominantly hydraulic modelling projects that go over budget ”


Is this a typical experience? 1. Inability of consultants to assist clients with a clear

definition of the scope of works, and to provide reasonable cost and time estimates for a staged approach to deliver appropriate information for decision-making. Very often, inclusions, exclusions, assumptions and limitations are not stated nor the scale of consequences quantified. Costs and timely delivery keep spiralling up, due to lack of clarity and forethought.

2. Inability of consultants to quickly assess the limitations with available data so that clients are provided with the scale of uncertainties with expected model outputs.

3. Lack of confidence by consultants in using the modelling outcomes for decision-making. Model verification and calibration needs needs to be viewed relevant to the scale of outputs required.

Auckland Region Asset Manager


Shortfalls No Structure Inconsistent use of guidelines Reliance on blackbox No industry-wide training Lack of information and clarity in

reports Budget control, spiralling costs Modellers inability to assist with

solutions


New Paradigm Firstly, what is the objective of any process that needs to

address these issues?

“A process that enables the modeller to respond to the needs of the client in a specific tangible manner that can then be measurable, transparent and transferable for future use.”


Characteristics Universal Comprehensive and comprehensible Teachable from management to technical level All modellers (experienced and inexperienced)

will be at the same communication level An over-arching umbrella for territorial laws,

codes and specifications and industry specifications

Creates accountability for quality Ensures client gets what client wants Strong communication tool


Structure


Creation of Modelling Policy Statement

A document that is

Specification Project Management Tool Quality Standard Document A future planning tool


Organise

The Thinking & Research Stage

The Client’s Job


Organise - elements

Target audience Standards Time horizon Experience requirement Expectations and “what success will look

like” Ownership and governance


Formalise

The point of greatest planning – and therefore benefit

The Design Stage

Client 80%, Modeller 20%


Formalise

Survey requirements and needs Catchment elements (area, slope, soils,

land-use) Rainfall inputs Calibration data quality (flow gauges,

water level gauges, rainfall gauges)


Implement

The Doing Stage

“Line in the Sand Reports” Processes

Client 10% , Modeller 90%


Implement - elements

Methodology statement Assumptions used Report of data used Calibration feedback Decision-making process


Monitor

The Review, Future Planning and Closing Stage

Client (and advisor) 80%, Modeller 20%


Monitor - elements Peer Review Process What improvement programmes need to

be considered? Do you need more or less modelling

expertise? Resource requirements Where is the model to be housed? How is the model to be updated?


Summary Hydraulic modelling has moved rapidly in the

past 25 years. However this does not mean the quality of the work has kept up with technological progress.

Clients experience a myriad of issues when procuring modelling work, often leading to cost over-runs, time delays, repetition and lost information.

A survey of asset managers highlighted that there is no standard methodology for checking modelling results and that 50% of consent reports require additional information due to the poor clarity of modelling results.

There are a number of guidelines for modelling but they can be too specific to the discipline or they do not clearly state the parameters.


Conclusion A Modelling Policy Statement Structure as been

developed It is a “live” document that responds to the needs

of the project Allows the client to clearly communicate what

they want Supports consultants to respond positively to

client needs Creates strong communication and collaboration Provides for an easier peer review process

(probably reduces its need) Ensures modellers are committed to project

outcomes – no excuses

The End

Contracting out Modelling Projects to Consultants Setting a Framework for Success Dr S A Joynes,...

Documents

Transcript of Contracting out Modelling Projects to Consultants Setting a Framework for Success Dr S A Joynes,...