Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service

Challenges in Extending, Testing, and Delivering FEWS

Lee Cajina National Weather Service, Office of Hydrologic Development,

U.S.A 2014 FEWS-User Days (10/28/2014)

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 1

Presentation Outline

• What is the Community Hydrologic Prediction System (CHPS) and How is it Related to FEWS? – Customers – Providers

• Developing Software Against FEWS – Challenges – Strategies

• Releasing Updates to FEWS – Challenges – Strategies

• Conclusion – Looking Ahead

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 2

Background • The NWS has almost completely transitioned from a mix of stand-

alone programs under the umbrella name of the National Weather Service River Forecast System (NWSRFS) to a FEWS-centric environment using a mix of functionality adapted to FEWS under the umbrella name of the Community Hydrologic Prediction Service (CHPS).

NWSRFS CHPS

Primary Drivers Were: • Future Maintenance and Support • Reducing the Time from Research to Operations

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 3

CHPS vs FEWS

1. FEWS 2. OHD-CORE-CHPS, 3. HEC-RAS, 4. HEC-RESSIM

CHPS Consists of 4 packages

OHD-CORE-CHPS, HEC-RAS, and HEC-RESSIM • Are external packages with functionality that can run within FEWS • Each has their own FEWS Adapter(s)

FEWS provides the CHPS modeling environment and primary graphical user interface

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 4

CHPS Customers

RFCs are federal entities tasked to provide fresh water-related information for the protection of lives and properties

CHPS is the primary system used at 13 NWS River Forecast Centers (RFCs) covering the US

NWS Headquarters (Silver Spring, MD)

Deltares USA (Silver Spring, MD)

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 5

CHPS Providers • NWS HQ has employees spread across two groups responsible for

the support, maintenance, enhancement and deployment of CHPS

Office of Hydrologic Development (OHD) Hydrologic Services Division (HSD) Jon Roe (Branch Chief) Mark Glaudemans (Branch Chief)

Neftali Cajina (Project Leader) Randy Rieman (Technical Support)

Mark Fresch (Project Leader) Bradley Mccuune (Technical Support)

Russ Erb (Content Manager)

Hank Herr (Senior Developer)

Cham Pham (Senior Developer)

Raymond Chui (Support Focal Point)

Xuning Tan (Junior Developer)

Gautam Sood (Build and Configuration Leader)

Only Federal Employees shown, NWS HQ has a team of supporting contractors also

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 6

CHPS - Providers • Office of Hydrologic Development – Roles

– A CHPS Software Development Organization • Develop and maintain software designed to run within FEWS

– Manage CHPS Deployment, • Includes updates to FEWS and the three other packages adapted to it • Monitor the progress of the other development organizations and coordinate beta testing

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 7

Developing Software Against FEWS

Type of Functionality Technology Used to Adapt to FEWS

Physical Process Hydrologic Models FEWS General Adapter

Flash Flood Guidance Algorithms FEWS General Adapter + FEWS Transformations

Graphical User Interface for Data and Product Preparation

FEWS PI service and GUI plugins

Automated Data Retrieving FEWS PI service

Customized Data Analysis FEWS API + FEWS Transformations

A complete list is available here: http://www.nws.noaa.gov/oh/hrl/general/indexdoc.htm#core

• Excluding some reservoir and channel routing (covered by HEC-RESSIM or HEC-RAS), OHD-CORE-CHPS consists of River Forecasting functionality used at RFCs not included in FEWS, but instead adapted to FEWS

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 8

Extending FEWS • OHD-CORE-CHPS makes use of the many approaches FEWS

provides to extend it with native functionality

executable call - no adapter

Java Adapter calls Java Program

Java Adapter calls C/Fortran executable

pi-service – no adapter

JAVAGUI plugin uses pi-service

JAVA CUSTOM FEWS API

FEWS

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 9

Extending FEWS

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 10

Extending FEWS - Challenges • Challenges for OHD related to extending FEWS include:

– Change in functionality • Even when it’s the right thing (i.e. a bug was fixed)

– Changes in shared code • Missing modules (e.g. deprecation) • Change in module signature

– Undocumented or Incompletely Documented Features • Schema Helps

– Changes in agreed upon data interface • Missing elements – This is not usually a problem

– Netcdf changes – Pi-XML changes

• Unknown changes – These are not always bad, but we don’t learn about enhancements until later – PI-version changes (e.g. “forecastDate”)

– Performance • Our users were spoiled by Fortran/C programs using custom formatted database files

– Maintenance • We often hear about the number of configuration files to maintain – “There’s Recent Improvements”

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 11

Extending FEWS -Strategies • To address challenges, OHD uses:

– Regression Testing • Quantitative • Qualitative • Performance

– Test-Driven Development – Bi-Weekly Meetings with Deltares, USA – Code Re-use and Code Sharing

• A majority of OHD functionality reuses a single FEWS Adapter – For example, code to read pi-xml files is common and shared

– Common CHPS software development practices

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 12

Strategies - Regression Testing • Static Data Sets (run during nightly builds and during integration testing)

– For Quantitative Comparisons • Actual - <event date="1988-11-01" flag="0" time="18:00:00" value=".440"/>

• Benchmark - <event date="1988-11-01" flag="0" time="18:00:00" value=".440"/> ☑ – Performance Testing – “Workflow Completed in 15 minutes 30 seconds” ☑ – “GUI Robot” ☑

The Robot program is written in Java. It uses several technologies, including 3rd party Java program Sikuli(http://www.sikuli.org/),which uses other GNU open source computer vision C programs (tesseract-OCR, openCV and other programs)

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 13

Releasing Updates to FEWS - Challenges • Challenges for OHD Related to Upgrading FEWS during a CHPS

Release, include: – Shaking out bugs

• Software and bugs go hand-in-hand, but change in functionality is especially noticeable when we “skip” a FEWS build

– Required Configuration Changes • Our users often complain about maintaining configuration files. In general we’ve heard

software changes are better than required configuration changes – Dependency To FEWS

• Our schedule is dictated by FEWS releases, but in general this has not been a problem – Coordinating with the number of partners involved in a CHPS release is not always

easy • Deltares is extremely flexible here

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 14

Releasing Updates to FEWS - Strategies • To address these challenges, during a CHPS release OHD uses

– Repeatable release process using fixed milestones – Methodical beta testing aimed at minimizing the number of moving parts

• FEWS only for several weeks • Non-FEWS packages later for a few weeks

– OHD assumes the responsibility to ensure all 13 RFC setups are considered during testing • Download latest configuration for SA testing

– In-Field beta testing using a subset of RFCs • An HSEB focal point coordinates communication during beta across all participants

– Regular teleconferences during Beta Testing, using tools useful for remote collaboration (Google Docs, list serve) to:

• Tracking issues, including progress on fixes and verification of fix • Test Spreadsheets, with enumerated tests covering most of CHPS functionality

– Mutually agreed on approach (“Free” fix) to give Deltares an incentive to ensure features that “used to work” “still work” new

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 15

Conclusion • By all accounts CHPS has been a success! • Because of our switch to FEWS, the research to operations path is

better understood and based on more modern technology – As an example, a major upgrade to CHPS is planned for Early 2015, Hydrologic Ensemble

Forecasting Service (HEFS) • With greater flexibility comes other challenges

– External Dependencies – Performance – “Newness”

• OHD has implemented some techniques to address these challenges

• Some ideas on a combined solution to these challenges are: – Developing a North American CHPS/FEWS community for the purpose of sharing ideas,

workarounds, tips – Deltares posting of alpha software, for the purpose of gathering earlier feedback – Investigate ways to improve/quicken the time for posting updates to FEWS documentation – Others?

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 16

Questions

Lee Cajina National Weather Service, Office of Hydrologic Development lee.cajina@noaa.gov

This image cannot currently be displayed.This image cannot currently be displayed.This image cannot currently be displayed.

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 21

• EXTRA SLIDES

Office of Hydrologic Development Silver Spring, MD

National Oceanic and Atmospheric Administration’s

National Weather Service Slide 22

CHPS Providers • Hydrologic Services Division

– Front line CHPS support for 13 River Forecast Centers • Hardware and Software • Work with CHPS Development Organizations (Deltares USA, HSEB, RMA) for requesting

additional support • Manage the Web-based Issue Tracking System - FogBugz