Water Management Reorganization Transition Management Committee

WCDS End-State Plan Water Control Data System

July 5, 2008 Preface As part of the Northwestern Division Water Management reorganization effort initiated in 2007, a WCDS Team was established to investigate and recommend an end-state condition for the WCDS for Northwestern Division. The primary purpose of the WCDS is to provide data and information used on a real-time basis to regulate water projects for which the Corps of Engineers is responsible including flood control, power, navigation, recreation, water supply, water quality, and environmental concerns. The WCDS also shares data with partner agencies and provides water management data to project operators and to the user community including; internal users in emergency management, dam safety, hydrologic and hydraulic engineering; external users such as other federal, state, and local agencies and the general public. The WCDS Team was instructed to focus on the WCDS, specifically acting on the recommendations from the WCDS Needs Assessment Gap Analysis and re-organization recommendations dated 21 August 2007. The WCDS Team was asked to describe several aspects of the end-state including; system architecture, data processing, quality control/quality assurance, data dissemination, and documentation. The Team completed and submitted a final report in May 2008. That report, in its entirety, is contained herein. The Technical Management Committee [TMC], which was responsible for coordination and direction of the effort and approving a final end-state for WCDS has reviewed the report and the recommendations provided. Based on the information provided by the WCDS Team, the TMC has developed the end state as described in this TMC End State Plan section immediately following. The TMC end-state plan does NOT match the WCDS Team report in all aspects. The TMC end-state plan supersedes that provided by the WCDS Team. Any contradictions between the two will be resolved by the TMC.

i <WCDS EndState070508.doc

TMC End State Plan for WCDS As part of the Northwestern Division Water Management re-organization initiated in 2007, a Water Control Data System [WCDS] Team was tasked with developing an end state description for the WCDS that would serve the needs of the region. The report and appendices following describe the end-state recommendation for the system as provided by the WCDS Team. The Technical Management Committee has provided the final WCDS end-state plan in this Executive Summary. This TMC Plan includes some limited but important changes to the WCDS Team End-State Recommendation Report findings. The following conclusions were developed by the Transition Management Committee based on input from the WCDS Team. Further discussions for these recommendations are contained below.

1. The 3-node bi-directional streaming architecture as described in Alternative 4 should be implemented based on a Transition Plan which will be developed in the next phase. The recommended architecture would fulfill the needs of the Columbia Basin region reservoir regulation offices and auxiliary users such as the dam operators and partnering agencies (BPA, RFC, and others). The recommended architecture;

a. minimizes failure points for key processes, b. has an integrated active COOP, c. aligns pertinent data with the appropriate reservoir regulation office, and d. aligns tasks such as QA/QC with the most knowledgeable office.

2. The overall Management of the WCDS would generally follow the Corps Project Management Business Plan.

a. A WCDS Steering Committee shall be established to set the vision, scope, and priority of work for the Columbia Basin Regional WCDS. Permanent committee members shall consist of the Chiefs of Reservoir Regulation function at each of the four NWD offices, a rotating representative of the broader H&H community, and the WCDS Project Manager (non-voting member). The Steering Committee may include representatives from other offices for technical support and recommendations as appropriate. [WCDS Team Terminology – Steering Committee]

b. A WCDS Product Development Team [PDT] shall be identified. The PDT would consist of all staff, contractors, other Federal and State organizations that provide technical and planning products for WCDS. The PDT activities will be led by a Project Manager through Technical Leads. In order to allow for flexibility in management within each office, it is not necessary that the team involved in and supporting the WCDS be located within and supervised by the Reservoir Regulation Chief in each office. [WCDS Team Terminology – WCDS Working Group]

c. A WCDS Project Manager [PM] shall be selected to lead the WCDS PDT, coordinate and lead the WCDS Technical Leads, coordinate and implement the Steering Committee vision, and coordinate development of the budget and annual work plans for WCDS. [WCDS Team Terminology – Technical Coordination Team Lead].

ii <WCDS EndState070508.doc

d. Technical Leads [TL] shall be selected to work with the WCDS Project Manager. The TLs will lead individual projects, coordinate WCDS functional areas throughout the region, provide technical recommendations to the PM, and provide product scopes and schedules as necessary. [WCDS Team – Technical Coordination Team]

e. The Steering Committee will report to the Chief of Columbia Basin Water Management. Chief of CBWM will act as an advocate for WCDS in the greater Corps Community and report to the Water Management Board on issues of direction, funding, or concern.

3. A Master Plan shall be prepared to document the system including annual or bi-annual reviews and updates of this Master Plan. The Master Plan will be coordinated with the Missouri Basin to assure that systems are being migrated toward a convergent path. There will also be periodic In Progress Review (IPR) meetings to review the status of the implementation of the current Master Plan. The Master Plan will be used as a basis for system documentation and programmatic budget development and coordination of the needs across the region. The Master Plan will include system performance metrics to clearly describe the key performance characteristics of the system and allowing tracking of performance and customer satisfaction in meeting those goals. Examples of metrics include things like data reliability, data accuracy, response time on data corrections, speed in posting data to the system, etc.

The proposed end-state System Architecture consists of three nodes; one each in Portland, Seattle, and Walla Walla. The three nodes will be configured similarly; each having the ability to collect, process, store, and disseminate data, as well as support modeling and all other necessary functions. All nodes will use the same version of the CWMS database. All data, processes, applications, and files will be synchronized in real-time across all three nodes. Since the database and file systems are replicated at each node, the four offices will have an active COOP with each of the three nodes able to act as a backup for one or both of the other two nodes. Each node will have the ability to operate independently in a network communication failure. The proposed architecture has similarities with the Missouri Basin WCDS and provides the potential for future collaboration on system and tool development. Each regulating office is responsible for the data collection for projects they regulate. The actual execution of the data collection function can be delegated to another office or a District in the case of RCC as agreed to by the Steering Committee and documented in the WCDS Master Plan. Assignment of responsibilities for QA/QC shall be documented in the Master Plan with the overlying concept being the regulating or end user office has final responsibility for QA/QC of the projects it is responsible for regulating. This could be reservoir regulation, water quality, fisheries biology, etc. The QA/QC changes made at one location will be automatically streamed to the other locations. Data exchange with partner agencies will occur via a file transfer protocol [ftp] server with one point of access. Changes to the existing data exchange process will be transparent to our partner agencies. Data dissemination internally and to the general public will continue to be provided via web reports, queries, and by specific request. Where appropriate, existing locally and regionally developed applications will be migrated to work with the CWMS. Future application development will be coordinated throughout the region to facilitate standardization and to make use of the CWMS Application Programming Interface (API) as appropriate. HEC will be informed about regional development efforts and provided any products of interest for inclusion

iii <WCDS EndState070508.doc

in the CWMS. The WCDS system architecture and development will also be coordinated with ACE-IT to ensure it fits within the ACE-IT guidelines for Corps Water Management agreement and the overall Corps IT system since ACE-IT will be supporting major aspects of the WCDS. An important aspect of the reorganization of WCDS is the development of a new management structure. A WCDS Steering Committee will be established to manage all aspects of the Water Control Data System. The Committee will be the representative of all users of the WCDS including partnering agencies and will be responsible for setting the vision, scope, and priorities of WCDS and balancing the data needs of all the regulating offices and incorporating Corps partnering agencies needs as appropriate. The Committee will consist of five voting members; the chief of the reservoir regulation function at each of the four offices, and a rotating representative of the broader H&H community [Branch Chief or other functional area chief].The Project Manager (PM) will be a non-voting member of the WCDS Steering Committee. The Committee will report to the Chief, Columbia Basin Water Management. The Committee may also select ad hoc members at its discretion. These members might be necessary to meet short-term or long-term planning or technical needs and will typically be non-voting members. The Committee may also establish sub-Committees if special projects or major investigations are deemed to be to the advantage of the WCDS community and additional specialized members are appropriate. To implement the Steering Committee directives, a Product Development Team (PDT) will be established consisting of all staff, contractors, other Federal and State organizations that provide technical and planning products for WCDS. Of course, PDT membership is flexible depending on current work activities. Due to the range of technical specialties needed to manage and maintain the WCDS, active PDT members will be dependent upon current work requirements. The PDT will be comprised of all Division and District staff that perform work on the WCDS with regard to development, maintenance, support, and management of the system. Members of the WCDS PDT include staff that work on the WCDS both full and part time. There is no requirement that PDT members necessarily be staffed in Reservoir Regulation Offices. Overall coordination of PDT activities will be handled by a Project Manager. The primary interface to the regional WCDS for stakeholders and ACE-IT will be through the Project Manager. The PM may also be responsible for technical development depending on the individual qualifications. This PM, through a number of Technical Leads, will implement Steering Committee directives, develop and coordinate scopes and schedules, manage products, develop annual budgets, develop five-year plans, and handle other actions as required. The PM will be responsible to coordinate staffing needs through resource providers. Technical Leads will be selected for ongoing or product specific activities. The TL’s will be chosen by the PM with concurrence from the Steering Committee. TL’s will typically be functional area experts or provide significant technical input to product development. The WCDS Transition Team expects there will be a reduction in the end-state WCDS level of effort. The average estimated reduction is 4,500 man-hours (2.5 FTE) without including work that could be shifted to ACE-IT. The WCDS Team identified 13 functional areas and provided a range of estimated labor resources required for each functional area (minimum, average, and

iv <WCDS EndState070508.doc

maximum – see table 6.2 of this report). In addition to the resources, the team further categorized what portion of each function’s resources had to be in each office/node or could be in any of the offices (floating work). The level of effort estimate is dependent on ACE-IT involvement and assumes the proposed WCDS standardization would allow for more system efficiencies and would provide an increased level of service to all users. The process the TMC used to determine the end state resource requirements is as follows: The average resources estimate from Table 6.2 was used, which resulted in about 9.3 FTE’s of work. Three of the functional areas were identified that could be accomplished by ACE-IT - System administration, hardware support and Security (Information Assurance). To estimate the ACE-IT contribution, the minimum resource estimate was assumed for those functions rather than the average resource. This resulted in an additional shift of 0.7 FTE of work that is now being performed by Corps WCDS staff to ACE-IT staff. [A Final Draft Issue Resolution Report has been completed and is awaiting official approval from Corps and ACE-IT upper management. The document does not however, provide significant details on cost and staffing. Some uncertainty remains as to level of effort that might be required by Corps water management staff as opposed to ACE-IT staff.] The recommended TMC WCDS end-state plan envisions a “final” requirement of approximately 9.3 FTE [15,136 annual hours] which includes 0.7 FTE of work that is assumed shifted to ACE-IT as compared to the existing level of 11.8 FTE’s [20,803 annual hours]. The end state is assumed to be effective on 01 October 2009 [FY2010] with FY2009 as a transition year. The end-state plan distributes current and future work functions among the four district/division offices. Level of effort for work that should be done at the local water management offices is estimated and distributed as: NWP – 1.5 FTE, NWS-1.7, NWW-1.0, and NWD-2.0. Work that could be distributed among staff at these four offices totals 2.4, for a total WCDS staff at all Corps offices of 8.6 FTE. Initially, the 2.4 FTE’s of work that could be distributed to various offices will remain at the NWD office, resulting in a total of 4.4 FTE’s at NWD. The District FTE’s will increase from current levels by 0.5 per office to reflect shifting of QA/QC functions and back-up duties for system and database administration. Since the NWD office currently has 9.1 FTE involved in the WCDS, transitioning to a future WCDS staff of 4.4 FTE represents a reduction of 4.7 FTE from the current staffing. In the long-term, this reduction will be addressed through attrition. In the short-term, it will be addressed by re-assigning staff currently working on WCDS work to other duties or to new WCDS duties in other offices. The WCDS work should be aligned with staff members having the appropriate skill sets regardless of office affiliation. As the skill set for a particular function migrates from one office to another the work should migrate in concert. (e.g. If the primary staff member leaves the backup member, regardless of their location, takes over as primary and a new backup is designated.) A smooth transition plan from existing skill sets, work load, and staff distribution to the end-state is essential and will be considered in the Transition Plan which will follow after approval of this end-state.

v <WCDS EndState070508.doc

Respectfully Submitted: Technical Management Team [Barton, Buchholz, Brettmann, Duffe, Lindgren, Mahar]

vi <WCDS EndState070508.doc

WCDS Team End-State Recommendation Report Water Control Data Collection and Management System

Executive Summary As part of the Northwestern Division Water Management re-organization initiated in 2007, the WCDS Transition Team was tasked with developing and finalizing an end-state description for the WCDS that would serve the needs of the region. The team was then further tasked with developing a transition plan to achieve the system end-state as approved by management. This report defines the end-state for the system. Once the Transition Management Committee (TMC) determines the WCDS end-state, the transition plan would then be developed as the next phase of this effort. The Needs Assessment Report from Phase-I provided a conceptual framework for the data system. The primary purpose of the WCDS is to provide data and information used on a real-time basis to regulate water projects for which the Corps of Engineers is responsible including flood control, power, navigation, recreation, water supply, water quality, and environmental concerns. The WCDS also shares data with partner agencies and provides water management data to project operators and to the user community including; internal users in emergency management, dam safety, hydrologic and hydraulic engineering; external users such as other federal, state, and local agencies and the general public. Currently, the four regulating offices use various and, in some cases, different methods for data processing, storage and dissemination. The proposed end-state System Architecture consists of three nodes located one each in Portland, Seattle, and Walla Walla. The three nodes would be configured similarly, with each having the ability to collect, process, store and disseminate data, as well as support modeling and all other necessary functions. All nodes would use the same version of the CWMS database. All data, processes, applications, and files would be synchronized in real-time across all three nodes. Since the database and file systems are replicated at each node, the four offices would have an active COOP with any node able to act as a backup for any or all of the other nodes. Each node would have the ability to operate independently in a network communication failure. The proposed architecture has similarities with the Missouri Basin WCDS and provides the potential for future collaboration on system and tool development. The office responsible for data collection would perform QA/QC on that data. The QA/QC changes made at one location would be automatically streamed to the other locations. Data exchange with partner agencies would occur via a COOPed ftp server with one point of access. Changes to the data exchange process would be transparent to our partner agencies. Data dissemination internally and to the general public would continue to be provided via web reports, queries and by specific request. Where possible, all locally and regionally developed applications would be migrated to work with CWMS. Future application development would be coordinated throughout the region to facilitate standardization and would make use of the CWMS Application Programming Interface (API) where appropriate. HEC would be informed about regional

i

development efforts and would be provided with any products of interest for inclusion in CWMS. Management of the WCDS would be performed by a Steering Committee and Technical Coordination Team. The WCDS Steering Committee would be responsible for balancing the data needs of all the regulating offices and Corps partnering agencies and would report to the Division Water Management Chief. The Steering Committee would set the vision, scope and priorities for the WCDS including the Technical Coordination Team. A Technical Coordination Team Lead would be assigned the duties of a Project Manager for the WCDS and would report to the WCDS Steering Committee. The Technical Coordination Team would manage and coordinate the development and maintenance of all aspects of the WCDS. The WCDS Transition Team expects there would be a reduction in the end-state WCDS level of effort. The average estimated reduction is 4,500 man-hours (2.5 FTE). The level of effort estimate is dependent on ACE-IT involvement and assumes the proposed WCDS standardization would allow for more system efficiencies and would provide an increased level of service to all users. WCDS documentation, planning, and development would occur in conjunction with the annually updated Columbia Basin WCDS Master Plan. Increased coordination and discussion between the Columbia Basin WCDS staff and the Missouri Basin WCDS staff would occur to leverage knowledge, lessons learned, and resources. The next step after the TMC determines an appropriate WCDS End-State, would be the development of a transition plan to provide direction and schedule for implementing the reorganization to the end-state.

ii

WCDS End-State Recommendation Report Water Control Data Collection and Management System

EXECUTIVE SUMMARY ........................................................................................................................... I 1. INTRODUCTION...........................................................................................................................- 1 - 2. PURPOSE OF A WATER CONTROL DATA SYSTEM (WCDS) ...........................................- 1 -

2.1. REAL-TIME REGULATION ...................................................................................................... - 1 - 2.2. PROJECT OPERATORS ............................................................................................................ - 2 - 2.3. HYDRAULIC & HYDROLOGY USER COMMUNITY.................................................................. - 2 -

3. EXISTING WATER CONTROL DATA SYSTEMS ..................................................................- 2 - 4. PURPOSE FOR THE WCDS TRANSITION TEAM EFFORT................................................- 3 -

4.1. OVERVIEW .............................................................................................................................. - 3 - 4.2. TRANSITION MANAGEMENT COMMITTEE DIRECTIVE ......................................................... - 3 - 4.3. WCDS NEEDS ASSESSMENT................................................................................................... - 3 -

5. APPROACH TO DEVELOPING WCDS END STATE .............................................................- 3 - 6. WCDS END STATE.......................................................................................................................- 4 -

6.1. OVERVIEW .............................................................................................................................. - 4 - 6.2. SYSTEM ARCHITECTURE AND CONTINUITY OF OPERATIONS............................................... - 5 -

6.2.1. Overview.............................................................................................................................- 5 - 6.2.2. General Configuration of WCDS ......................................................................................- 5 - 6.2.3. WCDS Node Configuration...............................................................................................- 8 - 6.2.4. WCDS Continuity of Operations (COOP).........................................................................- 8 -

6.3. DATA PROCESSING ............................................................................................................... - 10 - 6.3.1. Overview...........................................................................................................................- 10 - 6.3.2. Data Collection ................................................................................................................- 13 - 6.3.3. Data Acquisition ..............................................................................................................- 13 - 6.3.4. Data Posting to Database ................................................................................................- 14 -

6.4. QUALITY ASSURANCE AND QUALITY CONTROL (QA/QC)................................................. - 14 - 6.4.1. Overview...........................................................................................................................- 14 - 6.4.2. Automated QA/QC After Data is Posted to Database .....................................................- 15 - 6.4.3. Manual QA/QC ................................................................................................................- 15 - 6.4.4. QA/QC Other Agencies Data...........................................................................................- 16 -

6.5. APPLICATIONS ...................................................................................................................... - 16 - 6.5.1. Overview...........................................................................................................................- 16 - 6.5.2. System Monitoring and Maintenance Applications........................................................- 17 - 6.5.3. File Management and Sharing Applications ..................................................................- 17 - 6.5.4. User Applications.............................................................................................................- 17 -

6.6. DATA DISSEMINATION.......................................................................................................... - 17 - 6.6.1. Overview...........................................................................................................................- 17 -

6.7. SYSTEM MANAGEMENT........................................................................................................ - 19 - 6.7.1. Overview...........................................................................................................................- 19 - 6.7.2. The Role of the Division Water Management Chief in WCDS ......................................- 20 - 6.7.3. WCDS Steering Committee .............................................................................................- 20 - 6.7.4. WCDS Technical Coordination Team ............................................................................- 21 - 6.7.5. WCDS End-State Level of Effort ....................................................................................- 22 -

6.8. DOCUMENTATION, PLANNING, AND COORDINATION .......................................................... - 24 - 6.9. PERTINENT ENGINEERING REGULATIONS........................................................................... - 25 -

7. CONCLUSIONS ...........................................................................................................................- 26 -

iii

8. DEFINITIONS ..............................................................................................................................- 27 - APPENDIX A: DETAILED DESCRIPTION OF EXISTING WATER CONTROL DATA SYSTEMS ...............................................................................................................................................- 29 -

A.1 DIVISION-COLUMBIA BASIN................................................................................................. - 29 - A.2 PORTLAND DISTRICT............................................................................................................ - 29 - A.3 SEATTLE DISTRICT ............................................................................................................... - 31 - A.4 WALLA WALLA DISTRICT.................................................................................................... - 34 - A.5 DIVISION-MISSOURI BASIN .................................................................................................. - 35 - A.6 CURRENT APPLICABLE MEMORANDUMS OF AGREEMENT ................................................. - 36 -

APPENDIX B: DATA DISSEMINATION - EXTENT AND TYPES ...............................................- 39 - B.1 EXTENT OF DATA DISSEMINATION .............................................................................................. - 39 - B.2 DATA DISSEMINATION VENUES ................................................................................................... - 39 -

B.2.1 Models...................................................................................................................................- 39 - B.2.2 Internal Web .........................................................................................................................- 40 - B.2.3 Public Web ............................................................................................................................- 41 - B.2.4 CIAS......................................................................................................................................- 42 - B.2.5 Extranet ................................................................................................................................- 42 - B.2.6 Tools......................................................................................................................................- 42 - B.2.7 eGIS ......................................................................................................................................- 43 -

APPENDIX C: ACE-IT QUESTIONS AND INTERACTIONS........................................................- 44 - APPENDIX D: CATEGORIZED FUNCTION SPREADSHEET EXERCISE................................- 45 - APPENDIX E: WCDS STEERING COMMITTEE RECOMMENDATIONS................................- 47 -

E.1 WCDS STEERING COMMITTEE MEMBERS ................................................................................. - 47 - E.2 WCDS STEERING COMMITTEE RESPONSIBILITIES .................................................................... - 47 - E.3 COMMITTEE NORMS .................................................................................................................... - 48 -

APPENDIX F: TECHNICAL COORDINATION TEAM RECOMMENDATIONS......................- 49 - F.1 TECHNICAL COORDINATION TEAM STRUCTURE ........................................................................ - 49 - F.2 TECHNICAL COORDINATION TEAM ACCOUNTABILITY .............................................................. - 49 - F.3 COMMUNICATION ......................................................................................................................... - 49 - F.4 SUMMARY OF FUNCTIONS ............................................................................................................ - 49 -

APPENDIX G: COMPARISON OF PROPOSED WCDS END-STATE WITH ER 1110-2-249 ...- 51 - G.1 INTRODUCTION ............................................................................................................................ - 51 - G.2 CURRENT ER CONSIDERS A WCDS AT EACH LOCAL OFFICE .................................................. - 51 - G.3 ER SYSTEM ADMINISTRATOR – WCDS STEERING COMMITTEE CHAIRPERSON...................... - 51 -

G.3.1 Description of the System Administrator in the ER ............................................................- 51 - G.3.2 Responsibilities of the System Administrator in the ER......................................................- 51 -

G.4 ER SYSTEM MANAGER – WCDS TECHNICAL COORDINATION TEAM LEAD ............................ - 52 - G.4.1 Selection of the System Manager in the ER ........................................................................- 52 - G.4.2 Responsibilities of the System Manager in the ER .............................................................- 52 -

G.5 ER SITE MANAGER AND DATA ACQUISITION SYSTEMS MANAGER – MEMBERS OF TCT........ - 53 - G.6 ROLE OF THE DIVISION WATER MANAGEMENT CHIEF ............................................................. - 53 -

APPENDIX H: SYSTEM ARCHITECTURE CONSIDERATIONS AND ALTERNATIVES ......- 54 - 1. OVERVIEW ................................................................................................................................. - 54 - 2. ALTERNATIVES DISCUSSED....................................................................................................... - 54 -

2.1 Alternative 1: Post all data to a central node, Stream to the remote nodes............................- 55 - 2.2 Alternative 2: Post all data locally, Stream through a central node.......................................- 55 - 2.3 Alternative 3: Local Posting, Circular Streaming...................................................................- 56 - 2.4 Alternative 4: Local Posting, Fully Connected .......................................................................- 56 - 2.5 Alternative 5: Local and Remote Posting, No Streams...........................................................- 56 -

iv

2.6 Alternative 6: Central Data Acquisition, Stream Metadata ....................................................- 56 - 2.7 Alternative 7: Local Posting, Stream to a Central Node.........................................................- 57 -

3. CONCLUSION ............................................................................................................................. - 57 - APPENDIX H: NOTES ON ORACLE STREAMS ............................................................................- 58 -

H.1 ORACLE STREAMS OVERVIEW .................................................................................................... - 58 - H.2 MISSOURI RIVER REGION’S ORACLE STREAMS EXPERIENCE................................................... - 58 - H.3 EXAMPLE FROM ORACLE OF A 3-NODE ORACLE STREAMS CONFIGURATION.......................... - 59 -

v

WCDS End-State Recommendation Report Water Control Data Collection and Management System

1. Introduction As part of the on-going NWD Water Management re-organization, several teams were formed and tasked with providing transition plans for achieving the end-state defined in the Phase-I Implementation Plan. In the case of the Water Control Data System (WCDS), the end-state was not defined in the Implementation Plan, but a conceptual framework for the data system was outlined in the WCDS Needs Assessment Report, dated August 21, 2007. The WCDS transition team was tasked with continuing the work of the Needs Assessment team, finalizing the end-state plan, and then developing a transition plan which would detail how to migrate from the current WCDS to the recommended solution. This report details the data system end-state recommendation as defined by the WCDS transition team. More information and background on the reorganization can be found in the Water Management Reorganization Phase II Transition Plan Project Management Plan (PMP). The WCDS transition team responsible for developing the end-state report Team Lead:

Michael Swenson (NWD-MR) Team Members:

Arthur Armour (NWP) Dave Gustafson (NWS) Dave Portin (NWD-P) Mike Bainer (NWD-P) Ruth Abney (NWP) Stephen Hall (NWW)

2. Purpose of a Water Control Data System (WCDS)

2.1. Real-Time Regulation The primary purpose of the WCDS is to provide data and information to real-time regulators for the purpose of water management. Water Management includes flood control, power, navigation, recreation, water supply, water quality, and environmental concerns including endangered species and fish and wildlife issues. In order to provide the project regulators with all information necessary for regulation decision making, a large component of the WCDS is sharing data (providing and receiving data) with partner agencies. Depending on the hydrologic nature of the basin involved, an adequate monitoring of the basin may require more or less data reporting with varying degrees of timeliness. Snowmelt driven systems in Walla Walla District and Seattle District require hydrologically accurate historic data plus updated hourly values. Rain driven systems like those found in Seattle District require updated and transformed hourly data available for use within 15 minutes from the top of each hour. Data updated every

- 1 -

hour is hydrologically adequate for the river systems managed by Portland District and the Columbia Reservoir Control Center. 2.2. Project Operators A secondary purpose of either the data collection efforts of the district or of the WCDS is to provide water management data to project operators in a timely manner. Operators have expressed a desire to have the data either instantaneous or within five minutes of field measurement. This desire usually arises since all the other data that the operators work with in the project control room is instantaneous. It is not because monitoring of the hydrologic environment requires such a quick response from the operators. The viability of providing the project operators with hydro-meteorological data instantaneously or every five minutes would be evaluated on an individual case-by-case basis. 2.3. Hydraulic & Hydrology User Community A third purpose of a WCDS is providing data to the user community, beyond project regulators. The data is currently provided in the form of static reports and limited ad-hoc query capability. The user community includes internal users such as emergency management personnel, dam safety personnel, hydrologic engineers, and hydraulic engineers as well as external users such as Bonneville Power Administration (BPA), National Weather Service River Forecast Center, U.S. Geological Survey (USGS), Bureau of Reclamation, other federal, state, and local agencies and the general public. WCDS data provided to the user community takes the form of public and internal web pages, raw data feeds, model outputs, as well as static and ad-hoc reports/graphs.

3. Existing Water Control Data Systems Currently, the Columbia Basin Water Management offices (Columbia Basin Reservoir Control Center, Portland District, Seattle District, and Walla Walla District) make use of Corps and other agency resources to accomplish the Columbia River Basin Water Management function. The Columbia Basin district offices manage all data collection activities including maintenance of field equipment and agreements with partner agencies, with the exception of data collected via the Columbia Basin Teletype (CBT) system which is managed by the Columbia Basin WCDS. The district offices collect all data and provide it to the Division-Columbia Basin WCDS via various processes with the exception of satellite and CBT, which the Division-Columbia Basin WCDS acquires directly. Districts also collect satellite data via LRGS directly for local gaging stations. Due to regional and inter-agency interaction and dependencies there are a number of Memorandums of Agreement (MOAs) that exist. For a list of key memorandums refer to Appendix A, Section A.6. The four regulating offices each have various independent systems for data processing, storage, and dissemination. In the three district offices the data functions are co-located with the reservoir regulation staff. In the Division-Columbia Basin office, the regulation

- 2 -

function (Columbia Basin Reservoir Control Center) resides in a separate office from the data management functions (Columbia Basin WCDS). A detailed technical description of each of the current data systems can be found in Appendix A. 4. Purpose for the WCDS Transition Team Effort

4.1. Overview The WCDS Transition Team (The Team) was established as part of the Northwestern Division Water Management Reorganization. The Team was tasked with continuing the work of the WCDS Needs Assessment team as documented in their report dated August 21, 2007. The WCDS Needs Assessment team developed a conceptual framework for what the regulating offices wanted in the future WCDS. The Team is working from the defined conceptual framework to develop an end-state definition of the system. The next phase would continue the development of a transition plan to migrate from the current system to the new end-state. 4.2. Transition Management Committee Directive Overall purposes of the WCDS Transition Team effort as defined by the Transition Management Committee include:

• Improved communication and cooperation between all offices. High level of respect and trust between offices.

• Develop interdependent relationships among all offices. Technical support is shared between all offices including division support to districts where appropriate.

• Project authorized purposes would be met and organizational or process changes would not compromise Life, Safety, Health, Property, or legal and ethical standards.

• Ensure Water Management employees are taken care of.

4.3. WCDS Needs Assessment Purposes specific to the WCDS Transition Team as defined in the WCDS Needs Assessment report.

• Integration of WCDS management and development into the regulating offices.

• Standardization of WCDS database and software across the region. • Improved Continuity of Operations (COOP). • Improved capabilities with Corps Water Management System (CWMS)

software. • Standardization of new application development.

5. Approach to Developing WCDS End State The WCDS Transition Team began by clarifying any questions new members had regarding the concepts laid out in the WCDS Needs Assessment effort. The team then

- 3 -

proceeded to address the data system architecture in more detail since it set the framework and context in which other WCDS components would be discussed. Following agreement on system architecture, other key elements of the system were evaluated and an end state was developed. The key elements were categorized and addressed as follows:

• COOP • Data Processing including data collection, acquisition, and posting into the

database • Quality Assurance and Quality Control (QA/QC) • Applications • Data Dissemination including interaction with models, internal and external web

pages, tools, file sharing, and corporate eGIS integration • Management and technical team configuration and responsibilities, including

functions and staffing 6. WCDS End State

6.1. Overview The System Architecture consists of three nodes, one in Portland, one in Seattle, and one in Walla Walla. The three nodes would be configured similarly, with each having the ability to collect, process, and store data. All nodes would use the same version of the CWMS database. All data, processes, applications, and files would be synchronized among the nodes. All of these considerations reduce the maintenance workload and provides an active COOP. Since all three nodes are nearly identical, any node could act as a backup to any of the other nodes. The standard configuration at each node also lends itself to standardization in development of tools and other applications which would create efficiencies throughout the region. This architecture is very similar to the current Missouri Basin WCDS. The system architecture provides for key functions at all of the nodes so that each node could function independently in the event of a wide area network (WAN) failure. This assumes the locally collected data (data transmitted via satellite and/or line of sight (LOS) radios) continues to come into the node since they are independent of the WAN as well. Using this architecture, each regulating office, associated with a specific node, would be responsible for the QA/QC processes on the data that regulating office is responsible for. Further discussion of what capabilities and functions exist at each of the nodes follows in a subsequent section of this document. There would also be two regional ftp servers, one located in Portland and a backup server located in Walla Walla. This would provide partnering agencies one location (with a backup) where they can exchange data with the Corps.

- 4 -

A WCDS Steering Committee would be established which sets the vision, scope, and priorities of the WCDS. A Technical Coordination Team would be formed by the WCDS Steering Committee that would coordinate and carry out all technical aspects of the WCDS. WCDS documentation, planning, and development would occur in conjunction with the annually updated Columbia Basin WCDS Master Plan. Increased coordination and discussion between the Columbia Basin WCDS staff and the Missouri Basin WCDS staff would occur to leverage knowledge, lessons learned, and resources. More detailed discussion on the network configuration, node configuration, and Continuity of Operations (COOP) follows in the subsequent sections. 6.2. System Architecture and Continuity of Operations

6.2.1. Overview The WCDS Transition Team considered seven different system architectures. The benefits and drawbacks of each alternative were discussed using the following primary criteria for evaluation:

• Standard configurations at each node for ease of maintenance, ease of keeping each node in sync with the other nodes, and providing a environment for joint application development

• Replicating and standardizing technology to enhance reservoir regulation staff efficiencies within and between offices.

• Improved COOP, including integration of COOP (active), ease of recovering from COOP, limiting the points of failures between processes, and allowing for stand-alone operation at each node during crisis situations to accomplish the mission

• QA/QC performed by staff from each office on data collection stations primary to that office

• All data available at each node (Portland, Seattle, Walla Walla) • Simplicity, ease, and thoroughness of populating the databases and

handling metadata • Network reliability and performance

Refer to Appendix H for further information concerning the seven alternatives considered. In the following sections, the general system architecture and integrated COOP configuration will be described, and then the configuration at a node will be described in more detail. The third discussion addresses COOP in more detail. 6.2.2. General Configuration of WCDS The system architecture consists of three nodes, one in Portland, one in Seattle, and one in Walla Walla. At each of these nodes reside all applicable files, applications, and models for each of the four regulating offices in the region as well as a database with all of the data for the region. The team agreed that the

- 5 -

Columbia River basin should standardize on the CWMS database (Version 2.0 or later current HEC release), and that the transition plan and efforts would be shared with the Missouri River basin. The data system would include a complete CWMS installation at each office, but would not limit end-users to CWMS applications (see Application section). The system architecture concept diagram is shown in Figure 6-1 WCDS Architecture Overview. The fundamental design concepts captured in this schematic are:

• Each node is responsible for data collection • Each node would acquire all data or would have all acquisition

configuration information necessary to acquire all data • Each node would only post data to the database that it is responsible for

unless it is a COOP situation • In general, each regulating office (4 offices) would perform QA/QC on

stations for which it is responsible as determined by the WCDS Steering Committee

• The responsibility for data collection and data QA/QC can be performed by different offices as determined by the WCDS Steering Committee

• Data posted at each node would be streamed to the other two nodes using Oracle Streams

• Due to the use of Oracle Streams, the idea of separate databases is no longer applicable. In effect, there is only one database since any changes made at one location are automatically streamed to the other locations.

• Sharing of all water management information that is not stored in the database, that is needed for COOP, would be addressed in a file sharing process since the information would not be in the database to be streamed

• Any node can serve as the COOP site for any other node • COOP is integrated into the system design

- 6 -

Figure 6.1 WCDS Architecture Overview

- 7 -

The proposed system architecture would provide scalability for future data system needs. Within the system architecture, any of the three nodes would have the ability to provide data to a national COOP site and a regional and/or national public dissemination site. Only one node would routinely run processes for these additional sites. Further discussion of the standardized node configuration and integration of COOP follow in the next two sections. 6.2.3. WCDS Node Configuration Each of the three nodes (Portland, Seattle, and Walla Walla) would be very similar, with the main differences being in the local data collection methods, equipment, and servers. Each node would contain the following functionality:

• Data Collection • Data Acquisition • Posting of Data to the Database • CWMS Database • QA/QC Responsibilities and Applications • Data Monitoring and other Applications, Models, and File Sharing • Data Dissemination

The only functions that would not be at each node are the regional ftp server and public Internet pages. Figure 6-2 WCDS Data Processing displays the configuration of processes at each of the three nodes in the Columbia Basin WCDS architecture. Each of these functions is discussed in more detail in subsequent sections. 6.2.4. WCDS Continuity of Operations (COOP) COOP is the ability to continue to perform any key Water Control Data System function or set of functions such as data collection, acquisition, posting, QA/QC, execution of key applications, or dissemination of data when the regular mode of operation fails in whole or in part due to planned or unplanned events. Water management is an essential mission of the Corps and is supported by the WCDS. Loss of a critical system like WCDS could lead to loss of life and property. Therefore COOP of the WCDS is central to the responsibility the Corps has for operating water control projects. The system architecture, defined in the previous section, incorporates the idea of an “active COOP” in the system design. The goal of the WCDS Needs Assessment and Transition Teams was to move away from a COOP that was offline and needed to be supported separately and tested to be current, and move to an active COOP an integral part of the system architecture. The proposed system architecture includes active functioning redundancies which provide real-time performance benefits while satisfying COOP requirements. Including this active COOP concept in the system design process allows for seamless operations

- 8 -

in crisis situations. With the three-node architecture there does not have to be a primary COOP site since each node in the system architecture can serve as a COOP for any other node. With this three node architecture, the Columbia Basin WCDS would have equipment and capability located in three cities; Portland, Seattle and Walla Walla. Each node, acting as a COOP site, would have the ability to collect, acquire, and post all data to the WCDS database. During normal operation, each node can collect and acquire all data for the region but would only post the data that node is responsible for as determined by the WCDS Steering Committee. During an event when COOP is required, the designated COOP site would collect, acquire and post the data for the node that is down in addition to the standard data processing occurring at that node. The activation of COOP and the process for restoring normal system operations would be designed with safeguards in place so that only one node can post data from a given data collection station. This would avoid unintentional data conflicts, such as when the same piece of data from the same transmission source is posted multiple times. The data in the database at all three nodes would be synced via bi-directional streaming supported by Oracle Streams. As additional data streams or data collection stations are added to the system, their configurations and QA/QC specifications would be added by the appropriate office to the local node and synced with the other nodes via bi-directional streams and file transfer processes. Another aspect of the system architecture and COOP is file sharing between nodes. Each regulation office would have their files, procedures, tools, applications, models, and other necessary information stored on a local server as well as copied to each of the other nodes. These files, procedures, tools, applications, models, and other information would be regularly synced between nodes using an automated process. The team noted that there needs to be an uninterrupted power source for all key elements of the WCDS. This includes both automatic temporary battery supplies as well as having generator power immediately available upon a power failure for all key equipment. CWMS is an Automated Information System (AIS) supported by headquarters as part of the water management WCDS. CWMS development is the responsibility of the Hydrologic Engineering Center (HEC), and it is their responsibility to define a national CWMS COOP as part of the AIS. The national CWMS COOP is beyond the scope of this document other than to note that a connection to the national COOP would be accommodated from the Columbia Basin WCDS. The details of a national COOP are not available from HEC at this time.

- 9 -

The Columbia Basin Teletype (CBT) system, used by project operators, regulators, and partnering agencies for entering and transferring water management data, along with the data from GDACS and SCADA, needs to be evaluated for COOP so there is not a signal point of failure for retrieving data from these systems.

6.3. Data Processing

6.3.1. Overview The data processing aspects of the WCDS consist of three major phases: Data Collection, Data Acquisition and Posting to the database. A more detailed description of what is involved in these three phases is presented later in this section. All three phases would occur or be able to occur at each node. Figure 6.2 “WCDS Data Processing for One Node” depicts the three phases of Data Processing as they would occur at one of the three nodes. All nodes would have an identical structure. The three boxes inside the Data Acquisition (DA) box depict the processes running for data coming from each of the three nodes. The local DA is fed by the local data collection processes. The other two DA processes are fed by the Regional FTP Server which contains most data from the other two nodes’ local data collection processes. One exception is the LRGS (GOES) data which is not sent to the Regional FTP Server for processing at the other two nodes because each node as the capability of collecting all LRGS (GOES) data for the region. The two arrows leaving the Local Data Collection box show the data going to the local DA process as well as sending the data to the Regional FTP Server up above. The three parallel DA processes feed the posting processes on the local posting computer. The three posting streams (representing data received by the local node and two other nodes) are kept separate so that the streams from either of the other two nodes can be turned on or off, depending on COOP requirements for another node. The streams from the other two nodes would normally be turned off and only the local stream would post to the database, as depicted in the diagram. Figure 6.3 “WCDS Local and Regional Collection” shows how the above local node fits in with the overall architecture and flow of data. Each of the three nodes’ Local Data Collection processes feed the Regional FTP Server as well as the local Data Acquisition process. The Regional FTP Server then supplies each node with the local data sent to the Regional FTP Server by the other nodes. The dashed lines on the bottom of the diagram, between the “Oracle” cylinders, represent Oracle Streams which would be used to transfer data between the databases to keep them all in sync. The light gray large dashed line that cuts the figure in two horizontally shows that the Regional FTP Server is on the Controlled Internet Accessible Segment (CIAS) and the computers and servers below the line are on the Secure Production Network in the Corps.

- 10 -

Figure 6.2 WCDS Data Processing for One Node

- 11 -

Figure 6.3 WCDS Local and Regional Collection

- 12 -

6.3.2. Data Collection Data collection is the measurement of a value in the field, logging the value with equipment such as a Data Collection Platform (DCP), and all transmission or manipulation of the data prior to receipt by the data acquisition process. Each node would perform the data collection they determine to be valuable with coordination through the WCDS Steering Committee. Local data collection is sent directly to the local Data Acquisition Server and, at the discretion of the local regulating office, can also be sent to the regional ftp server. The local office needs to be aware that if they choose not send data to the regional ftp server it would not be available in a COOP situation. The regional ftp server, which would be located in Portland, would reside on the Controlled Internet Accessible Segment (CIAS) since it would be used to share data with stakeholders. A backup regional ftp server would exist at the Walla Walla node and the domain name would be managed in such a way that failover is transparent to the users. Local data collection servers at each node may be located on the CIAS, Extranet, or Production Network. Each regulating office is responsible for COOP of their data collection servers and processes. Discussion of data collection COOP should be addressed in the regulating office Data Collection Work Plan and is beyond the scope of this document. Transmission methods used during data collection include phone, radio, microwave, satellite, ftp and web-based transmissions. Examples of data collection servers include the LRGS server and the Sutron XConnect server. 6.3.3. Data Acquisition Data acquisition is the processes used to gather, reformat into SHEF, and move data from receipt via satellite (LRGS), regional ftp server, other ftp processes, Columbia Basin Teletype (CBT), or by some other means into the “incoming” directories where posting to the database begins. Each node would have the capability to acquire all data for the region. A separate data acquisition process would run for each datafeed instead of one process which cycles through the various datafeeds. The acquisition configuration at each node would be copied to the other nodes automatically on a regular basis. Each node would have the ability to send the reformatted data out to the regional ftp server for dissemination to partnering agencies. However, there would be some mechanism in the database to assure specific types of data, such as power production and forecasts, are not released to inappropriate agencies. Data acquisition would occur on the Production Network and the processes would reside on the same server where the posting processes and database reside. The acquisition process would store the raw data for a specified number of days. There should continue to be a way to display and retrieve this raw data. The program FileHunter would continue to be used for cleaning up and deleting files based on a designated look-back window until a better tool is found.

- 13 -

6.3.4. Data Posting to Database Posting is the process of moving the data from the acquisition process into the database. This process begins by pulling data out of the “incoming” directories. During routine operations, the local data is only posted at the local node and subsequently transferred to the other two nodes using Oracle Streams. The posting process would need to be turned on or off for specific datafeeds depending on what datafeeds and data collection stations are to be posted at that local node. When a posting process is updated at any of the nodes the changes would need to be copied to the other nodes. The posting processes and database would reside on the Production Network and would be on the same server where data acquisition resides. The posting process consists of the following processes:

• Posting is initiated by a process called DataCapture which routinely browses the incoming directories of the various data feeds for new files.

• When a new file is found DataCapture moves the file over to a daily file. • DataReader monitors the daily file for new messages and when a new

message is detected it sends it to the SHEFIT program for processing. • The SHEFIT program decodes the SHEF formatted records from the

incoming file. • The output from the SHEFIT program is sent to an application called

ProcessShefit which performs various processes on the data. • The application ProcessShefit then posts the data into the Oracle database. • It is not until the data is posted into the database that any automated

Quality Assurance and Quality Control (QA/QC) processes are performed and any transformations, such as the rating table process for producing flows, are initiated.

• Data dissemination to partnering agencies of the QA/QC data as well as any transformations and calculations, such as project inflows, would occur after the data is posted in the database.

Each node would use the CWMS V2 or later schema with the Oracle database. The database at each node would contain all the data for the region. Each node would have the ability to ftp processed data to the regional ftp server for dissemination to partnering agencies. However, there would be a mechanism set up to make sure proprietary or sensitive data, such as power production and forecasts, are not released to inappropriate agencies.

6.4. Quality Assurance and Quality Control (QA/QC)

6.4.1. Overview QA/QC is the automated or manual process by which data is reviewed, modified, or corrected. This data may be time-series data, metadata, or other information in the database such as pathnames.

- 14 -

Generally the office that performs data collection would have primary responsibility for monitoring the automatic processes and performing the manual processes associated with that data. The QA/QC criteria would be defined, maintained and managed by the office that is responsible for the specific location/data with regional coordination and data sharing. The QA/QC criteria would be regularly synced between nodes using Oracle Streams. In some cases the responsibility for certain data may not be clear. These situations would be resolved by the WCDS Steering Committee. Generally, if QA/QC is required for data received from sources other than the Corps it would be performed by the posting office unless otherwise determined by the WCDS Steering Committee. There should be cross-training between offices and sufficient information should exist in the QA/QC procedure manual so that alternative staff would be able to provide a basic level of QA/QC during a COOP event. To make sure the QA/QC methods and processes across the region are standard, a PDT with representative from each office should be established with a designated technical lead. The technical lead could represent the regional QA/QC issues on the Technical Coordination Team, which is discussed in a later section. This PDT would be responsible for sharing knowledge through out the region, developing the QA/QC SOP, and guiding the development of procedures and applications in support of the QA/QC process. The SOP would also include the roles and responsibilities of the various members and offices with regard to the QA/QC process. The SOP would be included as part of the Master Plan. There are two different processes involved in QA/QC including automated procedures as data is posted to the database and a manual process. The two procedures are described in further detail below.

6.4.2. Automated QA/QC After Data is Posted to Database Automated QA/QC processes which are performed as the data is posted to the database should assign quality flags to particular data so that questioned and rejected values, high, low and rate-of-change variances are tagged appropriately. An automated QA/QC process is built into CWMS and would likely be used. The local office would be responsible for maintaining all automated screening criteria for the locations they are responsible for as determined by the WCDS Steering Committee. 6.4.3. Manual QA/QC The manual QA/QC process occurs once the data is in the database. A knowledgeable user evaluates the data and determines if changes need to be made so the data is representative of what actually occurred. Since local knowledge of specific data locations adds significant benefit to the manual QA/QC process, the manual process would occur at the local office responsible for the specific data.

- 15 -

Applications to help the user perform manual QA/QC still need to be looked at by the team during transition planning. This may include off-the-shelf products, in-house development or HEC products. It would be highly desirable if the QA/QC tools are web based and have the ability to configure the time window displayed as well as what data is displayed. Standardization of a manual QA/QC application would be a high priority. A SOP addressing manual QA/QC procedures should include timeliness of corrections, criteria, approach, methods, and how data might be finalized. The SOP should allow for specific rules for different data and gaging locations. 6.4.4. QA/QC Other Agencies Data Data from other agencies should be corrected at the source or as close to the source as possible. Corrected and published data from other agencies can be included in the database. How that data is labeled would be decided in the implementation phase. An SOP needs to be written which spells out the policies of correcting other agencies data. The SOP also needs to include who would contact partnering agencies with regard to missing or incorrect data and how this would be done.

6.5. Applications

6.5.1. Overview As discussed in system architecture, each office would have a full CWMS installation. However, end-users would not be limited to CWMS tools and/or applications. In cases where it is reasonable, current applications would initially be migrated to work with the CWMS database. Any application development should be coordinated throughout the region and, to the extent possible, make use of the standard CWMS application programming interface (API). Using the CWMS API for development of applications that augment or replace CWMS functionality would help insulate the applications from CWMS database upgrades. A long-term effort to standardize applications, coordinate development and supplement individual office applications would occur. This effort would be coordinated with the Missouri River basin and with HEC. Coordination between the Columbia and Missouri offices would be advantageous when considering funding betterments, and/or encouraging HEC to incorporate any locally developed applications into the CWMS application software. Local application development, within a standardized framework, would leverage local knowledge and regional application sharing. HEC and ACE-IT roles in local development are still to be determined.

- 16 -

There are three general categories of applications that are needed 1) system monitoring and maintenance applications, 2) file management and sharing applications, and 3) user applications. Each category of application serves a vital role within the system. Below is a further discussion of these categories of applications. 6.5.2. System Monitoring and Maintenance Applications Applications that monitor and automatically maintain the system save many hours in manual labor, resulting in lower system maintenance. Some examples of work performed by automated applications are deletion of old files and data for disk space management, monitoring of automated processes to make sure specific data is coming into the system, generation of static user reports, and initiation of annual or seasonal data management. Monitoring and maintenance applications also provide a proactive approach to troubleshooting problems. It is important that these applications are developed and used. Big Sister is one such software package already in use that monitors the systems and sends out alerts when issues arise. A standard software package that performs such a function would reside at each node on the same server where the internal web resides. 6.5.3. File Management and Sharing Applications Models, files and tools, along with other key information would be regularly synced between nodes. Applications would be developed that keep track of which files are managed and shared as well as those applications that perform the sharing and automatic transferring of files. Processes and procedures would also be put in place to control the sharing of sequentially generated model data throughout the region. 6.5.4. User Applications Applications that allow the users to access and manage data within the system would be central to assuring quality data. Users would need applications to manage meta-data about projects, data collected, and alert thresholds, as well as tools for checking and reviewing automated and manual QA/QC designations (discussed in QA/QC). User applications would utilize the CWMS API to insulate the tools from changes to the underlying CWMS database. The CWMS CAVI and the Paging System are two examples of User Applications that are currently in use.

6.6. Data Dissemination

6.6.1. Overview Data dissemination is the retrieval and distribution of data during data processing as well as after the data is posted to the database. Data is distributed internally, to

- 17 -

trusted partners with some system access and to the general public. Data to be disseminated includes data to populate internal and public web pages; other files, tools, and applications; model results; as well as products produced by CWMS applications including the CAVI. With the exception of the regional ftp server and the public web server, all data dissemination capabilities would reside at each node on the Production Network. The regional ftp server, which would reside on the Controlled Internet Accessible Segment (CIAS), would provide a central distribution point for making data available to trusted partners. Files generated during data acquisition or from database processes would be pushed out to the regional ftp server for partner agencies. Modeling and sharing of modeling inputs, outputs, and configurations would need to be skillfully managed through documented procedures and automated processes since some models and their associated files would be used regionally. Many of these models run on a UNIX platform but reservoir regulators work in a Microsoft Windows environment. Therefore these UNIX model files would need to be accessible through the Corps Windows network. With regard to web applications such as report generation, graph generation, and internal web content, all capabilities to generate these products would exist at all locations but only local products of interest would be generated on a routine basis at each node. A web-based data dissemination management tool for identifying which products belong with which node would be developed with the capability of easily turning on or off specific products and groups of products. All content would be accessible to all nodes but may not reside locally. In a COOP situation where one node cannot generate their products, one of the other nodes would turn on the generation of those products. The public web server would likely be managed by ACE-IT and it would be the responsibility of ACE-IT to provide COOP for the public web server. The use of a public website like Rivergages.com may also prove to be advantageous since it is already a functioning Water Management web dissemination product with some capabilities to perform ad hoc data queries. However, before fully making use of a public website, the effort required to support the connection, transfer of information, maintenance, and development aspects need to be thoroughly evaluated. Each node would have the ability to ftp information out to the public web server. The data dissemination functionality that resides on the Production Network would exist on the same server as the models, other applications, the internal web pages, and the file sharing capabilities. Automated processes would be put in place to make sure that all the models, files, tools, and other applications at one node are replicated at the other two nodes.

- 18 -

Any development in the area of data dissemination needs to be maintainable, standardized, flexible, configurable, and if possible lend itself to be being incorporated into HEC development process or national water management efforts (example: Rivergages.com) either voluntarily or through organization momentum. Development would be coordinated with the Missouri Basin so that standardization between regions can progress. Refer to Appendix B for a listing of the extent and types of data dissemination discussed by the team.

6.7. System Management

6.7.1. Overview A Steering Committee and Technical Coordination Team would be established to manage all aspects of the WCDS. The WCDS Steering Committee is a managerial group responsible for balancing the data needs of all the regulating offices in each region as well as Corps partnering agencies. The WCDS Steering Committee reports to the Division Water Management Chief and sets the vision, scope and priorities of the Technical Coordination Team and WCDS working group. The Technical Coordination Team would have a technical lead assigned by the WCDS Steering Committee with the responsible of coordinating WCDS progress with the committee. The Technical Coordination Team Lead would be responsible for performing technical work in one or more functional areas of the WCDS and would also be assigned the duties of a Project Manager for the WCDS. The WCDS Technical Coordination Team is a regionally integrated team consisting of functional area leads that promotes regional solutions and equally serves the data needs of the staff in the four Columbia Basin Water Management offices; Division, Portland District, Seattle District, and Walla Walla District. The team coordinates efforts within the greater WCDS working group. The Technical Coordination Team reports to the WCDS Steering Committee. The WCDS working group would be comprised of all Division and District staff that performs work on the WCDS with regard to development, maintenance, support and management of the system. Members of the WCDS working group would include staff that work on the WCDS both full and part time.

- 19 -

Figure 6.4 WCDS End-State Organization Structure

Note: There is disagreement within the Team concerning the division representative on the steering committee. Some members of the team recommend that the offices have the latitude to select their representative from within Water Management. 6.7.2. The Role of the Division Water Management Chief in WCDS

• Advocate for WCDS • Resolves issues elevated by the WCDS Steering Committee • Impartial arbitrator with regard to WCDS related issues therefore does not

participate as a standing or rotating member of the WCDS Steering Committee

• Informs H&H Chiefs about WCDS activities, relaying information passed up from the WCDS Steering Committee Chair

• Relays pertinent Water Management information to the WCDS Steering Committee Chair

6.7.3. WCDS Steering Committee The WCDS Steering Committee would be responsible for defining the vision and scope for the WCDS and setting system priorities on an ongoing basis. With these

- 20 -

responsibilities, the committee participants would be a team that represents the user needs of the region. The committee would be considered one team, working together to build and manage a regional water management data system. The committee would also serve as a forum for accountability of the Technical Coordination Team. The WCDS Steering Committee would have a rotating Chair selected from the standing members of the committee but the Chair would not have authority over the other committee members. Committee members must have equal weight and participation regardless of status or grade. The additional responsibilities of the Chair include facilitating and fostering communication between the members of the committee, leading discussions and meetings, acting as the spokesperson for the committee where appropriate, and providing reports and updates to the Division Water Management Chief. The WCDS Steering Committee would have liaison responsibility including interaction with HQ, Northwestern Division Office, Missouri Basin Region, HEC, ACE-IT, and partner agencies on WCDS issues. Committee liaison functions would focus on system vision, scope and priorities. The liaison responsibilities may be assigned to committee members on a rotating basis. Refer to Appendix E for recommendations specific to the WCDS Steering Committee.

6.7.4. WCDS Technical Coordination Team Throughout the region there would be a greater WCDS working group that encompasses all people that perform some WCDS functions. The WCDS Steering Committee would select individuals from the WCDS working group to be leads for specific WCDS functional areas. The group of technical leads is called the WCDS Technical Coordination Team and is responsible for promoting, maintaining, managing, and developing the regional aspects of the WCDS. Members of the Technical Coordination Team must have demonstrated leadership, good communication and coordination skills, and have a good technical knowledge of the functional areas they would be responsible for coordinating. The leads would coordinate the regional and local activities associated with their functional area in addition to performing their normal WCDS functions on a day to day basis. The selection of leads is not limited to staff that spends 100% of their time in support of WCDS, however, is is expected that they would spend a significant part of their time performing WCDS functions. The technical leads, through their collective responsibilities, would oversee and coordinate all functional areas of WCDS on a day to day, and month to month basis. The WCDS Steering Committee sets the direction and priorities of the Technical Coordination Team and the team implements the plan.

- 21 -

A Technical Coordination Team lead would be selected by the WCDS Steering Committee, and would serve in that position at the discretion of the committee. The Technical Lead is responsible for the overall coordination and management of the activities and priorities identified by the WCDS Steering Committee, leadership of the Technical Coordination Team, monitoring schedule, budget and ensuring requirements are met. The lead would also be responsible for proposing updates to the WCDS PMP including budget, schedules, and annual work plans for review and acceptance by the WCDS Steering Committee. Additional responsibilities of the Technical Coordination Team lead include facilitating and fostering communication between the members of the Technical Coordination Team, leading discussions and meetings, acting as the spokesperson for the team where appropriate, and providing reports and updates to the WCDS Steering Committee. The Technical Coordination Team lead would also be a member of the WCDS Steering Committee. Team members are responsible for their contributions, and are responsible for evaluating activities and schedules against priorities and budget as established by the WCDS Steering Committee. Each team member is responsible for working with the Team Lead to ensure that their product supports the overall performance objectives of the project. Refer to Appendix F for recommendations specific to the Technical Coordination Team. 6.7.5. WCDS End-State Level of Effort

6.7.5.1.Current WCDS Estimated Level of Effort The WCDS Transition Team compiled a summary of the current level of effort for the WCDS to include all division and district staff. The Transition Teams definition of WDCS level of effort includes WCDS tasks performed at the district offices including items such as data collection. The following table shows the current level of effort as identified by each office. Some WCDS related tasks performed at the district offices are difficult to separate from other water management activities, especially when the tasks constitute a small percentage of an individual’s duties. The information below is the best available estimate of the level of effort; however it is only an estimate. Another caveat for these numbers is that they reflect the current effort committed to WCDS. They do not reflect items that have been identified as needed that are not currently being performed, such as data QA/QC.

Table 6.1 Current WCDS Estimated Level of Effort Office Level of effort (FTE) Level of effort (man-hours) * CENWD-HEB 8.6 15136CENWD-RCC 0.47 827CENWS 1.21 2130

- 22 -

CENWW 0.54 950CENWP 1.0 1760Total 11.82 20803 *1 FTE = 1760 man hours

6.7.5.2. End-State WCDS Level of Effort The WCDS Transition Team performed a detailed analysis of the current functions being performed within each office to develop, maintain and support the current WCDS. From this list of functions the team developed a comprehensive list of functions and duties required in the proposed end-state. The list of functions was grouped into thirteen categories of similar efforts. Based on personal experience, each member of the Transition Team estimated the required effort to perform the tasks in each category under the proposed end-state. The estimates in Table 6.2 give a reasonable range of effort required in the end-state. It should be noted there are many things that affect the required level of effort that are outside the control and influence of the WCDS Transition Team and could have a significant impact on the actual level of effort. One of the most significant and currently difficult to predict is the ACE-IT transition and its impact on WCDS. (Refer to Appendix C for a partial list of outstanding questions to address as ACE-IT is integrated into the Water Management organization). Also note that all the categories which are not affected by ACE-IT show a very tight range of level of effort required while the categories that are affected by ACE-IT have a much larger range in the estimate of level of effort based on differing opinions on the impact of working with ACE-IT and which elements would become the responsibility of ACE-IT. The following table shows the categories and the levels of effort compiled by the Transition Team. The total regional effort for the proposed end-state is between 13640 man-hours or 7 FTE and 22700 hours man-hours or 13 FTE with an average estimate of 16290 man-hours or just over 9 FTE.

Table 6.2 WCDS End-State Estimated Level of Effort

CBWM Regional End-State Level of Effort Estimate Category Min Max Ave Hours Hours Hours Data Collection 1760 4400 3230Data Acquisition 350 1410 880Posting 180 880 430DBA 1580 2290 2040Data Dissemination 350 1760 1100Application and Process Support 790 3520 2410On-going Technological Improvements 880 1760 1260QA/QC 1760 2640 1980

- 23 -

Documentation 180 880 380Coordination 350 1320 850System Admin (ACE-IT Role undefined) 350 3170 970Hardware Support/Maintenance (ACE-IT Role undefined) 180 880 480Security (ACE-IT Role undefined) 0 880 290 Total 13640 22700 16290

Looking at the functions that need to be performed, the conclusions of the WCDS Transition Team are that some functions performed in support of the WCDS could be centralized but that some should be performed in the 4 regulating offices or at the 3 computer nodes differing from the idea of a centralized team. See Appendix D for further information about where functions should be located throughout the region.

6.8. Documentation, Planning, and Coordination The system would be documented in the WCDS Master Plan as specified in ER1110-2-240. This master plan would be updated annually in coordination with the Missouri River Basin. From ER1110-2-240, Master Plans will

a) Outline the system performance requirements, including those resulting from any expected expansions of Corps missions,

b) Describe the extent to which existing facilities fulfill performance requirements, c) Describe alternative approaches which will upgrade the system to meet the

requirements not fulfilled by existing facilities, or are more cost effective then the existing system,

d) Justify and recommend a system considering timeliness, reliability, economics and other factors deemed important,

e) Delineate system scope, implementation schedules, proposed annual capital expenditures by district, total costs, and source of funding.

ER1110-2-240 was published in 1982 and is currently planned for revision. Aspects of the system that potentially should be addressed in the ER and are proposed here for the inclusion in the WCDS Master Plan are as follows:

• Regulations and Authorities • Management Structure including Technical Coordination Team configuration • Hardware and Software • Network Configuration • System Components • System Objectives • Processes and Procedures • SOPs • MOAs (See Appendix A.6) • System direction

- 24 -

• Short and long term plans (annual work plans and 5 year PMP) including appropriate schedules for improvements, changes, upgrades, and funding for the system

• Data Collection Work Plans • Operational Maintenance procedures

Master Plan development, planning and implementation efforts would be coordinated within the water management community throughout the entire division (Columbia and Missouri basins). On a more frequent basis, a forum needs to be established which facilitates coordination between the Columbia and Missouri basins technical and management staff to address current development efforts and future planning as well as relaying implementation schedules. An annual review and update of the WCDS Master Plan would be the responsibility of the WCDS Steering Committee. 6.9. Pertinent Engineering Regulations The most pertinent Engineering Regulation with regard to the reorganization of the WCDS is ER 1110-2-249 which pertains to management of Water Control Data Systems. The team noted that language in this ER is outdated and can be confusing due to changes in technology and terminology in recent years. The ER emphasizes the location of management personnel and hardware but could add more emphasis on key processes and systematic approaches. The data collection and quality checks as well as the maintenance of hardware were covered well but data dissemination was not emphasized as a function of WCDS. Therefore, it would be beneficial to update and add emphasis for other key functions of the WCDS. Comparing the currently proposed WCDS end-state with ER 1110-2-249 it is clear that the proposed management of the WCDS end-state is in agreement with ER 1110-2-249 in regard to aligning the management of WCDS with the reservoir regulation offices. Specific positions listed in the ER can be matched up with the Lead of the WCDS Steering Committee, the Lead of the Technical Coordination Team, and technical leads for various aspects of the WCDS. The functions described in the ER are also addressed in the proposed WCDS end-state. Further discussion regarding the similarities and differences between the proposed WCDS end-state and ER 1110-2-249 are presented in Appendix G. The list of Engineering Regulations below contains the main regulations that describe the purposes, processes, procedures, and equipment used in data management. These regulations identify the pertinent laws and other regulations which authorize the data management function and therefore the pertinent laws and other regulations are not listed specifically in this text. ER 1110-2-240. Water Control Management (8 Oct 82). This regulation prescribes the policies and procedures to be followed by the US Army Corps of Engineers in carrying out water control management activities, including establishment of water control plans for Corps and non-Corps projects, as required by Federal laws and directives.

- 25 -

ER 1110-2-248. Requirements for Water Data Transmission Using GOES/DCS (13 Mar 81). This Engineer Regulation establishes the requirements for data transmission and retrieval via the Geostationary Operational Environmental Satellite (~ES) Data Collection System (DCS) operated by the National Earth Satellite Service (NESS) of the National Oceanic and Atmospheric Administration (NOAA). ER 1110-2-249. Management of Water Control Data Systems (31Aug 94). This engineer regulation provides guidance for the management of water control data systems including the equipment and software used for acquisition, transmission, and processing of real-time data used to regulate water projects for which the Corps of Engineers is responsible. ER 1110-2-1455. Cooperative Stream Gaging Program (29 Jan 84). This regulation establishes policy, provides background, and discusses management of the Cooperative Stream Gaging Program between the U.S. Army Corps of Engineers (USACE) and the United States Geological Survey (USGS). It also presents the program requirements for Field Operating Activities (FOA). ER 1110-2-8155. Hydrometeorological Data Management and Archiving (31 Jul 96). This regulation establishes policy and provides guidance and procedures applicable to the management and archiving of hydrometeorological data used for planning/project studies and project operations. It provides a recommended format for archiving data. Other related ERs include: ER 1110-2-1400 Reservoir/Water Control Centers (30 Sep 93) ER 1110-2-1460 Hydrologic Engineering Management (07 Jul 89) ER 1110-2-1463 Hydrologic Engineering for Hydropower (30 May 92) ER 1110-2-1464 Hydrologic Analysis of Watershed Runoff (30 Jun 94) ER 1110-2-3600 Management of Water Control System (30 Nov 87) ER 1110-2-8156 Preparation of Water Control Manuals (31 Aug 95) 7. Conclusions The WCDS Transition Team recommends:

4. The establishment of the recommended WCDS Steering Committee to set the vision, scope, and priority of work for the Columbia Basin Regional WCDS. This committee should be rooted in the Reservoir Regulation offices as described in ER 1110-249.

5. The establishment of the recommended WCDS Technical Coordination Team lead that would function as the WCDS PM.

6. The establishment of the Technical Coordination Team which should function as a PDT under the WCDS PM with the responsibility of coordinating the functional areas of WCDS throughout the region.

7. The implementation of the recommended 3-node architecture, beginning with the development of a Transition Plan. The recommended architecture would fulfill the needs of the Columbia Basin region’s reservoir regulation offices and

- 26 -

auxiliary users such as the dam operators and partnering agencies (BPA, RFC, and others). The recommended architecture:

a. Minimizes failure points for key processes, b. Has an integrated active COOP, c. Aligns pertinent data with the appropriate reservoir regulation office, d. Aligns tasks such as QA/QC with the most knowledgeable office.

8. The documentation of the system by a Master Plan with a routine schedule for review and updates. Coordinate Master Plan with the Missouri Basin to assure that systems are being migrated toward a convergent path. Some key items to include in the Master Plan are:

a. Management Structure b. System Objectives c. SOPs d. MOAs e. Short and long-term plans f. WCDS PMP with 5 year outlook g. Programmatic Budget with schedules and work plans h. Data Collection Work Plans i. Operational Maintenance Procedures

8. Definitions CAVI (CWMS Control and Visualization Interface) - The CAVI component enables the user to perform CWMS command and control functions, visualize system status, and review output products. CEEIS – Corps of Engineers Enterprise Infrastructure Services is the internal Corps network (Intranet Segment) that can only be accessed by a Corps of Engineers computer. CIAS – Controlled Internet Accessible Segment with access granted to only specific IP addresses. Continuity of Operations (COOP) - The ability to continue to perform any key Water Control Data System function or set of functions such as data collection, acquisition, posting, QA/QC, execution of key applications, or dissemination of data when the regular mode of operation fails in whole or in part due to planned or unplanned events. CWMS (Corps Water Management System) - provides tools and information management for accomplishing the water management mission, including reservoir and river system status monitoring, flow regulation, and decision support. CWMS facilitates access to and sharing of water management-related information among district, division, HQUSACE staff, and staff of cooperating Federal, state, and local agencies. Data Acquisition – Data acquisition is the processes used to move data from receipt via satellite, ftp, etc… to the format needed to write it to the “incoming” directory (any data manipulation prior to Data Capture – need definition)

- 27 -

Data Collection – Data collection is the measurement of a value in the field, logging the value in the Data Collection Platform (DCP), and all transmission or manipulation of the data prior to receipt by the data acquisition process. Extranet – Access controlled via username and password on public internet website. LRGS – Local Readout Ground Station provides a means to receive hydro-meteorological data transmitted via GOES satellites. Posting – Moving the data from the acquisition process into the database. Obtaining data from an “incoming” directory is the beginning of the posting process. Production Network – Is any network or segment that is nested under the control and security of the CEEIS network. SHEF – Standard Hydrologic Exchange Format is used to exchange hydrologic data between agencies using a standard format.

- 28 -

Appendix A: Detailed Description of Existing Water Control Data Systems A.1 Division-Columbia Basin The Columbia Basin Reservoir Control Center (RCC) relies on the central database and the products generated by the Division-Columbia Basin WCDS. The Division-Columbia Basin WCDS office acquires a host of data from around the region, some of which is provided by the district offices. Upon acquiring the data, the division office makes the data available to a number of stakeholders as well as posting it to the central database. The Division-Columbia Basin WCDS also exchanges data with the River Forecast Center, BPA, and other partners. Reports and web pages are generated from the central database for the user community. See Diagram A-1 for a description of the Division-Columbia Basin WCDS. A.2 Portland District The Portland District office relies on the Division-Columbia Basin WCDS since it has no data management system of its own. To accomplish their water management mission, the Portland District relies heavily on the River Forecast Center and USGS website due to the usability of their products. Data display applications, reports, and web pages managed by the Division-Columbia Basin WCDS are also used to accomplish the district water management mission. There are several Excel spreadsheet tools that connect to the Division-Columbia Basin WCDS database and are used by the Portland District water management team as well. Data collected via line-of-sight radio at the Portland District is transferred to the Division-Columbia Basin WCDS database. Refer to Diagram A-1 “Portland District” to review Portland District data collection machines within the current WCDS framework. The Portland District data collected via radio can also be viewed during and after the collection process using the Sutron XConnect software installed on individual PCs accessing a central data collection database. The Portland District project operators and regulators have access to the hydro-meteorological radio data via the XConnect software and rely heavily on this data for real-time operation and regulation of the projects.

- 29 -

Diagram A-1: Division-Columbia Basin WCDS

- 30 -

A.3 Seattle District The Seattle District office has a WCDS which is independent of the Division-Columbia Basin WCDS. The Seattle District office acquires most of the data needed by the district, either through direct collection or agency data sharing, and posts the data to the local database as well as sending most of the data to the Division-Columbia Basin WCDS. The Seattle District does not provide all data to the Division-Columbia Basin WCDS due to data sharing agreements with some partners. The Seattle District office exchanges data directly with the River Forecast Center and other partners. The Seattle District office relies on the division office for CBT data and for maintaining web pages and reports. The Seattle District also maintains independent web pages to support district project regulators. See Diagram A-2 for a description of the Seattle District WCDS data collection and Diagram A-3 for a description of the Seattle District WCDS server configuration.

- 31 -

Diagram A-2: Seattle District WCDS – Data Collection

- 32 -

Diagram A-3: Seattle District WCDS – Server Configuration

- 33 -

A.4 Walla Walla District The Walla Walla District office has a WCDS which is independent of the Division-Columbia Basin WCDS. The Walla Walla District office acquires most of the data needed by the district either through direct collection or agency data sharing and posts the data to a local database as well as sending most of the data to the central database. The Walla Walla office has direct access to all data of interest with the exception of CBT data which is managed by the Division-Columbia Basin WCDS. The Walla Walla District office coordinates directly with partner agencies to share data. The Walla Walla District maintains independent web pages along with making use partner agency web pages and web pages and reports supported by the Division-Columbia Basin WCDS. Diagram A-4: Walla Walla District WCDS

- 34 -

A.5 Division-Missouri Basin Diagram A-5: Division-Missouri Basin WCDS

- 35 -

A.6 Current Applicable Memorandums of Agreement There are a number of Memorandums of Agreement (MOAs) currently in place. The key agreements include:

1. The CROHMS Design Memorandum No. 1, April 1974 2. Columbia Basin Water Control Data System Master Plan, 1990 3. CBT Manual 4. Portland District TMDLs (Applegate Temperature; Willamette River

Temperature, Mercury, Bacteria, and Turbidity; Willow Creek Temperature and pH; Lower Columbia River Projects Total Dissolved Gas and Temperature)

5. Seattle MFR (see below) The Seattle MFR is as follows: MEMORANDUM FOR RECORD SUBJECT: Agreements to Provide or Receive Water Management Data Seattle District Water Management Section is required to provide accurate and responsive hourly data in support of the Districts’ Flood Control mission. Several organizations make available the data necessary by automated methods to support this requirement. In addition, Seattle District also provides data to cooperating agencies and our partners in support of their water management, meteorological, data collection, and modeling efforts. The following list provides a brief summary of cooperating agency agreements and data sharing arrangements 1. National Weather Service, River Forecast Center (RFC)

• Seattle District provides real-time hourly data for approximately 30 stations located throughout Seattle District including parameters from Libby Dam, Howard Hanson Dam, Mud Mountain Dam, and Wynoochee Dam to the River Forecast Center.

• Seattle District provides Seattle City Light real-time data for stations in the Upper Skagit basin including parameters at Ross Dam, Gorge Dam, and Diablo Dam to the River Forecast Center.

• Seattle District receives real-time hourly meteorological and hydrological forecasts for all Seattle District basins and projects including Chehalis River, Skagit Rive, Olympic Peninsula rivers, Puget Sound and all of Western Washington basins alerts from the River Forecast Center.

• Seattle District receives real-time hourly-automated weather warnings and alerts from the River Forecast Center.

2. US Geological Survey (USGS) • Seattle District provides real-time hourly data for stations associated with

Howard Hanson Dam and Mud Mountain Dam to the USGS. • Seattle District receives real-time hourly data for all stations (approximately 130)

within Seattle District boundaries from the USGS. 3. Tacoma Public Utilities (TPU)

- 36 -

• Seattle District receives real-time hourly data for stations in the Chehalis River basin including parameters at Wynoochee Dam from Tacoma Public Utilities.

4. Corps of Engineers, Division Water Management (NWD) • Seattle District provides real-time hourly data for all locally collected

hydrological and meteorological stations to be used in the Division CWMS and WCDS databases to Division Water Management.

• Seattle District receives periodic USGS rating table updates for use in local flow calculations and studies from Division Water Management.

• Seattle District receives Chief Joe Power Loss program updates via automated system located at Division office.

5. King County • Seattle District provides real-time hourly data for stations along the Green River

basin used in real-time modeling to King County. 6. Puget Sound Energy

• Seattle District receives real-time hourly data for stations in the Baker Lake basin include parameters for Upper Baker Dam and Lake Shannon reservoir from Puget Sound Energy.

• Seattle District receives real-time hourly data for stations in the Skagit River basin from Puget Sound Energy.

7. Seattle City Light • Seattle District receives real-time hourly data for stations in the Upper Skagit

basin including parameters at Ross Dam, Gorge Dam and Diablo Dam from Seattle City Light.

• Seattle District receives hourly data for stations in the Cedar River basin from Seattle City Light.

8. Snohomish County • Seattle District receives real-time hourly data for stations in the Snohomish and

Stillaguamish basins. 9. Environmental Protection Agency (EPA)

• Seattle District provides hourly data for stations monitoring meteorological information for the Bunkerhill Superfund site.

10. US Forest Service • Seattle District provides real-time monitoring of the Bear Paw site including

periodic reports. 11. Howard Hanson Additional Water Storage, Shannon and Wilson

• Seattle District provides real-time hourly data for stations at Howard Hanson Dam to the contractor Shannon and Wilson.

12. Various local, state, federal and tribal agencies and organizations • Seattle District provides Lake Washington Ship Canal fish passage reports

including water quality and fish count data to various local agencies and organizations.

13. 3Tier Environmental Forecast Group • Seattle District provides real-time hourly data for stations along the Green River

basin for the purpose of producing independent meteorological and inflow forecasts to the 3Tier group.

- 37 -

• Seattle District receives real-time hourly meteorological and inflow forecasts as input into local CWMS for modeling and display purposes from the 3Tier group.

David A. Gustafson Water Management Section Seattle District

- 38 -

Appendix B: Data Dissemination - Extent and Types B.1 Extent of Data Dissemination Data Dissemination is very broad in its reach and can therefore can be initiated by various processes and applications. Data Dissemination can also take on many forms and therefore produce a variety of products. Below are some lists to help provide the breadth and scope of Data Dissemination. Data dissemination occurs during the following processes:

• data acquisition • data corrections • QA/QC • applications • web processes • LRGS decoding • modeling

Data dissemination products include:

• text files • static reports • static graphs • ad-hoc query tools that include tabular or graphic data • model input (files) • teacups • web reports • web graphs • web ad-hoc query tools that include tabular or graphic data • PDF files • web superpage and widgets

B.2 Data Dissemination Venues

B.2.1 Models Models are an important and necessary aspect of the Water Management System and create their own unique data management issues. Before models can be used the input data must be produced, retrieved, and formatted correctly. Preprocessing scripts and procedures are needed for this task. Automating these tasks as much as possible can save a substantial amount of time for those running the models. Interim model results may also need to be managed if those results are used as input to a sequential model run. Models related to the Columbia Basin as well as other models may need to be used by multiple offices in the region as well as the results from those models. Version control of models and model results would need to be managed as well as

- 39 -

the ability to lock model configurations when in use so one office does not step on another office for cases where models are used regionally. Procedures and automated processes for routinely, accurately, and consistently sharing model configurations and model results across the region would need to be implemented. The system architecture supports sharing of model results that are posted to the database. The regulators throughout the region also rely heavily on the RFC models and therefore require the ability to access and run the RFC models from any regulating office. Presently a dedicated line between the Corps and RFC is being used by the RCC and Portland District reservoir regulators. It is highly desirable that the other two regulating office would be able to use that dedicated line as well when running the RFC model. To capture and document the processes and procedures for processing the necessary model data and handle the file sharing correctly a SOP would need to be developed which describes these elements for running models, especially those models that are shared between offices. Below are some of the models that need to be considered when addressing handling of input, output, and configuration files. Regulation models:

• RES-SIM • HEC-RAS • HMS • NWS-RFS • Long Range Forecasts • Snowmelt • Flood Insurance Studies • Legacy models

Water Quality models:

• CE-QUAL-W2 • SYSTDG

B.2.2 Internal Web To remain consistent with the overall agreed upon system architecture and provide COOP for data dissemination there would be an internal web server at each node (Portland, Seattle, and Walla Walla). Each internal web server would be kept in sync with the others through an automated process. Each node would use the local internal web server but would have the ability to switch to either of the other internal web server at the other nodes if the local server is down. The internal web function would reside on the same server as the applications and file sharing functions.

- 40 -

A Superpage or Dashboard concept would be implemented which allows internal users to configure what they see on their internal web pages. A generic Superpage would be developed for each regulating office and users would be able to access the generic regulating office page or define a user configurable page. The Superpage implementation would transition the current content on the regulating offices pages, same look and feel, to the Superpage architecture. Eventually the regulating offices would be able to take advantage of the flexibility and configurability of the new Superpage architecture in coordination with the other regulating offices. Examples of general content types and functionality on the internal web server:

• Static reports • On-demand reports similar to static reports that can be run on the fly,

“update now” • Ad-hoc data queries • Superpage configuration • Superpage widgets • Sharing documents such as regulation schedules, Excel files, and planning

documents • Make use of WIKI • Teacup diagrams

File sharing functionality would reside on the same local server as the internal web functionality. The files would be synced between nodes for webpage support and other purposes. If temporary file locks are possible for such activities as editing of files then some of the file sharing issues may be solved. However, there would still be cases where the shared files would need to be organized in such a way as to account for the following permissions:

• shares that are read-only • shares that are read-write for all • shares that are read-write for a group • shares that are read-write for an individual

A robust file syncing utility would manage transfer of files between nodes. These files may be text files, models, and other applications. The syncing utility would have the ability to compare and manage content between multiple nodes without getting confused, similar to the Oracle Streams database functionality. B.2.3 Public Web External, static web pages (public dissemination) would occur through the use of a Water Management enterprise web server. The Columbia Basin region plans to continue to provide an ad-hoc query tool to the public. It is assumed that ACE-IT is responsible for the public web server and they would provide a COOP for the server.

- 41 -

A method for selecting what data can be distributed to the public needs to be established so that data such as forecasts and generation are not released to the public. Examples of general content types and functionality on the public web server:

• Static reports • Ad-hoc • FTP - Distribution to the public from the database • Teacups

B.2.4 CIAS Access to computers controlled by IP address. This is not as secure as the Internal Web but is used to allow partnering agencies access to pertinent information that is not appropriate for public release on a regular basis. Examples of applications and processes available on the CIAS:

• CBT • FTP - Distribution to partner agencies during data acquisition • FTP - Distribution to partner agencies after posting

B.2.5 Extranet Access controlled via username and password on public website. This type of site allows information to be stored on an internet site but limits the access to who can view that data by requiring a username and password to access the site. This type of site is not as secure as an internal site but provides greater flexibility for accessing information since a government computer is not required to access an Extranet site. Examples of possible extranet site content:

• Superpage • FTP - Distribution to partner agencies from the database (Seattle) • Teacups

B.2.6 Tools Tools are used to gather information from the database and format that information to provide users with pertinent and useful data. Examples of general types of tools include:

• Reporting tool • Graphing tool • Superpage widgets • Superpage configuration interface • QA/QC review and corrections tools • Paging system

- 42 -

B.2.7 eGIS WCDS would look to conform to eGIS standards as much as possible and share information with eGIS efforts so that Water Management can gain the greatest advantage possible of eGIS products.

- 43 -

Appendix C: ACE-IT Questions and Interactions

1. Is a UNC share (Samba mount) of a UNIX directory on a Windows PC acceptable?

2. Will ACE-IT manage the public web server? 3. Will ACE-IT allow an ad-hoc tool on the public web server that can access the

CWMS database on the Production Network? 4. How will ACE-IT impact the ftp/acquisition server (and server coop) and data

sharing with partner agencies? 5. Will storage be local or not? 6. DIACAP (security process and acronym that replaced DITSCAP): How will this

be handled under ACE-IT? Will there be a national one for all the Corps or for all of Water Management or will DIACAP be handled as separate local ones for Water Management? Is HEC looking into this?

7. Network stack: currently the Division WCDS has a Water Management network stack. Do we need/want to maintain this under ACE-IT? If so, do we want a separate network stack at each node? How does ACE-IT want to handle Water Management server connections to the Corps network?

- 44 -

Appendix D: Categorized Function Spreadsheet Exercise During team discussion a list of WCDS functions was generated and categorized. The categories the team agreed on for the discussion of the functions performed as part of the WCDS were:

1. Data Collection 2. Data Acquisition 3. Posting 4. Database Administration 5. System Administration 6. Data Dissemination 7. Application and Process support 8. QA/QC 9. Hardware support and maintenance 10. Security (Information Assurance) 11. On-going technical improvements 12. Documentation and 13. Coordination

During work on the spreadsheet there were a number of key assumptions made. These included:

1. This effort did not reflect COOP resources to perform the functions. 2. This effort does not quantify the level of effort to perform each task. 3. Specifying “any one location” does not indicate any function or task is limited

to a particular location. 4. Any task that is identified to be performed at any one location requires some

level of backup at a secondary location. The WCDS Transition Team worked through all the functions and identified

• whether the function needed a formal Point-of-Contact (POC) in the Columbia Basin

• whether ACE-IT would be involved • whether the function needed to be performed at each regulating office • whether the function needed to be performed at each computer system node • whether the function could/should be performed at a single location anywhere

in the region Recommendations on where functions should be performed are presented in the following table. Function Category Comments Application and Process Support

Regionally standardized applications and process support will be at any one location with locally specific elements performed at each office.

Coordination Generally, needs to be performed at each local office. This category reflects internal coordination tasks. ACE-IT coordination is captured under each specific category as necessary.

Security (Information Primarily performed by ACE-IT with coordination at each node and

- 45 -

Assurance) a primary lead. System Administration Performed at any one location with a lot of ACE-IT involvement. A

high level of regional coordination will occur between all staff assigned SA responsibilities.

QA/QC Primarily performed at each local office with regional coordination and no ACE-IT involvement.

Posting Posting configuration and monitoring performed at any one location. Monitoring and control of posting processes would reside at each node and would require extensive coordination.

On-going Technological Improvements

On-going technological improvements will be highly coordinated throughout the region at the Technical Coordination Team level and at the WCDS Steering Committee level. CWMS related development will be focused at any one location. Other development efforts could occur at all four offices with regional coordination.

Hardware Support/Maintenance

Primarily performed by ACE-IT with coordination at each node and a primary lead.

Documentation All offices will contribute to the development of the documentation products with the primary writing efforts occurring at any one location. The products address and represent the entire WCDS.

Database Administration (DBA) Primary responsibility will be assigned to one location with cursory capabilities at all nodes. A high level of regional coordination will occur between all staff assigned DBA responsibilities.

Data Dissemination In order to promote standardization, system related development and information sharing can occur at any one office with additional data dissemination product development occurring in the local offices. All efforts will include regional coordination and some level of ACE-IT involvement.

Data Collection Data collection responsibility is located at each office or node with regional coordination. Regional aspects of data collection, such as NESDIS and the regional ftp server, can be coordinated from any one location.

Data Acquisition Regionally standardized processes will be developed at any one location with locally specific elements performed at each office.

During the Transition Planning process it would be advantageous to determine any function that should be co-located with another function.

- 46 -

Appendix E: WCDS Steering Committee Recommendations E.1 WCDS Steering Committee Members

• 4 members, one from each of the four regulating offices (Division RCC, Portland, Seattle, and Walla Walla). These members would be the Reservoir Regulation Chiefs with authority to direct and commit resources and make decisions in their respective office. Each member should be able to adequately convey the needs of their Water Management office.

Note: The team did not reach consensus regarding the division representative on the steering committee. Some members of the team recommend that the offices have the latitude to select their representative from within Water Management.

• In compliance with the PMBP process, the Steering Committee is not intended to reflect all resource providers for the WCDS Technical Coordination Team or WCDS staff. The Steering Committee membership is intended to reflect the primary users of the system, setting system vision, scope and priorities.

• The WCDS Technical Coordination Team Lead who is selected by the WCDS Steering Committee

• 1 member from the H&H Corps community to provide a broader water resources perspective. This member may include but is not limited to an H&H Chief or equivalent. This member could help expand the usability of WCDS beyond real-time reservoir regulation. This team position would rotate between district and division offices. Each year a new member would be chosen by the committee. The district or division office with the responsibility to supply an H&H representative for that year would make a suggestion and then the committee would vote to accept or reject the representative.

• The committee may request ad-hoc knowledge experts to attend meetings and provide insight, perspective, and suggestions on various topics.

• Possibly one Missouri Basin WCDS Technical Coordination Team representative. • Potentially one representative from Operations may also be considered after the system is

meeting Water Management needs. • In the future the WCDS Steering Committee could be expanded to include ad-hoc

representation from offices beyond water management at the discretion of the WCDS Steering Committee.

E.2 WCDS Steering Committee Responsibilities

• Set vision, scope and priorities for the WCDS • Coordinate responsibilities between offices for data activities including which office does

QA/QC for which data collection stations. • Provides priorities for the system (reservoir regulation, water quality, treaty obligations,

dam safety, hydrology, hydraulic design, emergency management) • WCDS Steering Committee reports to Division Water Management Chief • Identifies necessary Standard Operating Procedures for the region • Updates and coordination WCDS Master Plan

- 47 -

E.3 Committee Norms • The committee would determine their decision making process. • The committee would select the committee chairperson, initially choosing one of the

Reservoir Regulation Chiefs. • The committee would determine frequency of meetings during transition to WCDS end-

state. • Routine quarterly face to face meetings after transition phase or as frequently as needed. • Ability for group or individual member to call for additional teleconference or face to

face meetings as needed. Examples why additional meetings may be needed include, but are not limited to, shifting WCDS priorities, another Libby type situation, or additional federal litigation requirements.

• The committee would be comprised of staff from different organizations with different organization structures and grade levels. Every effort must be made to assure that all members are equal participants.

- 48 -

Appendix F: Technical Coordination Team Recommendations F.1 Technical Coordination Team Structure

• Team lead chosen by the WCDS Steering Committee and continues to serve as lead at the discretion of the committee.

• Team lead serves as a member on the WCDS Steering Committee. • Team members are all chosen by the WCDS Steering Committee. • Team consists of between 3 and 7 technical leads that have leadership abilities

along with proven communication and coordination skills. • It is advisable but not necessary that team members be within one of the four

reservoir regulation offices. • The collective technical oversight of the team members encompasses all

functional areas of WCDS.

F.2 Technical Coordination Team Accountability • The team is a PDT with the PM being the WCDS Steering Committee. • The team lead submits routine reports to the WCDS Steering Committee. • Strongly recommend team members have a direct supervisor on the WCDS

Steering Committee. • If a team member does not have a direct supervisor on the WCDS Steering

Committee then the Committee Chair would interact with the team members direct supervisor, conveying job performance standards, goals, objectives, and work activities.

F.3 Communication

• Team lead communicates with the WCDS Steering Committee and represents the team on the committee.

• Team lead fosters and facilitates communication between team members. • Team members develop PDTs in their functional areas, incorporating persons

from the greater WCDS working group. • Team members interact with the user community.

F.4 Summary of Functions Depending on the nature of the function, some functions could be performed from one location while others were more closely tied to local offices and/or the user community and therefore it would be advantageous to perform those functions in closer proximity to their user base. The functions that could be performed at any one location include:

• Posting • Database Administration • System Administration • Hardware support and maintenance • Security • Documentation

- 49 -

The functions that are more closely tied to local offices and/or the user community and, therefore, advantageous to perform closer to their user base include:

• Data Collection • Data Acquisition • Data Dissemination • Application and Process support • QA/QC • On-going technical improvements • Coordination

There are still many unknowns regarding the role of ACE-IT and what support role they would have in regard to Water Management hardware, software, and security. Therefore not all the functions that Corps employees would be performing can be completely defined. The WCDS Transition Team, to the best of their knowledge, developed a spreadsheet with the WCDS functions and who might be responsible for executing those functions in the WCDS end-state.

- 50 -

Appendix G: Comparison of Proposed WCDS End-State with ER 1110-2-249

G.1 Introduction The regulation which addresses the management of Water Control Data Systems is ER 1110-2-249. It is the team’s perspective that this ER, which was published in August 1994, needs to be revised and updated. A general description of how the WCDS end-state defined in this report complies with and differs from ER 1110-2-249 is presented below. Though the discussion is not exhaustive it does demonstrate that the proposed WCDS end-state is in agreement with ER 1110-2-249 in regard to aligning the management of WCDS with the reservoir regulation offices. G.2 Current ER Considers a WCDS at Each Local Office The current ER considers a WCDS at each local office and does not address a regional system and its management. Therefore there has to be some acknowledgement and incorporation of the regional aspects of this proposed WCDS structure. Highlights (excerpts) of the similarities and differences are captured in the following discussion.

G.3 ER System Administrator – WCDS Steering Committee Chairperson The WCDS Steering Committee Chairperson in cooperation and consent of the rest of the WCDS Steering Committee would satisfy the roles and responsibilities of the position identified in the ER as the System Administrator. Below are excerpts from ER 1110-2-249 with comments in italics parenthesis that describe the similarity to the proposed WCDS.

G.3.1 Description of the System Administrator in the ER The WCDS System Administrator shall be a member of the water control management chain of command. (The WCDS Steering Committee Chairperson would be one of the Reservoir Regulation chiefs.) The System Administrator shall be either the person directly responsible for water control activities or that person’s immediate supervisor. Typical positions would be the Chief of Reservoir Regulation or the Chief of H&H. (The WCDS Steering Committee would consist of these persons within the region.)

G.3.2 Responsibilities of the System Administrator in the ER

1. Interpreting and administering this regulation. (The WCDS Steering Committee would do this while setting the vision and scope of WCDS.)

2. Appointing the System Manager. (The WCDS Steering Committee would choose the Technical Coordination Team and their lead.)

3. Ensuring that the WCDS is dedicated to the support of the water control activities. (Reservoir Regulation chiefs have this as their mission.)

4. Supervising the activities of the System Manager. (The WCDS Steering Committee would oversee the work of the Technical Coordination Team and lead. Since each member of the Technical Coordination Team would

- 51 -

have their direct supervisor on the WCDS Steering Committee the issue of supervision is represented well.)

5. Overseeing the management of the WCDS, including planning and budgeting, upgrading the WCDS, providing security and physical access to the WCDS computer system, determining the physical location of WCDS components, and operating and maintaining the WCDS. (The WCDS Transition Team agreed that the WCDS Steering Committee would set the vision, scope, and direction of the Technical Coordination Team.)

6. Ensuring that a WCDS continuity of operations plan (COOP) is developed and tested. (The proposed architecture has an integrated active COOP.)

G.4 ER System Manager – WCDS Technical Coordination Team Lead The Technical Coordination Team Lead in cooperation and consent of the rest of the Technical Coordination Team would satisfy the roles and responsibilities of the position identified in the ER as the System Manager. Below are excerpts from ER 1110-2-249 with comments in italics parenthesis that describe the similarity to the proposed WCDS.

G.4.1 Selection of the System Manager in the ER The WCDS System Manager (Technical Coordination Team and Lead) shall be appointed by the System Administrator (WCDS Steering Committee). This person shall be assigned within the chain of command of the System Administrator (WCDS Steering Committee. Again the Technical Coordination Team members would have their supervisors on the WCDS Steering Committee because each member of the Technical Coordination Team would reside in one of the four Reservoir Regulation offices.) G.4.2 Responsibilities of the System Manager in the ER

1. Maintaining routine operation of the WCDS. (The Technical Coordination Team would oversee the routine operation of the WCDS.)

2. Serving as the initial point of contact for access, problems, and issues related to the WCDS. (Either the Technical Coordination Team Lead or the team member that was the specific lead for the functional area of interest would be the initial point of contact.)

3. Selecting and implementing WCDS applications software. When acquisition of software is necessary – to include the specifications development and selection of COTS products, and/or the development of Corps-unique software, and/or selection and implementation of appropriate WCDS applications software – it must be acquired and implemented in accordance with all regulations. (The Technical Coordination Team would perform this function.)

4. Authorizing the establishment of USERIDs, passwords, and user numbers in accordance with Corps-wide standards to permit access to the WCDS. (A member of the Technical Coordination Team would do this most likely in coordination with ACE-IT.)

- 52 -

5. Preparing an operation procedures manual for the WCDS computer system. (The Technical Coordination Team would produce this manual and the WCDS Steering Committee would approve it.)

6. In coordination with the WCDS Site Manager and the WCDS Data Acquisition Systems Manager (these two persons would be leads on the Technical Coordination Team), preparing, validating, and testing the COOP.

7. Identifying and recommending, in consultation with leads necessary upgrades or expansions to the system. (The Technical Coordination Team would identify necessary upgrades and expansions and the Team’s recommendations would be reviewed by the WCDS Steering Committee.)

G.5 ER Site Manager and Data Acquisition Systems Manager – Members of TCT The WCDS Site Manager and the WCDS Data Acquisition Systems Manager are just two of the proposed leads on the Technical Coordination Team. The ER does not fully address all aspects of the WCDS in the roles specified and adding the additional leads identified for the Technical Coordination Team would help clarify and complete the ER. G.6 Role of the Division Water Management Chief The WCDS Transition Team did not include the Division Water Management Chief on the WCDS Steering Committee because it was felt that the Division Water Management Chief would better serve conflicts arising within the WCDS Steering Committee if the Chief was an outside observer rather than a committee member. Also in ER 1110-2-249 it states that there should be an annual review and inspection of the WCDS by the responsible division water control element. This could be the Division Water Management Chief. Therefore, reserving the Division Water Management Chief for use as an impartial inspector, reviewer, and conflict resolution resource would better server the WCDS overall.

- 53 -

Appendix H: System Architecture Considerations and Alternatives

1. Overview In determining a system architecture that would satisfy the needs of the region, representatives from the three district offices along with the division office brainstormed and come up with seven alternatives. The offices had different requirements and therefore varying criteria associated with the differing needs for judging if the architecture was adequate. Most of the key elements used to develop the proposed architecture are listed below.

The team agreed that the system architecture should accommodate these key elements:

• a WCDS node at each of the three sites (Portland, Seattle, and Walla Walla) • the complete suite of WCDS hardware and software at each node • an LRGS at each node for receipt of GOES data • the capability at each node to acquire most of the regions data • a CWMS v2 Oracle database and related software at each node • use Oracle Streams to synchronize the databases at each node in real time • replicate and synchronize system and user files across each node • have the ability for any node to operate stand-alone in a contingency situation

Most of the team also agreed the system architecture should accommodate these key elements:

• an active COOP • when operating in a contingency mode full, normal, production capabilities

should be maintained • QA/QC performed at each office with ability to perform QA/QC on local

computer In the end the architecture chosen met the needs of all offices. The alternative chosen was Alternative 4 which is described in the following section. The chosen architecture may not have been the best solution for any given office but since all the offices in the region agreed on this architecture it should serve the region well. All key elements from the two lists above can be handled well in the alternative chosen. The main point of concern was using the Oracle Bi-Directional Streams with three nodes. After much discussion, some research, and consulting with the Missouri Basin Region, which is using bi-directional Oracle Streams with two nodes, all members on the team were comfortable with the use of bi-directional streams and therefore Alternative 4 was the alternative all members could agree on. For further information on Oracle Streams refer to Appendix H. For additional information about the alternatives brainstorming effort refer to the whiteboard illustrations document in the notes associated with the End-State. 2. Alternatives Discussed

- 54 -

The team brainstormed and come up with seven possible system architectures. Each alternative was considered and the benefits and drawbacks of each were discussed. Since each office had differing criteria to meet their specific needs and since there were a number of reasons that were not easily quantifiable the team looked for the architecture that satisfied the needs of all offices. The alternatives considered follow.

2.1 Alternative 1: Post all data to a central node, Stream to the remote nodes The Alternative 1 architecture was proposed by the Division Columbia Basin Water Management in 2006 for the migration of the regional WCDS to one based on the CWMS v2 database. This alternative builds on the concepts of the current regional WCDS, having a central database in Portland, and adding one-directional streaming of data in real-time to remote nodes at Seattle and Walla Walla. All LRGS (GOES) data as well as other agency data would be collected at Portland and streamed out to Seattle and Walla Walla. Local data collected at Seattle and Walla Walla would first be sent via FTP to Portland where it would be posted to the database and then sent back to Seattle and Walla Walla via one-directional Streams. QA/QC of data would occur in the Portland database by each office and the updated data would be streamed out to Seattle and Walla Walla. Meta-data would also be maintained in the central database and streamed to the local nodes. Therefore, a complete database would reside at each of the three nodes. However the three nodes are not identical because the Portland database would be the only one that controls data flow and would likely have to be backed up somewhere for complete COOP. For normal operations except for QA/QC, users in Seattle and Walla Walla could access either the resources on their local server or at the server in Portland. The QA/QC process would have to access the Portland node since data only flows from the Portland node to the other nodes. In the event of a significant loss of network connectivity or processing capabilities in Portland, the nodes in Seattle and Walla Walla would activate posting of data to the local database and operate locally. During recovery from a Portland node failure it was not clear how data posted in Seattle and Walla Walla would get back to the Portland database. Also, during the time the Portland node was inoperable the local data normally collected in Portland would have to be sent to either Seattle or Walla Walla and Portland staff would work remotely until the Portland server was repaired.

2.2 Alternative 2: Post all data locally, Stream through a central node In Alternative 2 each node would collect the data it was responsible for and would post that data to the local database. QA/QC of data and posting of any corrected data would also occur at the responsible node. Data posted in the databases would be sent via bi-directional Streams between Seattle and Portland and Walla Walla and Portland. Meta-data would be maintained locally and sent out via Streams. Therefore, a complete database would reside at each of the three nodes. However the only way that Seattle or Walla Walla would get all the data for the region is through Portland. If either Seattle or Walla Walla went down then Portland would post all data for the failed node as long as any locally collected data from that node was sent to Portland. If Portland went down

- 55 -

then both Seattle and Walla would have to post all data collected locally at the other nodes because without Portland the databases are not connected via Streams.

2.3 Alternative 3: Local Posting, Circular Streaming In Alternative 3 each node would collect the data it was responsible for and would post that data to the local database. QA/QC of data and posting of any corrected data would also occur at the responsible node. Data posted in the databases would be sent via one-directional Streams. Seattle would send data to the Portland database which in turn would send it on to Walla Walla. Portland data would be sent to the Walla Walla database which in turn would send it on to Seattle. Walla Walla would send their data to the Seattle database which in turn would be sent to Portland. Meta-data would be maintained locally and sent out via Streams. Therefore, a complete database would reside at each of the three nodes. However if one of the nodes went down then the chain would be broken. Data for the lost node would have to be posted by the next node in the ring. 2.4 Alternative 4: Local Posting, Fully Connected In Alternative 4 each node would collect the data it was responsible for and would post that data to the local database. QA/QC of data and posting of any corrected data would also occur at the responsible node. Data posted in the databases would be sent via bi-directional Streams to the other two nodes. Meta-data would be maintained locally and sent out via bi-directional Streams. Therefore, a complete database would reside at each of the three nodes. If one of the nodes went down then either of the remaining nodes would post the lost nodes data. 2.5 Alternative 5: Local and Remote Posting, No Streams In Alternative 5 each node would collect the data it was responsible for and would then post that data to the local database as well as the databases at the other two nodes. QA/QC of data would also occur locally and posting of any corrected data would occur at all three nodes. Therefore Oracle Streams would not be used at all. Meta-data would have to be handled by some means to make sure all three databases have the same meta-data. Therefore, a complete database would reside at each of the three nodes without using Oracle Streams however in-house applications would have to be developed to manage data flow which Oracle Streams would have done. If one of the nodes went down then either of the remaining nodes would post the lost nodes data. 2.6 Alternative 6: Central Data Acquisition, Stream Metadata In Alternative 6 each node would collect the data it was responsible for and would send the data to a remote Data Acquisition and Posting Server which would format the data for posting to the database. The remote Server would then post all data to all three nodes. The remote Data Acquisition and Posting Server would have to have a backup as shown in the diagram. QA/QC of data would occur locally but posting of any corrected data would occur through the Data Acquisition and Posting Server. Bi-directional Oracle Streams would be used to handle the transfer of updated meta-data between the three nodes. Therefore, a complete database would reside at each of the three nodes. However, if the Wide-Area-Network went down then no acquisition or posting would

- 56 -

occur. If one of the nodes went down the remaining nodes would have the lost nodes data if the local data collection at the failed node was still operational. If the Data Acquisition and Posting Server went down the backup server would take over. 2.7 Alternative 7: Local Posting, Stream to a Central Node In Alternative 7 Seattle and Walla Walla would collect and post all data each office needed and send that data to Portland via one-directional Oracle Streams. QA/QC of data would occur locally and Seattle and Walla Walla would send corrected data and updated meta-data to Portland via one-directional Oracle Streams. Any data collected at the Portland node that needed to be sent to one of the other nodes (Seattle or Walla Walla) including data that was changed during the QA/QC process and meta-data would be sent via an in-house application. A complete database would only reside at the Portland node and therefore Portland would use a program like Oracle DataGuard to keep an offline backup system synchronized. This backup would likely reside in Walla Walla. If either Seattle or Walla Walla went down then the Portland node would have to post the lost node’s data if available and the down node would work remotely through Portland. If the Portland node went down then the system would switch over to the backup servers in Walla Walla.

3. Conclusion All offices could agree that the architecture described in Alternative 4 would satisfy the needs their office had. Alternative 4 incorporated the key elements that all team members acknowledged and also incorporated the additional key elements most of the team sought. Therefore Alternative 4 was selected as the recommended architecture.

- 57 -

Appendix H: Notes on Oracle Streams

H.1 Oracle Streams Overview Oracle Streams enables the propagation and management of data, transactions and events in a data stream either within a database, or from one database to another. The stream routes published information to subscribed destinations. This feature of the Oracle database provides greater functionality and flexibility than traditional solutions for capturing and managing events (such as an ingested piece of data), and sharing the events with other databases and applications. As users' needs change, they can simply implement a new capability of Oracle Streams, without sacrificing existing capabilities. Streams can be implemented in a one-directional or bi-directional mode. In a one-directional implementation, changes made to a primary database are streamed and applied to a secondary database. Changes made to the secondary database are not streamed back to the primary. In a bi-directional implementation, changes made to either database are streamed and applied to the other database. There is no primary or secondary distinction. Oracle provides mechanisms to prevent recursive streaming of data in the bi-directional case. Note that the use of Oracle Streams would produce a “replicated” database architecture as opposed to a distributed database architecture. In such architectures all data exists at every node; in a distributed database architecture a single copy of the data exists in the collection of nodes, with each datum typically located at the node closest to where it was collected or is used. H.2 Missouri River Region’s Oracle Streams Experience In the summer of 2007 the Missouri Region Water Management put a two-node CWMS v1.5 system architecture into production which made use of bi-directional Oracle Streams for database synchronization. Prior to that time the Missouri River Region worked through problems due to bugs in the immature CWMS software as well as issues implementing Streams with the CWMS schema. In the Missouri River Region network traffic is carried by two T1 lines. With only qualitative measurements, the Missouri Region reported that this configuration works very well, with streamed data appearing at the remote database within seconds after it is posted to the local database.

- 58 -

H.3 Example from Oracle of a 3-Node Oracle Streams Configuration Oracle® 11g Streams Replication Administrator's Guide Chapter 21 N-Way Replication Example

- 59 -