Hydrology SAF System Integration, Verification...

This System Design Document specifies the architectural concept design of the Satellite Application Facility on Support to Ope

Hydrology SAF

System Integration, Verification & Validation Plan

Reference Number: SAF/HSAF/SIVVP/1.0Issue/Revision Index: Issue 1.0Last Change: 08/02/2008

DOCUMENT SIGNATURE TABLE

Name Date Signature

Prepared by : H-SAF Project Team 08/02/2008 Approved by : H-SAF Project Manager 08/02/2008

DOCUMENT CHANGE RECORD

Issue / Revision Date Description

Version 0.5 07/11/2006 Preliminary Release prepared for PDR

file:///C|/Documents%20and%20Settings/Panti%20Massimo/Desktop/CDR2_DOCS/sivvp_1_0.htm (1 di 30)27/06/2008 9.49.19


Version 0.6 14/06/2007 Updated Release prepared for V1CP:● Review of Applicable Documents list and Reference Documents list

as acknowledgement of RID 114: transfer of items to the second list as [RD 25] and [RD 26]

● Modification of section 3.1 with a more appropriate explanation of the aims. This acknowledges RID 98

● Insertion of a clarification about the internal/external interfaces test into section 2.3 and section 3.4. This acknowledges RID 99

● Updating of the test schedule description with further details , as acknowledgement of RID 97.

Insertion of table 2 for roles/responsibilities for Cluster Leaders and Product Managers; insertion of section 2.5 to define error management procedures and table 3 to summarize them. These actions acknowledge RID 100.

● Clarification of the use of the different test technique in the various phases (integration, system, unit test etc.). This acknowledges RID 96

Version 1.0 08/02/2008 Baseline Release prepared for CDR

DISTRIBUTION LIST

Country Organization Name Contact

Austria ZAMG Veronika Zwatz-Meise [email protected]

ZAMG Alexander Jann [email protected]

TU-Wien / IPF Wolfgang Wagner [email protected]

TU-Wien / IPF Stefan Hasenauer [email protected]

Belgium IRM Emmanuel Roulin [email protected]

IRM Gaston Demaree [email protected]

ECMWF Philippe Bougeault [email protected]

Matthias Drusch [email protected]

Klaus Scipal [email protected]

Finland FMI Pirkko Pylkko [email protected]

FMI Jarkko Koskinen [email protected]

FMI Jouni Pulliainen [email protected]

FMI Panu Lahtinen [email protected]

TKK Juha-Petri Karna [email protected]

SYKE Sari Metsämäki [email protected]

France Météo France Jean-Christophe Calvet [email protected]

Météo France Laurent Franchisteguy [email protected]

CNRS-CETP Mehrez Zribi [email protected]

CNRS-CESBIO Patricia de Rosnay [email protected]

Germany BfG Thomas Maurer [email protected]

Hungary OMSZ Eszter Lábó [email protected]



Italy USAM Massimo Capaldo [email protected]

CNMCA Luigi De Leonibus [email protected]

CNMCA Costante De Simone [email protected]

CNMCA Francesco Zauli [email protected]

CNMCA Daniele Biron [email protected]

CNMCA Davide Melfi [email protected]

CNMCA Attilio Di Diodato [email protected]

CNMCA Lucio Torrisi [email protected]

CNMCA Massimo Bonavita [email protected]

CNMCA Adriano Raspanti [email protected]

CNMCA Alessandro Galliani [email protected]

DPC Roberto Sorani [email protected]

DPC Luca Rossi [email protected]

DPC Silvia Puca [email protected]

DPC Angela Corina [email protected]

DPC William Nardin [email protected]

CNR-ISAC Franco Prodi [email protected]

CNR-ISAC Bizzarro Bizzarri [email protected]

CNR-ISAC Alberto Mugnai [email protected]

CNR-ISAC Vincenzo Levizzani [email protected]

CNR-ISAC Francesca Torricella [email protected]

CNR-ISAC Stefano Dietrich [email protected]

CNR-ISAC Francesco Di Paola [email protected]

Ferrara University Federico Porcu' [email protected]

Ferrara University Davide Capacci [email protected]

DATAMAT Flavio Gattari [email protected]

DATAMAT Emiliano Agosta [email protected]

Poland IMWM Piotr Struzik [email protected]

IMWM Bozena Lapeta [email protected]

IMWM Jerzy Niedbala [email protected]

IMWM Jaga Niedbala [email protected]

IMWM Monika Pajek [email protected]

IMWM Jakub Walawender [email protected]

IMWM Jan Szturc [email protected]

Romania NMA Andrei Diamandi [email protected]

NMA Simona Oancea [email protected]

Slovakia SHMÚ Ján Ka•ák [email protected]



SHMÚ Marián Jurašek [email protected]

SHMÚ Marcel Zvolenský [email protected]

Turkey TSMS Fatih Demýr [email protected]

TSMS Ali Umran Komuscu [email protected]

TSMS Ibrahim Sonmez [email protected]

TSMS Ayd•n Erturk [email protected]

METU Ali Unal Sorman [email protected]

METU Zuhal Akyurek [email protected]

METU Orhan Gokdemir [email protected]

METU Ozgur Beser [email protected]

ITU Ahmet Oztopal [email protected]

Anadolu University Aynur Sensoy [email protected]

EUMETSAT Lorenzo Sarlo [email protected]

Frédéric Gasiglia [email protected]

Dominique Faucher [email protected]

Jochen Grandell [email protected]

Lothar Schüller [email protected]

Table of Contents

1 Introduction1.1 Purpose1.2 Document Overview1.3 Glossary1.4 References

1.4.1 Applicable Documents1.4.2 Reference Documents

2 System Integration, Verification & Validation Overview2.1 General2.2 Techniques and methods2.3 Master schedule2.4 Roles and Responsibilities2.5 Problems Management

2.5.1 Process description2.5.2 Problem Review Board

3 Unit Test plan3.1 Introduction3.2 Test items3.3 Features to be tested3.4 Approach



3.5 Item pass/fail criteria3.6 Test Deliverables3.7 Environmental needs3.8 Schedule3.9 Approval

4 Component test plan4.1 Introduction4.2 Test items4.3 Feature to be tested4.4 Approach

4.4.1 Test cases specification4.4.2 Component Test Procedures4.4.3 Component Test Reports

4.5 Item Pass/Fail criteria4.6 Test Deliverables4.7 Environmental needs4.8 Schedule4.9 Approval

5 Integration test plan5.1 Introduction5.2 Features to be tested5.3 Approach5.4 Item Pass/Fail criteria5.5 Test Deliverables5.6 Environmental needs5.7 Schedule5.8 Approval

6 System test plan6.1 Introduction6.2 Test Items6.3 Features to be tested6.4 Approach

6.4.1 System Test cases specification6.4.2 System Test Procedures6.4.3 Test Reports

6.5 Item Pass/Fail Criteria6.6 Test Deliverables6.7 Environment6.8 Schedule6.9 Approvals

7 System validation test7.1 Introduction7.2 Test items



7.3 Feature to be tested7.4 Approach7.5 Item Pass/Fail Criteria7.6 Test Deliverables7.7 Environmental needs7.8 Schedule7.9 Approvals

8 TBDs/TBCs list

LIST OF TABLES

Table 1 Schedule of H-SAF test phasesTable 2 H-SAF roles and responsibilitiesTable 3 Problem Review Board rolesTable 4 H-SAF Components vs. Responsible Institute/clusterTable 5 H-SAF Components vs. Subcomponents

LIST OF FIGURES

Figure 1 Problem Management Process flow diagram

1 IntroductionThis document is the System Integration, Verification & Validation Plan Document for Satellite Application Facilities on support to Operational Hydrology and Water Management (H-SAF). It describes all tests phases of the H-SAF project at system level.

1.1 PurposeThis document aims to describe test plan for generation of Precipitation, Soil Moisture and Snow Parameters products and for their centralization, archiving and dissemination that shall be provided at the H-SAF sites CNMCA (Italy), ZAMG (Austria), ECMWF (UK), FMI (Finland), TSMS (Turkey) in accordance with the System Design Document [AD 7] and the Component Design Document [AD 9].It also covers test activities for what concern Hydro Validation subsystem tasks (product validation, reports production, etc.) and Offline Monitoring subsystem tasks.

1.2 Document OverviewThe following sections compose this document:

• Section 2 provides a general description and schedule of the verification, validation and integration of H-SAF subsystems. Here applied methods and techniques are formalized.• Section 3 describes the unit test plan of each H-SAF subsystem. • Section 4 describes the component test plan of each H-SAF subsystem. • Section 5 describes the integration test plan of H-SAF subsystem. • Section 6 describes the system test plan of H-SAF system.



• Section 7 describes system validation plan of H-SAF system.• Section 8 summarizes the TBDs and TBCs items, providing the responsible of each item and a planned date for resolution.

1.3 GlossaryAAPP AVHRR and ATOVS Processing Package

ADEOS Advanced Earth Observation Satellite (I and II)

ALOS Advanced Land Observing Satellite

AMIR Advanced Microwave Imaging Radiometer

AMSR Advanced Microwave Scanning Radiometer (on ADEOS-II)

AMSR-E Advanced Microwave Scanning Radiometer - E (on EOS-Aqua)

AMSU-A Advanced Microwave Sounding Unit - A (on NOAA satellites and EOS-Aqua)

AMSU-B Advanced Microwave Sounding Unit - B (on NOAA satellites up to NOAA-17)

API Application Program(ming) Interface

ASAR Advanced SAR (on ENVISAT)

ASCAT Advanced Scatterometer (on MetOp)

ASI Agenzia Spaziale Italiana

ATDD Algorithms Theoretical Definition Document

ATMS Advanced Technology Microwave Sounder (on NPP and NPOESS)

ATOVS Advanced TIROS Operational Vertical Sounder (on NOAA and MetOp)

AU Anatolian University

AVHRR Advanced Very High Resolution Radiometer (on NOAA and MetOp)

BAMPR Bayesian Algorithm for Microwave Precipitation Retrieval

BfG Bundesanstalt für Gewässerkunde

BRDF Bi-directional Reflectance Distribution Function

BVA Boundary Value Analysis

CASE Computer Aided System Engineering

CBA Component-Based Architecture

CBSD Component-based Software Development

CDA Command and Data Acquisition (EUMETSAT station at Svalbard)

CDD Component Design Document

CDR Critical Design Review

CESBIO Centre d'Etudes Spatiales de la BIOsphere (of CNRS)

CETP Centre d’études des Environnements Terrestres et Planétaires (of CNRS)

CI Configuration Item

CMIS Conical-scanning Microwave Imager/Sounder (on NPOESS)

CMP Configuration Management Plan

CNMCA Centro Nazionale di Meteorologia e Climatologia Aeronautica

CNR Consiglio Nazionale delle Ricerche

CNRM Centre Nationale de la Recherche Météorologique (of Météo-France)

CNRS Centre Nationale de la Recherche Scientifique

COM Component Object Model

CORBA Common Object Request Broker Architecture

COTS Commercial-off-the-shelf

CPU Central Processing Unit

CRD Component Requirement Document



CVERF Component Verification File

CVS Concurrent Versions System

DCOM Distributed Component Object Model

DEM Digital Elevation Model

DFD Data Flow Diagram

DMSP Defense Meteorological Satellite Program

DOF Data Output Format

DPC Dipartimento della Protezione Civile

DWD Deutscher Wetterdienst

E&T Education and Training

EARS EUMETSAT Advanced Retransmission Service (station)

ECMWF European Centre for Medium-range Weather Forecasts

ECSS European Cooperation on Space Standardization

EGPM European contribution to the GPM mission

EOS Earth Observing System

EPS EUMETSAT Polar System

ERS European Remote-sensing Satellite (1 and 2)

ESA European Space Agency

EUR End-User Requirements

FMI Finnish Meteorological Institute

FOC Full Operational Chain

FTP File Transfer Protocol

GEO Geostationary Earth Orbit

GIS Geographical Information System

GMES Global Monitoring for Environment and Security

GOMAS Geostationary Observatory for Microwave Atmospheric Sounding

GOS Global Observing System

GPM Global Precipitation Measurement mission

GPROF Goddard Profiling algorithm

GTS Global Telecommunication System

GUI Graphical User Interface

HMS Hungarian Meteorological Service

HRU Hydrological Response Unit

H-SAF SAF on support to Operational Hydrology and Water Management

HSB Humidity Sounder for Brazil (on EOS-Aqua)

HTML Hyper Text Markup Language

HTTP Hyper Text Transfer Protocol

HUT/LST Helsinki University of Technology / Laboratory of Space Technology

HVR Hydrological Validation Review

HYDRO Preliminary results of Hydrological validation

HYDROS Hydrosphere State Mission

HW Hardware

ICD Interface Control Document

ICT Information and Communication Technology

IEEE Institute of Electrical and Electronics Engineers

IFS Integrated Forecast System



INF Progress reports in between meetings

INWM Institute of Meteorology and Water Management (of Poland)

IPF Institut für Photogrammetrie und Fernerkundung

ISAC Istituto di Scienze dell’Atmosfera e del Clima (of CNR)

ISO International Standards Organization

IT Information Technology

ITU Istanbul Technical University

JPS Joint Polar System (MetOp + NOAA/NPOESS)

J2EE Java 2 Enterprise Edition

KIDS Kestrel Interactive Development System

KLOC Thousand (Kilo) Lines Of Code

KOM Kick-Off Meeting

LAI Leaf Area Index

LAN Local Area Network

LEO Low Earth Orbit

LIS Lightning Imaging Sensor (on TRMM)

LLS Lower Level Specifications

LOC Lines Of Code

LST Solar Local Time (of a sun-synchronous satellite)

MARS Meteorological Archive and Retrieval System

MetOp Meteorological Operational satellite

METU Middle East Technical University (of Turkey)

MHS Microwave Humidity Sounder (on NOAA N/N’ and MetOp)

MIMR Multi-frequency Imaging Microwave Radiometer

MIN Minutes of Meetings/Reviews

MODIS Moderate-resolution Imaging Spectro-radiometer (on EOS Terra and Aqua)

MSG Meteosat Second Generation

MTBF Mean Time Between Failure

MTG Meteosat Third Generation

MTTR Mean Time To Repair

MVIRI Meteosat Visible Infra-Red Imager (on Meteosat 1 to 7)

N/A Not Available

N.A. Not Applicable

NASA National Aeronautics and Space Administration

NATO North Atlantic Treaty Organisation

NDI Non-developmental Items

NIMH National Institute for Meteorology and Hydrology (of Hungary)

NMS National Meteorological Service

NOAA National Oceanic and Atmospheric Organisation (intended as a satellite series)

NPOESS National Polar-orbiting Operational Environmental Satellite System

NPP NPOESS Preparatory Programme

NRT Near-Real Time

NWP Numerical Weather Prediction

OAR Options Analysis for Reengineering

OFL Off-line

OMG Object Management Group



OO Object Oriented

OP Proposal for H-SAF Operational phase

OPS Operational Product Segment

ORB Object Request Broker

ORR Operations Readiness Review

OWL Web Ontology Language

PAC Prototype Algorithm Code

PALSAR Phased Array L-band Synthetic Aperture Radar (on ALOS)

PAW Plant Available Water

PDR Preliminary Design Review

POP Precipitation Observation Production

PP Project Plan

PPR Products Prototyping Reports

PR Precipitation Radar (on TRMM)

QoS Quality of Service

R&D Research and Development

RCS Revision Control System

REP Report

RMI Royal Meteorological Institute (of Belgium)

RR Requirements Review

RT Real Time

SAAM Simulation, Analysis and Modeling

SAF Satellite Application Facility

SAG Science Advisory Group

SAOCOM Argentinean Satellite for Observation and Communication

SAR Synthetic Aperture Radar

SA/SD Structured Analysis / Structured Design

SCA Snow Covered Area

SCAT Scatterometer (on ERS-1 and 2)

SCM Source Configuration Management

SD Snow depth

SDAS Surface Data Assimilation System

SDD System Design Document

SDP Software Development Plan

SEI Software Engineering Institute

SEVIRI Spinning Enhanced Visible Infra-Red Imager (on MSG)

SHW State Hydraulic Works of Turkey

SHFWG SAF Hydrology Framework Working Group

SHMI Slovakian Hydrological and Meteorological Institute

SIRR System Integration Readiness Review

SIVVP System Integration, Verification & Validation Plan

SLAs Service-Level Agreements

SM Summary Report

SMART Service Migration and Reuse Technique

SMMR Scanning Multichannel Microwave Radiometer (on SeaSat and Nimbus VII)


http://en.wikipedia.org/wiki/Revision_Control_System


SMOS Soil Moisture and Ocean Salinity

SOA Service-Oriented Architecture

SoS System of Systems

SQL Structured Query Language

SR Snow Recognition

SRD System Requirements Document

SSM/I Special Sensor Microwave / Imager (on DMSP up to F-15)

SSMIS Special Sensor Microwave Imager/Sounder (on DMSP starting with F-16)

STRR System Test Results Review

SVALF System Validation File

SVERF System Verification File

SVRR System Validation Results Review

SW Software

SWE Snow Water Equivalent

SYKE Finnish Environment Institute

TBC To be confirmed

TBD To be defined

TC Test Case

TKK/LST Helsinki University of Technology / Laboratory of Space Technology

TMI TRMM Microwave Imager (on TRMM)

TP Test Procedure

TR Test Report

TRMM Tropical Rainfall Measuring Mission

TSMS Turkish State Meteorological Service

TU Wien Technische Universität Wien

UM User Manual

U-MARF Unified Meteorological Archive and Retrieval Facility

UML Unified Modelling Language

URD User Requirements Document

VIIRS Visible/Infrared Imager Radiometer Suite (on NPP and NPOESS)

VS Visiting Scientist

WBS Work Breakdown Structure

WMO World Meteorological Organization

WP Work Package

WPD Work Package Description

WS Workshop

W3C World Wide Web Consortium

XMI XML (eXtensible Markup Language ) Metadata Interchange

XML eXtensible Markup Language

ZAMG Zentral Anstalt für Meteorologie und Geodynamik

1.4 References

1.4.1 Applicable Documents[AD 1] H-SAF Development Proposal - Issue 2.1, 15 May 2005[AD 2] H-SAF Project Plan (PP). Ref.: SAF/HSAF/PP/2.1



[AD 3] H-SAF Configuration Management Plan (CMP). Ref.: SAF/HSAF/CMP/1.0[AD 4] H-SAF Users Requirement Document (URD). Ref.: SAF/HSAF/URD/2.1[AD 5] H-SAF SAF Preliminary Design Review Organization Note. Ref.: EUM/PPS/PRC/06/0125[AD 6] H-SAF System Requirements Document (SRD). Ref.: SAF/HSAF/SRD/2.0[AD 7] H-SAF System Design Document (SDD). Ref.: SAF/HSAF/SDD/2.0[AD 8] H-SAF Component Requirements Document (CRD). Ref.: SAF/HSAF/CRD/1.0[AD 9] H-SAF Component Design Document (CDD). Ref.: SAF/HSAF/CDD/1.0[AD 10] H-SAF Interface Control Document (ICD). Ref.: SAF/HSAF/ICD/1.0[AD 11] Algorithmic Software Development Guidelines Document (ASDGD). Ref.: SAF/HSAF/ASDGD/0.1[AD 12] System Integration, Verification & Validation Plan (SIVVP). Ref.: SAF/HSAF/SIVVP/1.0[AD 13] Guide to Software Configuration Management. Ref.: ESA PSS-05-09 Issue 1 Revision 1[AD 14] SAF Software Development Guidelines. Ref.: SAF/NET/EUM/SW/GD/MTR/01[AD 15] H-SAF Component Verification File (CVERF). Ref.: SAF/HSAF/CVERF/1.0[AD 16] H-SAF Algorithmic Software Development Guidelines Document (ASDGD) – internal issue – Ref.: SAF/HSAF/ASDGD/0.5[AD 17] H-SAF System Verification File (SVERF). Ref.: SAF/HSAF/SVERF/0.5[AD 18] Algorithms Theoretical Definition Document (ATDD). Ref.: SAF/HSAF/ATDD/1.0[AD 19] H-SAF System/Software Version Document (SSVD). Ref.: SAF/HSAF/SSVD/0.5

1.4.2 Reference Documents[RD 1] EUM.TD.08 MSG - Image Data Dissemination Service[RD 2] EPS AMSU-A L1 Product Generation Specification[RD 3] EPS AMSU-A L1 Product Format Specification[RD 4] EPS MHS L1 Product Generation Specification[RD 5] EPS MHS L1 Product Format Specification[RD 6] MetOp-AVHRR documentation VCS Engineering website (http://rst.vcs.de/)[RD 7] NASA-MODIS documentation at NASA website (http://directreadout.gsfc.nasa.gov/index.cfm)[RD 8] Meteosat-SEVIRI documentation at EUMETSAT website (http://www.eumetsat.int/Home/Main/What_We_Do/Satellites/Meteosat_Second_Generation/index.html=en)[RD 9] EUMETSAT-EUMETCast documentation at EUMETSAT website (http://www.eumetsat.int/Home/Main/What_We_Do/EUMETCast/index.html=en)[RD 10] EPS NRT User Interface Specification. Ref.: EPS-ASPI-IR-0648[RD 11] National Oceanic and Atmospheric Administration (NOAA) Low-Rate Information Transmission (LRIT) System. Ref.: http://noaasis.noaa.gov/LRIT/ [RD 12] WMO - WWWDM specifications. Ref.: http://www.wmo.ch/web/www/WDM/wdm.html [RD 13] MARS – Meteorological Archive and Retrieval System, MARS User Guide (for Data Retrieval), Revision 11 – September 1995, Meteorological Bulletin B6.7/3, ECMWF Reading, England. Ref.: http://www.wmo.ch/web/www/WDM/wdm.html [RD 14] MSG Level 1.5 Image Data Format Description. Ref.: ESA PSS-05-09 Issue 1 Rev.1[RD 15] ECSS Standard on Space engineering. Verification. Ref.: ECSS-E-10-02A[RD 16] ECSS Standard on Space engineering. Testing. Ref.: ECSS-E-10-03A [RD 17] ECSS Standard on Space engineering. Software - Part 1: Principles and requirements. Ref.: ECSS-E-40 Part 1B[RD 18] ECSS Standard on Space engineering. Software - Part 2: Document requirements definitions (DRDs). Ref.: ECSS-E-40 Part 2B[RD 19] Object Management Group, Inc. (OMG), “Unified Modelling Language: Infrastructure” Version 2.0 – 05/072005[RD 20] Object Management Group, Inc. (OMG), “Unified Modelling Language: Superstructure” Version 2.0 – 05/072005[RD 21] Object Management Group, Inc. (OMG), “Unified Modelling Language: Diagram Interchange” Version 2.0 – 05/072005[RD 22] Object Management Group, Inc. (OMG), “Unified Modelling Language: Business Modelling” Version 2.0 – 05/072005


http://rst.vcs.de/

http://directreadout.gsfc.nasa.gov/index.cfm

http://www.eumetsat.int/Home/Main/What_We_Do/Satellites/Meteosat_Second_Generation/index.html=en

http://www.eumetsat.int/Home/Main/What_We_Do/Satellites/Meteosat_Second_Generation/index.html=en

http://www.eumetsat.int/Home/Main/What_We_Do/EUMETCast/index.html=en

http://www.eumetsat.int/Home/Main/What_We_Do/EUMETCast/index.html=en

http://noaasis.noaa.gov/LRIT/


[RD 23] Brown, Alan W. & Wallnau, Kurt C. "Engineering of Component-Based Systems". Component-Based Software Engineering: Selected Papers from the Software Engineering Institute. Los Alamitos, CA: IEEE Computer Society Press, 1996.[RD 24] Jim Rumbaugh, Ivar Jacobson, Grady Booch. “The Unified Modeling Language Reference Manual (2nd edition)”

2 System Integration, Verification & Validation Overview

2.1 GeneralThe Test plan is focused on the following aspects of the test process:

• whole lifecycle;• types and phases;responsibility.

Test plan includes:

• test objectives;• schedule and logistics;• test strategies;• test cases:• test procedures;• test data;• test expected results;procedures for problems handling.•

2.2 Techniques and methods The testing techniques can be divided into:

Structural testing techniques, based on a “white box” testing, carried out by placing statements in the code and testing how a program/system does something. Functional testing techniques that refer to a “black box” testing, carried out working on inputs and outputs and testing the behaviour of a system or a program.

H-SAF is composed by 5 subsystems implemented in different institutes/clusters; due to this distribution, each institute has to execute verification and validation activities both locally/internally and in cooperation with other institutes/clusters, in order to verify the whole H-SAF system, according to the following phases:

• Unit test phase. During this phase each institute/cluster has to verify whether the program code implements the detailed design and whether it is free from faults. The unit test phase will not be formally reviewed, but each institute/cluster should produce an internal document where an overview of this test phase is given.• Component test phase. The component testing verifies that the components fulfil their component requirements, and that the products are generated according to the timeliness requirements. The component testing phase includes the algorithm test which verifies the correctness and accuracy of the generated products.• Integration test phase. During this phase all of the internal/external interfaces of each H-SAF subsystems have to be tested. Tests have to be done locally (internal interface) and among the institute/clusters (external interface) responsible for the H-SAF subsystems development. As far as the internal interfaces are concerned, the interface test phase will not be formally reviewed; anyway an internal document, describing an overview of this test phase, will be produced.• System test phase. During this phase the whole set of the H-SAF system interfaces have to be tested.



All institute/clusters must cooperate during this phase. System data flows, recovery, security, performance, reliability, resource usage, stress testing and redundant HW have to be tested during the System test phase.• System validation phase. During this phase each institute/cluster has to validate the User requirements.

As widely described in System Design Document [AD 7] the H-SAF subsystems are:

• Precipitation Subsystem: in charge of acquiring and pre-processing input data, generation of Precipitation (Observation and Assimilation) products, performing quality control processes and activities, and disseminating products to operational users;• Soil Moisture Subsystem, in charge of acquiring and pre-processing input data, generation of Soil Moisture (flat/forested, mountainous and combined) products, performing quality control processes and activities and disseminating products to operational users;• Snow Parameter Subsystem, in charge of acquiring and pre-processing of input data, generation of Snow Parameters products, performing quality control processes and activities, and disseminating products to operational users;• Hydro Validation Subsystem, in charge of providing independent assessment of the usefulness of H-SAF products in operational hydrology and water management, performing near-real-time acquisition of all products and carrying out hydrological validation activities using available tools and consolidated methods and models. It also provides valuable output for cal/val and validation reports used by production subsystems for possible products quality improvements;• Offline Monitoring Subsystem, in charge of acquiring products from other subsystems, storing them in an online/offline archive, providing offline dissemination to scientific users through UMARF interface and performing monitoring on dissemination results.

The test specifications consist of a test plan, test cases, test procedures and test reports. The test plan describes the test phases and their related activities. Although in general one test case is designed for each system or user requirement, one test case can even cover several requirements. Test results are detailed in test reports and this documentation is the main subject of the review. The tests are performed one by one and, if problems arise, they have to be solved before passing to the next test, unless the results of the next test are completely independent from previous one. The test case specifications in all test phases will define which tests can be done in parallel.If solving the problem causes the development and release of a new version of the tested item, the whole test on that item shall be rerun; moreover, for each changed unit, the unit testing shall be done again. For each component or interface containing the unit, the related component or interface test shall be rerun.

2.3 Master scheduleA master schedule table for planned integration, verification and validation activities is here reported:

Phase Milestone

1) Unit test -

2) Component test System test results review (STRR)

3) Integration test System Integration Readiness review (SIRR)

4) System test System test results review (STRR)Operation Readiness Review (ORR)

5) System validation System test results review (STRR)Operation Readiness Review (ORR)

Table 1 Schedule of H-SAF test phasesThe detailed schedule of testing tasks for the component testing, system testing and system validation testing shall be presented in the test plans peculiar to each phase.The phase numbering criteria intends to reflect the chronological order of test process execution.

2.4 Roles and Responsibilities



H-SAF is composed of 5 subsystems; each of them is developed under the responsibility of its own Institute/Cluster. Thus there are two levels of testing activities: an Institute/Cluster level and a System level where more institutes/clusters work together to perform the testing activities. To carry out testing activities, three organisational roles are needed at the Institute/Cluster level:

• Product Manager, responsible for supervision of the correctness of the Product disseminated to the users;• Cluster Leader, head of the overall cluster activities;• Operation Manager, responsible for coordination of the operational activities.

Sometimes other focal point people can be highlighted, i.e. Technical Leader, Test Manager, Impact Study Responsible etc.Moreover, it can happens that inside a Cluster more than one role of Product Manager can be associated to one only person; on the contrary the role of Operation Manager can be shared among several people.The following tables identify the specific responsibilities and roles for each Subsystem:

Role/Responsibility Name Institute / Organization

Country

Precipitation Subsystem

Cluster Leader Francesco Zauli CNMCA Italy

Operation Manager Francesco Zauli CNMCA Italy

PR-OBS-1 Product Manager Responsible of WP2100 (see Project Plan [AD 2])

CNMCA Italy


CNMCA Italy


CNMCA Italy


CNMCA Italy


CNMCA Italy

PR-ASS-1 Product Manager Responsible of WP2200 (see Project Plan [AD 2])

CNMCA Italy

Soil Moisture Subsystem

Cluster Leader Alexander Jann ZAMG Austria

Operation Manager Alexander Jann ZAMG Austria

SM-OBS-1 Product Manager Responsible of WP3100 (see Project Plan [AD 2])

ZAMG Austria

SM-OBS-2 Product Manager Responsible of WP3100 (see Project Plan [AD 2])

ZAMG Austria

SM-ASS-1 Product Manager Responsible of WP3200 (see Project Plan [AD 2])

ECMWF UK

Snow Parameters Subsystem

Cluster Leader Jouni Poullianen FMI Finland

Operation Manager Jouni Poullianen FMI Finland

SP-OBS-1 Product Manager Responsible of WP4000 (see Project Plan [AD 2])

FMI Finland

SP-OBS-1a Product Manager Responsible of WP4100 (see Project Plan [AD 2])

FMI Finland




FMI Finland


FMI Finland


FMI Finland


FMI Finland


FMI Finland


FMI Finland

SP-OBS-1b Product Manager Responsible of WP4200 (see Project Plan [AD 2])

TSMS Turkey


TSMS Turkey


TSMS Turkey

Hydro Validation Subsystem

Cluster Leader Bozena Lapeta IMWM Poland

Operation Manager Bozena Lapeta INWM Poland

Impact Study Responsible Responsible of WP5300 (see Project Plan [AD 2])

RMI Belgium


Meteo France France


BfG Germany


DPC Italy


IMWM Poland


SHMU Slovakia


METU Turkey

Offline Monitoring Subsystem

Subsystem Leader Adriano Raspanti CNMCA Italy

Operation Manager Adriano Raspanti CNMCA Italy

Archiving Responsible Responsible of WP1210 (see Project Plan [AD 2])

CNMCA Italy

Dissemination Responsible Responsible of WP1220 (see Project Plan [AD 2])

CNCMA Italy

Table 2 H-SAF roles and responsibilities

2.5 Problems Management

2.5.1 Process descriptionWhenever a SW or HW problem is detected during operations like data acquisition, product generation, archiving,



dissemination and so on, a Problem Report (PR) should be raised.This paragraph aims to describe the actions connected to the problems management in the H-SAF system context.A Problem Report should be focused on the detailed description of the observed problem, its reproducibility, collection of logs and relevant information.Once the Problem Report is submitted, it should be managed by Problem Review Board (PRB) following an assessment process.This process can consist of four phases:

• Investigation: this phase consists of collecting all data necessary to identify and characterize the issue raised. PRB reviews all new PR in order to assess the operational severity, to establish root cause and identify possible operational workaround solutions.

The following sources of problem can be identified:1. Problem due to external entities (e. g. from WMO-GTS)2. Operational problem (e.g. non nominal operation, accidental event, etc.)3. HW problem4. Configuration Problem5. Software Problem

Those may be on one or more systems in any combination.

• Disposition: This is the phase in which the PRB decides on the follow-up to be given to the investigation. The disposition can lead to a rejection of the PR, or to a corrective action.

The following decision can be taken:a. If the problem is identified on external entities, an investigation/corrective action is submitted to such external entityb. If the problem is identified as operational problem, a procedure will be provided to operation team. For example:

§ if the problem is identified as a non-nominal operation effect, an operational procedure update is disposed in order to clearly detail the operational steps;§ if the problem is identified as an accidental event effect (i.e. file system full, power failure, etc.), a contingency operational procedure to fix the problem is disposed;

c. If the problem is identified as a HW problem, a HW repair or replacement is disposed; d. If the problem is identified as a configuration problem (non-conforming or erroneously configured equipment), e. If the problem is identified as a SW problem, a solution requires software modification by SW maintenance responsible (to repair a non-conformance or to implement a required change). Note: for blocking problems an operational workaround solution should be proposed to the PRB within a prefixed time window before the release of a new official SW version that solve the identified problem.

The PRB can reject the PR relevant to the system (SW or HW) in the following cases: o the reported system behaviour is conforming to the approved baselines and no improvement in operations manual or system robustness to prevent the error is considered efficiento after some monitoring period the reported system behaviour is considered an isolated case due to exceptional circumstances (e.g. HW or network failure)o insufficient information is provided for investigation after several recalls

• Correction: If the source of the problem is recognized to be a configuration problem, or a non-conformance of the SW with respect to its specifications, or the result of a new operational requirement, then the PR is submitted to the SW responsible for a solution.

A classification of problem severity should be used to prioritise and plan the new SW release, according to a possible Service Level Agreement between EUMETSAT and the Contractors.

• Deployment and Verification: Once corrective action has been implemented, it is deployed to the operations team and the verification of the implemented solution is carried out on the operational platforms. This phase ends with the report on the action effectiveness produced by the operations team



and checked by PRB.If it is proved the new SW release is effective in curing the reported SW problem, PRB disposes the closure of the PR. Should it not prove to be effective, the PR is re-submitted for new investigation.

The flow diagram below shows the whole process:

Figure 1 Problem Management Process flow diagram

2.5.2 Problem Review BoardThe PRB is in charge of managing the Problem Review and Assessment Process and disposing Problem Reports impacting functionalities, performances or data quality.Each cluster can organize a proper PRB that could meet regularly, nominally weekly, and may be organised as meetings or teleconferences.The PRB is chaired by the PRB Chairman. As a guideline the following authorities should participate:

• Cluster Leader (who usually acts as Chairman)• Proper Product Manager for each product whose problem is involved• Operation Manager• Further people involved in specific problem reports

People invited at PRBs are usually part of the overall Cluster team, thus mentioned in Table 4.The following table states the roles inside the PRB of each H-SAF Cluster:

Name PRB role

Precipitation Subsystem PRB

Francesco Zauli Cluster Leader

Responsible of WP2100 (see Project Plan [AD 2]) PR-OBS-1 Product Manager





Responsible of WP2200 (see Project Plan [AD 2]) PR-ASS-1 Product Manager

Soil Moisture Subsystem PRB

Alexander Jann Cluster Leader

Responsible of WP3100 (see Project Plan [AD 2]) SM-OBS-1 Product Manager



Responsible of WP3100 (see Project Plan [AD 2]) SM-OBS-2 Product Manager

Responsible of WP3200 (see Project Plan [AD 2]) SM-ASS-1 Product Manager

Snow Parameters Subsystem PRB

Jouni Pulliainen Cluster Leader

Responsible of WP4000 (see Project Plan [AD 2]) SP-OBS-1 Product Manager

Responsible of WP4100 (see Project Plan [AD 2]) SP-OBS-1a Product Manager







Responsible of WP4200 (see Project Plan [AD 2]) SP-OBS-1b Product Manager



Hydro Validation Subsystem PRB

Bozena Lapeta Cluster Leader

Responsible of WP5300 (see Project Plan [AD 2]) Impact Study Responsible







Offline Monitoring PRB

Adriano Raspanti Subsystem Leader

Responsible of WP1210 (see Project Plan [AD 2]) Archiving Responsible

Responsible of WP1220 (see Project Plan [AD 2]) Dissemination Responsible

Table 3 Problem Review Board roles

3 Unit Test plan

3.1 IntroductionIn H-SAF system the term unit refers to items into which a component is decomposed. Components and their units are specified in Component Design Document [AD 9]. A unit test consists of the execution of the program associated to the component, with the main goal of finding errors. Each of the test cases is designed in order to maximize the probability of finding errors not still known, even though it can never prove that the program is free from faults.The unit testing phase starts after the coding and the debugging phase. It is useful to put SW items under configuration control before beginning unit tests, as stated in the Configuration Management Plan [AD 3].

3.2 Test itemsHerein the H-SAF system components are listed, with reference to the institute/cluster responsible of the component. Each unit item of the component shall be tested. Moreover the unit integration into component shall



also be tested during this phase.

Subsystem Component ID Abbreviations Responsible Institute/Cluster

Precipitation

PR-AP Data acquisition and pre-processing CNMCA/WP2000

PR-GC1 OBS Precipitation Product Generation Chain

CNMCA/WP2000

PR-GC2 ASS Precipitation Product Generation Chain

CNMCA/WP2000

PR-DIS Dissemination CNMCA/WP2000

Soil Moisture

SM-GC1 OBS Soil Moisture Generation Chain ZAMG/WP3100

SM-GC2 ASS Soil Moisture Generation Chain ECMWF/WP3200

SM-DIS Dissemination ZAMG/WP3100

Snow Parameters

SP-GC1 Snow Parameters in flat areas Generation Chain

FMI/WP4100

SP-GC2 Snow Parameters in mountainous areas Generation Chain

TSMS/WP4200

SP-DIS Dissemination FMI/WP4100

Hydro ValidationHV-AP Product Acquisition and Pre-processing Various/WP5000

HV-VPR Hydrovalidation Processing and Reporting

Various/WP5000

Offline Monitoring

OM-CEN Centralization CNMCA/WP1200

OM-ARC Archiving CNMCA/WP1200

OM-REP Report Analysis and Generation CNMCA/WP1200

OM-DIS Dissemination CNMCA/WP1200

Table 4 H-SAF Components vs. Responsible Institute/cluster

3.3 Features to be testedThe unit testing goal is to verify whether the program code implements the design and whether the code is free from faults.

3.4 ApproachDevelopers of a unit or an independent test group perform the unit tests. Each institute/cluster designs and executes relevant test case for relevant part of H-SAF subsystem.Units shall be tested using white box testing method. A white box test provides the information needed in order to test all the possible pathways. This includes not only correct inputs, but also incorrect inputs, so that error handlers can be verified as well. The goal of the tests is to execute all statements of the component; the results shall be documented in an internal institute/cluster review. Internal interface testing is part of a unit test phase. The goal of a unit test is to evaluate every single part (files, programs, modules, components, etc.) in isolation. According to the characteristics of the part subjected, techniques can be functional or structural. Concerning functional techniques, they include:

• input domain testing - test cases representative of the range of allowable input are picked, including high, low, and average values;• equivalence partitioning - the range of allowable input is partitioned so that the program is expected to



behave similarly for all inputs in a given partition, then a test case from each partition is picked;• boundary value - test cases are chosen with input values at the boundary (both inside and outside) of the allowable range;• syntax checking - test cases that violate the format rules for input are chosen;• special values - test cases that use input values representing special situations are designed;• output domain testing - test cases picked are projected to produce output at the extremes of the output domain.

Concerning structural techniques, there are:

• statement testing – ensures that the set of test cases exercises every statement at least once;• branch testing - each branch of an if/then statement is exercised;• conditional testing - each truth statement is exercised both true and false;• expression testing - every part of every expression is exercised.Path testing, where every path is exercised, is here considered impossible in practice.

3.5 Item pass/fail criteriaA unit passes the test if its outputs are the ones expected in its unit test design specification.

3.6 Test DeliverablesBefore starting the test the followings items are needed:

• Test configuration;• Test data;• Test tools.

At the end of this phase internal documents are produced.

3.7 Environmental needsTesting is performed in the development environment.

3.8 ScheduleThere is no formal review for test unit documentation.

3.9 ApprovalThe Cluster Leader approves all of the unit test results and communicates to H-SAF Project Manager that the unit tests are ended and endorsed.

4 Component test plan

4.1 IntroductionThe planning phase of the component testing method includes an analysis of the components under test in order to produce the test requirements. This analysis can begin as soon as the components’ specification is available and continues after the final implementation is available. The input to this phase includes the specification of the component, comprising its input data and its output data, its data flows and the set of interfaces that belong to the system component.

4.2 Test itemsThe whole of the requirements detailed in Component Requirement Document [AD 8] is involved during this phase; in CVERF [AD 15] the matrix that shows the dependence between requirements and components is traced.

4.3 Feature to be tested



Herein a table which details components and related subcomponents for each H-SAF subsystem is reported.

Subsystem Component Subcomponent

Identifier Description Identifier Description

Precipitation

PR-AP Data acquisition and pre-processing

PR-GC1 OBS Precipitation Generation Chain

PR-GC1-PG1 PR-OBS-1 Product Generation


PR-GC1-PG3 PR-OBS-3 and PR-OBS-4 Products Generation


PR-GC1-QC PR-GC1 Quality Control

PR-GC2 ASS Precipitation Generation Chain

PR-GC2-PG PR-ASS-1 Product Generation

PR-GC2-QC PR-GC2 Quality Control

PR-DIS Precipitation Dissemination

Soil Moisture

SM-GC1 OBS Global/Regional Surface Soil Moisture Generation Chain

SM-GC1-GPG SM-GC1 Global Product Generation

SM-GC1-RPG SM-GC1 Regional Product Generation

SM-GC1-QC SM-GC1 Quality Control

SM-GC2 ASS Volumetric Soil Moisture (root region) Generation Chain

SM-GC2-VPG SM-GC2 Volumetric Product Generation

SM-GC2-QC SM-GC2 Quality Control

SM-DIS Soil Moisture Dissemination

Snow Parameters

SP-GC1Flat/forested areas and combined Snow Parameters Generation Chain

SP-GC1-AP SP-GC1 Data acquisition and pre-processing

SP-GC1-PG SP-GC1 Product Generation

SP-GC1-QC SP-GC1 Quality Control

SP-GC2 Mountainous areas Snow Parameters Generation Chain

SP-GC2-AP SP-GC2 Data acquisition and pre-processing

SP-GC2-PG SP-GC2 Product Generation

SP-GC2-QC SP-GC2 Quality Control

SP-DIS Snow Parameters Dissemination

HV-AP-IS1 HV Impact Study 1 Belgium

HV-AP-IS2 HV Impact Study 2 France



Hydro ValidationHV-AP Product Acquisition and

Pre-processing

HV-AP-IS3 HV Impact Study 3 Germany

HV-AP-IS4 HV Impact Study 4 Italy

HV-AP-IS5 HV Impact Study 5 Poland

HV-AP-IS6 HV Impact Study 6 Slovakia

HV-AP-IS7 HV Impact Study 7 Turkey

HV-VPR Hydro Validation Processing and Reporting

Offline Monitoring

OM-CEN Centralization

OM-ARC Archiving

OM-REP Report Analysis and Generation

OM-DIS Dissemination

Table 5 H-SAF Components vs. Subcomponents

4.4 ApproachComponent tests are performed and results are obtained automatically, using scripts whenever it is possible. Procedures specify the expected output in correspondence of specified inputs, in order to pass the test. There is a failure when the SW is unable to fulfil the requirements: the test can continue if there is a work around, but it has to run again after SW modification is done and a new SW version is released. All tests shall run again for the new SW version.Component testing verifies that the components fulfil Component requirements. This phase starts after the coding and unit test phase. The whole component shall be put under configuration control before starting this phase, following what it is stated in the Configuration Management Plan [AD 3]. Component testing includes the design of:

• Component test case specifications;• Component test procedures;• Component test reports;

Wherever the test procedures design is omitted, it means that the requirement shall not be tested (T) but demonstrated (D), or inspected (I), or reviewed (R) as specified in Component Requirement Document [AD 8].Component tests can be performed using black-box or white-box techniques.

4.4.1 Test cases specificationFor each test case specification, information detailed in the following table shall be depicted:Test case identification

A unique identifier for test case (e.g. TC_subsystem_identifier (PR,SM,SP)_requirement identifier (GEN, PPR, PRG, etc), a progressive number)

Test case Name

A short description of the test

H-SAF Subsystem

List of the subsystems involved in the test

Component specification

List of the components involved in the test

Requirements to test

List of the component requirements involved in the test

Input

Input data

Output

Expected Output



Status

Pass/Fail

Test Method

List of used stubs, test data, etc

Dependency with other test

List of the test case specifications that have to be previously passed in order to carry out this test case

4.4.2 Component Test ProceduresFor each test procedure, information detailed in the following table shall be specified:Test procedure identification

A unique identifier for test procedure (e.g. TP_subsystem_identifier (PR,SM,SP)_requirement identifier (GEN, PPR, PRG, etc), a progressive number)

Test procedure description

A short description of the procedure

Test case identification

The identification of the test case covered by the procedure

Test procedure steps

List of instructions to be executed sequentially for the test execution

4.4.3 Component Test ReportsFor each test Report, information detailed in the following table shall be specified:Test report ID

A unique identifier for test report (e.g. TR_subsystem_identifier (PR,SM,SP)_requirement identifier (GEN, PPR, PRG, etc), a progressive number)

SW Version

SW version under testing

Test procedure step

Pass/Fail Comment

1 Pass/Fail Any comment

2 Pass/Fail

Procedure output

Expected output

Procedure result files

List of produced files

Test execution description

Description of the execution of the test

Overall test status Date Signature

Date of execution of the test

4.5 Item Pass/Fail criteriaA component passes the test if its outputs are the ones expected in test design document.

4.6 Test Deliverables



Before starting the component test, the following items are needed:• This document (SIVVP);• Component Verification File document containing test specifications, test case and procedures (CVERF);• Test tools.

When a component test ends, an update of the test specification document containing test reports and problem reports shall be performed.

4.7 Environmental needsTesting is done locally in the development environment.

4.8 ScheduleSchedule will be detailed in the baseline release of the document (SIVVP 1.0), expected to be delivered at CDR (t0+24).

4.9 ApprovalThe Cluster Leader approves all of the components testing results.

5 Integration test plan

5.1 IntroductionThe integration test focuses on testing of the following interfaces:

• H-SAF Internal Interfaces: the interfaces between H-SAF subsystems and between their internal components;• H-SAF External Interfaces: the interfaces between H-SAF and external entities.

Each institute/cluster has to edit a proper Integration Test Document where internal and external interfaces shall be identified, test cases and results shall be designed. Internal integration test shall be done locally at the institute/clusters. The external interface testing goal is to verify data flows from H-SAF subsystem and its client taking into account that a client can be another H-SAF subsystem.

5.2 Features to be testedData exchanged among the H-SAF subsystems.

5.3 ApproachEach institute/cluster has to produce the proper Integration test design specification where internal and external subsystem interfaces are detailed.Integration test is carried on using black-box techniques.

5.4 Item Pass/Fail criteriaA test is considered successful if the internal interface between two or more components works properly.

5.5 Test DeliverablesBefore starting an interface test, the following items are needed:

• This document (SIVVP);• System Verification File, (SVERF) containing test specifications, test case and test procedures specific for interface test;• Test tools.

When an interface test ends, SW problem reports shall be issued, if any. A test report containing all the problems occurred during the specific test session shall be issued also. Then an update of the test specification document



shall follow.

5.6 Environmental needsEnvironmental needs for Integration test plan will be detailed in the baseline release of the document (SIVVP 1.0), to be delivered at CDR (t0+24).

5.7 ScheduleThere is no formal review for interface test documentation.

5.8 ApprovalThe test shall be approved by responsible of H-SAF subsystem (Cluster leader).

6 System test plan

6.1 IntroductionDuring this phase, H-SAF system shall be tested in cooperation with all institute/clusters to verify tailoring with system requirements. During the system test, the ability of all H-SAF subsystems to work properly together shall be verified.

6.2 Test Items• Precipitation subsystem;• Snow parameter subsystem;• Soil moisture parameter subsystem;• Hydro Validation subsystem;• Offline Monitoring subsystem.

6.3 Features to be testedThe system requirements, as described in the System Requirement Document [AD 6].

6.4 ApproachSystem test is carried on using black-box techniques.The goal of this phase is to evaluate the system as a whole, not its parts. At this test level there are:

• stress testing - tests larger-than-normal capacity in terms of transactions, data, users, speed, etc.;• execution testing - tests performance in terms of speed, precision, etc.;• recovery testing - tests about how the system recovers from a HW failure, about how it handles corrupted data, etc.;• operations testing - tests about how the system fits in with existing operations and procedures in the user organization;• compliance testing - tests adherence to standards;• security testing - tests security requirements.

Techniques applied at this phase are:

• requirements testing – it can be considered the fundamental form of testing since it makes sure that the system does what it is required to do;• regression testing - make sure unchanged functionality remains unchanged;• error-handling testing - test required for error-handling functions (usually user error);• manual-support testing - test on the capability of the system to be used properly – it includes user documentation usage;



• intersystem handling testing - test whether the system is compatible with other systems in the environment;• control testing - test required for control mechanisms• parallel testing - feed same input into two versions of the system, in order to make sure that they produce the same output.

System testing includes the design of:• System test case specifications;• System test procedures;• System test reports.

Wherever the test procedures design is omitted, it means that the requirement shall not be tested (T) but demonstrated (D), or inspected (I), or reviewed (R) as specified in the System Requirement Document [AD 6].

6.4.1 System Test cases specificationFor each test case specification, information detailed in the following table shall be depicted: Test case identification

A unique identifier for test case (e.g. TC_ST_requirement identifier (GEN, PPR, PRG, etc), a progressive number)

Test case Name

A short description of the test

H-SAF Subsystem

List of subsystems involved in the test

Requirements to test

List of SW requirements involved in the test

Input

Input data

Output

Expected Output

Status

Pass/Fail

Test Method

List of used stubs, test data, etc

Dependency with other test

List of test case specifications that have to be previously passed in order to carry out this test case

6.4.2 System Test ProceduresFor each test procedure information detailed in the following table shall be specified:Test procedure identification

A unique identifier for test procedure (e.g. TP__ST_requirement identifier (GEN, PPR, PRG, etc), a progressive number)

Test procedure description

A short description of the procedure

Test case identification

The identification of the test case covered by the procedure



Test procedure steps

List of instructions to be sequentially executed for test execution

6.4.3 Test ReportsFor each system test Report information detailed in the following table shall be specified:Test report ID

A unique identifier for test report (e.g. TP_ST_requirement identifier (GEN, PPR, PRG, etc), a progressive number)

SW Version

SW version under testing

Test procedure step

Pass/Fail Comment

1 Pass/Fail Any comment

2 Pass/Fail

Procedure output

Expected output

Procedure result files

List of produced files

Test execution description

Description of execution of the test

Duration of test execution

Duration of test excluding the activities belonging to the test preparation

Overall test status Date Signature

Date of the execution of the test

6.5 Item Pass/Fail CriteriaSpecific pass/fail criteria for each item shall be described in the related test procedure.

6.6 Test DeliverablesBefore starting the system test, the following items are needed:

• This document;• System Verification File, (SVERF) containing test specifications, test case and procedures for H-SAF system test;• The configuration of test environment;• The data of the test.

The output of the system test is SVERF document filled with the test reports and problem reports.

6.7 EnvironmentThe environment of the system test execution is the operational environment where the system shall run. The input data shall be a representative set of data among those used when system is operating.

6.8 ScheduleThe system test starts after the completion of the integration tests.

6.9 ApprovalsThe H-SAF Project Manager approves all of the system testing results.



7 System validation test

7.1 IntroductionThe system validation testing is the final test phase among those having Operations Readiness Review as results. The goal of this phase is to verify whether the system satisfies the user requirements described in the User Requirement Document [AD 4].

7.2 Test items The whole set of the H-SAF system items, including its external interfaces.

7.3 Feature to be tested The user requirements in the URD ([AD 4].

7.4 ApproachDuring system validation test phase all of the user requirements shall be validated by means of test cases and procedures.

7.5 Item Pass/Fail CriteriaSpecific pass/fail criteria for each item shall be described in related test procedure.

7.6 Test DeliverablesBefore starting the system validation test, the following items are needed:

• This document containing the test plan; • System Verification File, (SVERF) containing Test specifications, test case and procedures for system validation test;• The configuration of the test environment;• The data of the test.

The output of the system test is SVERF document, filled with the test reports and problem reports.

7.7 Environmental needsThe test is done in the operative environment.

7.8 ScheduleSchedule will be detailed in the baseline release of the document (SIVVP 1.0), expected to be delivered at CDR (t0+24).

7.9 ApprovalsThe Cluster Leaders and H-SAF PM approve all of the system validation test results.

8 TBDs/TBCs listNo TBD/TBC items are currently pending.



- END OF THE DOCUMENT -


Hydrology SAF System Integration, Verification...

Documents

Transcript of Hydrology SAF System Integration, Verification...