Testing - an essential aspect of establishing an SDI

Clemens Portele, Anders Östman, Michael Koutroumpas, Xin He, Janne Kovanen, Markus Schneider, Andriani SkopelitiINSPIRE Conference 201130 June 2011

Reminder

RegistersService Metadata

Data Set Metadata

Registry Service Discovery Service

Metadata

Ser

vice

s

Access Control, Authentication

Application and Geoportals

Internet

InvokeSDService

Transf. Service

Spatial Data Set

Thematic DS

Framework for harmonized DS

Spatial Data

ViewService

DownloadService

INSPIRE Services• Based on OGC Services (WMS, WFS, WPS, etc)• Quality-of-Service requirements (performance,

capacity, availability)• Multilingual support • Additional service metadata

Reminder

RegistersService Metadata

Data Set Metadata

Registry Service Discovery Service

Metadata

Ser

vice

s

Access Control, Authentication

Application and Geoportals

Internet

InvokeSDService

Transf. Service

Spatial Data Set

Thematic DS

Framework for harmonized DS

Spatial Data

ViewService

DownloadService

Spatial data sets in INSPIRE• Data specifications based on ISO 19100 series• Using GML for encoding• Specifying additional requirements• Specifying layers in View Services

ESDIN• An eContentplus Best Practice Network project• September 2008 to February 2011• Coordinated by EuroGeographics• Key goal: help National Mapping and Cadastral

Agencies (NMCAs) member states, candidate countries and EFTA States prepare their data for INSPIRE Annex I spatial data themes and improve access

ESDIN• In total 7 ESDIN-related presentations at INSPIRE 2011• In particular:

• Thu 16:00, David Overton, ESDIN - the geospatial reference data and services for INSPIRE

• Fri 09:20, Antti Jakobsson, European Location Framework- How to provide geospatial reference data and services for Europe based on INSPIRE

• This talk focuses on the ESDIN test environment

ESDIN specifications• Use INSPIRE service specifications• Extend INSPIRE data specifications

– Additional information that already is part of current data products of EuroGeographics and which is used by existing customers has been added

– Information that may be void in the INSPIRE data specifications (i.e. is not necessarily provided in INSPIRE), but which can be provided by NMCAs is made mandatory

– Minimum data quality levels have been specified for European NMCA data

ESDIN testing• Provide a systematic approach to testing the technical

results of the ESDIN project and support implementation activities

• General testing approach:– Automated testing– Black box testing

OGC compliance testing (CITE)

OGC CITE tests vs ESDIN test requirements

• OGC CITE tests– Use a test data set– Test a software product– Not for all service and data standards available

• ESDIN Test requirements– Test a service instance, not the capabilities of a

software product• Products used in ESDIN have to pass CITE tests where they

are available

– Test against INSPIRE and ESDIN extensions

Publication data

Download Service

Transformations

ESDIN data production and publication workflow

Source to target model

Sourcedata

Coordinates

Generalise

Translate

Other data

Edge-matching

View Service

Authentication Service

Test targets – overview• Publication of spatial data in the SDI

– INSPIRE view services / OGC WMS– INSPIRE download services / OGC WFS– INSPIRE view/download services via

SAML/Shibboleth-based authentication services– INSPIRE coordinate transformation services– INSPIRE and ExM data sets

• Data production processes– Medium-to-small-scale generalisation– Data quality evaluation– Edge-matching processes

Focus on additional requirements not covered by CITE tests, e.g• Quality of Service requirements• Impact of access via authentication

facades• Schema conformity of data• Coordinate transformation

Test cases – overview• Test cases documented using

a common template for all test targets

• In total 165 pages of test cases

Publication data

Download Service

Transformations

ESDIN data publication test target (example)View services (WMS)

Source to target model

Sourcedata

Coordinates

Generalise

Translate

Other data

Edge-matching

View Service

Authentication Service

Publication data

Download Service

Transformations

ESDIN data publication test target (example)View services (WMS)

Source to target model

Sourcedata

Coordinates

Generalise

Translate

Other data

Edge-matching

View Service

Authentication Service

INSPIRE view service test cases• Service metadata validation• Support for INSPIRE service metadata extensions • Performance tests• Availability tests• Capacity tests• Scaling tests• Variations of valid and invalid requests• Support for INSPIRE layers

Publication data

Download Service

Transformations

ESDIN data publication test target (example)INSPIRE and ExM data sets

Source to target model

Sourcedata

Coordinates

Generalise

Translate

Other data

Edge-matching

View Service

Authentication Service

Publication data

Download Service

Transformations

ESDIN data publication test target (example)INSPIRE and ExM data sets

Source to target model

Sourcedata

Coordinates

Generalise

Translate

Other data

Edge-matching

View Service

Authentication Service

INSPIRE data set test cases• Basic validation

• against XML Schema• against OCL constraints

• Spatial and temporal information• Coordinates in ETRS89 and in Europe• Time zone information provided

• Links to other resources• Code list encoding• Encoding of references to other features• Locale provided for localised character strings

• Other tests• Encoding of nil values• Identifier provided• Correct feature collection container

© 2010 Open Geospatial Consortium 17

Using soapUI and loadUI(loadUI for load tests)

Modified version of soapUI to support additional capabilities for implementing test cases

Experiences in ESDIN• Very positive feedback from National Mapping and Cadastral Agencies

within the project transforming the data and setting up the ESDIN web services– Tests enable the discovery of problems

• by the developer• at an early stage of the implementation

– Automated tests allow monitoring the operation, too• Positive feedback from EU project review: „ESDIN is one of the first projects

that also provide appropriate testing tools that can check the conformity to the ESDIN data specifications as well as conformity to INSPIRE. These results should be forwarded to the INSPIRE community.“ one of the reasons we are here

• Additional work needed– Improve reporting of reasons for failed tests– Align with progress in INSPIRE specifications (new Technical Guidance

documents for network services, new data specifications, etc.)

Additional information• Test framework:http

://services.interactive-instruments.de/download/ETF.zip

• Deliverables:http://www.esdin.eu/project/summary-esdin-project-public-deliverables

– D12.4 Test cases– D12.5 Test components– D12.6 Test results

Final remarks• The ESDIN Test Framework is available under BSD

license• The ESDIN consortium is preparing the steps to maintain

and improve the project results in the future and the test components will be part of this– Currently the test framework has its home in a

Subversion server at interactive instruments, but we plan to move this as part of this process

– We are looking for opportunities for further collaboration to improve the test framework and make it available for future use in INSPIRE and national/community SDIs

Any questions?

Clemens Portele

portele@interactive-instruments.de