Testing - an essential aspect of

establishing an SDI

Clemens Portele, Anders Östman, Michael Koutroumpas, Xin He, Janne Kovanen, Markus Schneider, AndrianiSkopeliti

INSPIRE Conference 2011

30 June 2011

Reminder

RegistersService

Metadata

Data Set

Metadata

Registry Service Discovery Service

Metadata

Services

Access Control, Authentication

Application and Geoportals

Internet

InvokeSD

Service

Transf.

Service

Spatial Data Set

Thematic DS

Framework for

harmonized DS

Spatial Data

View

Service

Download

Service

INSPIRE Services

• Based on OGC Services (WMS, WFS, WPS, etc)

• Quality-of-Service requirements (performance, capacity,

availability)

• Multilingual support

• Additional service metadata

Reminder

RegistersService

Metadata

Data Set

Metadata

Registry Service Discovery Service

Metadata

Services

Access Control, Authentication

Application and Geoportals

Internet

InvokeSD

Service

Transf.

Service

Spatial Data Set

Thematic DS

Framework for

harmonized DS

Spatial Data

View

Service

Download

Service

Spatial data sets in INSPIRE

• Data specifications based on ISO 19100 series

• Using GML for encoding

• Specifying additional requirements

• Specifying layers in View Services

ESDIN

• An eContentplus Best Practice Network project

• September 2008 to February 2011

• Coordinated by EuroGeographics

• Key goal: help National Mapping and Cadastral Agencies (NMCAs) member states, candidate countries and EFTA States prepare their data for INSPIRE Annex I spatial data themes and improve access

ESDIN

• In total 7 ESDIN-related presentations at INSPIRE 2011

• In particular:

• Thu 16:00, David Overton, ESDIN - the geospatial reference data and services for INSPIRE

• Fri 09:20, Antti Jakobsson, European Location Framework- How to provide geospatial reference data and services for Europe based on INSPIRE

• This talk focuses on the ESDIN test environment

ESDIN specifications

• Use INSPIRE service specifications

• Extend INSPIRE data specifications

– Additional information that already is part of current data

products of EuroGeographics and which is used by

existing customers has been added

– Information that may be void in the INSPIRE data

specifications (i.e. is not necessarily provided in INSPIRE),

but which can be provided by NMCAs is made mandatory

– Minimum data quality levels have been specified for

European NMCA data

ESDIN testing

• Provide a systematic approach to testing the technical results

of the ESDIN project and support implementation activities

• General testing approach:

– Automated testing

– Black box testing

OGC compliance testing (CITE) vs

ESDIN test requirements

• OGC CITE tests

– Use a test data set

– Test a software product

– Not for all service and data standards available

• ESDIN Test requirements

– Test a service instance, not the capabilities of a software

product

• Products used in ESDIN have to pass CITE tests where they are

available

– Test against INSPIRE and ESDIN extensions

Publication

data

Download

Service

Transformations

ESDIN data production

and publication workflow

Source to

target model

Source

data

Coordinates

Generalise

Translate

Other data

Edge-

matching

View Service

Authentication

Service

Test targets – overview

• Publication of spatial data in the SDI

– INSPIRE view services / OGC WMS

– INSPIRE download services / OGC WFS

– INSPIRE view/download services via SAML/Shibboleth-

based authentication services

– INSPIRE coordinate transformation services

– INSPIRE and ExM data sets

• Data production processes

– Medium-to-small-scale generalisation

– Data quality evaluation

– Edge-matching processes

Test cases – overview

• Test cases documented using a

common template for all test

targets

• In total 165 pages of test cases

Publication

data

Download

Service

Transformations

ESDIN data publication test target (example)

View services (WMS)

Source to

target model

Source

data

Coordinates

Generalise

Translate

Other data

Edge-

matching

View Service

Authentication

Service

Publication

data

Download

Service

Transformations

ESDIN data publication test target (example)

View services (WMS)

Source to

target model

Source

data

Coordinates

Generalise

Translate

Other data

Edge-

matching

View Service

Authentication

Service

INSPIRE view service test cases

• Service metadata validation

• Support for INSPIRE service metadata extensions

• Performance tests

• Availability tests

• Capacity tests

• Scaling tests

• Variations of valid and invalid requests

• Support for INSPIRE layers

Publication

data

Download

Service

Transformations

ESDIN data publication test target (example)

INSPIRE and ExM data sets

Source to

target model

Source

data

Coordinates

Generalise

Translate

Other data

Edge-

matching

View Service

Authentication

Service

Publication

data

Download

Service

Transformations

ESDIN data publication test target (example)

INSPIRE and ExM data sets

Source to

target model

Source

data

Coordinates

Generalise

Translate

Other data

Edge-

matching

View Service

Authentication

Service

INSPIRE data set test cases

• Basic validation

• against XML Schema

• against OCL constraints

• Spatial and temporal information

• Coordinates in ETRS89 and in Europe

• Time zone information provided

• Links to other resources

• Code list encoding

• Encoding of references to other features

• Locale provided for localised character strings

• Other tests

• Encoding of nil values

• Identifier provided

• Correct feature collection container

© 2010 Open Geospatial Consortium 16

Using soapUI and loadUI

(loadUI for load tests)

Modified version of soapUI to support

additional capabilities for implementing

test cases

Experiences in ESDIN• Very positive feedback from National Mapping and Cadastral Agencies within the

project transforming the data and setting up the ESDIN web services

– Tests enable the discovery of problems

• by the developer

• at an early stage of the implementation

– Automated tests allow monitoring the operation, too

• Positive feedback from EU project review: „ESDIN is one of the first projects that

also provide appropriate testing tools that can check the conformity to the ESDIN

data specifications as well as conformity to INSPIRE. These results should be

forwarded to the INSPIRE community.“ � one of the reasons we are here

• Additional work needed

– Improve reporting of reasons for failed tests

– Align with progress in INSPIRE specifications (new Technical Guidance

documents for network services, new data specifications, etc.)

Additional information

• Test framework:http://services.interactive-

instruments.de/download/ETF.zip

• Deliverables:http://www.esdin.eu/project/summary-esdin-project-public-

deliverables

– D12.4 Test cases

– D12.5 Test components

– D12.6 Test results

Final remarks

• The ESDIN Test Framework is available under BSD license

• The ESDIN consortium is preparing the steps to maintain and

improve the project results in the future and the test

components will be part of this

– Currently the test framework has its home in a Subversion

server at interactive instruments, but we plan to move this

as part of this process

– We are looking for opportunities for further collaboration to

improve the test framework and make it available for future

use in INSPIRE and national/community SDIs

Any questions?

Clemens Portele

portele@interactive-instruments.de