Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data...
Transcript of Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data...
![Page 1: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/1.jpg)
Data quality control
Add value to Inspire services
![Page 2: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/2.jpg)
Presentation
Johan Esko
Project manager ISO 19157
Inspire
Data management
![Page 3: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/3.jpg)
Presentation
Anna Halvarsson
Project manager SDI
Inspire
Data management
![Page 4: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/4.jpg)
1´st May
2011
![Page 5: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/5.jpg)
Agenda
Introduction
Standards
Data quality
Example
Summary
![Page 6: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/6.jpg)
Introduction
![Page 7: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/7.jpg)
A year ago
Inspire conference Krakow 2010
• Inspire download service prototype
Publishing Workflow
Distribution Workflow
OGC
WFS
StagingDatabase
![Page 8: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/8.jpg)
Today
Krakow 2010 - Future development
• Integrate data quality control (QC)
• Integrate with OGC CSW Metadata
Publishing Workflow
StagingDatabase
Distribution Workflow
OGC
WFS
Metadata
Quality Control
![Page 9: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/9.jpg)
Standards
![Page 10: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/10.jpg)
Why use standards?
• Cornerstone of a SDI
• Inspire specifications
• Legible information
• Own development is
expensive
![Page 11: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/11.jpg)
From product specification to metadata
• Data product specification
ISO 19131, INSPIRE Data specifications
• Data quality
ISO 19113/19114/19138
(ISO 19157 will replace the standards above)
• Metadata
ISO 19115/19119/19139
Recipe: 4 eggs
4 tsp water
salt
Analyse of the
process:
3 eggs
6 tsp water
salt
Declaration of
contents:
3 eggs
6 tsp water
salt
![Page 12: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/12.jpg)
Data quality
![Page 13: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/13.jpg)
Definition of quality
“The degree to which a set
of inherent characteristics
fulfills requirements”
ISO 9000:2000
![Page 14: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/14.jpg)
Best quality?
![Page 15: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/15.jpg)
Swedish Technology Journal
“Short wheelbase and minimal overhang at the front and rear, combined with high ground clearance makes it extremely well suited to continue the drive where the road ends. Indeed the Niva outperform the Porsche Cayenne, which in comparison is clumsy as an elephant with a backpack in a supermarket”.
(Teknikens värld, 2006)
Comment:
“If it is the terrain capabilities that is evaluated, the Niva fulfils the quality requirements better than the Porsche.”
![Page 16: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/16.jpg)
Best quality?
Which one of these view services delivers data with the highest
positional accuracy?
![Page 17: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/17.jpg)
Geographic information - Data Quality
• Data quality elements
• Completeness
• Topological consistency
• Positional accuracy
• Temporal consistency
• Thematic accuracy
• Non-quantitative overview statements (usage, purpose, lineage)
Data quality can be described by:
![Page 18: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/18.jpg)
Geographic information - Data Quality
ISO 19157 CD
• Metaquality
• Usability element
• New data quality measures
• etc
• Schedule ISO 19157 CD
• Text sent to ISO for DIS
• DIS 2011-06
• FDIS 2012-06
• IS 2013-01
Some news
![Page 19: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/19.jpg)
Quality of data used in Inspire
services
• To evaluate if the data provided by an Inspire service is of sufficient quality
• Lack of quality information:
• can make data “useless”
• can cause misleading results
• Inspire data specifications specifies quality elements and data quality measures
![Page 20: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/20.jpg)
Data management
![Page 21: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/21.jpg)
Metria - Data management
Data
Input Processes
Base layer
Refining Processes
Product layer
Metadata
Portal
Web Services
Metria’s data warehouse
stores more than 2 Tb
geographical data
Automatic data
management processes are
important
![Page 22: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/22.jpg)
ISO 19131
INSPIRE Data specifications
ISO 19113, 19114, 19138
(ISO 19157)
ISO 19115, 19139
Product
SpecificationQuality Control Metadata
Input and refining processes
![Page 23: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/23.jpg)
Example – Data mangement
INSPIRE Protected Sites
![Page 24: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/24.jpg)
Product
SpecificationQuality Control Metadata
Data management – Step 1
![Page 25: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/25.jpg)
Requirement – Completeness commission
Conformance level = 0
![Page 26: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/26.jpg)
Requirement – Completeness ommission
Conformance level = 0
![Page 27: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/27.jpg)
Overall Requirement - Data product specification passed
Conformance level = True
![Page 28: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/28.jpg)
Requirement - schema
![Page 29: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/29.jpg)
Product
SpecificationQuality Control Metadata
Data management – Step 2
![Page 30: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/30.jpg)
Quality control routines
Automatic tests with FME
• Schema control
• Completeness
• Omission
• Comission
• Aggregation
•Data product specifikation
passed
(Check if all conformance levels
passed) Quality results reported as XML in metadata
(ISO 19115,19139)
![Page 31: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/31.jpg)
Product
SpecificationQuality Control Metadata
Data management – Step 3
![Page 32: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/32.jpg)
Quality result reported in
metadataXML format ISO TS 19139
![Page 33: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/33.jpg)
Update metadata in portal using FME and
CSW
![Page 34: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/34.jpg)
Summary
![Page 35: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/35.jpg)
Summary
• Different users have
different requirements
• Report data quality in
metadata
• Automated quality control
processes integrated with
the data workflow
![Page 36: Data quality control - INSPIRE · Presentation Johan Esko Project manager ISO 19157 Inspire Data management. Presentation ... Metria’s data warehouse stores more than 2 Tb geographical](https://reader034.fdocuments.in/reader034/viewer/2022042710/5f662b1ddbf1606fc56997cc/html5/thumbnails/36.jpg)
QuestionsVisit our booth in the
exhibition area
Anna Halvarsson
Johan Esko
http://www.metria.se