Validation and Quality Control with FME and FME Server
-
Upload
safe-software -
Category
Technology
-
view
879 -
download
0
description
Transcript of Validation and Quality Control with FME and FME Server
![Page 1: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/1.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Validation and Quality Control with FME and FME Server
![Page 2: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/2.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Validation and Quality Control with FME and FME Server
![Page 3: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/3.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Validation and Quality Control with FME and FME Server
![Page 4: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/4.jpg)
Outline
Why are we talking about Validation and QC ? What can we check?
Schema Attribute Values Geometry Etc.
Validation Workflows on FME Server Partner Examples
![Page 5: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/5.jpg)
Why are we talking about this? Bad data costs time, money and even lives.
![Page 6: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/6.jpg)
Why are we talking about Validation and Quality Control?
FME and FME Server are powerful tools for checking data and providing feedback
Many partners, integrators and users use FME today for Validation and Quality Control
![Page 7: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/7.jpg)
What can we check?
þ Schema Feature Type Name Attribute Present Attribute Names Attribute Data Types Coordinate System
![Page 8: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/8.jpg)
What can we check?
Schema Checking – Key Tools Schema (Any Format) Reader FeatureTypeFilter StatisticsCalculator AttributeClassifier transformer XMLValidator Transformer JSONValidator Transformer
![Page 9: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/9.jpg)
What can we check?
þ Attribute Values Values exist Data Type (numeric, alpha etc) Value within in range No duplicates
![Page 10: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/10.jpg)
What can we check?
Attribute Values – Key Tools Tester and TestFilter AttributeClassifier AttributeRangeFilter & AttributeFilter NullAttributeMapper Matcher DuplicateRemover StringSearcher StatisticsCalculator
![Page 11: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/11.jpg)
What can we check?
Geometry 2D and 3D Intersections Orientation Duplicate Vertices Missing Surface Normals Invalid Solid Voids etc
![Page 12: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/12.jpg)
What can we check?
Geometry Checking – Key Tools GeometryValidator (12 tests and counting)
Detection and repair old transformers bundles in eg Self-Intersector
GeometryFilter Matcher SpatialFilter (Sweco example) SliverRemover & SpikeRemover AreaCalculator & LengthCalculator OrientationExtractor
![Page 13: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/13.jpg)
Workspace Example
Demo 1 – FME Workbench
![Page 14: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/14.jpg)
Validation Workflows on FME Server
Submit data via email, upload, directory watcher, scheduled task
Provide report or results via email, form, data download, PDF
![Page 15: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/15.jpg)
Validation Workflows on FME Server
Demo 2 – FME Server
![Page 16: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/16.jpg)
Partner Validation Examples#1 Axmann GeoChecker
1. Users upload data2. Workspaces Check Schema, Geometry, Attributes3. Email sent with PDF Report and Bad Data in AutoCAD
![Page 17: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/17.jpg)
Partner Validation Examples#1 Axmann GeoChecker
Detailed Reports & AutoCAD File of Errors
![Page 18: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/18.jpg)
Partner Validation Examples#2 Veremes - Qualigéo
Generic Approach to avoid “Chamber of Horrors”
![Page 19: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/19.jpg)
It's easy
Constraint FME tool
Geometry GeometryValidator
Spatial Relationship SpatialFilter/SpatialRelator
Integrity Constraint Feature Merger
Topology
Attribute Domain String Searcher
Statistic Statistic Calculator
Schema Schema reader
Unique Key Duplicate Remover
![Page 20: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/20.jpg)
It's complicated
![Page 21: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/21.jpg)
Hyper generic workspace• 1 unique workspace to validate all formats, all
data models, all constraints...
Using an external description of the data model and constraints
Solution
![Page 22: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/22.jpg)
Partner Validation Examples#3 Metria SwedenEnvironmental Monitoring Data – ISO
QC-service on FME server
Metadata
XML
INSPIREWFS/SOS
QC-report
Metadataportal
DatasetGML
![Page 23: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/23.jpg)
Partner Validation Examples#3 Metria SwedenEnvironmental Monitoring Data – ISO
![Page 24: Validation and Quality Control with FME and FME Server](https://reader036.fdocuments.in/reader036/viewer/2022081512/5560c1d0d8b42aef3b8b53d1/html5/thumbnails/24.jpg)
Partner Example #4
1Spatial - UKPostcodeValidator