ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating...
Transcript of ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating...
![Page 1: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/1.jpg)
ArcGIS Data Reviewer
Advanced Data Validation Concepts
Francisco Perez & Shankar Chandrasekaran
![Page 2: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/2.jpg)
Workshop Agenda
• Why Quality Control Matters ?
• What is ArcGIS Data Reviewer?
• Quality Control - A Seamless Experience
• Challenges in Production and Management of Data
• Advance Data Validation Process/Building an Integrated QA Platform
• Best Practices
![Page 3: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/3.jpg)
Why Quality Control Matters
• In 2010 BP Deepwater Horizon Rig exploded due to a number of quality failures.
• In 2015 Aliso Canyon gas leak
• In 2015 Refugio State Beach Oil Spill -The pipeline did not have an automatic shut-off valve that would
have been required on an intrastate pipeline
![Page 4: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/4.jpg)
Garbage in Garbage Out
Buildings
Roads
Water Bodies
![Page 5: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/5.jpg)
What is ArcGIS Data Reviewer?Data Quality Management for ArcGIS
• Provides
- Rule-based validation
- Interactive tools
- Track errors
• For individuals and enterprise
- Saves time/money
- Less rework
• For multiple domains
- Configurable
- Extendable
Enterprise
Workflow
Validate
ReviewSummarize
![Page 6: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/6.jpg)
Garbage in Quality Out
Buildings
Roads
Water Bodies
![Page 7: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/7.jpg)
What is ArcGIS Data Reviewer?Data quality management in the ArcGIS platform
• Data Reviewer for Desktop
- ArcGIS Pro
- ArcMap
• Data Reviewer for Server
- ArcGIS Server (standard or higher)
• Data Reviewer API
- JavaScript
- ArcGIS Pro SDK for Microsoft .NET
• Web AppBuilder for ArcGIS
Server
Online Content and
Services
Desktop Web Device
ArcGIS Online
Portal
![Page 8: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/8.jpg)
Using ArcGIS Platform
Quality Control - A Seamless Experience
![Page 9: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/9.jpg)
Requirements Quality Assurance Plan
Stake holder Inputs
Business Requirements
Technical Data Quality
Requirements
Quality Assurance Plan
![Page 10: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/10.jpg)
Requirements Quality Assurance Plan - Cyclic
Stake holder Inputs
Business Requirements
Technical Data Quality
Requirements
Quality Assurance Plan
![Page 11: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/11.jpg)
Requirements Quality Assurance Plan - Challenges
Dynamic
Change at different points in time
Change triggered by
• New business requirements
• Internal Process Improvements
Change across organization
Common
Disruptive in Nature
Requires more changes in QC process
![Page 12: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/12.jpg)
Requirements Quality Assurance Plan – Manage Change
Collaborate changes authoritatively
Unified experience for change
Visualize results
Assess & share results
SDK
✓
![Page 13: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/13.jpg)
Shankar Chandrasekaran
Quality Control - Demo
![Page 14: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/14.jpg)
Key Components Quality Assurance Plan – ArcGIS Platform
Provides ability to create
configuration add-ins
Ability to configure and execute rules
Ability to visualize results
Ability to create charts to access
data quality
Ability to share authoritatively
Ability to share results as map
packages
Ability to share charts as map packages to
portal / online
SDK
✓
![Page 15: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/15.jpg)
Questions ?
![Page 16: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/16.jpg)
Francisco Perez
Challenges in Production/Management of Data
![Page 17: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/17.jpg)
Challenges in Production and Management of Data
ArcGIS
Data validation is time-consuming
Data being created is of poor quality
Understanding data quality can be difficult
Data
Pro
du
cti
on
Knowing if we will meet our deadlines is difficult
There is no consistency in how work is done
Takes too long for staff to become effectiveData
Man
ag
em
en
t
Data
Reviewer
Workflow
Manager
ArcGIS Pro
Tasks
![Page 18: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/18.jpg)
Advance Data Validation Process = Building an Integrated QA Platform
• ArcGIS Pro
• Workflow Manager
• Data Reviewer
• ArcGIS Pro Tasks
ArcGIS Pro
Workflow
Manager
Data Reviewer
Editors & QA
Managers
= Standardization
= Tracking Tasks
= Categorization/ Validation
= Fix/Edit
Configured
Tasks
![Page 19: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/19.jpg)
Francisco Perez
QA Platform - Demo
![Page 20: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/20.jpg)
QA Platform Demo
ArcGIS Pro
Data Reviewer
Pro Tasks
Workflow Manager
![Page 21: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/21.jpg)
QA Platform Demo – Key Setups
1. WMX-Reviewer Integration (Map Level)
2. WMX-Reviewer Integration (Workflow Level)
3. ArcGIS Pro Tasks (Map Level)
![Page 22: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/22.jpg)
Questions ?
![Page 23: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/23.jpg)
Best Practices
![Page 24: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/24.jpg)
Best Practices
Organize QC Checks
Results Storage
Sharing Results
![Page 25: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/25.jpg)
Organize QC ChecksBest Practices
Type of Checks
Medium to Large Inputs
Importance of configuration
Frequency of execution
Time of Execution
![Page 26: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/26.jpg)
Results StorageBest Practices
• Find and fix errors at the same time.
• No Error tracking
• No Error reporting in organization
• When reporting and history of errors matters
• Always use Non-Versioned Reviewer
• Requires managing/tracking error lifecycle
FGDBs
Enterprise
GDB
![Page 27: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/27.jpg)
Sharing ResultsBest Practices
As Map layers
• Map Packages
• Map/Map Documents
• Project Packages
Using custom script tools
• Managing Data Reviewer
Records
http://www.arcgis.com/home/item.html?id=651e07ac692d4be2bf9d682968a75079#
![Page 28: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/28.jpg)
Best Practices - Demo
![Page 29: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/29.jpg)
Where to startLeveraging templates
• Data Reviewer Templates
- Electrical utilities
- Gas utilities
- Local government
- Topographic mapping
- Water resources
- Water utilities
• Based on Esri industry models
• Use as starting point
solutions.arcgis.com
![Page 30: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/30.jpg)
Day and Description Type Time Location
Tuesday, July 11
ArcGIS Data Reviewer: An IntroductionTechnical
Workshop10:15 – 11:30am Room 31B
Wednesday, July 12
ArcGIS Data Reviewer: An IntroductionTechnical
Workshop3:15 – 4:30pm Room 31B
ArcGIS Data Reviewer: Integrating Data Validation Capabilities into
Web ApplicationsDemo Theater 3:30 – 4:15pm
Demo Theater 8
Enterprise
ArcGIS Data Reviewer: Implementing Data Quality Reporting in Web
ClientsDemo Theater 4:30 – 5:15pm
Demo Theater 8
Enterprise
Thursday, July 13
ArcGIS Data Reviewer: Advanced Data ValidationTechnical
Workshop8:30 – 9:45am Room 31A
ArcGIS Data Reviewer: Leveraging Geoprocessing for Data
ValidationDemo Theater 9:30 – 10:15am
Demo Theater 6
Spatial Data Mgmt
ArcGIS Data Reviewer: Validating Linear-Referenced Events Demo Theater 10:30 – 11:15amDemo Theater 6
Spatial Data Mgmt
Want to learn more this week?
![Page 31: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/31.jpg)
Want to learn more this week?
Data Reviewer Experts
![Page 32: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/32.jpg)
Want to learn more?
• Product informationwww.esri.com/datareviewer
![Page 33: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/33.jpg)
Want to learn more?
• Documentation
- Desktop (desktop.arcgis.com)
![Page 34: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/34.jpg)
Want to learn more?
• Documentation
- Desktop
- Server (server.arcgis.com)
![Page 35: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/35.jpg)
Want to learn more?
• Documentation
- Desktop
- Server
• Training (training.esri.com)
- Assessing Data Quality using ArcGIS Data Reviewer (Seminar)
- Evaluating Positional Accuracy Using ArcGIS Data Reviewer for Desktop (Seminar)
- Data QC with ArcGIS: Automating Validation (Web Course)
- Data QC with ArcGIS: Visual Review (Web Course)
- Quality Control Using ArcGIS Data Reviewer for Desktop (Instructor-Led)
![Page 36: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/36.jpg)
Want to learn more?
• Documentation
- Desktop
- Server
• Training
- Assessing Data Quality using ArcGIS Data Reviewer
- Evaluating Positional Accuracy Using ArcGIS Data Reviewer for Desktop
- Data QC with ArcGIS: Automating Validation
- Data QC with ArcGIS: Visual Review
- Quality Control Using ArcGIS Data Reviewer for Desktop
• GeoNet (geonet.esri.com)
- Data Reviewer place
![Page 37: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/37.jpg)
Please Take Our Survey on the Esri Events App!
Select the session you attended
Scroll down to find the survey
Complete Answersand Select “Submit”
Download the Esri Events app and find
your event
![Page 38: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/38.jpg)
As “GIS - The Science of Where”becomes an integral part of society
Food for thought as we conclude..
Shouldn’t the accuracy of “WHERE”be an integral part of it as well….
More to come …. Adios
![Page 39: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/39.jpg)
Questions ?
![Page 40: ArcGIS Data Reviewer: Advanced Data Validation Concepts€¦ · ArcGIS Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 3:30 –4:15pm Demo](https://reader030.fdocuments.in/reader030/viewer/2022013002/5ec5e549dd25fa06af0cb7ea/html5/thumbnails/40.jpg)