International Standards for Data Interoperability: Earth Sciences and GIS models interoperability
Data Interoperability: An Introduction - Esri · Data Interoperability Differentiators . For Data...
Transcript of Data Interoperability: An Introduction - Esri · Data Interoperability Differentiators . For Data...
Esri UC 2014 | Technical Workshop |
Data Interoperability: An Introduction Bruce Harold [email protected] Dale Lutz Safe Software
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Why Data Interoperability?
Background
ArcGIS + FME
Esri UC 2014 | Technical Workshop |
What is the Data Interoperability Extension?
• Connect to Formats - 200+ added file formats - Feeds & Web sources - Custom formats
• Quick Translation - Any format to File GDB - Any format to another Format
• Transformation - Schema transformation - Geometry transformation
Connects your World of Apps
File, Photo, Parcel, BIM, MS Office, DBMS, CSV, Network, Military, Survey, Sensor, GPS…
Esri UC 2014 | Technical Workshop |
ArcGIS Data Interop Common Tasks
• Integration • Migration • Cleanup • Validation • Distribution • Merging Changes
Connected workflows
Spatial ETL
Esri UC 2014 | Technical Workshop |
Data Interoperability Differentiators For Data Triathletes
• Feature by Feature Processing • Inline SQL across all your data • Data Inspection • Deep format & standards support • XML Documents as Features or Attributes • JSON Documents as Features or Attributes • Protocol Support (FTP, HTTP, RSS…) • URL and Compressed Inputs • Flow Control & Parallel Processing • No Coding
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
What you get just by installing
Basic OGC Support
Esri UC 2014 | Technical Workshop |
WFS and GML
• GML - Simple Features Profile
• WFS in Data Interoperability - Cache lives 24 hours by default - Numeric Identifier forces live access
• You must install the extension
Supported without Licensing
OGC GML & WFS suitable for Geoprocessing
Type Presentation Title Here
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Data Interoperability as an OGC client
OGC GML & WFS
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop |
Other OGC Stuff
• KML - Schema object preserved - KML Properties - KML Styling
• CityGML - CityEngine Tools coming - To & From File GDB
But wait there’s more
OGC Formats in Data Interoperability
Type Presentation Title Here
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Ways to read any format data
Using Formats
Esri UC 2014 | Technical Workshop |
Direct Read/Write From Disk
• Catalog browse tree - No need to add file type - Feature Dataset icon - Available feature types shown - Format defaults used
• As geoprocessing input - Tools - Models - Scripts (enable the extension!)
File Based – recognized by extension
Recognized formats ‘just work’
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Interoperability Connections
• When default format connection is: - Infeasible, e.g. authentication required - Undesirable, e.g. optional parameter wanted
• Authored in Catalog tree • Off-disk feeds & web sources • May view source data prior to connecting • Not cached, read when used • Coordinate system may be selected • May be shared (.fdl file)
Manually set up and persistent
Yes, Excel can be read as Point data!
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Selecting Formats
• Browse • Search • Customize
Or, ‘What was that “FME Reader Gallery” thing in the previous slide?’
Don’t see what you’re looking for? Ask us!
Type Presentation Title Here
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Using any format in ArcGIS for Desktop
Direct Read & Connections
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Quick Import & Quick Export GP Tools
Quick Translation
Esri UC 2014 | Technical Workshop |
Quick Import & Quick Export
• Quick Export - To Any Format
• Quick Import - To File Geodatabase
Geoprocessing tools in Data Interoperability
Rapid format conversion using format defaults
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Automate Quick Translations
• Easy way to ship multiple formats • Retain your GIS schema
Use the Quick tools in ModelBuilder
Supply popular formats with GIS-friendly properties
Type Presentation Title Here
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Automated Format Supply
Quick Tools
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Custom Formats
Leveraging Formats
Esri UC 2014 | Technical Workshop |
What if you need your own format?
• Based on a known format • Build format definition workflow once • Remains available in the gallery
Save time by “canning” a custom format workflow
You can share a custom format with other Data Interoperability users
Based on JSON
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
JSON read from the web
Custom Formats
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Spatial ETL
The Power of Transformation
Esri UC 2014 | Technical Workshop |
The Full Power of Spatial ETL
• Format reading and writing is only a beginning - Information has no ‘format’ - Formats are only languages; many have dialects - Formats merely facilitate information (or wreck it)
• Information is often hidden or requires business logic • Transformation reveals and derives information
- Calculate stuff, mess with words, ignore junk - Use relational, dictionary, list & network structures - Way, way more…
• Stop. Writing. Code. (Unless you find it fun)
Connect. Transform. Automate.
Transformation is the real power of Data Interoperability
Esri UC 2014 | Technical Workshop |
Workbench Application
• Create via context menu • Follow the wizard
Graphically Design Data Flows
The Workbench application authors Spatial ETL tools
Esri UC 2014 | Technical Workshop |
Workbench Authoring
• Read • Transform
- Attributes - Geometry - Style
• Write
• Run standalone • Run as tool
Design & Document
Workbench has a canvas and supporting windows
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Transformers
• Eponymous - Named after their function
• Categorized • Browse • Search • Popup
Hundreds of functions at your fingertips
Drag or click transformers onto the canvas or use the popup functionality
Type Presentation Title Here
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Connect. Transform. Automate.
ETL
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
A few more observations…
Wrapping Up
Esri UC 2014 | Technical Workshop |
Data Interoperability in ArcGIS for Server
• Map services can use supported formats - 64 bit only
• Spatial ETL tools must be in Models - Publish service as usual from Result
• Good Zip & Ship solution - Generic Writer into ZIP file - Output format made parametric
Considerations
Data Interoperability is available in ArcGIS for Server
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Coming in 10.3
• FME 2014 SP 2 Engine • Write images & hyperlinks to Excel (maps!) • Data Interoperability for ArcGIS Pro
- 64 bit
• More…
And it’s service packs…
Data Interoperability will be released for ArcGIS Pro
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Learn More
• At the User Conference - Esri Showcase Tuesday 9am until Thursday 1:30pm - After this presentation
• 60 Day Software Evaluation - http://www.esri.com/software/arcgis/extensions/datainteroperability/evaluate
• Training - http://training.esri.com/gateway/index.cfm?fa=search.results&searchterm=data+inter
operability
Data Interoperability
Talk to us now!
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Thank You! Now its Question Time!
• Please fill out the session survey:
First Offering ID: 1221 Second Offering ID: 1402
Online – www.esri.com/ucsessionsurveys Paper – pick up and put in drop box
Data Interoperability – An Introduction
Have a great conference!
Type Presentation Title Here
Esri UC 2014 | Technical Workshop | Type Presentation Title Here