Data Interoperability

Jason Schroeder

Solution Engineer – Local Government GA & East TN

Best Practices

Agenda

1. What Data Interoperability isA. How to license, install, openB. Why use?

2. Example use casesA. TransformersB. Settings

3. Automation – how to script with Task Scheduler

4. Resources

What is Data Interoperability? Why use it?

ETL – Extract, Transform, Load

1. Can read and write to hundreds of different formats (spatial and non-spatial)

2. Transform data (data not be in a useful format)A. No coding necessaryB. Validate / reject bad data

3. Validate data (Change detection)A. INSERTSB. UPDATESC. DELETES

Data Interoperability 101

• How to:- License – Data Interoperability is an extension for ArcGIS Pro

- Install – Download separate install file from My Esri

- Open – From ArcGIS Pro Analysis ribbon

3 basic ways to use DI demoDive in to Data Interoperability

1. Quick Import & Export2. Spatial ETL

1. User Parameters – pass to Model Builder

3. FME Standalone

Create Spatial ETL Tool:

• A new workbench file will be created or embedded (check ‘Import’) by default

• For existing workbench files:1. To embed the workbench (fmw format will be embedded in toolbox)

A. Browse to the existing workbench file (*.fmw), click Import

2. To use existing workbench independently A. Workbench file can be opened independently with FME Desktop / Data Interoperability

A. Can share with others if they have FME Desktop

B. Browse to the existing workbench file, leave Import unchecked

Use case – Overwrite or Append

• When to use Data Interop over Desktop/Online tools?

• Data is not acceptable in its current form1. Attribute scrubbing2. Spatial scrubbing

Material Date State

1 1232019 Georgia

2 20190124 Mississippi

2 1-24-2019 Texas

2 1-24-2019 Texas

Coded values1 = Plastic2 = Iron

Date formatting issuesFormat with DIRinse and Repeat

Change data Use State Acronym(GA, MS, TX, etc.)

Bad coordinates- 0, 0- NULL

DuplicatesDetect and Remove

Attribute handling demoDive in to Data Interoperability

Use Case – Enrich

1. We may want to enrich the dataA. Intersect with attributes from some other Geometry

Join Council Districts

Enrichment demoDive in to Data Interoperability

This is a simple demonstration- FME can be used for geoprocessing so sky is the limit- Based mostly on convenience

Use Case – Repeatable process

import arcpyarcpy.ImportToolbox("C:/Users/YourUser/Documents/ArcGIS/Projects/DataInterop/DataInterop.tbx")arcpy.OpenData_DataInterop("C:/Users/YourUser/Documents/RaleighCrimes.csv")

Python Script filename(In this case, it is the same

name as the tool)

Path to folder containing the python script

(OpenData.py)

Path to Python

Parameter(input file for Spatial tool)

Path to toolbox

OpenData.py

Resources- Esri Data Interoperability Training:

https://www.esri.com/training/catalog/5c4b6c7f2c10d337f906f885/arcgis-data-interoperability-in-action/

- Bruce Harold (Data Interoperabilty Product Manager) Blog:https://community.esri.com/people/bharold-esristaff

- FME training site:https://www.safe.com/training/recorded/

Please Take Our Survey on the AppDownload the Esri Events app and find your event

Select the session you attended

Scroll down to find the feedback section

Complete answersand select “Submit”