Post on 06-Jul-2020
High Performance Batch Geocoding with
ArcGISVictor Bhattacharyya
Goal: Get the best possible
batch geocoding
performance from your
system, be it Desktop or
Enterprise
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
Batch Geocoding: Turns a spreadsheet of addresses into points on the map
• Your spreadsheet or table
• An ArcGIS Application such as ArcGIS Pro, Map Viewer or API and SDKs
• And a locator
CSV, XLS
FGDB Table
Portal Table
Big Data File ShareArcGIS Map Viewer
ArcGIS Pro
Geoanalytics Extension
World Geocoding Service
StreetMap Premium
World Geocoder for ArcGIS
Create Your Own
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
Desktop Geocoding RecommendationsBest practices for local geocoding
• Use the latest version of ArcGIS Pro (2.4)
• Use new locators
- Newest StreetMap Premium locators or locators built with the
CreateLocator tool in ArcGIS Pro 2.4
• Set the number of threads on your locator to “Auto”
- This uses 1 less than the number of cores on your machine
• Put the locator and input table on a solid state drive
Desktop Geocoding Recommendations: Use Geocode TableArcGIS Pro 2.4
• Guided workflow steps you through batch
geocoding
• More batch geocoding options
- Location Type
- Category filtering
- Country filtering
• Works with local locators and Portal geocoding
services (including the World Geocoding Service)
• Geocode Addresses can be used for automation
Automating Desktop Geocoding: Geocode Addresses Best practices for local geocoding
• Run Geocode Addresses GP Tool in ArcGIS Pro
• Right-click -> “Copy Python command”
• Paste into your favorite IDE
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
System RecommendationsBest practices for Enterprise geocoding
• ArcGIS Server
- Multiple cores (the more the better)
- Enough memory for those cores (locator size + 500MB per core)
- SSD Drive
- Fast processors
- Physical hardware (not a VM)
ArcGIS Enterprise Batch Geocoding SystemArchitecture
• Perform batch geocoding behind your
organizational firewall
• Get the most performance out of your
enterprise geocoding services
• Manage backend resources
ArcGIS Portal
GIS Servers
Geocode Server
World Locator Service
SMP Locator Service(s)
GeocodingToolsGeoprocessing Service
Hosting
Server
How the system works
GIS Servers
Geocode ServerGeocoding Tools
Geoprocessing Service
Hosting
Server
Behind the scenes
Input Table
Output Feature Layer
World Locator Service
SMP Locator Service(s)
Custom Locator Service
1. GP Service takes input table and chunks requests into batches of 1000
2. GP Service uses geocodeAddresses API to call the Geocode Server geocoding service
3. GP Service implements retry logic to retry failed requests
4. GP Service receives geocoding results & stitches together results
5. GP Service makes output available for download by clients
Configuring the system for your Organization’s use
• Publish a locator to Portal via ArcGIS Pro
• Add the locator as a Utility Service so
your organization can work with them
• Scale up the Geocoding Service to meet
Enterprise needs in Server Manager
- More cores = more geocoding throughput
• Configure batch geocoding resources
available to clients
- numBatchThreads configured per locator
ArcGIS Enterprise Large Batch Geocoding
https://bit.ly/2UspGqa; https://bit.ly/2KF3h7M; https://bit.ly/2C65UJF; https://bit.ly/2WYNHWg
Configure Threads
Configuring the system: A closer look
• Scale up the Geocoding Service to meet
Enterprise needs in Server Manager
- More cores = more geocoding throughput
ArcGIS Enterprise Large Batch Geocoding
GeocodingToolsGeoprocessing Service
GeocodingService
GeocodingToolsGeoprocessing Service
Geocoding Service Instances
Set Geocoding ServiceMin instances = 4Max instances = 4
https://<machine_name>/server/manager
Configuring the system: A closer look
• Configure batch geocoding resources
available to clients
- numBatchThreads configured per locator
ArcGIS Enterprise Large Batch Geocoding
Configure Threads
Geocoding Service Instances
GeocodingToolsGeoprocessing Service
Set numBatchThreads = 4
https://<machine_name>/portal/sharing/rest/portals/self/update
Configuration
Demos
https://<machine_name>/portal/sharing/rest/portals/self/update
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
Clients for geocoding data using Portal
Clients that use Enterprise Batch GeocodingBest practices for Enterprise geocoding
• Use the latest version of Asynchronous Batch Geocoding enabled clients
- Geocode Table in ArcGIS Pro 2.4 -> Desktop
- Geocode Locations from Table Mapviewer tool -> Web
- arcpy.geocoding.GeocodeLocationsFromTable() and arcpy.geocoding.GeocodeFile() -> Scripting
Demo in Pro
Demo in
MapViewer
Automating Enterprise Geocoding: Geocode Locations from Table Python
Automating Enterprise Geocoding: Geocode File Python
Clients for geocoding data on a Big Data File Share
(BDFS Input)
GeoAnalytics: Geocode Locations from Table Data in HDFS -> Geocode Locations from Table GeoAnalytics tool
• If you have the GeoAnalytics Extension enabled
in ArcGIS Enterprise this tool appears in the map
viewer
• Has chunking and retry logic built-in
• Takes input table from a BDFS in ArcGIS
Enterprise
GeoAnalytics: Geocoding via the Python APIData in HDFS -> Geocode Locations from Table GeoAnalytics tool
• GeoAnalytics tool exposed via the
“geoanalytics” module in the Python API
• Has chunking and retry logic built-in
• Takes input table from a BDFS in
ArcGIS Enterprise
Configuring Geocode Locations GeoAnalytics ToolData in HDFS -> Geocode Locations from Table GeoAnalytics tool
• Add the geocoding service as a utility service
in your organization
• Set the
“percentageMaxAllowedComputeCores” and
“percentageMaxAllowedComputeMemory”
system properties
• Read more at:
http://enterprise.arcgis.com/en/server/latest/g
et-started/windows/geoanalytics-settings.htm
https://<machine_name>/server/admin/system/properties/update
Please Share Your Feedback in the App
Download the Esri
Events app and find
your event
Select the session
you attended
Scroll down to
“Survey”
Log in to access the
survey
Complete the survey
and select “Submit”
Questions?