Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables...

42
Rohit Singh (@geonumist) Cédric Despierre Corporon (@ceddc) Geospatial Deep Learning with arcgis.learn

Transcript of Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables...

Page 1: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Rohit Singh (@geonumist)Cédric Despierre Corporon (@ceddc)

Geospatial Deep Learningwith arcgis.learn

Page 2: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Session Overview

• What is AI, Machine Learning & Deep Learning• Machine Learning & Deep Learning

- What is Machine Learning?- How is it different from Deep Learning

• Deep Learning workflow in ArcGIS Pro• Geospatial Deep Learning with arcgis.learn• Types of models and their applications

- Training and deploying deep learning models- Scalable deep learning with Image Server

• Resources / Getting started

Page 3: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

What is AI?

Page 4: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Neural Networks

TensorFlow

CNTK

Natural Language Processing

Cognitive Computing

GeoAI

Computer Vision

Dimensionality Reduction

Object Detection

Support Vector Machines

Object Tracking

Keras

PyTorch scikit-learn

fast.ai

Random Forest Machine Learning

Deep Learning

Artificial IntelligenceCaffe

Page 5: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Machine Learning

Deep Learning

Artificial Intelligence

Page 6: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Machine Learning

Deep Learning

Artificial Intelligence

Page 7: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Machine Learning

Deep Learning

Artificial Intelligence

CNTK TensorFlowPyTorch

Video game behavioral AI

Keras

ConvolutionalNeural Networks

fast.ai scikit-learn

Computer Vision

Natural Language Processing

Speech Recognition

Page 8: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Machine Learning

Page 9: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Labelled Data Sets: Features + Output

Call Drops

# Complains Subscribed Package

Call Rate Decline

Churned?

4 5 ABC 20% Yes

6 2 ABC 5% No

9 4 XYZ 12% Yes

Features Output (label)

Training Data(Historical)

For Learning..

New Data Prediction

Learning Models

Trained Model

Problem: Predicting Churn

7 5 KLM 8% Yes (75%)

Page 10: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Labelled Data Sets: Features + Output

Segment Type Proximity to Intersection

Time of Day Weather Accident

Highway 0.1 M Morning Raining Injury

Tunnel 0.3 M Evening Sunny Property

Inner 0.2 M Noon Foggy Injury

PipeAge Depth Temperature Pressure Break

20 3 m 95 F 20 P Yes

15 4 m 5 F 35 P Yes

6 3.5 m 2 F 17 P Yes

IS Indirect Fire

IS Explosive Attack

IS Vehicle Attack

Coalition Air Ops

ISIS Camp

0.2 M 2 M 3 M 0.7 M Yes

4 M 9 M 6 M 1 M No

1.2 M 5 M 5 M 0.5 M Yes

# Females 35 - 50 F&B Sales % Urban

Chic#

Competitors Sales

35,000 $400M 55% 15 $20M

14,000 $150M 15% 27 $5M

27,000 $210M 26% 9 $12M

Road Accidents Prediction Water Leakages Prediction

ISIS Location Prediction Retail Sales Prediction

Page 11: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

ArcGIS has Machine Learning Tools

ArcGIS

Classification

Clustering

Prediction

Page 12: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...
Page 13: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Deep Learning Workflowin ArcGIS

Page 14: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Train DL Model

Detect Objects

Export Training Data For DL Model

Training Samples

ArcGIS Pro ArcGIS Proarcgis.learn

Classify Pixels

Collect Samples Export TrainingSamples

TrainPerform Inference

ArcGIS Enterprise ArcGIS Enterprise

ArcGIS Deep Learning Workflow

Page 15: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Data Labeling: Training Samples Manager

• Add Labels • Quickly Collect Samples• Save Samples to a Feature Class

Collect Samples

Export TrainingSamples

Train Perform Inference

Page 16: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Export Training Data for Deep Learning Tool

Collect Samples

Export TrainingSamples

Train Perform Inference

• Exports Samples to Training Images• Each Image has Labels• Supports various formats

Page 17: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Train Model

• arcgis.learn module in ArcGIS API for Python- No installation (ArcGIS Notebooks)- Easy to use - 3-5 lines of code- Models:

- Object Detection- Pixel Classification- Feature Classification

• External Deep Learning Frameworks- Tensorflow- PyTorch…

Collect Samples

Export TrainingSamples

Train Perform Inference

Page 18: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Consume Deep Learning ModelsPerform Inference

Model Definition

ArcGIS User

Inference results

Input Images

InferenceTools

• ArcGIS Image Analyst in Pro• ArcGIS Image Server on Enterprise

Inference Tools

• Classify Pixels Using Deep Learning

• Object Detection Using Deep Learning

Non Maximum Suppression

Collect Samples

Export TrainingSamples

Train Perform Inference

Page 19: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Classify Pixels Using Deep LearningRuns the model on an input raster to product a classified raster, each valid pixel has an assigned class label.

• Mini-batch support• Processor type: CPU or GPU• Parallel processing in ArcGIS Pro and distributed

raster analysis on Enterprise

Inference Tools

• ArcGIS Image Analyst in Pro• ArcGIS Image Server on Enterprise

Page 20: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Detect Objects Using Deep LearningRuns the model on an input raster to produce a feature classcontaining the objects it finds. • Batch processing

• Optional Non Maximum Suppression

• Processor type: CPU or GPU

• Parallel processing in Pro and distributed raster analysis

on Enterprise

Inference Tools

• ArcGIS Image Analyst in Pro • ArcGIS Image Server on Enterprise

Page 21: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

arcgis.learn moduleDeep Learning using ArcGIS Python API

Page 22: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

gis

geometry

network

schematics

features

realtime

widgets

mapping

env

geocoding

geoenrichment

geoprocessing

raster

geoanalytics

learn

Page 23: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

arcgis.learn

The arcgis.learn module in ArcGIS API for Python enables GIS analysts and data scientists to easily adopt and apply deep learning in their workflows. It enables training state-of-the-art deep learning models with a simple, intuitive API.

Page 24: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Beyond Detections: End to End GeoAI Lifecycle enabled by ArcGIS Imagery AI Capabilities

ImageryAccess

Imagery Prep

Training Data Prep

Train & ConsumeModels

Run Inference at SCALE

Feedback Loop

TakeAction

Deploy Models to Production

Before After

• Installing External DL Frameworks

• Dozens of lines of Code

• HARD!

• No Installation (Notebooks)

• 3-5 lines• EASY

Page 25: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Not just “Training”!

Exporting Training Data arcgis.learn.export_training_data

Training DL Modelsarcgis.learn.SingleShotDetectorarcgis.learn.UnetClassifierarcgis.learn.FeatureClassifierarcgis.learn.PSPNetClassifierarcgis.learn.RetinaNetarcgis.learn.MaskRCNNarcgis.learn.EntityRecognizer

Preparing Data (Augmentation)arcgis.learn.prepare_data

Model Managementarcgis.learn.list_modelsarcgis.learn.Model

Model.installModel.uninstallModel.query_info

Inference APIsarcgis.learn.detect_objectsarcgis.learn.classify_pixels

Page 26: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Things you can do today with ArcGIS.LearnObject Detection using SSD, Pixel Classification using Unet, Feature Classification

Damaged Structures

Roads

Swimming Pools

Building Footprints

Oil Pads

Land Cover

Road CracksCars Palm Trees

Pipeline Encroachment

Page 27: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Computer Vision TasksApplied to GIS

Page 28: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Why use Deep Learning for Imagery

ImageNet Visual Recognition Challenge error rate

• Better than human accuracy at vision tasks

Page 29: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Semantic SegmentationAssign a label to each pixel

Cat

Ground

Sky

Turf/Grass

Building

Water

Pixel Classification

Applications:- Land Cover Classification- Pervious/Impervious mapping…

Models:- UNetClassifier- PSPNetClassifier

Page 30: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Image ClassificationAssign a label to a given image

Cat

Applications:- Damaged building classification- Clean or ‘green’ pools…

Undamaged Damaged

Feature ClassificationAssign a label to a given feature

Models (from torchvision):- Inception- ResNet- VGG…

Page 31: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Object DetectionFind objects and their location (bounding boxes)

Applications:- Detect trees, cars, airplanes, …

Models:- SingleShotDetector- RetinaNet

Page 32: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Instance SegmentationFind objects and their precise locations (masks or polygonal features)

Applications:- Building footprint extraction

Models:- MaskRCNN

Page 33: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Scalable Deep Learning

Page 34: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

ArcGIS Enterprise for Scaling Deep Learning

• Leverage Raster Analytics to scale inference

• All desktop inferencing tools are accessible through enterprise

• Clients to invoke distributed inferencing – map viewer, Pro, notebooks

• Multi GPU support

• Requires the ArcGIS Image Server license

Page 35: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

ExportTrainingDataforDeepLearning Uses a remote sensing image to convert labeled vector or raster data into deep learning training datasets. The output is a folder of image chips and a folder of metadata files.

DetectObjectsUsingDeepLearning Runs a trained deep learning model on an input raster to produce a feature class containing the objects it finds. The features can be bounding boxes or polygons around the objects found, or points at the centers of the objects.

ClassifyPixelsUsingDeepLearning Runs a trained deep learning model on an input raster to produce a classified raster with each valid pixel having a class label assigned.

QueryDeepLearningModelInfo Extracts the model specific settings from the model package item or model definition file.

InstallDeepLearningModel Installs the model package item from portal to the Raster Analysis Image Server.

UninstallDeepLearningModel Uninstalls the model package from portal to the Raster Analysis Image Server

ListDeepLearningModels Lists all the installed model packages on the Raster Analysis Image Server

Pro

+ Se

rver

Too

lsSe

rver

Onl

y To

ols

ArcGIS Enterprise Deep Learning Tools / Services

Page 36: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Image ServerImage Server

Image Server

ArcGIS Pro

Portal

Apps

DesktopAPIs

Web Apps, Maps, Dashboards

ArcGIS Server

ArcGIS Data Store

ArcGIS Services

Image Server

GDB

Content StoreDistributed Raster Data StoreImagery Storage

Notebook Server

Deep Learning Models on Portal

GPU per Server

ArcGIS Enterprise Deep Learning – System Architecture

Page 37: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

60,000 buildingsArcGIS Pro: 1 GP100 GPU (16 GB): 4.5 hoursArcGIS Enterprise: 4 nodes RA server with 3 x P40 GPUs (24GB): 20 minutes

Building Detection – Large Scale ProcessingObject Detection using SSD, Pixel Classification using Unet, Feature Classification

Page 38: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Impervious Surface Classification

Coconut Tree Detection

Building Footprint Extraction

Damaged House Classification

Applications of Deep Learning to Imagery

Pixel Classification Object Detection Instance Segmentation Image Classification

End to End Deep Learning – Wide spectrum of deep learning models

Page 39: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Text / NLP

• arcgis.learn – model for extracting location and other entities- EntityExtractor model

Page 40: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Resources for Getting Started

arcgis.learn:• https://developers.arcgis.com/python• https://medium.com/geoai• https://github.com/Esri/arcgis-python-api

Practical deep learning for coders:• https://course.fast.ai/

Page 41: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...

Complete answersand select “Submit”

Scroll down to find the feedback section

Select the session you attended

Download the Esri Events app and find your event

Please Take Our Survey on the App

Page 42: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API. ...