Image Server

Imagery and Raster AnalyticsPeter Becker

Manage and process imagery into authoritative data

sources that are appropriately and efficiently disseminated

to those that need access

Enable access to imagery and analysis through a wide

range of integrated desktop, mobile, and web applications

that are interactive, informative, and engaging

Derive actionable information from imagery and

rasters by performing analytics on massive

volumes of data available from multiple sources

ArcGIS

GIS Server

ArcGIS

GeoEvent ServerArcGIS

Image Server

ArcGIS

GeoAnalytics Server

ArcGIS

Business Analyst ServerArcGIS

Image Server

Desktop

Server

Enterprise / Online

System of

Systems

A Distributed Platform built on Open Geospatial Services

ArcGIS Supports Multiple Implementation Patterns

An Interconnected Network Is EmergingServices

DataData

Web / Mobile / Cloud

Services

Information Model

Data

Identity

Apps

APIs & SDKs

ArcGIS Desktop

The ArcGIS Platform Information Model

ArcGIS EnterpriseArcGIS Online

Scene Layers

Scenes

http:

http:http:

TileLayers

Maps

Services

Data

FeatureLayers

ImageLayers

Analytics Analytics

ArcGIS Image Server 10.5

Image Server

fast on-the-fly dynamic processing

caching and serving tiled

maps

OGC services

serve and analyze scientific

data

scalable raster

analysis and image processing

weather & climate

WCS

NetCDF

HDF

GRIB

vegetation analysis

spectral processing

suitability analysis

terrain analysis

multidimensional analysis

WMS

KML

multidimensional

custom algorithms with Python

design multi-source, multi-LOD tiled services

burn geographic featuresand text into tiles

watermarking

update AOIs

scale tile creation with addition servers

persistent product generation

compliance & standards

store once, many products on-the-fly

reduce storage costs

only process what’s being looked at

100+ analytic functions

http

http

Web Maps(reports)

Web Apps

Story Maps(reports)

Powerful Desktop Apps

Mobile Apps & Devices

Developer Apps

Production Systems(automation)

Systems

Integration

new

compression control for low bandwidths

orthorectifcation and mosaicking

Imagery ArcGIS Can Process Massive Imagery in the Cloud

• . . . A key aspect of the ArcGIS Platform

Enabling

• Imagery Dissemination

• Raster and Image Analysis

• Content Access and Management

Massive

Image Storage

ArcGIS

Image ServerAny Image Source

Process imagery dynamically and serve directly into applications

Process imagery at source resolution and save for multiple uses

Image Services

ArcGIS 10.4 Image Processing

• ArcGIS 10.4 has scalable high

performance analysis of big rasters

and imagery for visual analytics

• On-the-fly processing of massive

images and massive image collections

• Desktop and server

• Visual results can be exported

Raster Functions

Desktop

Server

can be run on the server

Dynamic Raster Models

Mosaic Datasets Image Servicespublished

can be run on the desktop

ArcGIS 10.4 Geoprocessing

• ArcGIS 10.4 has high performance analysis

of standard rasters and imagery for

persistent analytics

• Processing of single images or spatial

subsets of massive images or mosaics

• Desktop and server

• Persistent results

Geoprocessing Models

Geoprocessing Tools

can be run on the desktop

Desktop

GP / SA / Data

Server

GP Servicespublished

can be run on the server

. . . Processing is distributed to Servers in a Cluster. Each process creates its own output file

What is ArcGIS Image Server 10.5

Server dedicated to the

Efficient Processing, Analysis and Disseminationof Imagery and Rasters

• Dynamic Image Services – ‘Making your imagery accessible’

- Serve large collections of imagery and rasters with dynamic mosaicking and on-the-fly

processing

• Raster Analytics – ‘Extracting information from imagery’

- Enabling massive distributed processing and analysis of imagery and rastersnew

Image Server is an additional server capability available with ArcGIS Enterprise 10.5 new

What is Raster Analytics?

A new way to create and execute

spatial analysis models and raster processing chainswhich leverages distributed storage and analytics

• Works with your existing imagery, rasters and GIS data

- Register your local data, use mosaic datasets or image services

- Optionally import data into distributed storage

- Process single massive rasters or large collections

• Enable massive distributed processing and analysis

- Persist dynamic imagery products

- Run spatial analysis as predefined models

• Integrate Analytics into your workflows

- Use ArcGIS Pro to Author models

- Outputs are rasters or features layers on your portal

Foundational Concepts

Raster Analytics adds to existing ArcGIS foundational concepts

Dynamic Raster

Models

on-the-fly processing

Geoprocessing

Models / spatial

analysis

powerful analytics Scalable distributed analytics

with persisted storage

Server-based distributed

processing and storage

Portal

Web GIS Layers

newmoremore extends

Solve New Problems with Raster Analytics

• Run models against data that is too big for single desktop

- Global rasters (big geography)

- Large Scale (high resolution)

- Large Collections (many)

• Run models and meet time constraints

months weeks days hours minutes

Raster Analytics Conceptual Overview

Enterprise

GIS

Web GIS Layers

GIS Data & Imagery

ArcGIS Pro

Developers &

System Integrators

GdbFiles WCS ServicesArcGIS Services

Web Analysis Tools

on Portal MapViewer

Desktop Web Device

Design & Run Models

Image Server

enable Raster Analytics

Raster Analytics Conceptual Overview

Enterprise

GIS

Web GIS Layers

GIS Data & Imagery

GdbFiles WCS ServicesArcGIS Services

Desktop Web Device

Design & Run Models

import and optimize

(optional)

distributed raster analytics cluster

Model Execution Distribution

distributed raster datastore

ArcGIS Pro

Web Analysis Tools

on Portal MapViewer

Developers &

System Integrators

Model Execution Distribution

Raster Analytics Conceptual Overview

Enterprise

GIS

Web GIS Layers

GIS Data & Imagery

New Web GIS Layers

GdbFiles WCS ServicesArcGIS Services

Desktop Web Device

Design & Run Models

import and optimize

(optional)

distributed raster analytics cluster

Model Execution Distribution

distributed raster datastore

analysis results as a new Web GIS Layers

ArcGIS Pro

Web Analysis Tools

on Portal MapViewer

Developers &

System Integrators

Model Execution Distribution

Using Your Own Data

• Your own registered data

- Registered data can be used as input but not output

- Models running against single rasters can be parallelized by block (*as long as the model allows it)

- Models running against a collection of rasters will be parallelized per raster in the collection

- Performance can be susceptible to underlying image format (TIFF vs. JP2)

• ArcGIS distributed storage

- Easy to use import tool gets your data into Raster Analytics optimized storage

- CRF (Cloud Raster Format)

- multi-band, block based, multiple readers, multiple writers, fast

- CRF is a format optimized for Raster Analytics computations

• All outputs of Raster Analytics are written in parallel to ArcGIS distributed storage

- Running models on new Web GIS layers is inherently optimized

Raster Analytics is Powerful

Large Collection of Raster Functions

Chain functions together into Raster Models and apply them to

answer complex questions

Math

Abs

Arithmetic

Band

Arithmetic

Calculator

Divide

Exp

Exp10

Exp2

Float

Int

Ln

Log10

Log2

Minus

Mod

Negate

Plus

Power

Round Down

Round Up

Square

Square Root

Times

Bitwise And

Bitwise Left

Shift

Bitwise Not

Bitwise Or

Bitwise Right

Shift

Bitwise Xor

Boolean And

BooleanNot

Boolean Or

Boolean Xor

Equal To

Greater Than

Greater Than

Equal

Is Null

Less Than

Less Than

Equal

Not Equal

ArgStatistics

Cell Statistics

Statistics

ACos

ACosH

ASin

ASinH

ATan

ATan2

ATanH

Cos

CosH

Sin

SinH

Tan

TanH

Data Management & Conversion

Raster to Vector

Vector to Raster

Colormap

Colormap To RGB

Complex

Grayscale

Remap / Reclass

Spectral Conversion

Unit Conversion

Vector Field

LAS to Raster

LAS Dataset to Raster

Clip

Composite

Extract Bands

Mask

Mosaic Rasters

Rasterize Features

Reproject

Interpolation

Interpolate Irregular Data

Nearest Neighbor

IDW

EBK

Swath

Correction

Apparent Reflectance

Geometric Correction

Speckle Filtering (Lee,Frost,Kuan)

Analysis: Image Segmentation & Classification

Segmentation (Mean Shift)

Training (ISO, SVM, ML)

Supervised Classification

Visualization & Appearance

Contrast and Brightness

Convolution

Pansharpening

Resample

Statistics and Histogram

Stretch

Surface Generation & Analysis

Aspect

Curvature

Elevation Void Fill

Hillshade

Shaded Relief

Slope

Viewshed

Analysis: Overlay

Weighted Sum

Weighted Overlay

Analysis: Band Math & Indices

NDVI / NDVI Colorized

SAVI / MSAVI / TSAVI

GEMI

GVI (Landsat TM)

PVI

Tasseled Cap (Kauth-Thomas)

Binary Thresholding

Analysis: Distance & Density

Euclidean Distance

Cost Distance

Least Cost Path

Kernel Density

Analysis: Zonal

Zonal Statistics

Python

Custom Algorithms

Conditionals

Con

Set Null * Does not contain all capability of Spatial Analyst

Raster Analytics is Easy

• Easy to get started, it is “out of the box analytics”

- Install Image Server on nodes -> start Raster Analytic services -> go

- Your own infrastructure, Amazon, Azure

- Scale up & down as required

• Great user experience

- ArcGIS Pro: visual modeler to design simple and complex models

- Portal Map Viewer: Run Raster Analytics Tools

- Python and REST based API for developers

• Results are immediately available in your Web GIS

- No publishing workflow required

- All outputs are written in parallel to ArcGIS distributed storage

Raster Analytics on Different Infrastructure

• Deployed as Enterprise / Web GIS On-premise | Cloud

• Infrastructure can be…

- your hardware

- your Amazon

- your Azure

• Deployment tools

- ArcGIS Enterprise Cloud Builder for Microsoft Azure

- Amazon Cloud Formation Templates

Portal

Raster Analytics Test Case: Terrain Suitability

Global SRTM 90m

0

100

200

300

400

500

600

700

800

1 2 4 8 16

790

425

252

126

80

Min

ute

s

Raster Analytics Processorsesri virtual machine (Image Server)

• 16GB RAM, 8 cores, NAS storage

13.12 hours

80 minutes

terrain suitability model• compute slope

• compute aspect

• remap

• overlay

global terrain suitability raster

Raster Analytics Test Case: Solar Power Plant Suitability

WebGIS (Image Server cluster) on Amazon

• 8 c3.2xlarge instances (8 vCPUs, 16GB RAM)

Mean Rainfall

Mean Temperature

Elevation

Landcover

30m National Solar Plant Suitability Raster

Raster Analytics

9 minutes

ArcGIS Pro

5 hours 45 minutes

suitability model

Raster Analytics Test Case: Landsat Processing

(foreach) input scene

top of atmosphere

correction

modified soil adjusted

vegetation index

remap to classes

mask

no data

output thematic raster

ArcGIS Enterprise GIS on AWS

Distributed Raster Analytics (Image Server) Cluster

• single node

• AWS c3.8xlarge

• 60GB RAM, 32 cores, 500GB SSD

• 200 Raster Analytics Processors

Infrastructure ProcessingInput Collection Output

Landsat GLS 1990

• 7422 Multispectral Scenes

• S3 storage

Thematic Rasters

• 7422 Thematic Rasters

• Distributed Raster Datastore

2 hours 48 minutes44 scenes per minute

¾ scene per second

Raster Analytics: Penn State Watershed Processing

Input data

False color composite

Segmentation

Classification

Brightness/Veg/Chromaticity

output LandCover map

ProcessingInput data Output

Penn State Watershed

• 397 GB

• Distributed datastore

LandCover Map

• Distributed Raster Datastore

100 Billion pixels!1 hour 13 minutes

10 – 20 core Azure instances

ArcGIS Enterprise on Azure

Image Server Cluster

10 Azure instances – 20 cores each

Infrastructure

Thank You to Our SponsorsE

ME

RA

LD

SA

PP

HIR

E

Questions?

Select the session

you attended

Scroll down to

find the survey

Complete Answers

and Select “Submit”

Download the Esri Events

app and find your event

Please Take Our Survey on the Esri Events App!