GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984,...
Transcript of GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984,...
![Page 1: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/1.jpg)
GeoEvent & GeoAnalytics:Leveraging the spatiotemporal big data store
Adam MollenkopfReal-Time & Big Data GIS Capability Lead, Esri
@amollenkopf
Suzanne FossReal-Time & Big Data Product Engineer, Esri
@sfoss_esri
![Page 2: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/2.jpg)
1
2
3
4
5
6
Agenda
why a new type of data store?
visualizing observation data
writing observation data from GeoEvent Server
on-the-fly aggregations
writing analytic results from GeoAnalytics Server
deeper dive
![Page 3: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/3.jpg)
why a new type of data store?1
![Page 4: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/4.jpg)
Web GISwith real-time capabilities
Web GIS
ingest visualize
analyze store
real-time & big data
thousands3K e/s
3K e/s 200 e/s
BYO-RDBMS
X
X
200 e/s
GeoEvent
Server
![Page 5: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/5.jpg)
Web GISwith real-time capabilities
Web GIS
ingest visualize
analyze store
real-time & big data
GeoEvent
Server
thousands3K e/s
3K e/s 200 e/s
relational
data store
X
X
200 e/s
![Page 6: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/6.jpg)
Web GISwith real-time & big data capabilities
Web GIS
spatiotemporal
big data store
ingest visualize
analyze store
real-time & big data
GeoEvent
Server
millions4K e/s
4K e/s 10Ks e/s
4K e/s
![Page 7: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/7.jpg)
Web GISwith real-time & big data capabilities
Web GIS
spatiotemporal
big data store
ingest visualize
analyze store
real-time & big data
millions4K e/s
4K e/s 10Ks e/s
GeoEvent
Server
4K e/s
4K e/s
4K e/s
![Page 8: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/8.jpg)
ArcGIS Enterprisewith real-time & big data capabilities
ArcGIS
Enterprise
GeoEvent
Server
spatiotemporal
big data store
Big DataIoT
ingest visualize
analyze store
real-time & big data
millions4K e/s
4K e/s 10Ks e/s
GeoAnalytics
Server
![Page 9: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/9.jpg)
ArcGIS Enterprisewith real-time & big data capabilities
ArcGIS
Enterprise
GeoEvent
Server
spatiotemporal
big data store
Big DataIoT
GeoAnalytics
Server
1
2 43
MINIMUM environment
functional servers & spatiotemporal big data store
SHOULD BE on ISOLATED machines!!!
![Page 10: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/10.jpg)
ArcGIS Enterprisewith real-time & big data capabilities
ArcGIS
Enterprise
GeoEvent
Server
spatiotemporal
big data store
Big DataIoT
GeoAnalytics
Server
1
2 3 64 75 8
RECOMMENDED environment
functional servers & spatiotemporal big data store
SHOULD BE on ISOLATED machines!!!
![Page 11: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/11.jpg)
ArcGIS Enterprisewith real-time & big data capabilities
DesktopWeb Device
visualization
live & historic
aggregates & features
map & feature service
• Ingest high velocity real-time
data into ArcGIS
• Perform continuous analytics
on events as they are received
• Store observations in a
spatiotemporal big data store
• Run batch analytics on
stored observations
• Visualize high velocity &
volume data:
- as an aggregation
- as discrete features
• Notify those who need to
know about patterns of
interest
stream service
live features
ArcGIS
Enterprise
GeoEvent
Server
spatiotemporal
big data store
storage
ingestion
analytics
GeoAnalytics
Server
analytics
![Page 12: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/12.jpg)
visualizing observation data2
![Page 13: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/13.jpg)
Observation datatypes of observation data
movingsomething
that moves
• planes
• vehicles
• animals
• satellites
• storms
stationarystands still but
attributes change
• water gauges
• weather stations
• traffic sensors
• air quality
discretesomething that
“just happens”
• crimes
• lightning
• accidents
![Page 14: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/14.jpg)
Observation datadefined
• An observation is a recording of a feature’s attribute values and location at a specific
moment in time.
- Observations are immutable, they happen and are typically not edited.
- Observations can be replayed over space & time.
- Moving observations are identifiable by a unique attribute, known as a TRACK_ID.
space & time
AM
Observations can be
shown at a specific time
PM
Moving observations can
be identified by a unique attribute
tracks
1
1
1
2
22
3
3
3
the tracks of moving observations
Moving observation illustration for two tracks
over space (X, Y) and time (T)
can be reconstructed
T
![Page 15: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/15.jpg)
On-the-fly aggregationsof observations
• The spatiotemporal big data store enables you to aggregate data on-the-fly
- aggregating on-the-fly accommodates for real-time data
- you can perform exploratory analysis via any combination of space, time and attribute queries
- toggle from aggregation to raw feature view
- access feature level attributes in aggregate or feature level view
![Page 16: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/16.jpg)
Demo
map & feature service
visualizing observation data
![Page 17: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/17.jpg)
writing observation data
from GeoEvent Server3
![Page 18: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/18.jpg)
Writing observation datafrom GeoEvent Server
ArcGIS
Enterprise
GeoEvent
Server
spatiotemporal
big data store
IoT
map & feature service
data source
input output
GeoEvent Service
![Page 19: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/19.jpg)
Demo
into a spatiotemporal big data store
from GeoEvent Server
Writing observation data
![Page 20: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/20.jpg)
on-the-fly aggregations4
![Page 21: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/21.jpg)
Preparing data for on-the-fly aggregationsspatial indexing: geohash
• as data is written to a data source in the spatiotemporal big data store:
- a spatial index for geohash aggregation is continuously updated
geohash aggregation (based on a geohash index)
![Page 22: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/22.jpg)
Preparing data for on-the-fly aggregationsspatial indexing: geohash & square
• as data is written to a data source in the spatiotemporal big data store:
- a spatial index for geohash aggregation is continuously updated
- a spatial index for square aggregation is continuously updated
square aggregation (based on a square index)geohash aggregation (based on a geohash index)
![Page 23: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/23.jpg)
Preparing data for on-the-fly aggregationsspatial indexing: triangle
• as data is written to a data source in the spatiotemporal big data store:
- a spatial index for ‘pointy’ triangle aggregation is continuously updated
- a spatial index for ‘flat’ triangle aggregation is continuously updated
flat triangle aggregation (based on a flat triangle index)pointy triangle aggregation (based on a pointy triangle index)
![Page 24: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/24.jpg)
Preparing data for on-the-fly aggregationsspatial indexing: hexagon (same as triangle)
• as data is written to a data source in the spatiotemporal big data store:
- a spatial index for ‘pointy’ hexagon (pointy triangle) aggregation is continuously updated
- a spatial index for ‘flat’ hexagon (flat triangle) aggregation is continuously updated
flat hexagon aggregation (based on a flat triangle index)pointy hexagon aggregation (based on a pointy triangle index)
![Page 25: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/25.jpg)
Preparing data for on-the-fly aggregationsspatial indexing
• as data is written to a data source in the spatiotemporal big data store:
- up to four types of spatial indices are supported: geohash, square, pointy & flat hexagon/triangle
- this is in addition to a temporal index on the time field
- and an inverted index on each of the attribute fields
geohash
square
trianglepointy
triangleflat
hexagonflat
hexagonpointy
![Page 26: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/26.jpg)
Visualizing on-the-fly aggregationsgeohash
geohash aggregation response geohash aggregation response
• map & feature services can query a data source in the spatiotemporal big data store with results
aggregated on-the-fly.
map servicefeature service
lodType=geohash&lod=2
feature servicelodType=geohash&lod=2&returnGeometry=false
geohash aggregation response
![Page 27: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/27.jpg)
Visualizing on-the-fly aggregationssquare
map servicefeature service
lodType=square&lod=2
square aggregation response square aggregation response
• map & feature services can query a data source in the spatiotemporal big data store with results
aggregated on-the-fly.
![Page 28: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/28.jpg)
Visualizing on-the-fly aggregationshexagon aggregation
map service
flat hexagon aggregation response
feature servicelodType=flatHexagon&lod=2
flat hexagon aggregation response
• map & feature services can query a data source in the spatiotemporal big data store with results
aggregated on-the-fly.
![Page 29: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/29.jpg)
Visualizing on-the-fly aggregationsspatial projection
• some of the spatial indices support projection(s):
- the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly
- square, pointy & flat hexagon/triangle spatial indices are defined with a spatial reference
geohash aggregation in world mollweide projection (wkid = 54009) square aggregation in world mollweide projection (wkid = 54009)
![Page 30: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/30.jpg)
Demo
geohash, square & hexagon
spatial projection
Visualizing on-the-fly aggregations
http://github.com/esri/aggregation-viewer-server-map-service http://github.com/esri/aggregation-viewer-client-feature-layer
![Page 31: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/31.jpg)
writing analytic results
from GeoAnalytics Server5
![Page 32: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/32.jpg)
ArcGIS
Enterprise
GeoEvent
Server
spatiotemporal
big data store
Big Data
GeoAnalytics
Server
Writing analytic resultsfrom GeoAnalytics Server
IoT
![Page 33: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/33.jpg)
GeoAnalytics Serveranalytic capabilities
Summarize DataAggregate PointsSummarize NearbySummarize WithinReconstruct TracksCreate Panel
Find LocationsFind Existing LocationsFind Similar Locations
Analyze PatternsCalculate DensityFind Hot SpotsCreate Space Time Cube
Use ProximityCreate Buffers
Manage DataCopy to Data StoreJoin Features
* time aware tools
![Page 34: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/34.jpg)
ArcGIS
Enterprise
GeoEvent
Server
spatiotemporal
big data store
IoT
GeoAnalytics
Server
Writing analytic resultsfrom GeoAnalytics Server
• GeoAnalytics Server has the ability to:
- perform analytics against data sources in the spatiotemporal big data store
- and write the analytic results to the spatiotemporal big data store
Web GIS layers
new Web GIS layers
use your Web GIS layers through Pro, Portal,Python notebooks or the REST API
![Page 35: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/35.jpg)
ArcGIS
Enterprise
GeoEvent
Server
spatiotemporal
big data store
GeoAnalytics
Server
big datafile shares
featureservice
.shp
shapefiles
.csv
text files
Hadoopfile system
Hive
Writing analytic resultsfrom GeoAnalytics Server
• GeoAnalytics Server has the ability to:
- perform analytics against files in an external big data file share
- and write the analytic results to the spatiotemporal big data store
new Web GIS layers
Web GIS layers
![Page 36: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/36.jpg)
Loading datavia ‘Copy to Data Store’ GeoAnalytics tool
GeoAnalytics tool interface
output features
big datafile shares
.shp
shapefiles
.csv
text files
Hadoopfile system
Hive
featureservice
relationaldata store
spatiotemporalbig data store
![Page 37: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/37.jpg)
Writing analytic resultsvia ‘Reconstruct Tracks’ GeoAnalytics tool
GeoAnalytics tool interface
big datafile shares
.shp
shapefiles
.csv
text files
Hadoopfile system
Hive
featureservice
relationaldata store
spatiotemporalbig data store
![Page 38: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/38.jpg)
Writing analytic resultsvia ‘Reconstruct Tracks’ GeoAnalytics tool
output features
input features GeoAnalytics tool interface
![Page 39: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/39.jpg)
deeper dive6- operating system requirements
- resource requirements
- data reliability
- ObjectId
![Page 40: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/40.jpg)
Operating system requirements64-bit Windows or Linux
resource minimum RECOMMENDED notes
64-bit
operating system
(Windows)
Windows Server:
2012 R2 Standard & Datacenter (Sep 2016)
2012 Standard & Datacenter (Sep 2016)
2008 R2 Standard, Enterprise & Datacenter (sp1)
2008 Standard, Enterprise & Datacenter (sp2)
Windows Server:
2012 R2 Standard & Datacenter (Sep 2016)
‘_’ characters are
not allowed in
machine names
64-bit
operating system
(Linux)
Red Hat Enterprise Linux Server 7 (update 8)
Red Hat Enterprise Linux Server 6 (update 2)
CentOS Linux 7 (7.2)
CentOS Linux 6 (6.8)
Scientific Linux 7 (7.2)
Scientific Linux 6 (6.8)
Ubuntu Server LTS (16.04.1)
SUSE Linux Enterprise Server 12 (sp1)
SUSE Linux Enterprise Server 11 (sp4)
Oracle Linux 7 (update 2)
Oracle Linux 6 (update 8)
Use your Linux flavor of choice with the
latest update version Esri has certified.
http://server.arcgis.com/en/data-store/latest/install/windows/arcgis-data-store-system-requirements.htm
http://server.arcgis.com/en/data-store/latest/install/linux/arcgis-data-store-system-requirements.htm
![Page 41: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/41.jpg)
Resource requirementsprocessors
http://server.arcgis.com/en/data-store/latest/install/windows/arcgis-data-store-system-requirements.htm
http://server.arcgis.com/en/data-store/latest/install/linux/arcgis-data-store-system-requirements.htm
resource minimum RECOMMENDED notes
processors 8 cores 16 cores
More processors results in
faster writes (indexing time),
higher write throughput &
faster query response time.
![Page 42: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/42.jpg)
Resource requirementsprocessors, memory
http://server.arcgis.com/en/data-store/latest/install/windows/arcgis-data-store-system-requirements.htm
http://server.arcgis.com/en/data-store/latest/install/linux/arcgis-data-store-system-requirements.htm
resource minimum RECOMMENDED notes
processors 8 cores 16 cores
More processors results in
faster writes (indexing time),
higher write throughput &
faster query response time.
storage 400 GB 1 TB or moreStorage need is dictated by data velocity & size.
SSD results in much faster writes & queries.
![Page 43: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/43.jpg)
Resource requirementsprocessors, memory, storage
http://server.arcgis.com/en/data-store/latest/install/windows/arcgis-data-store-system-requirements.htm
http://server.arcgis.com/en/data-store/latest/install/linux/arcgis-data-store-system-requirements.htm
resource minimum RECOMMENDED notes
processors 8 cores 16 cores
More processors results in
faster writes (indexing time),
higher write throughput &
faster query response time.
storage 400 GB 1 TB or moreStorage need is dictated by data velocity & size.
SSD results in much faster writes & queries.
memory 16 GB 32 GBMore memory results in
higher likelihood of query cache hits.
32 GB is the maximum due to JVM limitations.
![Page 44: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/44.jpg)
Resource requirementsprocessors, memory, storage & instances
resource minimum RECOMMENDED notes
processors 8 cores 16 cores
More processors results in
faster writes (indexing time),
higher write throughput &
faster query response time.
storage 400 GB 1 TB or moreStorage need is dictated by data velocity & size.
SSD results in much faster writes & queries.
memory 16 GB 32 GBMore memory results in
higher likelihood of query cache hits.
32 GB is the maximum due to JVM limitations
# of
ISOLATED
instances
1 3 or more
With one instance there is risk of data loss.
With three instances replicas are written
increasing data reliability.
The # of instances will vary based on the velocity of
incoming data as well as the data retention policy
configured on each data source.
http://server.arcgis.com/en/data-store/latest/install/windows/arcgis-data-store-system-requirements.htm
http://server.arcgis.com/en/data-store/latest/install/linux/arcgis-data-store-system-requirements.htm
![Page 45: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/45.jpg)
spatiotemporal
big data store
GeoEvent
Server
Data reliabilityshards & replication factor
GeoAnalytics
Server
node 1
node 2
node 3node 4
node 5
T1
T3
T2
T2
r = 1
T1
T3
![Page 46: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/46.jpg)
spatiotemporal
big data store
Data reliabilityauto-rebalancing of data upon node membership changes, + or -, in the big data store
GeoEvent
Server
GeoAnalytics
Server
node 2
node 3node 4
node 5
T1
T3
T2
T2
r = 1
x
T1
T3
T1
T1T3
![Page 47: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/47.jpg)
purge based on
data retention
spatiotemporal
big data store
GeoEvent
Server
Data reliabilitydata retention policies, configured per data source
GeoAnalytics
Server
node 1
node 2
node 3node 4
node 5
r = 1
![Page 48: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/48.jpg)
purge based on
data retention
spatiotemporal
big data store
GeoEvent
Server
Data reliabilityrolling index option, set appropriately to the velocity of your observation data
GeoAnalytics
Server
node 1
node 2
node 3node 4
node 5
r = 1
indices indices
indices indices
indices
![Page 49: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/49.jpg)
purge based on
data retention
spatiotemporal
big data store
GeoEvent
Server
Data reliabilityautomatic data backups using periodic snapshots, including ability to restore from a snapshot
GeoAnalytics
Server
node 1
node 2
node 3node 4
node 5
r = 1
snapshot-2016-05-17-11-0-0.snapshot
snapshot-2016-05-17-12-0-0.snapshot
…
![Page 50: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/50.jpg)
ObjectIdchoosing an Object Id option
![Page 51: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/51.jpg)
ObjectIdchoosing an Object Id option
![Page 52: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/52.jpg)
ObjectIdchoosing an Object Id option
Max Value # of IDs ArcGIS Clients
Int32 2,147,483,647 2.1 billion Pro, Desktop, Ops Dashboard, …
events per day Int32
1,000 e/s 86,400,000 25 days
10,000 e/s 864,000,000 2.5 days
100,000 e/s 8,640,000,000 6 hours
1,000,000 e/s 86,400,000,000 36 minutes
10,000,000 e/s 864,000,000,000 4 minutes
![Page 53: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/53.jpg)
ObjectIdchoosing an Object Id option
Max Value # of IDs ArcGIS Clients
Int32 2,147,483,647 2.1 billion Pro, Desktop, Ops Dashboard, …
Int64 (signed) 9,223,372,036,854,775,807 9.2 quintillion JavaScript, custom apps
events per day Int32
1,000 e/s 86,400,000 25 days
10,000 e/s 864,000,000 2.5 days
100,000 e/s 8,640,000,000 6 hours
1,000,000 e/s 86,400,000,000 36 minutes
10,000,000 e/s 864,000,000,000 4 minutes
![Page 54: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/54.jpg)
ObjectIdchoosing an Object Id option
Max Value # of IDs ArcGIS Clients
Int32 2,147,483,647 2.1 billion Pro, Desktop, Ops Dashboard, …
Int64 (signed) 9,223,372,036,854,775,807 9.2 quintillion JavaScript, custom apps
events per day Int32 Int64 (signed)
1,000 e/s 86,400,000 25 days 292,472,000 years
10,000 e/s 864,000,000 2.5 days 29,247,200 years
100,000 e/s 8,640,000,000 6 hours 2,924,720 years
1,000,000 e/s 86,400,000,000 36 minutes 292,472 years
10,000,000 e/s 864,000,000,000 4 minutes 29,248 years
![Page 55: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/55.jpg)
ObjectIdchoosing an Object Id option
Max Value # of IDs ArcGIS Clients
Int32 2,147,483,647 2.1 billion Pro, Desktop, Ops Dashboard, …
Int64 (signed) 9,223,372,036,854,775,807 9.2 quintillion JavaScript, custom apps
UniqueStringID n/a unlimited JavaScript, custom apps
events per day Int32 Int64 (signed)
1,000 e/s 86,400,000 25 days 292,472,000 years
10,000 e/s 864,000,000 2.5 days 29,247,200 years
100,000 e/s 8,640,000,000 6 hours 2,924,720 years
1,000,000 e/s 86,400,000,000 36 minutes 292,472 years
10,000,000 e/s 864,000,000,000 4 minutes 29,248 years
![Page 56: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/56.jpg)
Summary
http://links.esri.com/geoevent-sbds
spatiotemporal big data store
• The spatiotemporal big data store enables:
- GeoEvent Server to write high velocity & volume observation data
- On-the-fly aggregations to be visualized using Map & Feature Services
- GeoAnalytics Server to read & write high volume analytic results
• To learn more:
- See the ‘Spatiotemporal Big Data Store’ tutorial
![Page 57: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/57.jpg)
Real-Time & Big Data GIS
• Applying Real-Time Analytics Mon, 1:45pm-2:45pm, Room 147B
Tue, 11:00am-12:00pm, Room 151B
• Leveraging the Spatiotemporal Big Data Store Mon, 3:00pm-4:00pm, Room 147B
Tue, 1:30pm-2:30pm, Room 151B
• ArcGIS GeoAnalytics: Intro to Big Data Feature Analytics Mon, 4:15pm-5:15pm, Room 147B
Tue, 2:45pm-3:45pm, Room 147A
• Leveraging Stream Services Tue, 2:45pm-3:45pm, Room 151A
• ArcGIS GeoEvent Server: Best Practices Tue, 4:00pm-5:00pm, Room 151A
• Real-Time & Big Data GIS at a Massive Scale Tue, 5:15pm-6:15pm, Room 151A
other sessions
![Page 58: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/58.jpg)
Please Take Our Survey on the Esri Events App!
Select the session
you attended
Scroll down to find
the survey
Complete Answers
and Select “Submit”
Download the Esri Events
app and find your event
![Page 59: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/59.jpg)
Print Your Certificate of Attendance
Print stations located in the 140 Concourse
Monday
12:30 PM – 6:30 PM
GIS Solutions Expo,
Hall B
5:15 PM – 6:30 PM
Expo Social,
Hall B
Tuesday
10:45 AM – 5:15 PM
GIS Solutions Expo,
Hall B
6:30 PM – 9:30 PM
Networking Reception,
Smithsonian National Air
and Space Museum
![Page 60: GeoEvent & GeoAnalytics - Esri€¦ · -the geohash spatial index only supports GCS WGS 1984, however you can project on-the-fly-square, pointy & flat hexagon/triangle spatial indices](https://reader031.fdocuments.in/reader031/viewer/2022011905/5f2779144161bb74ba536713/html5/thumbnails/60.jpg)
Questions / Feedback?
http://links.esri.com/geoevent
http://links.esri.com/geoevent-forum
To learn more:
Adam MollenkopfReal-Time & Big Data GIS Capability Lead, Esri
@amollenkopf
Suzanne FossReal-Time & Big Data Product Engineer, Esri
@sfoss_esri