Location analytics by Marc Planaguma at Big Data Spain 2014

38
SPATIAL DATA ANALYTICS MARC PLANAGUMÀ LEAD DATA ENGINEER AND RESEARCHER BDIGITAL

Transcript of Location analytics by Marc Planaguma at Big Data Spain 2014

Page 1: Location analytics by Marc Planaguma at Big Data Spain 2014

SPATIAL DATA ANALYTICS

MARC PLANAGUMÀLEAD DATA ENGINEER AND RESEARCHER BDIGITAL

Page 2: Location analytics by Marc Planaguma at Big Data Spain 2014

Ciudad 2020Exploratory lines and further work

Joana Simoes, BDigital27/05/2014

Scaling up a in a world of geolocated dataMarc Planagumà, BDigital Technology Centre

Page 3: Location analytics by Marc Planaguma at Big Data Spain 2014

Our team

Page 4: Location analytics by Marc Planaguma at Big Data Spain 2014

Our work

Page 5: Location analytics by Marc Planaguma at Big Data Spain 2014

A world of geolocated data

Page 6: Location analytics by Marc Planaguma at Big Data Spain 2014

Applications

Page 7: Location analytics by Marc Planaguma at Big Data Spain 2014
Page 8: Location analytics by Marc Planaguma at Big Data Spain 2014
Page 9: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Spatial analysis or spatial statistics includes any ofthe formal techniques which studies entities usingtheir topological, geometric or geographicproperties

Page 10: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Basic Spatial Types

Line Point Polygon

* plus others

Page 11: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Geometries

Page 12: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Operations

Contains Within

Page 13: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Operations

Equals Disjoint

Page 14: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Operations

Crosses

Touches Overlaps

Page 15: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Operations

Distance

Page 16: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Operations

Union Difference

Page 17: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Operations

Intersection Symmetric Diff

Page 18: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Complex operations

BufferingConvex Hull

Page 19: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Complex operations

Area

Page 20: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Geospatial data mining techniques

Density-based clustering Hierarchy-based clustering

Page 21: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Geospatial data mining techniques

Heatmaps

Page 22: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Geospatial data mining techniques

Trajectory mining

Page 23: Location analytics by Marc Planaguma at Big Data Spain 2014

Spatial data analytics

Geospatial data mining techniques

Geospatial queryingSpatial anomalies

Geographically weighted regressionsSelf-organizing Maps

Agent-based modeling

Page 24: Location analytics by Marc Planaguma at Big Data Spain 2014

Our Use Cases

Customeractivitytracking

Where does people buy?

What is the profile of the buyer?

Traffic agenciesCan we predict the

traffic status?

Can we predict the traffic incidences?

Spatial data analytics

Page 25: Location analytics by Marc Planaguma at Big Data Spain 2014

Our Datasets

Twitter (Point)Meteo (Polygon)Traffic status (Multi-Line-String) Traffic incidence (Multi-Point) Credit card ops (Multi-Point) Demographic (Point & Polygon)Shapefiles (Multi-Polygon)

Spatial data analytics

Page 26: Location analytics by Marc Planaguma at Big Data Spain 2014

Our Tools

Queries

Clustering

Heat maps

TrajectoryAnomaly detection

Spatial data analytics

Page 27: Location analytics by Marc Planaguma at Big Data Spain 2014

Our Stacks

Page 28: Location analytics by Marc Planaguma at Big Data Spain 2014

Old School Stack - Why Not?

PostGIS Hadoop + GIS Tools ElasticSearch

Kibana

Spatial data analytics

Page 29: Location analytics by Marc Planaguma at Big Data Spain 2014
Page 30: Location analytics by Marc Planaguma at Big Data Spain 2014

Hadoop Stack - The new Classic

PostGIS Hadoop + GIS Tools ElasticSearch

Kibana

Spatial data analytics

Page 31: Location analytics by Marc Planaguma at Big Data Spain 2014
Page 32: Location analytics by Marc Planaguma at Big Data Spain 2014

Complete Stack - Big Data GIS

PostGIS Hadoop + GIS Tools ElasticSearch

Kibana

Spatial data analytics

Page 33: Location analytics by Marc Planaguma at Big Data Spain 2014
Page 34: Location analytics by Marc Planaguma at Big Data Spain 2014

Do’s and Don’ts

Geo Batch

Near Real-Time

Basic Geo Operations

Workflow Composition

Geo Streams

Real-Time

Advanced Geo Op.

Unattended Workflow

Spatial data analytics

Page 35: Location analytics by Marc Planaguma at Big Data Spain 2014

What’s next?

Page 36: Location analytics by Marc Planaguma at Big Data Spain 2014

RFC ;)

Spatial data analytics

Page 37: Location analytics by Marc Planaguma at Big Data Spain 2014

Moltes gràcies!

[email protected]@bdigital.org@djkram @bdigital

Page 38: Location analytics by Marc Planaguma at Big Data Spain 2014

17TH ~ 18th NOV 2014MADRID (SPAIN)