Post on 03-Jun-2020
Final presentation
wherecamp berlin 2019
1
Design Patterns of GeoData Driven Apps
Shoaib Burqhttps://geografia.com.au
http://deeplearningbox.com
WhereCamp 2019 (Berlin)
Complexity in Data Driven Apps
3
greater number of data sources
more stakeholders
more applications
bigger datasets
4
5
6
7
Client Name
v1 (2017) - Traditional Geo Applications
8
v1: Challenges
9
No Quality Checks
Poor Schema Enforcement
Time Consuming
Not Scalable for Growth
10
11
v2: add workflow system
12
v2: add workflow systemAdd S3 to Store
Raw Data Trigger Workflow
(AWS lambda)
13
v2: add workflow systemAdd S3 to Store
Raw Data Trigger Workflow
(AWS lambda)
https://docs.prefect.io/core
14
Forecasting with Machine / Deep Learning
v3: Doing Data Science
15
v3: Doing Data Science
16
Workstation with GPUs to train ML &
DL Models ————————— deeplearningbox.com —————————
v3: Doing Data Science
17
Workstation with GPUs to build Deep Learning Models
Competing for Resources with
applicatioonManual Deployment
of ML Models
v4: ML Model Tracking + Deployments
18
v4: PostgreSQL read only replicas
19
v4: MLFlow + RDMBS scaling
20
v4: MLFlow + RDMBS scaling
21
Fix: Competing for ResourcesFix: ML
Experiment Track & Deploy
Futureexpected demands from customers & team
22
FutureInteractive Big Data Queries via Hadoop + Presto
23
FutureReal-Time events integration (e.g. social media feed)
24
Thank you! Questions?
Shoaib Burqhttps://geografia.com.au | http://
deeplearningbox.com
25