Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and...
Transcript of Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and...
![Page 1: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/1.jpg)
Deep Learning and GeoSpatial Data
@o_courtin
@foss4g.be 2019
![Page 2: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/2.jpg)
AlgorithmOutput
Input
![Page 3: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/3.jpg)
NeuralsNetwork
Output
Input
Loss Function
AlgorithmOutput
Input
![Page 4: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/4.jpg)
NeuralsNetwork
Output
Input
Loss Function
AlgorithmOutput
Input
TrainedModel Output
![Page 5: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/5.jpg)
Computer Vision at scale with RoboSat.pink
![Page 6: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/6.jpg)
Detect inconsistencies between two DataSets
Train on a small area, predict on a larger one.
Goals
![Page 7: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/7.jpg)
RoboSat.pink
@RoboSatPink Computer Vision ecosystem for GeoSpatial Imagery
DataSet Quality Analysis
Change Detection highlighter
Features extraction
![Page 8: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/8.jpg)
RoboSat != RoboSat.pink
https://github.com/mapbox/robosat/issues/184
https://github.com/mapbox/robosat https://github.com/datapink/robosat.pink
![Page 9: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/9.jpg)
RoboSat.pink
GeoJSON
PostGIS
Raster
Coverage
WMS
OSM PBF
XYZMasks
Prediction
MasksCompare
VectorPrediction
Spotifydifferences
areas
![Page 10: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/10.jpg)
Compare Predicts against alternate datasets
Pink : Predicted by trained model
Green : Alternate dataset
Grey : Both agree
Pink squares : Significant differences
![Page 11: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/11.jpg)
Command Line Interface
![Page 12: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/12.jpg)
https://github.com/datapink/robosat.pink/blob/master/docs/101.md
RoboSat.pink 101
![Page 13: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/13.jpg)
Easy to deploy
pip3 install RoboSat.pink
![Page 14: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/14.jpg)
So all you need is :
- Imagery - GPU - Labels
![Page 15: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/15.jpg)
So all you need is :
- Imagery → any file format readable by GDAL
- GPU → NVIDIA > 8Go RAM
- Labels → that’s often the key point
![Page 16: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/16.jpg)
From OpenData to OpenDataSet
https://github.com/datapink/robosat.pink/blob/master/docs/from_opendata_to_opendataset.md
![Page 17: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/17.jpg)
Unet Like with ResNet50 Encoder
Image Credit: https://divamgupta.com
![Page 18: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/18.jpg)
Image Label Cross Entropy mIoU Lovasz
http://www.cs.toronto.edu/~wenjie/papers/iccv17/mattyus_etal_iccv17.pdfhttp://www.cs.umanitoba.ca/~ywang/papers/isvc16.pdfhttps://arxiv.org/abs/1705.08790
Surface based semantic Loss
![Page 19: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/19.jpg)
https://arxiv.org/abs/1809.06839https://github.com/albu/albumentations
Data Augmentations
![Page 20: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/20.jpg)
More than an application, an easy to extent framework
https://github.com/datapink/robosat.pink/blob/master/docs/extensibility_by_design.md
![Page 21: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/21.jpg)
Stacks
Proj 4
GEOS GDAL
Rasterio
CUDAcuDNN
PyTorch
NumPy
OpenCV
RoboSat.pink
PillowShapely Osmium
Mercantile
SuperMercado
Albumentations
LeafLet+ VectorGrid
![Page 22: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/22.jpg)
Open Source
![Page 23: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/23.jpg)
Request For Funding
- Increase (again) prediction accuracy : - on low resolution imagery- even with few labels - feature extraction when they are (really) close- with multibands and Data Fusion
- Add support for :- MultiClass- PointCloud data support- Time Series Imagery
- Improve (again) performances
![Page 24: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/24.jpg)
Few performances Metrics
rsp train ~5 Mp/s, -per epoch-rsp download ~1 to 5 Mp/s rsp tile ~5 Mp/srsp predict ~10 Mp/srsp compare ~50 Mp/srsp rasterize ~50 Mp/s rsp vectorize ~50 Mp/s
8 cores CPU, single GPU (either RTX or V100), SSD
![Page 25: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/25.jpg)
How to scale it, or improve it again ?
rsp train add more GPU, reduce dataset redundancy, improve model, loss or optimizer
rsp tile add more CPUuse raster compression
rsp predict export model to ONNX or JIT, then use an high performance inference solution.
![Page 26: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/26.jpg)
Why performances matters ?
- Playful and Human Learning
- Time and money saver
- No Planet B
![Page 27: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/27.jpg)
RoboSat.pink powered by @data_pink
Computer Vision Take Away
- Industrial OSS state of art GeoSpatial Imagery framework available
- Plain OpenData can be use to train accurate model
- Performances already OK to use it for region / small country area, even on cheap GPU server
- Scale if you provide better hardware…
![Page 28: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/28.jpg)
From text to map, a state of art !
![Page 29: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/29.jpg)
Text 2 Map
![Page 30: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/30.jpg)
Which part of the text refer to a spatial stuff ?
![Page 31: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/31.jpg)
Which part of the text refer to a spatial stuff ?
Then, is this toponym, findable in a Gazetter ?
![Page 32: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/32.jpg)
Which part of the text refer to a spatial stuff ?
Then, is this toponym, findable in a Gazetter ?
Named Entity Recognition
![Page 33: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/33.jpg)
Which part of the text refer to a spatial stuff ?
Then, is this toponym, findable in a Gazetter ?
Named Entity Recognition
GeoNames query
![Page 34: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/34.jpg)
Which part of the text refer to a spatial stuff ?
Then, is this toponym, findable in a Gazetter ?
Named Entity Recognition
GeoNames query
![Page 35: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/35.jpg)
Which part of the text refer to a spatial stuff ?
Then, is this toponym, findable in a Gazetter ?
Named Entity Recognition
GeoNames query
![Page 36: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/36.jpg)
Why NLP hard ?
![Page 37: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/37.jpg)
Let’s eat grandma !
Let’s eat, grandma !
![Page 38: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/38.jpg)
Irony
![Page 39: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/39.jpg)
« The meaning of a word is its use in the language. »
Ludwig Wittgenstein
![Page 40: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/40.jpg)
Source: https://wiki.openstreetmap.org/wiki/OSM_Semantic_Network
![Page 41: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/41.jpg)
![Page 42: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/42.jpg)
NLP Open Source libs
![Page 43: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/43.jpg)
Source: https://spacy.io/usage/facts-figures
![Page 44: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/44.jpg)
Source: https://spacy.io/usage/facts-figures
SacreMoses https://github.com/alvations/sacremoses
PyTorch Text https://github.com/pytorch/text
DeepPavlov https://github.com/deepmipt/DeepPavlov
![Page 45: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/45.jpg)
![Page 46: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/46.jpg)
![Page 47: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/47.jpg)
PerspectivesNER :
- Use more rich structured text, as dbpedia to train models
- Anything helpful for multilang handling
- Use latest papers: https://nlpprogress.com/english/named_entity_recognition.html
Gazetteer
- OCR existing maps for completion ?
- Other kind of data alowing massive completion ?
Toponym matching
- Batch to use text context
- Use others dimensions than only population, and obviously geographical one...
![Page 48: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/48.jpg)
NLP TakeAway
NLP is harder than any others patterns recognition domains
NER softwares and models already availables
Academic NLP R&D is an hot and vivid topic
Firsts Text2Map tools arises
But still lot to do !
![Page 49: Deep Learning and GeoSpatial Data - FOSS4G Learning and GeoSpatial Data... · Deep Learning and GeoSpatial Data @o_courtin @foss4g.be 2019. Algorithm Output Input. Neurals Network](https://reader033.fdocuments.in/reader033/viewer/2022042911/5f453e5a1056b457795a1066/html5/thumbnails/49.jpg)
Extract insights from GeoSpatial data with Deep Learning
Development, Expertise, Support and Training, on : - Computer Vision
- Natural Language Processing - Time Series Analysis
@data_pink
www.datapink.com