Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME,...
Transcript of Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME,...
![Page 1: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/1.jpg)
Data preparation,tools & formats
Warren VickEuropa Technologies Ltd.
![Page 2: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/2.jpg)
Better Mapping 2017
What’s your poison?
• We use a variety of methods and technologies• Straw poll
• Hand drawn• Non-GIS software (whole process)• Esri• PB / MapInfo• Open source• Other
• Your choice?
![Page 3: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/3.jpg)
Better Mapping 2017
Ken Field, 2017
![Page 4: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/4.jpg)
Better Mapping 2017
What is the best GISfor cartography?
• There is no single product or platform• Where possible, use the right tool(s) for the job
![Page 5: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/5.jpg)
Better Mapping 2017
Data Preparation
• Crossing the data chasm• Not a task to be underestimated!• Plan• Document
(especially if a repeated process)
Source data Data ready & ideal for use
Effort Results
Disproportionateeffort
![Page 6: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/6.jpg)
Better Mapping 2017
What might need to be changed?
• Data Format• Data Type (raster / vector)• Data Projection• Inherent styling• Geometry joins / splits
(Good example in my second presentation)• Topology
![Page 7: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/7.jpg)
Better Mapping 2017
Formats
• There are many formats for geospatial data:• Based on published standards• De facto vendor standards
• Data suppliers support limited number of formats• Choose the best format for your software• Some may be better than others:
• Active vs. transfer formats• Know the limitations of your chosen format
• 2Gb file size limits are common
![Page 8: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/8.jpg)
Better Mapping 2017
Projection
• Choosing the right projection for a map isvery important
• Nearly all GIS software can reproject on the fly…• … but this is a computational burden• If you have a known destination projection,
it is best to process to it• Not all reprojection algorithms are the same…• … which can lead to misalignment…
![Page 9: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/9.jpg)
Better Mapping 2017
![Page 10: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/10.jpg)
Better Mapping 2017
Spatial ETL• Extract / Transform / Load• Promoting greater interoperability• Building a bridge over our data chasm• Common tasks:
• Reprojection• Transformations• Restyling
• Other tasks:• Geocoding• Merging / conflict management• Publishing / dissemination
![Page 11: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/11.jpg)
Better Mapping 2017
Developing a workflow
• Programming/scripting vs. Visual workflows• Programming/scripting is difficult but becoming
more accessible. e.g. Python• Visual programming is being made available on
many technology platforms• Building blocks (readers, transformers & writers)
with an intuitive drag-and-drop user interface• Examples…
![Page 12: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/12.jpg)
Better Mapping 2017
Alistair Aitchisonhttps://alastaira.wordpress.com/2011/04/11/an-unlikely-replacement-for-the-bing-maps-3d-control/
FME, Safe Software (www.safe.com)
![Page 13: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/13.jpg)
Better Mapping 2017
Geoprocessing in ArcGIS, Esri (www.esri.com)
Just like programmers creating code,add comments to workflows so that they
may be better understood by others.
![Page 14: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/14.jpg)
Better Mapping 2017
Spatial Model Builder, ERDAS Imagine, Hexagon (www.hexagonegeospatial.com)
![Page 15: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/15.jpg)
Better Mapping 2017
Graphical ModellerQGIS (www.qgis.org)
![Page 16: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/16.jpg)
Better Mapping 2017
Introducing GDAL & OGR
• Open source software (www.gdal.org)• GDAL for raster processing• OGR for vector data processing• Multiple pronunciations
Gee-Dhal vs. Goo-Dull, even Gee-Dull!• Included with QGIS package (www.qgis.org)• Command line tools
OSGeo4W Shell
![Page 17: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/17.jpg)
Better Mapping 2017
Introducing GDAL & OGR
• Open source software (www.gdal.org)• GDAL for raster processing• OGR for vector data processing• Multiple pronunciations
Gee-Dhal vs. Goo-Dull, even Gee-Dull!• Included with QGIS package (www.qgis.org)• Command line tools
OSGeo4W Shell
![Page 18: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/18.jpg)
Better Mapping 2017
OS Terrain 50
Hillshade Slope Base colour
Using gdaldem command
![Page 19: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/19.jpg)
Better Mapping 2017
Using gdaldem command
LIDAR – Digital Surface Model (DSM)
![Page 20: Data preparation, tools & formats€¦ · • Not all reprojection algorithms are the same ... FME, Safe Software () Better Mapping 2017 Geoprocessing in ArcGIS, Esri () Just like](https://reader033.fdocuments.in/reader033/viewer/2022043008/5f98b427907f245591496b95/html5/thumbnails/20.jpg)
Better Mapping 2017
GDAL/OGR Cheat SheetGDAL• Get vector information• Print vector extent• List vector drivers• Convert between vector formats• Print count of features with attributes matching
a given pattern• Read from a zip file• Clip vectors by bounding box• Clip one vector by another• Reproject vector• Add an index to a shapefile• Add a spatial index• Merge features in a vector file by attribute
("dissolve")• Merge features ("dissolve") using a buffer to
avoid slivers• Merge vector files• Get the difference between two vector files
OGR• Get raster information• List raster drivers• Force creation of world file• Convert between raster formats• Reproject raster• Clip raster by bounding box• Crop raster dimensions to vector bounding box• Merge rasters• Stack grayscale bands into a georeferenced RGB• Export a raster for Google Earth• Raster calculation (map algebra)• Create a hillshade from a DEM• Apply color ramp to a DEM
Derek Watkins (http://blog.dwtkns.com)https://github.com/dwtkns/gdal-cheat-sheet