Geoprocessing with GDAL and Numpy in Python
description
Transcript of Geoprocessing with GDAL and Numpy in Python
Geoprocessing with GDAL and Numpy in Python
Delong Zhao11-03-2011
Geoprocessing with GDAL and Numpy in Python
• GDAL - Geospatial Data Abstraction Library• Numpy - the N-dimensional array package for
scientific computing with Python. • Both of them are open source software
Read raster dataset using
GDAL
Do some calculation using
Numpy
Output to geospatial dataset using GDAL
GDAL
• Supports about 100 raster formats– ArcInfo grids, ArcSDE raster, Imagine, Idrisi, – ENVI, GRASS, GeoTIFF– HDF4, HDF5– USGS DOQ, USGS DEM– ECW, MrSID– TIFF, JPEG, JPEG2000, PNG, GIF, BMP– See http://www.gdal.org/formats_list.html
Numpy
• the fundamental package needed for scientific computing with Python.
• a powerful N-dimensional array object• sophisticated (broadcasting) functions• tools for integrating C/C++ and Fortran code• useful linear algebra, Fourier transform, and
random number capabilities.• http://numpy.scipy.org/
Installation
• 1. Enthought python scientific computing package, includes numpy– http://www.enthought.com/
• 2. GDAL - Geospatial Data Abstraction Library– http://www.lfd.uci.edu/~gohlke/pythonlibs/
• Or all of these has been installed on EOMF and Cybercommons servers
Tutorial
• http://itmetr.net/itmetr.cgi/PyIntro• http://www.gis.usu.edu/~chrisg/python/2009
/• http://www.scipy.org/NumPy_for_Matlab_Us
ers• https://www.cfa.harvard.edu/~jbattat/comput
er/python/science/idl-numpy.html
Sample 1
• Read two tif files (red band and nir band)• Calculate
• Output NDVI in same projection and georeference as the input file.
• Numpy example
Algorithm development for global cropping intensity from 2000-2011
1-crop per year2-crop per year
3-crop per year
(b) 6/11/99 rice field preparation
(c) 7/3/99 2-weeks after rice transplanting
(d) 9/6/99 rice plant heading
Dynamics of winter wheat and paddy rice fields in Nanjing, Jiangsu, China
Field site in Jiangsu
Time (8-day interval)
1/1/02 3/1/02 5/1/02 7/1/02 9/1/02 11/1/02 1/1/03
Veg
etat
ion
indi
ces
0.0
0.2
0.4
0.6
0.8
1.0
NDVILSWIEVI
winter wheat paddy rice
MODIS 8-day composites of surface reflectance product (MOD09A1)
NDSI NDVI, EVI, LSWI
Snow mask
Cloud mask Permanent water mask
Evergreen vegetation mask
Cropping intensity ( # of crops per year)
Crop calendar (planting & harvesting dates)
Inundation and paddy rice
Temporal profile analysis of individual pixels
one MODIS pixel in Bangkok area
Month in 2004
1 2 3 4 5 6 7 8 9 10 11 12
Veg
etat
ion
Indi
ces
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
NDVILSWIEVI
Starting date Starting date
One MODIS pixel in Mekong basin
Month in 2004
1 2 3 4 5 6 7 8 9 10 11 12
Veg
etat
ion
Indi
ces
-0.2
0.0
0.2
0.4
0.6
0.8
1.0NDVILSWIEVI
flooding &transplanting
flooding &transplanting
flooding &transplanting
Cropping Intensity map in 2004
Global Mapping of Croplands
Python multiprocessing
• http://docs.python.org/library/multiprocessing.html
• import multiprocessing• pool = multiprocessing.Pool(processes=multiprocessing.cpu_count())• pool.map(doprocess,findfiles(root_dir))
Benchmark
• I did some benchmark. By using all 8 cpu and 16G memory on one eomf server
• can finish 1 MODIS tile NDVI, EVI, CLOUD,SNOW, LANDWATER, FLOOD, Drought products in 15 minutes.
• This means we can finish global 296 tiles 2000-2011 MODIS data in 786 hours (32 days) with one server.
• And we have 6 computation servers, we can improve it to 6 days if all the servers can do the job.