Post on 12-Jan-2016
description
Web-Based Open Source GIS:
Decision Support Tools Explaining the Software Stack
Presented by Aaron Racicot – GIS Programmer
aaronr@ecotrust.orgApril 19th, 2006
A Citizen of Salmon Nation
Outline
Introduction - Ecotrust Benefits/Limitations of Open Source OCEANSystem
Problem description Technology Overview - Software Stack
FORESTSystem Problem description Technology Overview - Software Stack
Where is it all going?
Who am I…B.S. Computer
Science
Split Personality
Telemark/BC SkiingRafting Guide Mountain Rescue
M.S. Environmental Science
Open Source User/Developer
GIS Programmer
Ecotrust - Salmon Nation
Benefits/Limitations of Open Source
Benefits Software cost = $0 Source code is available and modifiable User and development communities flourish Development cycles are VERY fast
Limitations Total cost is NOT = $0 Many applications are not as polished as the
proprietary counterparts Compatibility with proprietary software can be an
issue
Software Stacks… Desktop
Generic Stack FOSS4G Stack Grouping
Application QGis, Grass, OSSIM, JUMP, UDig
User Interface
Application Dev. Environment
Eclipse, QT, OpenGL
High Level Utilities GeoTools, PostGIS Data Serving
High Level Scripting Languages
PHP, PERL, Python
Low Level Utilities Shplib, GEOS, OGR/GDAL, PostGIS, R-Statistics, GMT
Data Processing
Low Level Languages
C, C++, Java, Fortran System Software
Operating System
Linux, Darwin, Cygwin
Hardware Drivers
32-64 bit processor drivers
Desktop Software Stack
Software Stacks... Server
Server Software StackGeneric Stack FOSS4G Stack Grouping
Client Side Browser Firefox, Safari, Netscape User Interface
Client Side Scripting
Java Script, Java Applets
~~~~~~~~~~~WEB~~~~~~~~~~~
Server Side Application
Chameleon, Cartoweb3, Custom Data Serving
High Level Utilities Mapserver, PCO, Grass
Low Level Utilities Shplib, GEOS, OGR/GDAL, PostGIS, R-Statistics, GMT
Data Processing
High Level Scripting Languages
PHP, PERL, Python
Low Level Languages
C, C++, Java, Fortran System Software
Operating System/Drivers
Linux, Darwin, Cygwin
Hardware Drivers 32-64 bit processor drivers
Software Stack... DSTs (Ecotrust)
DST Software StackGeneric Stack FOSS4G Stack Grouping
Client Side Browser Firefox, Safari, Netscape User Interface
Client Side Scripting
Chameleon/Custom
~~~~~~~~~~~WEB~~~~~~~~~~~
Server Side Application
Chameleon Data Serving
High Level Utilities Mapserver, GRASS, PostGIS
Low Level Utilities Shplib, GEOS, R-Statistics, GMT, OGR/GDAL
Data Processing
High Level Scripting Languages
PHP, PERL
Low Level Languages
C, C++, Java, Fortran System Software
Operating System/Drivers
Linux (Fedora)
Hardware Drivers 32 bit processor drivers
Work Flow – Real-Time Web DST
Web User Request
Web User Response
StaticData Storage
WebService
sData Gathering
Data Formatting
Data Processing
Map Formatting
Map Production
Real-Time Web Decision Support Tools
Work Flow – What OSGIS is good at
Web User Request
Web User Response
StaticData Storage
WebService
sData Gathering
Data Formatting
Data Processing
Map Formatting
Map Production
The Problem Is The Arrows! Connecting a web request to server side GIS analysis is tricky
Real-Time Web Decision Support Tools
Data Flow... DSTs (Ecotrust)
Raster based DSTSiuslaw Watershed Restoration Initiative
Data Gathering
Data Formatting
Data Processing
PHP Mapscript
GRASS
Raster based DST - Code
class Grass_GIS { var $data_dir; // Where the GRASS data
dir is var $rc_dir; // Where the GRASS rc dir is var $bin_dir; // Where the GRASS bin dir
is var $location; var $mapset; function Grass_GIS($data_dir,
$rc_dir, $bin_dir, $location, $mapset){
$this->data_dir = $data_dir; $this->rc_dir = $rc_dir; $this->bin_dir = $bin_dir; $this->location = $location; $this->mapset = $mapset; } function
Grass_GIS_copy($grass_gis){ $this->data_dir = $grass_gis-
>data_dir; $this->rc_dir = $grass_gis->rc_dir; $this->bin_dir = $grass_gis->bin_dir; $this->location = $grass_gis-
>location; $this->mapset = $grass_gis->mapset; } function set_location($location) {$this->location = $location;} function set_mapset($mapset) {$this->mapset = $mapset;} function set_data_dir($data_dir) {$this->data_dir = $data_dir;} function set_rc_dir($rc_dir) {$this->rc_dir = $rc_dir;} function set_bin_dir($bin_dir) {$this->bin_dir = $bin_dir;}
function run_command($cmd){ $std_output = ""; $grass_exec = ""; // Check that all the required info is available.. if ($this->data_dir != NULL && $this->rc_dir != NULL &&
$this->bin_dir != NULL && $this->location != NULL &&$this->mapset != NULL){// Here we will write out the script to run...$fp_data = fopen($this->data_dir."/grass_run.sh","w");// write out text headerfwrite($fp_data,"export GISBASE=".$this->bin_dir." \n");fwrite($fp_data,"export GISDBASE=".$this->data_dir." \n");fwrite($fp_data,"export GISRC=".$this->rc_dir."/.grassrc6 \n");fwrite($fp_data,"export ETC=".$this->bin_dir."/etc \n");fwrite($fp_data,"export PATH=".$this->bin_dir."/bin:".$this->bin_dir."/scripts:".$this->bin_dir. "/garden/bin:\$PATH:/usr/bin:/usr/local/bin/ \n");fwrite($fp_data,"export LOCATION_NAME=".$this->location." \n");fwrite($fp_data,"export MAPSET=".$this->mapset." \n");fwrite($fp_data,"export LOCATION=".$this->data_dir."/".$this->location."/".$mapset." \n");fwrite($fp_data,"\n");fwrite($fp_data,$cmd."\n");fflush($fp_data);fclose($fp_data);$fp_data = fopen($this->rc_dir."/.grassrc6","w");fwrite($fp_data,"GISDBASE: ".$this->data_dir."\n");fwrite($fp_data,"LOCATION_NAME: ".$this->location."\n");fwrite($fp_data,"MAPSET: ".$this->mapset."\n");fwrite($fp_data,"PAINTER: ppm\n");fwrite($fp_data,"MAPLP: stuff.ppm\n");fflush($fp_data);fclose($fp_data);$grass_exec = sprintf("chmod u+x ".$this->data_dir."/grass_run.sh");exec($grass_exec,$std_output,$std_error);
$grass_exec = sprintf($this->data_dir."/grass_run.sh");exec($grass_exec,$std_output,$std_error);
}
Raster based DST - Codedefine("GRASSDATA_DIR", "/var/www/html/apps/siuslaw_target/data/GRASSDATA/");$location_name = "OR_siuslaw“;$mapset_name = "land_targets“;$session_grass_dir = $session_tmp_dir."/GRASSDATA";
// Make a Dir to work inecho " Make a directory to work in inside this session<br>";$dir_exec = sprintf("mkdir ".$session_grass_dir);exec($dir_exec,$arr,$err1); $dir_exec = sprintf("mkdir ".$session_grass_dir."/rc");exec($dir_exec,$arr,$err1); $template_path = GRASSDATA_DIR."siuslaw_epa_template";echo " Copying over new GRASS GIS template<br>";$dir_exec = sprintf("cp -R ".$template_path." ".$session_grass_dir."/".$location_name);exec($dir_exec,$arr,$err1); // Make a MAPSETecho " Setting up the siuslaw specific area<br>";$dir_exec = sprintf("mkdir ".$ session_grass_dir."/".$location_name."/".$mapset_name);exec($dir_exec,$arr,$err1); // Copy over the projection info from the PERMANENT areaecho " Setting projections<br>";$dir_exec = sprintf("cp ".$template_path."/PERMANENT/PROJ* ".$ session_grass_dir."/".
$location_name."/".$mapset_name."/");exec($dir_exec,$arr,$err1); $grass_session = new Grass_GIS($session_grass_dir, $session_grass_dir."/rc", "/usr/local/grass-6.0.0", $location_name,
$mapset_name);$grass_session->run_command("g.list type=rast");
Vector based DST
Mapserver
PHP -> PHPMapscript -> PostGIS
PostGIS
Vector based DST - Code// Now connect to the db$host_str = "host=localhost";$db_str = "dbname=shelf_closure_db";$usr_str = "user=web_user";$string = $host_str." ".$db_str." ".$usr_str;$connection = pg_connect($string);if (!$connection){ pg_close($connection); echo "Error: Cannot connect to the database <br>\n";} else { echo "Connected to the SCA database <br>\n";}
// Create a dynamic VIEW in PostGISecho "<b>* Create postgresql VIEW * </b><BR>\n";$exec_str = sprintf("CREATE VIEW ". $previous_session_name."_".$previous_session_id." AS ". "SELECT bk.the_geom, sum(tr.total_lbs*tb.percent) as total_lbs ". "FROM tow%02d_3kblk_intersect as tb, trawl%02d". " as tr, cablk3km as bk, port as pt ". "where tb.blk_gid=bk.gid ".$species_sql." ".$port_sql." ".$gear_sql. " and tb.tow%02d_id=tr.tow_id2 ". "GROUP BY bk.the_geom HAVING count(tr.tow_id2)>2", $start_year_short,$start_year_short,$start_year_short);$result_pg_exec = pg_exec($connection, $exec_str);if (!$result_pg_exec) { printf("<B>%s</B><BR>\n", pg_errormessage());}
Vector based DST - Code
// Execute the pgsql2shp to create the outputecho "<B>* Execute pgsql2shp create the shapefile in session directory *</B><br>";$dir_exec = sprintf("/usr/local/pgsql/bin/pgsql2shp -u web_user -f ". $session_tmp_dir."data/target_areas/test_output.shp shelf_closure_db ". "\"SELECT * from ".$previous_session_name."_".$previous_session_id."\"");exec($dir_exec,$arr,$err1); for ($ii=0;$ii<count($arr);$ii++){ echo $arr[$ii]."<br>\n";} // Drop the dynamic table echo "<b>* DROP the postgresql VIEW * </b><BR>\n";$result_pg_exec = pg_exec($connection, "DROP VIEW ". $previous_session_name."_".$previous_session_id);if (!$result_pg_exec) { printf("<B>%s</B><BR>\n", pg_errormessage());}
13 Step Tool Install
1) First install Fedora Core4 with updates2) Ming
wget http://easynews.dl.sourceforge.net/sourceforge/ming/ming-0.3beta1.tar.gz
tar -xzvf ming-0.3beta1.tar.gz
cd ming-0.3beta1
make
make static
cd php_ext/
make
make install
echo "Don't forget to add 'extension=php_ming.so' to php.ini!"
3) Pdflib
wget http://www.pdflib.com/products/pdflib/download/602src/PDFlib-Lite-6.0.2.tar.gz
tar -xzvf PDFlib-Lite-6.0.2.tar.gz
cd PDFlib-Lite-6.0.2
./configure
make
make install
4) Update ldconfig
Add /usr/local/lib to /etc/ld.so.conf
Run : /sbin/ldconfig
5) Proj.4
tar -xzvf proj-4.4.9.tar.gz
cd proj-4.4.9
cd nad/
ls
cp ../../proj-nad27-1.2.tar.gz .
tar -xzvf proj-nad27-1.2.tar.gz
cd ..
./configure
make
make install
6) GDAL
Installed the ECW SDK (do a configure, make and install)
Installed the MrSid SDK (moved to /usr/local)
wget http://gdal.org/dl/gdal-1.3.1.tar.gz
tar -xzvf gdal-1.3.1.tar.gz
cd gdal-1.3.1
./configure --with-ecw --with mrsid=/usr/local/GeoExpressSDK/
make ogr-all
make install
which gdalinfo
7) GEOS
wget http://geos.refractions.net/geos-2.2.1.tar.bz2
tar -xjvf geos-2.2.1.tar.bz2
cd geos-2.2.1
./configure
make
make install
8) POSTGRESQL
tar -xzvf postgresql-8.1.2.tar.gz
cd postgresql-8.1.2
NOTE the LDFLAGS here... this is to support GEOS for postGIS
LDFLAGS=-lstdc++ ./configure --with-perl --prefix=/usr/local/pgsql_8_1_2
gmake
gmake install
Add the shared library path to /etc/ld.so.conf :
/usr/local/pgsql_8_1_2/lib
Run /sbin/ldconfig
Run make install in the ./doc directory to install the documentation
Finally add the following to the profile file to make paths available:
PATH=$PATH:/usr/local/pgsql/bin
MANPATH=$MANPATH:/usr/local/pgsql/man
export MANPATH
Add the postgis user
useradd postgis
passwd
[postgis@localhost pgsql]$ postmaster -D ./data/ > ./data/logfile.txt 2>&1 &
13 Step Tool Install9) POSTGIS
Just make sure that the Postgresql is configured with the LDFLAGS variable set
and that the GEOS software is downloaded and installed from Refractions.
./configure --with-proj --with-geos --with-pgsql
make
make install
Now we can create the database:
createdb -O aaronr test_db
createlang -U aaronr plpgsql test_db
psql -f /usr/local/pgsql_8_1_2/share/postgresql/contrib/lwpostgis.sql -d test_db
10) PHP
./configure --prefix=/usr/local/php4 --program-suffix=4 --enable-force-cgi-redirect --with-config-file-path=/etc/httpd/ --with-gd=/usr/local/ --with-jpeg --with-png --with-tiff --with-zlib --with-freetype-dir --without-ttf --with-mysql --with-regex=system --enable-dbase --enable-dbx --enable-versioning --with-pgsql=/usr/local/pgsql_8_1_2/
make
make install
strip sapi/cgi/php
cp sapi/cgi/php /var/www/cgi-bin/php4
cp php.ini-dist /etc/httpd/php.ini
Made the following mod to /etc/httpd/php.ini:
; Directory in which the loadable extensions (modules) reside.
;extension_dir = "./"
extension_dir = "/etc/httpd/php_mods"
mkdir /etc/httpd/php_mods
Added the following to /etc/httpd/conf/httpd.conf
#
# For PHP scripts as CGI-BIN
#
AddType application/x-httpd-php-cgi .php4 .phtml
Action application/x-httpd-php-cgi /cgi-bin/php4
Now re-start the server:
/etc/rc.d/init.d/httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]
11) MAPSERVER
./configure --with-jpeg --with-gd --with-freetype --with-zlib --with-png --with-pdf --without-tiff --with-proj --with-threads --with-ogr --with-gdal --with-postgis --with-wfs --with-wmsclient --with-wfsclient --enable-debug --with-php=/src/php/php-4.4.2
make
cp legend mapserv scalebar /var/www/cgi-bin/
cp mapscript/php3/php_mapscript.so /etc/httpd/php_mods/
12) GRASS
CFLAGS="-g -Wall" ./configure --with-gdal=/usr/local/bin/gdal-config --with-postgres-includes=/usr/local/pgsql/include/ --with-postgres-libs=/usr/local/pgsql/lib/
make
make install
13) R-Statistics
tar -xzvf R-2.1.1.tar.gz
cd R-2.1.1
./configure
make
make install
Where to go for more infoOSGIS
Maptools - http://www.maptools.org FreeGIS - http://freegis.org/ Open Source GIS - http://opensourcegis.org/
Standards OGC - http://www.opengeospatial.org/
Desktop GRASS - http://grass.itc.it/ QGIS - http://qgis.org/ UDIG - http://udig.refractions.net/confluence/display/UDIG/Home JUMP – http://jump-project.org/ OpenEV - http://openev.sourceforge.net/
Server/Web Mapserver - http://mapserver.gis.umn.edu/ GRASS - http://grass.itc.it/ PostGIS - http://postgis.refractions.net/
Tools Remote Sensing - http://remotesensing.org/tiki-index.php GDAL/OGR - http://gdal.maptools.org/index.html PROJ.4 - http://proj.maptools.org/ R-Statistics - http://www.r-project.org/ GMT - http://gmt.soest.hawaii.edu/
The End
Tool Screen Shots Follow
Backup - GRASS
Backup – R-Statistics
Backup – PostGIS
Geometry
WKT Geometry
Backup - QGIS