NC GIS Confernce 2009 - The Right Tool For The Job
28
City of Asheville Information Technology Services The Right Tool for the Job
-
Upload
dave-michelson -
Category
Technology
-
view
74 -
download
1
description
Finding the right tool for the job. Using metrics to determine when tools work best. In this case looking at PostGIS and WebService vs ArcGIS (2010) server.
Transcript of NC GIS Confernce 2009 - The Right Tool For The Job
City of AshevilleInformation Technology Services
The Right Tool for the Job
City of AshevilleInformation Technology Services
The Wrong ToolYes you can use this hammer, but…
City of AshevilleInformation Technology Services
The Wrong Tool This could happen.
City of AshevilleInformation Technology Services
The mentalityThere are a lot of tools out there to accomplish a task, some are commercial and some are OpenSource
Each is:●good at specific tasks●great at specific tasks●and not so good at specific tasks
But a lot of us still view it as one or the other and sometimes even one versus the other. We would never think of using them….together.
City of AshevilleInformation Technology Services
The mentalityNo don’t do it.
City of AshevilleInformation Technology Services
The mentalityGreat Dave, now you caused the end of the Geo-Spatial world as we know it.
City of AshevilleInformation Technology Services
The MentalityBut really the only thing that is going to happen is we are going to get a different result.
Maybe we will even find a better tool, a better hammer.
City of AshevilleInformation Technology Services
Why should we do this.Why not is the real question.
And according to a recent study by Gartner found this:
“A recent survey by Gartner, Inc. found that more than half of organizations surveyed have adopted open-source software (OSS) solutions as part of their IT strategy. Nearly one-third of respondents cited benefits of flexibility, increased innovation, shorter development times and faster procurement processes as reasons for adopting OSS solutions. However, the survey revealed that only one-third of responding organizations had a formal OSS policy in place.” *
* http://www.gartner.com/it/page.jsp?id=1541414
City of AshevilleInformation Technology Services
The Great ReviewI was tasked with reviewing alternatives to our normal work flow.
We were looking for ways to save money while increasing GIS usage while increase GIS options/flexibility to our users. while not removing anything we currently offered.
The review included commercial and open source GIS software
City of AshevilleInformation Technology Services
The Great ReviewWe reviewed GIS for the inter-web
ArcGIS serverGeoServer
MapServerMapGuideFeature Serverand a few others…
City of AshevilleInformation Technology Services
The Great ReviewWe reviewed GIS for Databases
SDE (MS-SQL SERVER)PostGISMySQLSpatial MS-SQL Server
City of AshevilleInformation Technology Services
The Great ReviewWe reviewed GIS for the Desktop
ArcMapManifoldQ-GISudigand a few others…
City of AshevilleInformation Technology Services
We Found some SurprisesMost of these products had advantages and disadvantages over each other which for the most part made each product comparable.
Except - It was clear that PostGIS offered something beyond what we could currently do, and it meet the goals we set out to accomplish.
City of AshevilleInformation Technology Services
We Found some SurprisesWe found on a desktop pc* with PostGIS loaded we could:
●Do things faster then SDE on a high end server.
●Do things that SDE just could not accomplish. Such as what I am about to demonstrate.
●Reduce our cost.
●We could use PostGIS with SDE. Not taking away anything we currently offered.
* 1 ghz - 500 megs of ram
City of AshevilleInformation Technology Services
In the real worldWe were at a point were we needed to see what PostGIS could do in the real world with a real world project.
City of AshevilleInformation Technology Services
In the real worldAt roughly the same time we were working on joint project with Buncombe County, NEMAC*, - and us the City of Asheville.
*National Environmental Modeling and Analysis Center
We were tasked with building an application that provided the average natural slope of a Geographic area. And then check if the resulting slope qualified under a new steep sleep ordinance developed by Asheville City Council and/or Buncombe County Commissioners.
The calculation using an ArcGIS Server Geo-Process that was taking 10 – 30 seconds. This is a really, really, longtime on the inter-web.
City of AshevilleInformation Technology Services
The TestsWe started looking at PostGIS to do the calculation
●I converted the data into PostGIS.
●Created some PostGIS spatial SQL queries that accomplished the same thing as the Geo-Process.
City of AshevilleInformation Technology Services
The TestsThe results
●On an old desktop, 1 ghz - 500 megs of ram machine, the calculation took 2-5 seconds. From 10-30 seconds, on a machine with a 2 ghz dual quad core processor and 16 gb ram, 1 to 2-5 seconds on desktop machine with 1 ghz single core processor with 500 megs ram!
●We have a few 10,000+ acre parcels in the National Forests that took 15 minutes in PostGIS – on the same machine. This never worked in ArcGIS server, ArcGIS desktop, or anything ArcGIS.
City of AshevilleInformation Technology Services
And...I optimized the query
●Added SQL functions using PostgreSQL – PostGIS (spatial SQL) which resulted further enhancements to the performance to 1 – 3 seconds.
City of AshevilleInformation Technology Services
And...I found I could further improve the performance by caching the calculation results.
The idea is if the search geometry matches a previous search geometry the slope is the same. So why do the calculation again.
Check the search geometry before executing the calculation and if it matches a geometry searched previously, we just retrieve the old result. This really improves performance for large areas – for small areas there was really only minor improvements.
The caching for the NF parcels meant instead of 15 minutes it took less than 1 second!
City of AshevilleInformation Technology Services
What Next?It was clear that the results were game changing, at least for us.
But the group had already spent a significant amount of time on the project.
So the challenge was to integrate the PostGIS process into the existing application.
City of AshevilleInformation Technology Services
What Next?●We built a web service that looks just like an ArcGIS geo-processing web service but based on the PostGIS methodology.
●We Replaced the ArcGIS geo-processing service with the new PostGIS web service.
This worked!
Web App
Geo-Process
PostGIS Web Service
City of AshevilleInformation Technology Services
Show the demoDemo WMC
City of AshevilleInformation Technology Services
WOW
City of AshevilleInformation Technology Services
Why•Applications are logic, or code. Applications make decisions based on input. The ArcGIS Geo-Process, ArcGIS server, and even SDE are applications and is not really the “best” tool for doing this.
•PostGreSQL-PostGIS is a database. Yes a database is an application also, but it is specially optimized to store, query, and analyze data. This is what it is designed to do. And databases will always outperform other applications attempting to do the same thing.
City of AshevilleInformation Technology Services
Why•Native Spatial Data Type.
•Spatial SQL means no need for a secondary application or api to do spatial queries.
•This uses a database (the right tool for this job) to draw on the strengths of a database.
Use the database to do what a database does best.Use the application to do what a applications does best.
City of AshevilleInformation Technology Services
What it doesModel and Formula PDF
City of AshevilleInformation Technology Services
The End
