Spatial Database for Postgres and ArcGis Users_ How to Choose

Thinking in GISa blog about GIS from a urban geogeek living at the countryside

Feed, Categories, Archives

Spatial Database for Postgres and ArcGis users: how to choose

Posted: June 06, 2008Categories: GeoServer, GIS, ArcIMS, QGIS, FeatureServer, uDig, ArcObjects, PostGIS, gvSIG, ArcGisDesktop, ZigGis, ArcSdeFeedback: 14 Comments and 0 Reactions

As many of you can have already read, ArcSDE for Postgres is coming out at ArcGIS 9.3 (it is currently inthe release candidate state). It will let you store geometries in two formats, Esri geometry and PostGISgeometry, in the same fashion ArcSde for Oracle is letting Esri or Oracle geometries be stored. I have seensome interest in the GIS community about this new, and i was reading interesting posts by Bill Dollins,Paul Ramsey, James Fee and Dave Bouwman, so I thought i would post here my opinion. Plus, as some ofyou may already know, zigGis 2.0 is out, so I am very interested in understanding where it would be betterto use one (ArcSde) or the other (zigGIS 2.0) solution.

First let me introduce you my point of view: i am since the ARC/INFO era a great fan of Esri software,and I am one of the many users in the GIS community saying that Esri is much much better vs OpenSource in the GIS desktop products sector. In fact, while also in the GIS OS scenario there are goodproducts like QGIS, UDig, gvSIG (and some other), in my opinion no one of them still can even fairlyreach what ArcGis Desktop and its extensions can actually do (with the big limit that you can use it only ina Microsoft box, but that is another story).

At my job place, working for the public sector and willing to follow the CEE recommendations, I am alsoan advocate of Open Source software, and I truely believe that -not like for desktop products - the GIS OSscenario for server side products its perfectly close to commercial solutions, in some case even better.

Products like MapServer, GeoServer, PostGis and their skeleton libraries like proj and GDAL (just to namea few), are a wonderful example of outstanding results obtained within the Open Source philosophy from afantastic GIS community of developers and very passionate users. I came to the OS side some years ago,when at my office we couldn't afford anymore the Esri product suite (it was the 9.0 era, then). To gostraight, we decided to switch to Open Source for the server family (choicing PostGis and MapServerreplacing ArcSde for SQL Server and ArcIMS), while we decided to keep the ArcView's licenses.Nowadays i think this was a decision that saved us money and still let us to keep a powerful GISenvironment. What I was missing then, though, was the Esri GeoDatabase model, and the possibility toview and edit data via ArcMap without using complicated solutions implemented by using some sort ofinterchange files. Regarding the GeoDatabase model, I quickly found that using Postgres SQL and PostGisSpatial functions I could almost obtain what i really needed from the Geodatabase stuff. Regarding thepossibility to use PostGis in ArcMap, that days where the times when I started to think about writing aPostGis connector for ArcGis Desktop in my spare time (typically at the weekends and with long no-sleepnights). Not being totally satisfied of PgArc, and before I could even start with my project, I discovered

Spatial Database for Postgres and ArcGis users: how to choose http://www.paolocorti.net/2008/06/06/spatial-database-for-postgres-and-...

1 of 9 03-Feb-13 11:55 AM

the magnificent ArcObjects code that Abe Gillespie wrote to implement its first zigGis version (0.3 if idon't remember badly). The code Abe wrote was really a wonder at my eyes: a completed PostGisWorkspace implementations developed from scratch by using ArcObjects in an extreme, original and notat all documented way (at that time, but maybe even now). To get an idea, here is how looks the code toadd a PostGis layer in ArcMap using the zigGis library:

'Create PostGisWorkspaceFactory Dim wksf As IWorkspaceFactory Set wksf = New PostGisWorkspaceFactory

'Open the PostGIS Workspace from a from IPropertySet (in the SDEWorkspaceFactory fash Dim ps As IPropertySet Set ps = New PropertySet With ps .SetProperty "server", "localhost" .SetProperty "database", "TUTORIAL" .SetProperty "user", "psqluser" .SetProperty "password", "psqluser" .SetProperty "port", "5432" 'specify the configfile property if you want to log zigGis actions for debugging '.SetProperty("configfile", @"C:ziggisZigGislogging.config") End With Dim ws As IWorkspace Set ws = wksf.Open(ps, 0)

'Alternatively you can open the PostGIS Workspace from a zigFile 'ws = wksf.OpenFromFile(@"C:ziggisZigGisexample.zig", 0);

'Open the PostGIS feature class Dim fwks As IFeatureWorkspace Set fwks = ws Dim fc As IFeatureClass Set fc = fwks.OpenFeatureClass("zone")

'Create the new layer (default renderer is ISimpleRenderer) Dim layer As IFeatureLayer Set layer = New PostGisFeatureLayer Set layer.FeatureClass = fc layer.Name = fc.AliasName

'Add the layer in ArcMap focus map Dim pMxDoc As IMxDocument Set pMxDoc = ThisDocument pMxDoc.FocusMap.AddLayer layer

(more samples here)

So I just fell in love with that project, and as far Abe had no time to keep it in life because of other workshe was heavily involved in, I decided to inherit its work and to keep it in life. Almost at the same time, alsoBill Dollins, one of the most experienced GIS people I have ever known, joined the project, and together


2 of 9 03-Feb-13 11:55 AM

we released zigGis 1.0, the first zigGis version with an installer. Quickly other versions came out, untilzigGis 1.2, the first version massively installed by the community (almost 5,000 downloads). One big thingwas missing then: zigGis was read only, and for editing data you needed to go out from ArcMap and gostraight with some OS alternatives like QGIS, gvSIG or uDig. Abe rejoined the project, and a lot ofdevelopment time was spent for implementing editing and make zigGis performance much better. NowzigGis 2.0 is out, giving the user a full GIS experience with PostGis in ArcMap.

Having talked about the release of ArcSde for Postgres and of zigGIS 2.0, let's give a close look at somekind of combinations you can adopt for storing GIS data in Postgres and access them by ArcGis Desktop.The typical every day Esri user may very possibly not ever heard about Postgres and PostGis. In a fewwords you can consider the Postgres/PostGis combination an Open Source equivalent of thexRDBMS/ArcSde product (where x=Oracle, SQL, DB2, Informix). Now, at ArcGis 9.3 release, the x inxRDBMS will also host Postgres.

I would say that when ArcSde for Postgres will be out (soon), there will be at least four ways to go forusing Postgres in ArcGis Dekstop:

1. Using ArcSde for Postgres storing data as Esri geometries (Esri-centric with Postgres view)2. Using ArcSde for Postgres storing data as PostGis geometries using the Esri Geodatabase model(Esri-centric hybrid view)3. Using ArcSde for Postgres storing data as PostGis geometries using the OGC Simple Featuremodel (PostGis-centric hybrid view)4. Using PostGis on Postgres with zigGIS (PostGis-centric view)

(3) this is the combination that Bill is currently showing in his blog.

I will go now more in details on what you get and what you loose by chocing each of this combination.Note that I am going in an Esri-centric to PostGis-centric order. Note once more that what i am writinghere is based on my understanding, as far as i could not try ArcSde for Postgres: so if some of you hassome correction to show, drop a note in the comments please.

1. Using ArcSde for Postgres storing data as Esri geometries (Esri-centric with Postgres view) Gowith this if you:

need Geodatabase goodness (domains, subtypes, networks, etc...)want store raster in the Geodatabasewant use versions, (long transactions)want to read and edit the Geodatabase only using Esri software like ArcGis Desktop (with aminimum of an ArcEditor license if you need to edit data)want to serve GIS data on the web only by ArcGis server (until someone will write the 9.3 bindingsfor MapServer, GeoServer and so on)don't need the powerful PostGis SQL Spatial Functions and you are going to do everything by Esriclients and ArcObjects (though maybe Esri is going to provide its own implementation, like theyrecently did for Oracle and SQL Server â€“ not sure about this)

2. Using ArcSde for Postgres storing data as PostGis geometries using the Esri Geodatabase model(Esri-centric hybrid view) Go with this if you:

need Geodatabase goodness (domains, subtypes, networks, etc...)want store raster in the Geodatabasewant use versions (long transactions)want to be able to read data with major Open Source clients (QGIS, uDig, gvSIG, MapServer,


3 of 9 03-Feb-13 11:55 AM

GeoServer, FeatureServer and many others). You will need at least an ArcView license for readingdata with Geodatabase goodness (ie: to navigate relationship class)want to edit with an ArcEditor license (with the other clients you would surely compromise theGeodatabase structure)want to serve GIS data on the web by ArcGis server or by any important OS map server(MapServer, GeoServer and so on)want to use PostGis SQL Spatial Functions (but without doing INSERT, UPDATE and DELETEoperations otherwise you would still destroy the Geodatabase structure)

3. Using ArcSde for Postgres storing data as PostGis geometries using the OGC Simple Featuremodel (PostGis-centric hybrid view) Go with this if you:

dont' need Geodatabase goodness (domains, subtypes, networks, etc...)want store raster in the Geodatabasewant use versions (long transactions)want to be able to read and write data with major Open Source clients, if not using versions (QGIS,uDig, gvSIG, MapServer, GeoServer, FeatureServer and many others).want to edit in ArcGis desktop with an ArcEditor licensewant to serve GIS data on the web by ArcGis server or by any important OS map server(MapServer, GeoServer and so on)want to use PostGis SQL Spatial Functions, if not using versions (including INSERT, UPDATE andDELETE operations)

4. Using PostGis on Postgres with zigGIS (PostGis-centric view) Go with this if you:

don't need Geodatabase goodness (domains, subtypes, networks, etc...)don't want store raster in the Geodatabasedon't want use versions (long transactions)want to be able to read and write data with major Open Source clients (QGIS, uDig, gvSIG,MapServer, GeoServer, FeatureServer and many others)want to read and edit data with an ArcView license via zigGIS, saving the costs for ArcEditor andArcSde licenseswant to serve GIS data by any important OS map server (MapServer, GeoServer and so on)want to use PostGis SQL Spatial Functions

Now many points are obvious, but I need to clarify some of them, specially for PostGis users not knowingwhat you will get adopting the Esri GeoDatabase implementation.

Considerations if you are already a PostGis user

First: what you get if you use ArcSde? You will be able to store your GIS data in an RDBMS like Oracle,SQL Server, Informix, DB2 and Postgres (from 9.3). Possibly Postgres, if you are reading this post ;-)

The geodatabase is able to contain three dataset types:

Feature classesRaster datasetsTables

Regarding feature classes, you may opt to use some of the Esri Geodatabase Model benefits, like:

Feature datasetsSubtypes


4 of 9 03-Feb-13 11:55 AM

Attribute domainsRelationship classesTopologyNetwork datasetsGeometric networksTerrain (TIN)Address locatorLinear referencingVersioning

for more info you may look the ArcGis documentation.

Keep in mind that doing so you are highly tied to the Esri suite. In fact the geodatabase model is managedat server side level by ArcSde with a lot of metadata tables and stored procedures in the database, thatkeep track of all of this stuff, and at client side (for example from ArcMap) by the ArcObjects API. Themodel is very powefull and gives the developer and even the analyst a lot of GIS out of the box benefits,but - being based on this Esri structure - you are then forced to use Esri software for accessing the data.For instance, if you have created a relationship class between two feature classes, you will need ArcMapor some other ArcObjects application to navigate the relationship or to edit it.

Considerations if you are already an ArcSde user PostGIS will let you store data in the PostgresRDBMS. You can store Feature classes and Tables according to the OCG Simple Feature implementation.There is still not the possibility to store raster, but my opinion is that rasters should be better stored in thefilesystem than in a RDBMS (well, it is just my opinion â€“ of course). And â€“ as far as Open Sourcesoftware evolves quickly, it is even possible that a solution for storing raster in PostGis will be soonavailable.

You obviously cannot use the GDB features: if you need them you have two options: 1)use ArcSde forPostGIS (only from 9.3, of course); 2)write the logic of your model by using Postgres objects or by codingit in your application (instead than in the data).

If you opt for the first solution, be sure to read this Bill Dollins post about it. It is a very good solution fortaking in your organization other cool Open Source software, desktop clients like uDig, QGIS, and gvSIG(you will be able to effectively read and edit data with them), and outstanding (and free!!!) server productslike MapServer, GeoServer, FeatureServer and SharpMap.

If you still would like to use PostGis from within ArcGis desktop but you can't afford the ArcSde andArcEditor licenses, zigGis could be the way to go. But if that is your choice and you miss the intelligentdata (GDB), how can you obtain similiar GDB behaviours in PostGis? Well, mostly is just a question to usesome DDL, Postgres SQL and PostGis Spatial SQL. For example:

GDB Subtypes: Postgres Primary and Foreign Keys. and some triggers if you want to differentiatebehaviour for each feature class subtype. But ArcMap won't display the different subtypes in theTOC, you may obtain this only by customizationGDB Attribute Domains: Postgres Check Constraints. But ArcMap won't show you the combobox inthe edit data form. Again: you may obtain this only by customizationGDB Relationship Classes: Postgres Primary and Foreign Keys, with cascade behaviour. Triggerwith spatial SQL for spatial relationship. ArcMap won't show the relationship, only if customizedGDB Topology: Postgres triggers with spatial SQL. The topologic dataset will be updated in ArcMapas soon as the triggers fire and the map is refreshedGDB Network Datasets: Postgres triggers with spatial SQL. The network connectivity rules can beimplemented from ArcCatalog with customization


5 of 9 03-Feb-13 11:55 AM

Like

GDB Geometric Network: Implementable like for network datasets

Fot this time is all, I hope i could clarify some points to you.

Disclaimer: I still couldn't directly test ArcSde for Postgres 9.3 (and not sure I can ever be able to), and iam one of the zigGis developers.

Showing 14 comments

4 people liked this.

LoginAdd New Comment

Paolo,

This is an excellent write up. Thanks for sharing. Will be sure to check out zigGis for any futureprojects that may require the ArcGIS/PostGIS combo.

--Andres

Excellent article :-)I've tried a little bit the demo version of ZigGIS 2.0 and I'm surprising of the capabilities: multi-userediting functionalities are impressive :-0For my (little) experience, ArcMap is necessary for a bunch of important functions (like advancedediting and plottings) that gvSIG (the best OS Desktop GIS AFAIK) don't implement at this time, soZigGIS looks extremely useful to maintain a OpenSource architecture (likePostGIS+GeoServer+gvSIG) without lose any advantage (using ArcGIS only when I really need).


6 of 9 03-Feb-13 11:55 AM

Great article; I look forward to sharing it with some of our clients wondering what the best mix willbe for them.

Great work,

Brian

Nice work, great comparisons and synopses of options.

Very interesting article Paolo!

Great article Paolo. But I guess you've heard a lot of that already. Still need to kick my zigGISinstall around. Hopefully I'll be able to in the coming weeks.

I guess the only thing I see charming about the ArcSDE is the raster part and even that isquestionable if there is no easy way to query it except via ArcGIS tools. My main reason for thatironically is because of firewalls. The DMZ for example can hit the database but things like rastersare a pain since they need to be uploaded to the website.

Has anyone tried storing rasters in PostGIS with ArcSDE?

Thanks to all of you, glad that you liked it! :D

@Diego: ArcGis desktop excels not only at Plotting and Editing, but also has a lot of spatialanalysis functions that are still not integrated in the other desktop GIS. Just think of the ArcGisextensions: there are a lot of GRID, TIN, Network and Geostatistic functions out there ;-)

@Regina: I agree on your doubts about raster. The DMZ issue is effectively something that could


7 of 9 03-Feb-13 11:55 AM

be effectively solved by loading the raster in the RDBMS. But are you sure that copying the imagesto a RDBMS in the DMZ would make so much difference in respect of uploading the images to afilesystem in the DMZ? (this i guess could depend heavily in the internet protocol you are using toaccess the file system).

@Regina: ok, i read again your comment. Looks like your RDBMS is out of DMZ, but the port isopen so web server can query it. In this case, yours is a good point, and i cannot be helpful to you.

Great Work, Paolo!

Paolo,

Excellent summary all around. The introduction of ArcSDE to PostgreSQL definitely will createsome interesting technology mixes.

Many folks who were locked in before will get to experience PostGIS and may just begin towonder. Tom Waits once did a song that started with the line "Small Change got rained on with hisown .38" ....

@Regina. I did do a quick test of loading rasters into ArcSDE on PostgreSQL and creating a rastercatalog. Seemed to work like it does on the other platforms. I will discuss that in my next post butI've had a deadline pressing so it's had to wait.

Bill

Nice to hear that. I wait with baited breath. Its nice to know others have listened to Tom Waits.Haven't heard his songs in a while though.


8 of 9 03-Feb-13 11:55 AM

M Subscribe by email S RSS

Trackback URL http://disqus.com/forums/thinkingingis/thinking_i

blog comments powered by DISQUS

Powered by Blogofile.Some right reserved by Paolo CortiRSS feeds for Entries and Comments.

Bill, the sense of that song's words is not very clear to me, but I also like Tom Waits, whatever he issinging there :D

It's more the idea that, by supporting PostgreSQL, ArcSDE may lead people not otherwise inclinedto an open-source platform where they may begin to question other technology choices. I guess thekey part is "with his own .38". :)

@Regina. It had been a while for me too but he snuck onto my iPod recently.

ZigGIS is end. An alternative tool is PgMap from ST-Links.


9 of 9 03-Feb-13 11:55 AM

Spatial Database for Postgres and ArcGis Users_ How to Choose

Documents

Transcript of Spatial Database for Postgres and ArcGis Users_ How to Choose