Combining ArcGIS Server and Other SOAP Services to Display

Bond Project InformationAn example of mashing up web services

Eric Dean KauffmanState of California – The Resources Agency

The Bonds, The Complexity

$42.7 Billion (give or take)• 1B Transportation - $19.925 billion (Dept. of

Transportation)• 1C Housing - $2.85 billion (Housing and Community

Development/California Housing Finance Agency)• 1D Education - $10.4 billion (Dept. of

Education/University of California/California State University/California Community Colleges)

• 1E Flood Control - $4.09 billion (Resources Agency/Dept. of Water Resources)

• 84 Natural Resources - $5.388 billion (Resources Agency/Dept. of Parks and Recreation)

January 2007• Specifically, the executive order directs government agencies that

spend bond funds to institute a three part accountability structure that includes:(Video)– Front-End Accountability: Create a strategic plan with performance

standards for projects prior to the expenditure of funds. – In-Progress Accountability: Document what ongoing actions it will

take to ensure that the infrastructure projects or other activities funded from bond proceeds are staying within the scope and cost that were identified. Additionally, each department shall make semi-annual reports to the Department of Finance to ensure that the projects and activities funded from bond proceeds are being executed in a timely fashion and achieving their intended purposes.

– Follow-Up Accountability: Audit completed projects to determine whether the expenditures were in line with the goals laid out in the strategic plan.

Dept. of Finance Gives Orders• Directs all responsible agencies to work together

to build a common web site• Web site is hosted among departments

– Virtual web site• Directs agencies to follow template• Directs agencies to provide accountability

information that is regularly updated• They wanted it ready by summer• Used the Resources Agency’s/ Dept. of Parks

and Recreation proposition 40/50 as an example

Proposition 40/50 Web Site

Prop 40/50 Lessons

• Lots of lessons regarding database design• Goal of mapping project locations never lived up

to expectations– Original database design not quite right– ArcIMS didn’t lend itself to online (web browser)

mapping of projects– Developer tools not quite there– Design hampered by certain policy decisions

• Tracking bonds, their programs, and projects is easier said then done

1E and 84 Complexity

• 38 pages of business rules• Dozens of terms being defined

– For example, what is a “project?”• Dozens of tables, hundreds of fields• Crazy relations between tables

Follow the Money

Prop. 1 E

Program A

Project on the ground

Prop. 84

Program B

Matching Money

Single project ID number(prevents double counts)

Assets • Dept. of Parks and Recreation (DPR)

– Expertise in managing bond money– Prior work with 40/50 web sites– Established IT infrastructure and support

• Resources Agency (RA)– Expertise with geospatial data– Prior work with 40/50 web mapping– Established IT infrastructure and support

• Combined– Both work with ASP.NET– Both use SQL Server– Both use Visual Studio– We are both in the same governmental “family”

The Solution

• Distribute the bond accountability application– Bond project tracking database built at DPR– Mapping database built at RA– Public web site hosted at RA– Intranet web site hosted at DPR

• Share data through SOAP• One public web site, is, well, “virtually” one

web site

Key Benefits

• Development is done locally with experts• Servers managed by experts• Services are managed by the experts

Major Downside

• Stay synchronized– Development– Services– Data

SOA, Suds, and FUD

SOAP – The Big Idea“Simple Object Access Protocol”

SOAP…is a protocol for exchanging XML-based messages over computer networks, normally using HTTP/HTTPS. SOAP forms the foundation layer of the web services protocol stack providing a basic messaging framework upon which abstract layers can be built.

- http://en.wikipedia.org/wiki/SOAP (Feb 7 2008)

History of SOAP• Wasn’t originally tied to XML

– In fact it predated it– Interfaces where “primitive types”– Eventually grew to include “methods”– Now can emit data using JSON strings, and more

• SOAP originated with support from Microsoft– 1998 Dave Winer, Don Box, Bob Atkinson, and

Mohsen Al-Ghosein (latter two were at Microsoft at the time … right about 1998)

– Release was delayed because internally, developers Microsoft couldn’t agree on the details

1999-2000 Intra-Company Politics

The biggest non-technical issue that faced SOAP in 1999 and 2000 was the hideous nature of vendor wars. The FUD that flew around the trade press and vendor web sites was downright embarrassing. I recently ran across a Sun Reality Check that made me ill. In particular, the following quote blew me away:

SOAP has changed a lot. It started to become interesting to us when IBM made additions to the mediocre specification that Microsoft initially championed (you're right, we thought that specification was a bad idea).

- Don Box (2001)

http://webservices.xml.com/pub/a/ws/2001/04/04/soap.html

W3C Protocols

• Version 1.1 W3C Note 08 May 2000• Version 1.2 (Second Edition) April 2007

SOAP Is Dead!

While much of the hype around Service Oriented Architectures (SOA) revolves around SOAP and its request/response custom-API based brethren, REST-based SOAs are quietly, but quickly, permeating the web to such an extent that one wonders whether SOAP-based SOAs have already been permanently eclipsed.

-Burnham's Beat (2004)

http://billburnham.blogs.com/burnhamsbeat/2004/12/rest_vs_soap_wh.html

(not!)

SOAP and the Application

• ArcGIS Server makes great use of SOAP• Integrated Development Environment

– Sun’s FORTE (2001)– Microsoft’s Visual Studio (2000)– WebSphere Studio– CodeWarrior

• Overall, seems that Java and .Net programmers have great tools

Architecture

DPR RA

SQL Server 2005

SQL Server 2005With ArcSDE

Web Server Web Server

IntranetInternet

Benefits of Our Approach

• SOAP with ASP.NET and Visual Studio is easy to use

• SOAP on ASP.NET integrates very well with data sources

• Is fast (with small amounts of data)• Keeps content, software development

physically close to the experts• Makes it possible to combine existing

information in ways not imagined before

Things We’ve Learned

• Coordination between data providers and consumers is really important

• Something breaks on a data provider that shows up with data consumer

• Building good trust relationships between parties a must

• Performance is boosted with caching of SOAP data

• We’re going to do this more often

Questions?

Email: eric.kauffman@resources.ca.gov