Geocoder Web Service Geocoder Web Service for the Metrofor the Metro Area

Nancy Read Nancy Read Metro. Mosquito Control DistrictMetro. Mosquito Control District

nancread@mmcd.orgnancread@mmcd.org

MetroGIS Policy Board, MetroGIS Policy Board,

January 27, 2010January 27, 2010

Geocoder

Background – how it works Update Addition – Landmarks

– Parks, Schools, City Halls, …

Text string Locationcoordinates

Metro Counties Govt. Center2099 University Ave. W 44.959665 °

St. Paul, MN 55104 -93.187222 °

Coordinates used to put point on map:

Geocoder

Text string Locationcoordinates

2099 University Ave. W 44.959665 °

St. Paul, MN 55104 -93.187222 °

Match Street NameInterpolate along Range

Address – Streets File

2001 2201

University Ave. W

2099

Text string Locationcoordinates

2099 University Ave. W 44.959665 °

St. Paul, MN 55104 -93.187222 °

Match a Point

Address – Parcel Points

2001 University Ave. W

2099 + University Ave W

2099 University Ave. W

2151 University Ave. W

Text string Locationcoordinates

Metro Counties Govt. Center2099 University Ave. W 44.959665 °

St. Paul, MN 55104 -93.187222 °

Match a Point

Point of Interest / Landmark

BakerElem.School

Text match

Metro CountiesGovt. Center

McDonald’s

Geocoder: 3 Parts

Data

• Format• Accuracy• Up-to-date• Abbreviations• Spelling errors

Geocoder Parts

Match, Score Data

ST = StreetST = Saint

• Geocoder EngineStandardizersComparison

• SettingsWeightsRules

Geocoder Parts

Web Service Match, Score Data

ST = StreetST = Saint

• Input, Output formats

Why build a geocoder service?

Use point data as well as interpolated Local data more current, “Better” Unlimited use, no fees or license issues

Complex tool, only build once One data set to update

MetroGIS Geocoder ProjectMetroGIS Geocoder Project

2007 - MetroGIS project funding2007 - MetroGIS project funding

Dave Bitner (MAC)Dave Bitner (MAC) Jim Maxwell (TLG)Jim Maxwell (TLG) Mark Kotz (Met.Co.)Mark Kotz (Met.Co.) Gordy Chinander (MESB)Gordy Chinander (MESB) Bob Basques (St. Paul)Bob Basques (St. Paul) Chris Cialek, Jim Dickerson & Pete Olson (LMIC)Chris Cialek, Jim Dickerson & Pete Olson (LMIC) Kent Treichel (MN Dept. of Revenue) Kent Treichel (MN Dept. of Revenue) Nancy Read (MMCD)- Project ManagerNancy Read (MMCD)- Project Manager

MetroGIS Geocoder ProjectMetroGIS Geocoder Project

PAGC open source geocoderPAGC open source geocoder– reworked from batch to multi-userreworked from batch to multi-user– created code for a servicecreated code for a service– has community of interested developershas community of interested developers

Set up service with Metro dataSet up service with Metro data– TLG StreetsTLG Streets– 7-county 7-county Parcel Layer (centroids)Parcel Layer (centroids)– Address Points (as available)Address Points (as available)

House + Street or House + Street or IntersectionIntersection

Host Host – MnGEO MnGEO

(Minnesota Geographic Information Office)(Minnesota Geographic Information Office)

MetroGIS Geocoder ProjectMetroGIS Geocoder Project

Metro GeocoderMetro Geocoder

Up continuously since June 2008Up continuously since June 2008 Automated data updateAutomated data update

Streets – weekly from NCompass (TLG)Streets – weekly from NCompass (TLG)

Parcels – quarterly from MetroGIS/ Met. Council Parcels – quarterly from MetroGIS/ Met. Council (as provided by counties)(as provided by counties)

Usage 10,000 – 90,000 hits per monthUsage 10,000 – 90,000 hits per month

Tools, Docs, UsersTools, Docs, Users

Sample form - try address, see responses Sample form - try address, see responses Technical documentsTechnical documents

– Geocoder Request Protocol documentation Geocoder Request Protocol documentation – Geocoder Response API documentation Geocoder Response API documentation

SOAP wrapper for .Net (Matt McGuire, Metro Council)SOAP wrapper for .Net (Matt McGuire, Metro Council) ArcMap ToolArcMap Tool (Steve Jakala, Scott Co.) (Steve Jakala, Scott Co.) Public sites using Geocoder Web Service: Public sites using Geocoder Web Service:

– Metro Mosquito Control web map (Houston Engineering) Metro Mosquito Control web map (Houston Engineering) – DNR LandView (Craig Perrault, MnDNR)DNR LandView (Craig Perrault, MnDNR)– Carver County Carver County

Using Geocoder in ApplicationsUsing Geocoder in Applications

Web Service takes urlencoded Address or Web Service takes urlencoded Address or Intersection requestsIntersection requests

returns closest matches with latitude-returns closest matches with latitude-longitude, in XML, CSV or JSON.longitude, in XML, CSV or JSON.

Example RequestExample Requesthttp://geoserver.state.mn.us/geocoder/geocode_response?http://geoserver.state.mn.us/geocoder/geocode_response?methodName=GeocodeRequestmethodName=GeocodeRequest&Version=1.1&Version=1.1&CountryCode=US&CountryCode=US&maximumResponses=10&maximumResponses=10&CompleteAddressNumber=&CompleteAddressNumber=20992099&CompleteStreetName=&CompleteStreetName=UNIVERSITYUNIVERSITY%20%20AVEAVE%20%20WW&PlaceName=&PlaceName=SaintSaint%20%20PaulPaul&ZipCode=&ZipCode=5510455104%20%2034313431&ResponseFormat=XML&ResponseFormat=XML

Response (xml)Response (xml)

<?xml version="1.0" encoding="UTF-8" ?> ….<?xml version="1.0" encoding="UTF-8" ?> ….

- <GeocodeResponse>- <GeocodeResponse>

- <GeocodeResponseList numberOfGeocodedAddresses="- <GeocodeResponseList numberOfGeocodedAddresses="11">">

- <GeocodedAddress> -<Address>- <SiteAddress>- <GeocodedAddress> -<Address>- <SiteAddress>

   <CompleteAddressNumber><CompleteAddressNumber>20992099</CompleteAddressNumber> </CompleteAddressNumber>

- <CompleteStreetName> <StreetName>- <CompleteStreetName> <StreetName>UNIVERSITYUNIVERSITY</StreetName> </StreetName>

   <PostType><PostType>AVENUEAVENUE</PostType> <PostDirectional></PostType> <PostDirectional>WESTWEST</PostDirectional> </PostDirectional>

   </CompleteStreetName></CompleteStreetName>

   <PlaceName><PlaceName>SAINT PAULSAINT PAUL</PlaceName> </PlaceName>

   <PlaceName_USPS><PlaceName_USPS>SAINTSAINT PAULPAUL</</PlaceName_USPSPlaceName_USPS> >

   <ZipCode><ZipCode>5510455104</ZipCode> </ZipCode>

   <ZipPlus4><ZipPlus4>34313431</ZipPlus4> </SiteAddress> </Address></ZipPlus4> </SiteAddress> </Address>

- <gml:Point> <gml:pos>- <gml:Point> <gml:pos>-93.186111 44.959949-93.186111 44.959949</gml:pos> </gml:Point></gml:pos> </gml:Point>

   <GeocodeMatchCode <GeocodeMatchCode accuracyaccuracy="="1.0000001.000000" matchType="" matchType="PRECISEPRECISE" note="" /> " note="" />

- <source> <- <source> <dataSourcedataSource>>parcels_all7_points_lat83_pointparcels_all7_points_lat83_point</dataSource> </dataSource>

   <<addressIdentifieraddressIdentifier>>123-332923220025123-332923220025</addressIdentifier> </source></addressIdentifier> </source>

   </GeocodedAddress></GeocodedAddress> </GeocodeResponseList></GeocodeResponseList>   </GeocodeResponse> </GeocodeResponse>

Geocode by IntersectionGeocode by Intersection

Geocode by IntersectionGeocode by Intersection

Call Tracking applicationCall Tracking application

Geocode while customer is on the phoneGeocode while customer is on the phone

Assigns Service AreaAssigns Service Area

uses “st_within” PostGIS SQL function with uses “st_within” PostGIS SQL function with lat,long from geocoder service to populate lat,long from geocoder service to populate other fields from spatial tablesother fields from spatial tables

Field staff select their calls, mapField staff select their calls, map

Geocoding calls reduced time field staff spend locating call by 80%

North service area calls, 2008North service area calls, 2008

How well is Geocoder working?

Web Service Match, Score Data

• Inputs, parsingSeparate fields or one string

• Geocoder EngineStandardizersComparison

• SettingsWeightsRules

• Point or Ranges• Loc. Accuracy• Current• Parsing• Abbreviations • Free of spelling errors• “Strange but true”

PAGC

ST = StreetST = Saint

How well is Geocoder working?

Web Service Match, Score Data

• Inputs, parsingSeparate fields or one string

• Geocoder EngineStandardizersComparison

• SettingsWeightsRules

• Point or Ranges• Loc. Accuracy• Current• Parsing• Abbreviations • Free of spelling errors• “Strange but true”

PAGC

ST = StreetST = Saint

How well is Geocoder working?

Web Service Match, Score Data

• Inputs, parsingSeparate fields or one string

• Geocoder EngineStandardizersComparison

• SettingsWeightsRules

• Point or Ranges• Loc. Accuracy• Current• Parsing• Abbreviations • Free of spelling errors• “Strange but true”

PAGC

ST = StreetST = Saint

Points of Interest / Landmarks

Change match/scoring engine (PAGC)– International interest – some areas do not use

house numbers + street name addressing– Scores based on

Each (significant) word Soundex Approximate (whole name)

Web Service Match, Score Data

One service handles address, intersection, and named feature

Change web service inputs allowed– LandmarkName, LandmarkType, CountyName

Web Service Match, Score Data

Points of Interest / Landmarks

Try data file with x,y (not shp file) Data available?

Web Service Match, Score Data

Points of Interest / Landmarks

Data GNIS – Geographic Names, USGS NCompass / TLG Landmarks

– Part of MetroGIS streets package– Some points, some polygon centroids (water)

Metro. Council – Transit, from bus route requests– Other data?

911– Each Public Service Answering Point (PSAP) has their

own data HSIP + state creation/update CAP grant

– Hospitals, Fire Stations, Police, Schools Commercial data sources?

Data: examples

Data: examplesTLG

Data: examplesGNIS

Data: examplesTransit

Data: examples

Data

Data for testing: TLG Landmarks Could assemble a more complete dataset

– Eliminate duplicates– Check positional accuracy– Check spelling and consistency– Feature types?

Data maintenance??

Points of Interest / Landmark Example of raw output

<?xml version="1.0" encoding="UTF-8" ?> - <GeocodeService xmlns="http://www.metrogis.org/geocode" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.metrogis.org/geocode GeocodeService.xsd" xmlns:gml="http://www.opengis.net/gml">  <ResponseHeader /> - <Response Version="1.1" RequestID="">- <GeocodeResponse> - <GeocodeResponseList numberOfGeocodedAddresses="30"- <GeocodedAddress>- <Address>- <LandmarkSiteAddress>  <LandmarkName>EAST BETHEL CITY HALL</LandmarkName>   <FeatureType>D65A</FeatureType>   <CountyName>ANOKA</CountyName>   <PlaceName>EAST BETHEL</PlaceName>   <StateName>MINNESOTA</StateName> </LandmarkSiteAddress></Address>- <gml:Point> <gml:pos>45.371203 -93.214304</gml:pos> </gml:Point>  <GeocodeMatchCode accuracy="0.507497" matchType="LANDMARK" note="" /> - <source> <dataSource>clipped_to_metro2_lat_point</dataSource> <addressIdentifier />   </source>  </GeocodedAddress>- <GeocodedAddress>- <Address>- <LandmarkSiteAddress>  <LandmarkName>BETHEL CITY HALL</LandmarkName>   <FeatureType>D65A</FeatureType>   <CountyName>ANOKA</CountyName>   <PlaceName>BETHEL</PlaceName>   <StateName>MINNESOTA</StateName>   </LandmarkSiteAddress>  </Address>- <gml:Point>  <gml:pos>45.404547 -93.269909</gml:pos>   </gml:Point>  <GeocodeMatchCode accuracy="0.478886" matchType="LANDMARK" note="" /> - <source>  <dataSource>clipped_to_metro2_lat_point</dataSource>   <addressIdentifier />   </source>  </GeocodedAddress>- <GeocodedAddress>- <Address>- <LandmarkSiteAddress>  <LandmarkName>EAST BETHEL ICE ARENA</LandmarkName>   <FeatureType>D67</FeatureType>   <CountyName>ANOKA</CountyName>   <PlaceName>EAST BETHEL</PlaceName>   <StateName>MINNESOTA</StateName>   </LandmarkSiteAddress>  </Address>- <gml:Point>  <gml:pos>45.344766 -93.234857</gml:pos>   </gml:Point>  <GeocodeMatchCode accuracy="0.437920" matchType="LANDMARK" note="" /> - <source>  <dataSource>clipped_to_metro2_lat_point</dataSource>   <addressIdentifier />   </source>  </GeocodedAddress>

Geocoder Enhancements

Change candidate matches returned – alt. street names before alternate numbers

Fix corrected names returned with pre-types (no “County Road County Road B”)

Allow entry of House # + Street Name as continuous text string

MetroGIS Geocoder Project

Continuing Work– Address points dataset, with

automated data update– Adjusting performance– More Hosts? Batch service?– Open source – anyone can

contribute to code

Web Service – Contacts List ? Outreach ?