CHALLENGES OF UTILITY DATA REPOSITORIESPhil Meis, P.E. Utility Mapping Services, Inc.

Jim Anspach, P.G. J.H. Anspach Consulting / So-Deep, Inc.

Maintenance,Safety, Damage Prevention, Emergency Response

Permitting

Data Integration with DOT Enterprise GIS

Planning/Coordination

SUE Utility Data Acquisition

Construction As-BuiltsTied to NGS HARN

Utility GIS

Design

Utility Data Repositories are Changing

• 1980s and before– Used to be collection of paper maps maintained

and controlled by individual utility owners

• 1990s– Digital CADD files and paper maps

• Now– All of the above, plus GIS

– Who is in control of data?

– Who keeps data current?

CADD & GIS are Merging

• CADD is Graphical

• GIS is Database + Graphics

• GIS has much more available data, and data can be organized and searched

• Both CADD & GIS are going 3-D

• GIS will serve many more masters than CAD, and therein lies the challenge

How Close is GIS to Taking over the Utility Engineering Field?

• FAA already requiring all airports to do their layout plans for design in GIS rather than CAD

• SHRPII R-01A Research project is about developing standards and protocols for GPS/GIS to build Utility Data Repositories for state DOTs

A Geographic Information System (GIS) assembles information from several sources, including ground surveys, existing maps, aerial photos, and satellite imagery. In a GIS, specific information about a place, such as the locations of utility lines, roads, streams, and buildings, is layered over a set of geodetic data.

Because a GIS stores data digitally, information can be quickly and economically updated, easily reproduced, and made widely available. In fact, because of its power and speed, GIS technology is doing most of the cartographic (mapmaking) work that, in the past, was laboriously done by hand on paper charts and maps.

Utility Management System Architecture

Geographic Information Systems

• Challenges– system architecture for creating robust legacy

data inventory and data integration• GIS, Geodesy, National Standards, State Statutes

– management buy-in and funding– initial data collection – data maintenance

Everybody needs to understand lingo and basics.

Good idea to run everyone through basics.

Need multi-discipline team to manage initiative – don’t just turn it over to GIS nerds.

Leader needs to have multi-discipline background to understand and guide effort.

GIS, PLS, and engineering communities have in general not yet fully meshed. Throw on top of that, all of them lack an understanding of utilities.

Managing three data types:I. attribute (what it is)II. coordinate (where it is)III. metadata (data about

the data)

Need to be able to access, update, query, and retrieve data readily, and push it back into variety of CADD platforms, tied and projected to desired project coordinate system.

TransportationInstitute

Texas

What defines robust, legacy data?• Need essential attribute data fields

– e.g., size, capacity, material, ownership, conduit, quality levels, coordinates

• Need essential meta-data– e.g., data source, acquisition campaign,

survey control and projection, engineer-of-record

• Need coordinate data tied to national datum

• Comply with Federal and international data standards (FGDC and ISO)

UtilityCompanyUsers

PK UserProfID

EventDateFirstNameLastNameTitle

FK2,I4,I3 UtilCoIDI5 Division

AddressCity

FK1,I2 StateI6 ZipCode

PhoneNoFaxNo

I1 EmailUserNamePasswordConfirmPWNotes

TxDOTUsers

PK UserProfID

EventDateFirstNameLastNameTitleRInitRevRFieldVerRApprovalRABuiltRevRGISDocRArchivalDistrictIDAreaOfIDMaintOfIDPhoneNoFaxNoEmailUserNamePasswordConfirmPWNotes

PermitReviewAttachments

PK,FK1,I2,I1 PermitApNoPK AttachName

AttachmentUserProfID

I3 AtSubmDate

AreaOffices

PK AreaOfID

I1 AreaOfNameAreaEgName

FK1,I3,I2 DistrictIDPhStreetPhCityPhStatePhZipMaStreetMaCityMaStateMaZipPhone1FaxComment

ActionTypes

PK ActionType

FacilityActionConfigurations

PK FacActConf

CommentExamples

PermitApplAttachTypes

PK AttachType

CommentOrder1

Permits

PK,I2 PermitApNo

U1 PermitIDFK3,I10,I9 UserProfID

AccessTypePrefix

I3 RoadNoI4 Direction

TLMSNoBegTLMSDisEndTLMSDisLatitudeLongitudeDescription

FK1,I6 StatusI1 SubmDate

SubmTimeI7 InRevDateI5 FldVerDate

DecDateNotifDateComplDateFinDateDocumDateArchivDateExpirDate

FK2,I8 InRevUserFldVerUserDecUserFinUserDocumUserArchivUserFldVerNeedPropBgDateActBgDatePropFnDateActFnDateTxDOTRep

FK4 DistrictIDCountyIDAreaOfIDMaintOfIDComments

MaintenanceOffices

PK,I2 MaintOfID

MaintOfNamSupervisor

FK1,I1 DistrictIDPhStreetPhCityPhStatePhZipMaStreetMaCityMaStateMaZipPhone1FaxComment

PermitLog

PK,FK1,I1,I2 PermitApNoPK LogID

LogDateFK2,I4,I3 StatusID

MsgFromMsgToMsgCcTitleDescription

PermitApplDetails

PK,FK3,I6,I4 PermitApNoPK,I3 ActionID

FK1,I1 ActionTypeI5 PermitIDI7 UtilClass

UtilSubClsFeatClassFeatureLocationMaterialCapacityCapUnitsCasingCasingSizeCasSzUnitsMnDpthHghtMxDpthHght

FK2,I2 FacActConfCdFileUserCdFileServSharedUCapComment

PermitRevegetationProvisions

PK,FK2,I3,I1 PermitApNoPK,FK1,I2 ProvID

Comment

Districts

PK,I1 DistrictID

DistrNameDistrictNoEngineerPhStreetPhCityPhState

I3 PhZipMaStreetMaCityMaState

I2 MaZipPhone1FaxComment

PermitStatus

PK,I3 StatusID

I1 StatusFK1,I2 RespID

Order1Comment

PermitRevegetationProvisionTypes

PK,I2 ProvID

DistrictIDI1 FormID

RevDateProvTitleStatusDocumentFileNameUserIDComment

PermitApplAttachments

PK,FK2,I3,I2 PermitApNoPK AttachID

FK1,I1 AttachTypeAttachmentAttachNameAttachUsNm

PostalAbbreviations

PK,I1 AbbrevID

NameAACRAbbrev

PermitGeneralProvisions

PK,FK2,I3,I1 PermitApNoPK,FK1,I2 ProvID

Comment

UtilityCompanies

PK,I1 UtilCoID

I2 UtilCoNameAcronym

Responsibilities

PK,I1 RespID

I2 ResponComment

PermitGeneralProvisionTypes

PK ProvID

DistrictIDI1 FormID

RevDateProvTitleStatusDocumentFileNameUserIDComment

PermitApplAttachmentsAsBuilt

PK,FK1,I2,I1 PermitApNoPK AttachID

FK2 AttachTypeAttachmentAttachNameAttachUsNm

PermitApplDetailsAsBuilt

PK,FK1,I4,I2 PermitApNoPK,I1 ActionID

FK3 ActionTypeI3 PermitIDI5 UtilClass

UtilSubClsFeatClassFeatureLocationMaterialCapacityCapUnitsCasingCasingSizeCasSzUnitsMnDpthHghtMxDpthHght

FK2 FacActConfCdFileUserCdFileServSharedUCapComment

FKPK

FK = Foreign keyPK = Primary key

I = Index

ClientWeb browser

(Internet Explorer, Netscape)

Web server(Internet Information Server -IIS)

File upload(AspUpload)

Application serverconnector(ServletExec)

Server

Map server(ArcIMS)

Application server

Spatial server

Map service

Spatial data(ArcView format)

Database(Access)

Uploaded files(Server hard drive)

Email transfer protocol(SMTP)

Email client

Databaseconnector

(ODBC)

Virtualfolders

File systemobject(FSO)

Positional accuracy is paramount for utilities.

Ensure system development includes individuals who understand the science of geodesy. Otherwise data usage will be limited to inventory type applications.

Geodesy - the science of measuring and monitoring the size and shape of the Earth and the location of points on its surface. NOAA's National Geodetic Survey (NGS) is responsible for the development and maintenance of a national geodetic data system that is used for navigation, communication systems, and mapping and charting.

Positional data and accuracy is the foundation for a GIS based utility data management system.

National Spatial Reference System

Datum to securely support other spatial information

NSRS Horizontal Datum

In 1927 the U.S. Coast and Geodetic Survey, the predecessor of the National Geodetic Survey, "connected" all of the existing horizontal monuments together and created the North American Datum of 1927 (NAD 27). This datum was used extensively during the next 60 years as the primary reference for horizontal positioning. In 1983, NAD 27 was adjusted to remove inaccuracies and to correct distortions. The new datum, called NAD 83, is the most commonly used horizontal positioning datum today in the United States.

NSRS Vertical Datum

In 1929, the National Geodetic Survey (NGS) compiled all of the existing vertical benchmarks and created the National Geodetic Vertical Datum of 1929 (NGVD 29). Since then, movements of the Earth's crust have changed the elevations of many benchmarks. In 1988, NGVD 29 was adjusted to remove inaccuracies and to correct distortions. The new datum, called the North American Vertical Datum of 1988 (NAVD 88), is the most commonly used vertical datum in the United States today.

Positional data and accuracy is the foundation for a GIS based utility data management system.

Taking it to Next Level - CORS and GIS

The Continuously Operating Reference Stations (CORS) system is a network that continually corrects GPS signals, and provides these corrections to GPS users over the Internet.

Relatively simple to tie all observations to the NSRS

National Committees and Centers• Federal Geographic Data Committee - representatives from Dept.’s of Ag, Commerce, Defense, Energy, Housing and Urban Development, Interior, State, Transportation, EPA, FEMA, Library of Congress, NASA, National Archives, and Tennessee Valley Authority.

• CADD/GIS Technology Center for Facilities, Infrastructure, and Environment, located at the U.S. Army Engineer Research and Development Center’s Information Technology Laboratory in Vicksburg, MS - a consortium for joint development of CADD and GIS standards by the Army Corps of Engineers, Army, Air Force, Navy, Marines, and other federal, state, local government agencies.

Spatial Data Standard for Facilities, Infrastructures, and Environment (SDSFIE)

CADD/GIS Technology Center, Information Technology Laboratory (ITL), U.S. Army Engineer Waterways Experiment Station (WES) Vicksburg, MS, a complex of five laboratories of the U. S. Army Engineer Research and Development Center (ERDC). The CADD/GIS Technology Center was chartered in 1992 to promote the use of CADD and GIS technologies for life-cycle facilities management within the Army, U.S. Army Corps of Engineers, Navy, and Air Force.

ESRI ArcGIS Survey Analyst (Oct ‘02) developed under contract with US Army Corps of Engineers as a result.

National GIS Data Standards

Need business processCapture data from permitting, construction, and

SUE activities

Need digital data standards, special provisions, and templates for data submittals

Need repository

Be mindful of state statutes

regulating publication of mapped fixed

features of engineering.

“Model Law” for management of survey data in a

GIS

Utility Management System Buy-In

• Need management to support need.• Need cost / benefit analysis and business process.• Need funding.

– Consider banding with regional metropolitan entities, cities, counties, transit authorities, and other state agencies with identical needs - support a single system with pooled permit fees.

• Need to educate continuously on why this is not a trivial exercise.

• May need legislative changes and utility lobby support.

Immediate GIS Database Benefits for Big Projects

Flag Utility Conflicts

Rapid Query of Database for Conflicts

Cost Sensitivity Analysis

Value Engineering (mitigate conflict cost)

Recommend Resolutions

Exacting Summary Reports for Utility Coordinator

Track Conflict Resolutions

Monitor Critical Path

Utility Management System Buy-In

Port to handheld

PC for Field Staff

Extended Advantages of a Utility GIS Inventory

Utility Management System Buy-In

Retain Investment of SUE InvestigationsROW Management Tool

Facilitate PermittingShift SUE Data Burden to Utilities Easement CommodityMaintenance UsePlanningSafetyMove SUE Effort Up to Planning or EA / EIS StageGASB 34 ComplianceFHWA Office of Asset Management Data Integration

FHWA Office of Asset Management - 1999

Pavements Bridges

ROW

RoadwayHardware

Utilities

Materials

Vehicles

Data

Utility Management System Buy-In

Asset Management Goal

PavementsBridges

ROW

RoadwayHardware

Utilities

Materials

Vehicles

Data

Data Integration

Utility Management System Buy-In

Don’t Miss the Boat!

Utilize Existing/New GeoSpatial Data Standards Generate Legacy Data Sets Share O&M Costs with Other Agencies Utilize & Share Data with National/State/Local Sources Harness the Digital World and Enable Planners,

Designers, and Maintenance to Perform Fantastic GeoSpatial Data Analyses and Use Emerging Technologies (e.g., BIM, 3D Modeling, etc.)

Utility Management System – NOW!

“Captain, sensors indicate an oil pipeline ofprobable Romulan origin is in conflict with our new storm drainage system. However, I’ve instructed the computer to develop a new optimal alignment that allows the pipeline to remain without threat.”

The New Utility ManagementSystem

Your time is very much appreciated.

Utility Mapping ServicesPhil Meis, P.E.Office: 406.933.5300Cell: 801.209.2032email: pjmeis@umsi.us