The Design of a Framework and Associated Tools to Facilitate the

Integration of Multi-sourced Built and Natural Datasets within

National SDI Initiatives

Submitted by:

Hossein Mohammadi

Supervisors:

Dr Abbas Rajabifard

Prof Ian Williamson

May 2006

PhD Confirmation

2 of 36Geomatics Department-Melbourne University

Acknowledgement

Uni Melb, Dept. Geomatics Research Partners of ARC-Linkage

project on Integration (DSE, DoL, GA)

WA and PSMA

3 of 36Geomatics Department-Melbourne University

Acknowledgement

Confirmation Panel Prof Ian Bishop Prof Ian Williamson Dr Abbas Rajabifard Ms Jude Wallace

4 of 36Geomatics Department-Melbourne University

Presentation Outline

Introduction Overview Problem Hypothesis Aim Objectives Background Integration Issues Integration and Interoperability Research Design Progress To-date Time Table Overall Thesis Structure

5 of 36Geomatics Department-Melbourne University

Introduction

6 of 36Geomatics Department-Melbourne University

Definition

Data Integration The integration is not only to geometrically and

topologically match data, and having the correspondence of attributes, but also the establishment of all legal, policy, institutional and social mechanisms in order to facilitate the integration of data.

7 of 36Geomatics Department-Melbourne University

Overview

Spatial Technology is growing dramatically

Citizens and governments are recognizing the value of “Spatially-enabled” Societies

Demands are growing among Broader Range of SI Users towards more value-added spatial products & services

8 of 36Geomatics Department-Melbourne University

Overview

Sustainable Development through its Social, Economical and Environmental objectives & Governance is not achievable unless there is the ability to monitor the Impacts of Human Activities on the Environment.

Sustainable Development

9 of 36Geomatics Department-Melbourne University

Overview

SDI development and progress to-date (Evolving)

SDIs aims to Facilitate Access, Sharing and Use of spatial data and services by a broad range of users (SI specialists, governments, citizens,…)

But

10 of 36Geomatics Department-Melbourne University

Problem

Inconsistencies, heterogeneities, interoperability, vertical integration, legal, institutional, policy and social issues hinder effective data integration of built and natural datasets.

Land-Use

Cadastre

Transportation

Topography

Satellite Image

Utility

GPS Network

DEM

Pilot Area

11 of 36Geomatics Department-Melbourne University

Problem

Emergency management

Land Administration

Navigation Services

Utility management

Urban Planning

Environmental Protection

…

12 of 36Geomatics Department-Melbourne University

Scenario 1

Nationally consistent data (PSMA’s Experience)

Inconsistencies in Data Model, Metadata, Standards, Specifications, Collaboration Model, and IP issues in Victoria, NSW, GA, …

• G-NAF• CadLite• Transport and Topography• Admin Boundaries• Points of Interest

IDM

13 of 36Geomatics Department-Melbourne University

Scenario 2

Hazard management, Tsunami (EICU’s Experience)

Utilities

Bathymetry

Buildings Data

Critical Infrastructures

Datasets:

Cadastre

Access Network

Underground Features

…

Gaps of Data, No Vertical Topology, Currency, Collaboration, Data Exchange Policy,…

Bathymetry Building Data

Cadastre Utilities

Created and Owned by Users

NO Nation-wide DataNO Metadata

Data Barely Exists

Not Completely Fits to Utility and Access Network

PrivacyPoor Attribution

Restrict PrivacyDifferent Custodians

14 of 36Geomatics Department-Melbourne University

Problem

Despite the growth in spatial technology and the development of SDIs to facilitate

the sharing, access and use of spatial datasets, the integration of multi-sourced

datasets of the built and natural environments, is not being achieved

effectively from both technical and non-technical perspectives.

Despite the growth in spatial technology and the development of SDIs to facilitate

the sharing, access and use of spatial datasets, the integration of multi-sourced

datasets of the built and natural environments, is not being achieved

effectively from both technical and non-technical perspectives.

15 of 36Geomatics Department-Melbourne University

Hypothesis

The development of a framework and an improved data model focussing on both technical and non-technical aspects of data integration within the context of a National SDI will facilitate more effective integration of built and natural environmental datasets.

The development of a framework and an improved data model focussing on both technical and non-technical aspects of data integration within the context of a National SDI will facilitate more effective integration of built and natural environmental datasets.

16 of 36Geomatics Department-Melbourne University

Aim

This research aims to develop a framework, and integration data model together with associated

tools to facilitate the integration of built and natural environmental datasets (in particular

cadastral and topographical data) in the context of National SDI initiatives.

This research aims to develop a framework, and integration data model together with associated

tools to facilitate the integration of built and natural environmental datasets (in particular

cadastral and topographical data) in the context of National SDI initiatives.

17 of 36Geomatics Department-Melbourne University

Objectives

1. Investigate and understand the problems and issues in integrating multi-sourced datasets in National SDIs within Australia and related International case study countries.

2. Identify barriers and criteria for data integration 3. Develop a framework and guidelines for the integration of

built and natural environmental datasets.4. Develop an integration data model and associated tools

(eg web-service) for the integration of built and natural environmental datasets.

5. Test the framework, data model and tools through the development of a prototype.

1. Investigate and understand the problems and issues in integrating multi-sourced datasets in National SDIs within Australia and related International case study countries.

2. Identify barriers and criteria for data integration 3. Develop a framework and guidelines for the integration of

built and natural environmental datasets.4. Develop an integration data model and associated tools

(eg web-service) for the integration of built and natural environmental datasets.

5. Test the framework, data model and tools through the development of a prototype.

18 of 36Geomatics Department-Melbourne University

Background Overview

Built and Natural Datasets

Built Natural

CadastreUtilitiesPoints of InterestsTransportAddressesPlanningGeodetic Network…

TopographyWater Bodies/HydrographyVegetationBiodiversity (Flora and Fauna)EnvironmentalImagery..

19 of 36Geomatics Department-Melbourne University

Background Overview

Spatial data integration issues Technical Legal Policy Institutional Social

20 of 36Geomatics Department-Melbourne University

Integration Issues

Social Issues Cultural Issues Historical Issues Capacity Building and Raising Awareness

21 of 36Geomatics Department-Melbourne University

Integration Issues

Institutional Issues Collaboration Funding Model Business Model Awareness of Data Integration and Proper

Data

22 of 36Geomatics Department-Melbourne University

Integration Issues

Policy Issues Major Policy Drivers (eg Sustainable

Development, Counter Terrorism, Environment)

Legislation Pricing

23 of 36Geomatics Department-Melbourne University

Integration Issues

Legal Issues Rights, Restrictions and Responsibilities on

Data Intellectual Property Rights (IP/Copyright) Licensing Access and Privacy

Integrated Data{RRR(ID), IP(ID), …} = Data1{RRR(1), IP(1), …} + Data2{RRR(2), IP(2), …} + … + Data n{RRR(n), IP(n), …}

Data 1 Data 2 Data n

Integrated Data

24 of 36Geomatics Department-Melbourne University

Integration Issues

Technical Issues Data Model Interoperability Standards and Specifications Vertical Topology (Across Layers) Coordinate System Format Metadata Data Quality (Currency, Logical Consistency, Accuracy, …)

Geometry

25 of 36Geomatics Department-Melbourne University

Technical Issues

Data Model (Feature Type as a Class or as an Attribute: Roads)

NSW’s Data ModelHarmonized Data Model (ICSM)

Integrated Data Model (PSMA)

26 of 36Geomatics Department-Melbourne University

Technical Issues

There are at least seven sources of road centre line in Victoria with the different road alignments and mis-matched road names.

27 of 36Geomatics Department-Melbourne University

Technical Issues

Feature Level and Theme Level Metadata

NSW: Feature LevelVictoria, GA: Theme Level

(ANZLIC Guidelines)

28 of 36Geomatics Department-Melbourne University

Da

ta In

teg

rati

on

Integration and Interoperability

Interoperability is the ability of a system or a product to work with other systems or products without special effort

on the part of the customer Effective Data Integration Needs Interoperability in

Different Dimensions

Policy Interoperability

Institutional Interoperability

Social Interoperability

Legal Interoperability

Technical Interoperability

29 of 36Geomatics Department-Melbourne University

Research Design

Step 2

Step 3

Step 4

Step 5

Step 1 Literature Review and Strategy Development

Case Studies Investigation

Case Study, Tools and User Needs Analysis

Framework/Tools Development and Validation

Documentation of Results

30 of 36Geomatics Department-Melbourne University

Research Design

Step 2

Step 3

Step 4

Step 5

Step 1 Literature Review and Strategy Development

Case Studies Investigation

Case Study, Tools and User Needs Analysis

Framework/Tools Development and Validation

Documentation of Results

NationalInternational

SDI and its Components, SDI Hierarchy, Organizational Behaviour (Related to Data Integration), Built and Natural Environmental Datasets, Applications, Characteristics, Technical and Non-technical Issues, Spatial data Integration and Interoperability at different levels

SDI and its Components, SDI Hierarchy, Organizational Behaviour (Related to Data Integration), Built and Natural Environmental Datasets, Applications, Characteristics, Technical and Non-technical Issues, Spatial data Integration and Interoperability at different levels

Australian Case study State and national organization visits (Vic, NSW, GA, PSMA) Short-Term projects

Technical assessment of built and natural datasets integration using Victoria and GA datasets

Technical assessment of built and natural datasets integration of NSW and GA datasets

Geo-webservice requirements for data integration (in progress)

Ongoing

ARC-Linkage/ PCGI-AP

31 of 36Geomatics Department-Melbourne University

Progress To-date

Some Early Findings of Case Studies Different agencies are at different levels of maturity

in database management and collaborations Cadastral data model is time-variant in contrary to

topographic data model Different data models are used by different

jurisdictions Content of metadata is not adequate for proper

integration & Feature level metadata is more efficient rather theme-based metadata

32 of 36Geomatics Department-Melbourne University

Progress To-date

Some Early Findings of Case Studies Diversity and inconsistencies in specifications,

feature categorizing, standards and topology, format, precision, currency, coordinate system, IP, pricing and privacy, etc.

Social characteristics do play a key role in sharing and integrating data

33 of 36Geomatics Department-Melbourne University

Progress To-date

Publications and Presentations Mohammadi H, Rajabifard A., Binns A., Williamson, I.P. (2006),

“Bridging SDI Design Gaps with Facilitating Multi-source Data Integration”, Coordinates, May 2006

Mohammadi H, Rajabifard A., Binns A., Williamson, I.P., National SDI Framework to Facilitate Multi-source Data Integration, (in progress) for ISI journal

Presentation on “ The Integration of Built and Natural Environmental Datasets Within NSDI”, Workshop on the Launch of Project, September 2005, Melbourne

Presentation on “ The Integration of Built and Natural Environmental Datasets Within NSDI”, Department of Sustainability and Environment (DSE), October 2005, Melbourne

Presentation on “ The Integration of Built and Natural Environmental Datasets Within NSDI”, Land Admin Lecture Series, March 2006

Conferences and Seminars Attended Meeting International Experts and

Research Discussion

34 of 36Geomatics Department-Melbourne University

Time Table

Activity2005 2006 2007 2008

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

Literature Review

Case Study Visits and Analysis

Confirmation report

Data Model Analysis

Geo-WebService Analysis

Data Model Development

Integration Geo-WebService Development

Framework Development

Thesis Writing

Thesis Submission

International Workshop

35 of 36Geomatics Department-Melbourne University

Overall Thesis Structure

Chapter 1 Introduction Statement of Research Problems, Aims, Hypothesis and Objectives

Chapter 2 Spatial Data Infrastructures and Related Concepts and Theory

SDI and its Components, SDI Hierarchy, Organizational Behaviour (Related to Data Integration)

Chapter 3 Built and Natural Environmental Data Production, Characteristics and Applications

Built and Natural Environmental Datasets, Applications, Characteristics, Technical and Non-technical Issues and Arrangements

Chapter 4 Spatial Data Integration Spatial data Integration and Interoperability at different levels

Chapter 5 Methodology and Case Studies Design Methodology Development for National and International Case Studies, Methodology for Analysis and Developments

Chapter 6 Case Study Projects and Analysis Analysis of the Observations, Current Situation, Issues and Problems

Chapter 7 Framework, Design Criteria and Integration Data Model Development

Overall Spatial Integration Framework, Integration Data Model, Integration Geo-webservice Design

Chapter 8 Prototype Development and Testing Development and Testing Framework, Integration Data Model and Geo-webservice

Chapter 9 Conclusion and Future Directions Discussion, Recommendation, Conclusion

36 of 36Geomatics Department-Melbourne University

Thanks for your attention

37 of 36Geomatics Department-Melbourne University

Outcomes

An Overall Framework to Address Issues and Problems

Integration Data Model

Integration Data Model

38 of 36Geomatics Department-Melbourne University

Outcomes

Integration Geo-webservice Federated Databases (Virtual) Physical Integration

Vertical Integration = Cross-thematic topological integrity

Horizontal Integration = Same-theme

Federated Database

DCDB

DTDB

External Data

Interoperability, O

GC Standards

39 of 36Geomatics Department-Melbourne University

Overview

Cadastral and Topographic Data Characteristics

Cadastral Topographic

Scale Large - Medium Medium - Small

Custodian State/Local Governments National Mapping Agencies to Councils Different Levels within Jurisdictions

Data Model Time-variant Not Time-variant Not Child-parent Hierarchy

Coordinates Distance and Angle of Coordinates Easting, Northing

Legal Restrict Privacy and IP IssuesFree in Small Scales and IP Rights for Large Scales

Major Use Legal, Fiscal Planning, Military, Development

Currency Short Update Period Long Update Period

40 of 36Geomatics Department-Melbourne University

Technical Issues

Discrepancy of Vector and Raster Data

GA: Boundary

Victoria: Image

41 of 36Geomatics Department-Melbourne University

Technical Issues

Different Specifications and Standards: Railway, Tank Point and Cableway have been Categorized in Topography category in NSW, No Railway Category in NSW

Data Victoria NSW GA

File NameNo particular category for Tank points

(HY-WATER_STRUCT_POINT)Beside watering places and swimming pools

TankPoint Point_watertank

Category Hydro TopoFD Drainage

42 of 36Geomatics Department-Melbourne University

Overview

Completeness

43 of 36Geomatics Department-Melbourne University

Overview

Quantitative Benchmarking Through Indicators Duplication of Effort Heterogeneity among Datasets Number of Disparate Non-interoperable

Databases

44 of 36Geomatics Department-Melbourne University

Research Design

Step 2

Step 3

Step 4

Step 5

Step 1 SDI, Integration and Interoperability

Technical assessment of

Case Study, Tools and User Needs Analysis

Framework/Tools Development and Validation

Documentation of Results

45 of 36Geomatics Department-Melbourne University

Progress To-date

Conferences and Seminars Attended Spatial Interoperability Demonstration Project (SIDP),

June 2005, Melbourne ARC-Linkage Launch Project Workshop, September

2005, Melbourne Simulating Urban Neighbourhood dynamic, August

2005, Melbourne Spatial Information in Victoria, June 2005, Melbourne SSC (Spatial Sciences Institute) Conference,

September 2005 , Melbourne Expert Group Meeting on Sustainability and Land

Administration Systems, November 2005, Melbourne

46 of 36Geomatics Department-Melbourne University

Progress To-date

Meeting International Experts and Research Discussion

Mr. Mathew Kistler, Federal Office of Topography, Switzerland, 2005

Prof Michael Goodchild, University of California, USA, 2005 Prof Holger Magel, Technical University Munich, Germany ,

2005 Prof Paul Van der Molen, ICT, The Netherlands , 2005 Dr. Daniel Steudler, Federal Office of Topography, Switzerland ,

2005 Prof Bas Kok, Technical University of Delft, The Netherlands ,

2005 Prof Gerrit Pienaar, South Africa , 2005 Prof Ian Masser, United Kingdom , 2006

47 of 36Geomatics Department-Melbourne University

Progress To-date

Some Early Findings of Case Studies Different agencies are at different levels of maturity

in database management and collaborations Cadastral data model is time-variant in contrary to

topographic data model Different data models are used by different

jurisdictions Content of metadata is not adequate for proper

integration & Feature level metadata is more efficient rather theme-based metadata

48 of 36Geomatics Department-Melbourne University

Progress To-date

Some Early Findings of Case Studies Diversity and inconsistencies in specifications,

feature categorizing, standards and topology, format, precision, currency, coordinate system, IP, pricing and privacy, etc.

Social characteristics do play a key role in sharing and integrating data

Virtual data repository is a more proper solution for integration of different systems

49 of 36Geomatics Department-Melbourne University

Background Overview

SDI is an initiative to facilitate data sharing and access through the creation of an environment in which all stakeholders can co-operate with each other and interact with technology to better achieve their objectives.

Rajabifard et al (2001)