Interoperable Information System of Systems

for

HTAP

Rudolf B. Husar and Rich Scheffe

With Erin Robinson

Presented at

HTAP Workshop, WMO, Geneva, January 26, 2006

Intercontinental Transport Events have

Influenced AQ Regulations ..and now Policy? –> Air Quality ManagementWeb-Based Collab. in Virtual Workgroups –> Science, the ProcessNear-Real-Time Data Access & Integration –>Technologies and Tools

US-Daily Avg. PM25 Dust: Seasonal + Events

HTAP Characteristics

• Stakeholders are autonomous countries, organizations• Diverse perspectives on the issues, solutions• Pollution sources and impacts are hemispheric• Information integration is key to success• Observations and models are ready for integration

The HTAP - GEOSS• Stakeholders are autonomous countries, organizations• Diverse perspectives on the issues, solutions• Recognition of the need to understand, act • Pollution sources and impacts are hemispheric• Observations and models are ready for integration• Information integration is key to success

Hence

GEOSS is a good system model for HTAPHTAP is a suitable use case for GEOSS

Some Characteristics of System of Systems (SoS)

• Autonomous constituents managed independently• Independent evolution of each constituent• Displays emergent (novel, unpredictable) behavior

Summary

There is an ‘alignment of stars’ for integrating heterogeneous HTAP data

• System of Systems architecture is suitable for integrating HTAP data– Standard data access is a key interoperability protocol– Heterogeneous data can be non-intrusively standardized by mediators– Service-based software architecture delivers tailored products to diverse uses

• Federated data and shared web-based tools are in use– DataFed already includes over 100 datasets (emissions, ground, satellite) – The system has been applied to EPA regulatory development

• However, – DataFed is just one of the many mediator nodes, but these need to be connected– Much more data would need to be federated – HTAP model-data comparison would be an attractive use case

WMO-GAW could be an effective neutral facilitator for global data integration using GEOSS (system of systems) principles

GEOSS Architecture Framework - DSS for HTAP

ObservationsHTAP Knowledge into the Minds of

Policy Analysts

HTAP Reports:Model Estimates,

Obs. Evidence

Models

Knowledge into the Minds of

AQ Analysts

Decisions

HTAP Knowledge into the Minds of

Policy Makers

Decision Support System

GEOSS Architecture Framework

System of Systems: Interoperability Connecting Machines and People

IP – Internet Protocol

Open Architecture Data Standards Service Based

Amplify Individuals Connect Minds

Interoperability Stack: Key concept of the Web System components have to be interoperable at each layer

How do you agree on Interoperability?

What few things must be the same so that everything else can be different?

WCS/WMS

App

App

App

App

Are WCS/WMS Space-Time-Parameter queries

Convergence Protocols

GetCapabilities

GetData

Capabilities, ‘Profile’

Data

Where? When? What? Which Format?

Server

Back End S

td.

Inte

rface

Client

Front EndS

td.

Inte

rface

Query GetData Standards

Where? BBOX OGC, ISO

When? Time OGC, ISO

What? Temperature CF

Format netCDF, HDF.. CF, EOS, OGC

T2T1

Lets agree onSpace-Time-Parameter

Data Access Query Protocol

ScientistScience

DAACs

• Current info systems are project/program oriented and provide end-to-end solutions

Info UsersData Providers Info System

AIRNowPublicAIRNow

ModelCompliance

Manager

From ‘Stovepipe’ to Federated Architectures

• Part of the data resources of any project can be shared for re-use through DataFed

• Through the Federation, the data are homogenized into multi-dimensional cubes

• Data processing and rendering can then be performed through web services

• Each project/program can be augmented by Federation data and services

Web Services and Workflow for Loose Coupling

Integrated DataDatasets

Std

. In

terf

ace

Data ViewsS

td.

Inte

rface

Data

Control

Reports

Reports

Obs. & Models Decision Support System

Service Broker

Service Provider

PublishFind

BindServiceUser

Web Service Interaction Service Chaining & Workflow

Collaborative Reporting and Dynamic Delivery

Integrated DataDatasets

Std

. In

terf

ace

Data ViewsS

td.

Inte

rface

Data

Control

Reports

Reports

Obs. & Models Decision Support System

Co Writing - Wiki

Screencast

Analysis Reports:

Information supplied by manyNeeds continuous program feedbackReport needs many authorsWiki technologies are for collaborative writing

Dynamic Delivery:

Much of the content is dynamicAnimated presentations are compellingMovies and screencasts are for dynamic delivery

Model-Data Comparison Workflow Software

Workflow Flow Program Lego-like assembly of component

WMS, WCS OGC Services

AeroComChemical Models

Paris, FR

VIEWSChemical DataFt. Collins, CO Model-

Data ComparisonWorkflow

Std

I/

OS

td

I/O

Sample of Federated Datasets

• HTAP Model-Obs “Test/Benchmark”ed– A virtual data center design to meet the needs of the TF HTAP

model intercomparison and evaluation– Archives of key observations and model data – Tools for model intercomaprison

Tools..Analyst Console

User Layout

Imagine…More Shared Global Obs & Models

2007++More Global Data & Models

2007Global Data & Models

The Network Effect:Less Cost, More Benefits through Data Reuse

ProgramPublic

Data Organization

Data

Data Program

Program

OrganizationData

Data

ProgramData

Orgs Develop ProgramsPrograms ask/get Data Public sets up Orgs

Pay only once Richer content

Less Prog. Cost More Knowledge

Less Soc. Cost More Soc. Benefit

Data Re-Use Network Effect

VIEWS

AeroCom

Federated Information System

Observations, Models, Services

CAPITA

Other Federations

Obs

ModelsObs &Services Services

Obs & Model

Models & Services

Agile Information System Design Principles

Based on P. Senge et. al, 1994: Architecture of Learning Organizations(Link)

Guiding Idea: System of Systems

GEOSS

Methods, Tools: Standards, Use Cases

Infrastructure: Internet, Middleware,

Web 2.0

Domain of Interaction

WMO/GAW & Stakeholders

Apparent divergences?Organizations have different missions

NPSProtect ecosystems, AQ WQ

NASAExplore fundamental Earth System Properties

USDAProtect/optimize Ag and forest resources

EPA (Protect human health & envi.

Improve air, water, ecosystem

CDCTo promote health and quality of life ….

NOAATo understand and predict changes in the Earth’s environment and …

R. Scheffe

Collaboration (culture) empire building?

Observation technologies {e.g., satellites}

Computational power

Science, talent{embodied in AQ models and young geniuses}

Budgets, agency collaboration resource/program accountability

Accountability, ↓ regulatoryassessments {e.g., NAS, CASAC}

Information technologies{e.g., data sharing protocols}

Alignment

Stars aligned?Confluence again..

R. Scheffe

Summary

There is an ‘alignment of stars’ for integrating heterogeneous HTAP data

• System of Systems architecture is suitable for integrating HTAP data– Standard data access is a key interoperability protocol– Heterogeneous data can be non-intrusively standardized by mediators– Service-based software architecture delivers tailored products to diverse uses

• Federated data and shared web-based tools are in use– DataFed already includes over 100 datasets (emissions, ground, satellite) – The system has been applied to EPA regulatory development

• However, – DataFed is just one of the many mediator nodes, but these need to be connected– Much more data would need to be federated – HTAP model-data comparison would be an attractive use case

WMO-GAW could be an effective neutral facilitator for global data integration using GEOSS (system of systems) principles