Web Services: ES Rationale and Assertions

Provider Push Science Pull

Flow of DataFlow of Control

DATA 1

Data 2

Data 2

Knowledge Prod. 1

Knowledge Prod. 2

Knowledge Prod. 4

Knowledge Prod. 3

Knowledge Prod. 5

Web Services for Refining Data to Knowledge

Prepared for:

Technology Infusion Web Services Sub-groupMarch 17, 2004 Telecon

R. Husar, rhusar@me.wustl.edu

[Better Earth-Sun] Science is the DRIVER!

The Researcher’s Challenge

“The researcher cannot get access to the data;if he can, he cannot read them;if he can read them, he does not know how good they are;and if he finds them good he cannot merge them with other data.”

Information Technology and the Conduct of Research: The Users ViewNational Academy Press, 1989

These resistances can be overcome through

• A catalog of distributed data resources for easy data ‘discovery’

• Uniform data coding and formatting for easy access, transfer and merging

• Rich and flexible metadata structure to encode the knowledge about data

• Powerful shared tools to access, merge and analyze the data

OGC & OpenDAP are addressing the Finding and Reformatting tasks

The custom processing of data into knowledge is still a major burden

Petabytes 1015 Terabytes 1012 Gigabytes 109 Megabytes 106

Calibration, Transformation To Characterized

Geophysical Parameters

Filtering, Aggregation, Fusion, Modeling,

Trends, Forecasting

InteractiveDissemination

ACCESS

Multi-platform/parameter, high space/time resolution,

remote & in-situ sensing

Sensing Analysis & Synthesis

Earth Science Data to Knowledge Transformation:Value-Adding Processes

Data Acquisition Value Chain (Network)

InfoSystem Goal: Add as much value to the data as possible to benefit all users

Data Usage Value Network

Flexible data selection, and processing to to deliver right knowledge, right place right time

Data - L1 Information – L2 Knowledge – L3-6? Usable Knowledge

Query

Data

Distributed, DynamicMore Local, DAAC

Processing Knowledge Use

Value-Added Processing in Service Oriented Architecture

Control

Data

Chain 1

Chain 2 Chain 3

Peer-to-peer network representation

Data ServiceCatalog

User

Data, services and users are distributed throughout the network

Users compose data processing chains form reusable services

Intermediate data are also exposed for possible further use

Chains can be linked to form compound value-adding processes

Service chain representation

User Tasks:

Find data and services

Compose service chains

Expose output

Chain 2

Chain 1 Chain 3

Data

Service

User Carries less Burden

In service-oriented peer-to peer architecture, the user is aided by software ‘agents’

Data Flow and Flow Control in AQ Management

Provider Push User Pull

Data are supplied by the provider and exposed on the ‘smorgasbord’

However, the choice of what data and processes to be is made by the userThus, the info system consists of autonomous data users, providers, mediators and their linkages

Flow of DataFlow of Control

AQ DATA

METEOROLOGY

EMISSIONS DATA

Informing Public

AQ Compliance

Status and Trends

Network Assess.

Tracking Progress

Data to Knowledge ‘Refinery’

Data Flow and Flow Control in AQ Management

Relationship between different information activities

States

Regions

AIRS AQSEPA Air Portal

EPA Science Portal

VIEWS

AIRNOW

Data Flow and Flow Control in AQ Management

The data ‘refining’ process is not a chain but network connection processing nodesThe value is created by the dynamic exchanges between individuals, groups or organizations

Like on the Internet, new nodes and connections are added continuouslyThus, the infosystem needs to support the dynamics

addition of new nodes and connectionsshorten the ‘distance’ and matching finding and allow interaction between consumers and users

A Sample of Datasets Accessible through DataFed/ESIP MediationNear Real Time (~ day)

It has been demonstrated (project FASTNET) that these and other datasets can be accessed, repackaged and delivered by AIRNow through ‘Consoles’

MODIS Reflectance

MODIS AOT TOMS Index

GOES AOT

GOES 1km Reflec

NEXTRAD Radar

MODIS Fire Pix

NRL MODEL

NWS Surf Wind, Bext

Assertions on Web Services Technology• Currently Web Services are the leading (and only?) technologies for building software

applications in autonomous, heterogeneous networked environment

• The Web Services future is promising since:– businesses are driving the WS technologies and the community is benefiting from the

increasingly ‘semantic web’– a growing array of resources (data and processes) are exposed as ‘services’ – WS-based ES application frameworks are being developed (e.g. SciFlo, DataFed)

WS Adaptation Issues• Catalogs for finding and using services are grossly inadequate• The semantic layers of the interoperability stack are not yet available• Network Technology issues or ‘fallacies of distributed computing’:

– Network is reliable – Latency is zero – Bandwidth infinite – Network is secure – Topology stable – One administrator – No transport costs – Network uniform

Interoperability Stack

Layer Description Standards

Semantics Meaning WSDL ext., Policy, RDF

Data Types Schema, WSDL

Protocol Communication behavior SOAP, WS-* ext.

Syntax Data format XML

Transport Addressing, Data flow HTTP, SMTP

Kickoff Questions• What is a Web Service?

– e.g. 'A programming module with a well-defined, web-based I/O interface' (operating on well structured data??)

– Examples of what is/is not a WS

• WS Classification by Interoperability Layer– Transport– Interface Syntax

• Strongly typed interface (e.g. SOAP, WSDL)• Weakly typed interface (e.g. arbitrary CGI? URL interface)

– Protocol/Data– Semantics

• WS Classification by Architecture– Services for Tightly Coupled applications (e.g. URL service called from IDL)– Services for Loosely Coupled (e.g. application composed from SOAP services)

Value Network• The primary activities of a value network are concerned with mediating exchanges between customers and managing relationships with them.

• Fjeldstad and Stabell define value networks as organizations that create value for customers by linking them together or mediating exchanges between them. This can be direct, as in a telephone service, or indirect, as in retail banking where a group of customers (borrowers and lenders) are linked together through a common pool of funds. In each case, it is the customers who are the network. The value-network organisation provides a networking service, typically through some form of infrastructure. Fjeldstad and Stabell note that value-network companies:

• Operate like clubs – the mediating organization establishes, monitors and terminates direct or indirect relationships among members. It admits members who complement each other, and in some cases excludes those that do not.

• Facilitate matching and monitoring through standardization – value networks use standardization (of infrastructure or customer records, for example) to help them mediate interactions between customers.

• Increase value to customers by enlarging the network – the more customers a value network has or is able to connect, the more value it represents to each individual customer. Thus many new value networks adopt a ‘giveaway’ strategy to build a critical mass of customers.

• Perform mediation activities simultaneously at multiple levels – to mediate exchanges between a large number of customers efficiently, value networks need a simultaneous and layered set of activities. For example, a bank mediating electronic payment between a buyer and a store must simultaneously link multiple payment systems (other banks and credit card companies) and communications networks. The resulting set of activities is performed by multiple, layered networks that co-produce value for the customer.

• Value networks must excel at matching customers and multiplying connections between them. The management agenda for value networks (opposite) focuses on such imperatives as utilising the network infrastructure fully, maintaining the exclusivity of the network, devising innovative service provisioning and pricing, assessing long-term value of customers, and identifying clusters and connections between customers and network layers.

Value Network• The management agenda for value networks

Value networks must understand how its customers add value to each other, and identify which customers to bring in to the network, and which to exclude.

• Optimising utilisation of the network infrastructure is a key economic imperative, achieved by both building scale and encouraging the right type of usage. (Airlines could fill every flight by discounting every seat, but they would not make money.) Value networks must encourage good use of the network.

• Value networks seek to sell excess capacity (off-peak power, unused telecommunications network capacity, flights at inconvenient times) in ways that attract new customers without undermining existing relationships.

• The more connections they offer, the more valuable networks are to their customers – but customers also value exclusivity. How do networks resolve this paradox? Frequent-flyer programmes are a good example of an attempt to maintain uniqueness in the face of pressure to become more open and link with competing airlines.

• Network infrastructure and customer acquisition both carry high fixed costs, so networks seek ways to build and sustain long-term revenue streams and customer bases. Thus they favour subscription pricing rather than pay-per-transaction. Subscriptions can be direct (an annual subscription to use a cellular network or join a club) or indirect (minimum monthly bank balances or fees).

• A crucial opportunity for networks to increase revenue is to move beyond generic exchanges between customers and identify unique opportunities to bring them together or to increase the value of their exchanges. To gain this vital knowledge, networks must monitor and understand customer behaviour.

• The more network companies know about the relationships between customers and their network usage patterns, the more easily they can find opportunities to add new value. A successful example is the Friends and Family Program introduced by MCI (a long-distance telecommunications carrier). This offers special discounts to customers calling the numbers they dial most often.

• The more services you add to your network, the more value it has for customers. It also may be advantageous to connect to new customers through other value networks. Credit-card companies, for example, now offer special card ‘affinity’ programmes for groups and clubs of various kinds.

Strategy Tradeoffs in the Knowledge and Network Economy

• In traditional “value chain” firms, the main activity tradeoff is between differentiation and low cost.

• Increasingly, however, firms are creating customer value through networks (eg AOL) or by providing knowledge-based solutions for customers (eg venture capital firm Kleiner Perkins).

• This article discusses the quite different activity tradeoffs faced by these “value networks” and “value shops”.

• It then explores the tradeoff between exploitation (focusing on short-term performance) and exploration (focusing on transcending short-term activity tradeoffs).

• Finally, in reviewing the implications for managers, it discusses the problem of trying to manage different types of business (value chains, networks and shops) within the same corporation.