Oct 1999 Gio XIT 1

Extrapolating Trends for Information Technology

Gio Wiederhold Stanford University

September 1999

Based on “Trends for Information Technology” 1999

www-db.stanford.edu/pub/gio/1999/miti.htm

Oct 1999 Gio XIT 2

T r e n d s 1998 : 1999

• Users of the Internet 40% 52% of U.S. population

• Growth of Net Sites (now 2.2M public sites with 288M pages)• Expected growth in E-commerce by Internet users [BW, 6 Sep.1999]

segment 1998 1999– books 7.2% 16.0%– music & video 6.3% 16.4%– toys 3.1% 10.3%– travel 2.6% 4.0%– tickets 1.4% 4.2%– Overall 8.0% 33.0% = $9.5Billion

An unstainable trend cannot be sustained [Herbert Stein]

new services

98 99 00 01 02 03 04 0.3 1 3 9 27 81 **

90 80 70 60 50 40 30 20 10

0

Year / %

%

Centroid, in 1999 ~1% of total market

E-penetration Toys

Oct 1999 Gio XIT 3

Interactions

Consumer

Pull

Research &

Inno -vation

Toolbuilding

Product building &

marketingGeneralTechnology

Push Businessneeds

Governmentresponsibilities

InformationTechnology

Oct 1999 Gio XIT 4

Assumptions

• Hardware technology will continue to lead and encourage broader usage

• Communication technology will continue to lead and become more economical

• User interfaces will improve and not be a barrier to the acceptance of technology

• Government policies will not hinder open interaction - or not be able to

Oct 1999 Gio XIT 5

The Problem of Information Growth:

"We are drowning in information but starved for knowledge. This level of information is clearly impossible to be handled by present means. Uncontrolled and unorganized information is no longer a resource in an information society, instead it becomes the enemy."

-- John Naisbitt, author of 1982 bestseller Megatrends

. . . and it’s not getting better

Dealing with this issue requires Precision:• Helpful for casual users• Essential for business

Oct 1999 Gio XIT 6

Precision in:

• Search for Information– recall versus precision

• Relevance of Information for the Customer– modeling the customer

• Meaning of the Information– resolving semantic mismatch

• Timeliness of Information– resolving temporal mismatch

Service model to achieve these objectives services add value by increasing precision

Oct 1999 Gio XIT 7

Search techniques to add value

Yahoo catalogues and organizes useful web sites.

Junglee integrates diverse sources.

AltaVista automatically surfs and indexes the web.

Excite also tracks queries and classifies customers.

Firefly provides customer control over their profiles.

Cookies track users’ activities between sessions.

Alexa collects webpages and their usage.

Google ranks the reference importance of web pages.

. . .

Oct 1999 Gio XIT 8

Problems for search engines and progress

• Unsuitable source representations• part classification: HTML --- XML• print formats: postscript, adobe PDF• non-text: images, sound, video• hidden in databases behind CGI scripts

• Inconsistent semantics • context distinct / scope / view

• Naïve modeling of customers• roles & growth

Search engines cannot solve all problems

Being improved.

Rate?

Oct 1999 Gio XIT 9

The world wide information network and its participants

External:

sourcesand / orsinks

Internal:

transformers

and memory.

_…._

….

_…._

….

_…._

….

_…._

….

_…._

….

_…._

….

_…._

….

_…._

….

_…._

….

_…._

….

_…._

…._

…._

….

data, meta-data,knowledge

Oct 1999 Gio XIT 10

Understand the Architecture forInformation Technology:

Sources

Services

Customers

SourcesSources

ServicesServices

CustomersCustomersCustomersCustomers

Component Classification

Oct 1999 Gio XIT 11

Specifications for the components

Sources

Services

Customers

SourcesSources

ServicesServices

CustomersCustomersCustomersCustomers

Metadata

Customermodels

CatalogsContent

&Methods

Progress

Oct 1999 Gio XIT 12

Functional Service Layers

Service Service interfaceinterface

Resource accessResource access interfaceinterface

User interfaceUser interface

Real-worldReal-world interfaceinterface

Human-computerHuman-computer InteractionInteraction

Application-Application- specific codespecific code

Domain-Domain- specific specific codecode

Source-Source- specificspecific codecode

MEDIATIONMEDIATION ServicesServices

Available Sources

Client

Oct 1999 Gio XIT 13

Modeling: sources• Models provide abstractions

• abstractions represent a point of view• Models of databases are schemas and E-R models

• well established• constraints - references, uniqueness

• scopes remain implicit• Information systems have meta-data• XML has DTD’s

• under discussion, still limited

Focus on resources

Meta data

Oct 1999 Gio XIT 14

Customer modelsCustomer is a person one specific task

• arranging a vacation trip• activity ˆ location town ˆ hotel by grade ˆ flight ˆ public transport ˆ rented car

•arranging a business trip• location ˆ hotel by plan ˆ flight ˆ taxi or rented car

• getting a computer for Joe Cheap• search CPU by price ˆ modem ˆ display

• getting a computer for Peter Fast • search CPU by speed ˆ storage ˆ display ˆ network

Hierarchical alternatives at each level ( evaluate, commit, rollback )

Oct 1999 Gio XIT 15

Personal vs. Customer Model

Actual Person has multiple roles how to switch

explicitly implicitly

keep past contexts

Switching rate will differ• work versus fun• adequacy of models

Oct 1999 Gio XIT 16

Service layer

Customer Customer Service Service

Resource Resource accessaccess

MEDIATIONMEDIATION

Multipledomains !

Shared software,standards ?

Oct 1999 Gio XIT 17

Value-added intermediate services 1

Filters attached to the customer model; balance relevant volume and precision

Wrap resources to make them compatible, exploit wrapper templates, skip unavailable sources

Match available metadata and indices of resource contents to leaf nodes in the customer model

Monitor and index public metadata, describe resource capabilities, contents & methods

Needs Technologies extant and new

Describe customer model

Discover new resources

Select relevant resources

Easy access to resources

Filter out excessive data

Build interpretable workflow model with meta-specifications for selection

Oct 1999 Gio XIT 18

Value-added intermediate services 2

Automatic abstraction to match sources at

articulation points within the customer model Attach data instances to articulation points, combine elements , link to customer model

Match data for content, omit overlap, report inconsistencies in overlapping sources

Summarize according to customer model, rank information at each level

Present information according to model hierarchy, consider bandwidth

Needs Technologies, extant and new

Identify articulations *

Match level of detail *

Integrate information

Omit redundant data, documents

Reduce customer overload

Inform customer

Matching of related concepts, use articulation rules to match nodes

Oct 1999 Gio XIT 19

Abstraction layers differ:

Example in medical research• Individual patient records• Family based genetic traces• Disease-based summaries• Genetically-linked disease data• Ligand-based genomic segments• Aggregated gene sequences • 3-D configurations of segments• Drug-gene interactions

All have their own hierarchies, roots

Oct 1999 Gio XIT 20

Combining the models* Identify articulations

• Match customer and resource terms• semantic mismatches

• thesauri, matching rules

Match level of detail • Match customer and resource values,

summarize numbers, result ranks • completeness, unit mismatches, text

• indicate constraints in models• textual abstraction • input for visualization

Oct 1999 Gio XIT 21

Mediator Service Design Principle

Transform Data into Information

Match

User Model

Hierarchical

to

Resource Model

General network

(and maintain models)

Oct 1999 Gio XIT 22

Result modes for ranking

Databases:• Completeness• All the answers

Prolog• Correctness• The first answer

Optimization• The best one• Assumes all factors are known, no human decision

Customer:

• wants choices

• explanation

• background

Oct 1999 Gio XIT 23

Ranking

Qualitative Significant Differences:

in terms of the customer model

Plan 1. UA59 dep.Wash.Dulles 17:10, arr. LAX 19:49

Plan 2. AA75 dep.Wash.Dulles 18:00, arr. LAX 20:10

Plan 3. UA119 dep.Wash.Dulles 9:25, arr. LAX 12:00

Busy Joe:

P1= P2, P3

Speedy Mike:

P2, P1=P3

Greedy Pete:

P1=P3, P2

Oct 1999 Gio XIT 24

Mediation for Quality

User Modelf(S,C,T)

Assessments: S1=.8 S2=.9 S3=8

User Modelf(S,C,T)

Assessments: S1=.8 S2=.9 S3=8

BEST=low costrapid responsereliable deliverytrustworthiness

C3= 10+_1T3=50+_80

Estimates: C1= 5+_1 T1=100+_160

C2= 8+_1T2=70+_30

S1 S3S2

S= source reliabilityC= confidenceT=

Oct 1999 Gio XIT 25

Computing Projections

timepast now future

01.3

Next period alternatives and subsequent periods

0.40.4

0.60.6

0.20.2

0.50.50.30.3

0.250.25

0.10.1

0.050.05

0.30.3

0.070.07

0.30.3

For decision-making: not just past data

Integrate simulation results into information systems: SQL SimQL

Oct 1999 Gio XIT 26

Extending the support into the future

Must manage multiple projected futures ---Novel tools needed to help the decision maker:

1. Assess the likelihood of a branch being taken (if not controlable)

2. Compute probabilities into the future, up to desired/final endpoints

3. Compute results at each node, by backtracking from the endpointsand considering the probabilities

4. Compare the associated costs and benefits for the alternatives at any future time

5. Recalculate to get new, better values, less uncertainty

• Trim or summarize unlikely branches to reduce the complexity

• Prune to the current state and delete all but one actual path

Oct 1999 Gio XIT 27

Architecture instances

Applications . . . .

Mediators . . . . . .

Resources . . ._

….…. .

_….…. .

_….…. .

include computational resources

Oct 1999 Gio XIT 28

Assigning maintenance responsibility

a. Source data quality –supplier database, files, or web pages

b. Interface to the source – wrapper, supplier or vendor for supplier

c. Source selection – expert specialist in mediator

d. Source quality assessment – customer input to mediator

e. Semantic interoperation – specialist group providing input to the mediator

f. Consistency and metadata information – mediator service operation or warehouse

g. Informal, pragmatic integration – client services with customer input

h. User presentation formats – client services with customer input

Services

Sources

Customers

Oct 1999 Gio XIT 29

Summary

To sustain the trend 1. The value of the results has to keep increasing

precision, relevance not volume2. Value is provided by experts,

encoded as models of diverse resources, customersProblems to be addressed mismatches quality temporal extensions maintenance

} Clear models

Oct 1999 Gio XIT 30

Technology Transition .

• Economic drivers have to be considered. • Three party model

• Industry: need-based invention• academia: formalization• innovators: new technology

• New Service models provide new Opportunities• supply innovative tools to industry• supply specialized information to industry

I i

a

Oct 1999 Gio XIT 31

Understanding the other parties

Motivation is profit and loss avoidance of • Industry: investment --

– payoff to stockholders / retain value / stable • Academia: prestige -- (leads to continuing funding)

– visibility, not stability or reliability• Innovative businesses: leverage -- not sustainable

– low downside cost, high upside risk, – change expected and needed

• Government research: – technology dissemination & shelving service ?

Oct 1999 Gio XIT 32

Research economy transfer paths

Pro

du

cts

Tool suppliers (TS) versus Product suppliers (PS)

high-valuemodest volume

Customers

Research

GovernmentTeaching

Ta

xes

highvolume

people results

Oct 1999 Gio XIT 33

Operating Systems

• Microsoft Windows, personal computer and WS.

proprietary product, no obligations to hardware,

rapidly adapted to new requirements • UNIX, an open systems, consensus and takes time.

• SUN servers• LINUX clients and servers, free, low entry cost• ….

• Mainframe operating systems, little growth expected• VMS (COMPAQ) reliable 24 hour / 7 day

Oct 1999 Gio XIT 34

1 Pre-competitive development.2 Integration and Marketing3. Problem: Asynchrony.

3.1 Industry-driven. research. 3.2 Curiosity-driven research. 3.3 Fundamental research3.4 Transition windows

4 Transition agents. 4.1 Link academic researchers to industry 4.2 Link academic and industrial research. 4.3 Startup companies. 4.4 Incubator services. 4.5 Research stores.

Commercial Technology Transfer Company. Governmental Technology Transfer Institute. Other candidate organization models for research stores.

5 Research Venues and Technology Transfer.6 Summary

Oct 1999 Gio XIT 35

Alternative solutions• A Super Database

– unwieldly

– obsolete before it is established

• Distributed, free standing databases (today)– awkward for sharing information

(much knowledge derives from the intersections)

– hyperlinks and shared references allow navigation

• Distributed databases with a single standard allowing interoperation

– standards follow progress, cannot lead it

• Distributed databses with published formats – requires rapid adaptation to keep up with resources

(but the number of resources per project will be limited)

with mediators to isolate projects from resources

Oct 1999 Gio XIT 36

Paying• Free goods (as information), supported by advertisers• The referred service pays for references made• After contact and selection direct by credit card

at some processing overhead and delay

• Customer trust for tolerable losses • Audited ba mediator, violators are blacklisted only • Escrow for substantial value: more delay

• Very small transactions use walletsa. Risk is assumed by the vendor:

b. Risk is assumed by the customer:

• Subscriptions for long-term interactions