Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In...

84
Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool, Hong Su, Amy Lee, and others. Dept. of Computer Science Worcester Polytechnic Institute

Transcript of Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In...

Page 1: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

1

Generic Change Management:

The SERF Project and Beyond

Elke A. Rundensteiner

In collaboration with students at WPI including:

Kajal Claypool, Hong Su, Amy Lee, and others.

Dept. of Computer ScienceWorcester Polytechnic Institute

Page 2: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

2

“Nothing is as constant as change itself”

Christina Baldwin

Page 3: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

3

Motivation: Change is Everywhere

Persistent data everywhere! And it changes ...

Why? Mistakes in database design or modeling Changes in real world (understanding of domain) Inherent in design and exploratory processes Addition of new applications to system

Support for comprehensive and generic change management needed !!!

Page 4: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

4

Issues in Change Management

How to specify change? How to execute

change? How to manage effect

on dependent information?

DBAs

Views

DBMSStructurechange

DataChange

Constraintchange

Applications

Web pagesDBMS

DBMS

Structurechange

DataChange

Page 5: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

5

Specification of Change

Page 6: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

7

Schema EvolutionPerson

EmployeeStudent

namePerson

EmployeeStudent

nameaddressadd_attribute(address, Person)

add_attribute(address, Person)add_attribute(address, Employee)add_attribute(address, Student)

Object transformations to conform to schema changes

Changes to structure of class and class hierarchy

Page 7: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

8

Issues – Specification of Change

Traditional change Structural change and data change

Schema evolution Pre-defined fixed operations

[Objectstore,Versant,Objectivity] E.G.: Add or delete of an attribute

Pre-defined complex operations [Breche96, Lerner96] E.G.: Merging two classes, split a class

Not enough – hard to meet all users’ needs...

Goal 1: need flexible mechanism

Page 8: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

9

Issues – Specification of Change

Many types of change: Structural, data, constraint, rules, time, model

Related work: Schema evolution [Objectstore94, Versant92, Objectivity93,

Breche96, Lerner96, O2-94] Constraint evolution [Mayol99]

Time evolution [Nascimento95]

Problem: Individual similar solutions for each change

type [ER -Roddick99]

Goal 2: Need to different kinds of changes

Page 9: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

10

Limitations of Current Evolution Technology

lack of flexibility a fixed set of primitives

lack of extensibility pre-determined semantics

lack of re-usability ad-hoc programs for complex

transformations

Page 10: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

11

SERF:

Schema Evolution through an Extensible, Re-usable and Flexible

Framework

Page 11: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

12

Goal – Generic and Flexible Change Specification

Framework Flexible:

user or domain-specific transformation Extensible:

user can specify new semantics Reusable:

generic and not application-specific library

Portable: not tied to a particular system

Page 12: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

16

SERF TransformationSERF Transformation :

combination of evolution primitives

+specification of object transformations

+ OQL as the “glue logic”

SERF schema evolution framework sits on top of existing OODB systems.

Page 13: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

17

Person

name

Address

address street

city

state

Person

name

street

city

state

Example of Inlining

Page 14: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

18

Inline Transformation// Add required attributes to Person classadd_attribute (Person, Street, String, "");add_attribute (Person, City, String, "");add_attribute (Person, State, String, "");

// Get all objects for Person classdefine extents() as select c from Person c;

// Update all objectsfor all obj in extents(): obj.set (obj.Street, valueOf(obj.address.Street)); obj.set (obj.City, valueOf(obj.address.City)); obj.set (obj.State,valueOf(obj.address.State)); // Delete the address attributedelete_attribute (Person, address);

Step A

Step B

Step C

Step B

Page 15: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

19

SERF Template

SERF Transformation+

Re-callable (Named)

+Generally Applicable (MetaData)

+Re-usable (Parameters)

Page 16: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

20

Naming and Parameterizing a Transformation

Re-callable named

begin template inline (…..)end template

Parameterized input and output parameters(className, refAttr)

Example of Instantiation inline (Person, address)

Page 17: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

21

Generalizing a Transformation

Remove dependency to specific classes by using metadata (ODMG System Repository)

define localAttrs($cName) as select c.localAttrList from MetaClass c where c.metaClassName = $cName;

for all attrs in localAttrs(refClass) add_attributeadd_attribute ($className, attrs.attrName, attrs.attrType, attrs.attrValue);

Page 18: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

22

begin template inline (className, refAttrName){ refClass = element ( select a.attrType from MetaAttribute a where a.attrName = $refAttrName and a.classDefinedIn = $className; ) define localAttrs(cName) as select c.localAttrList from MetaClass c where c.metaClassName = cName; for all attrs in localAttrs(refClass) add_attributeadd_attribute ($className, attrs.attrName, attrs.attrType, attrs.attrValue); define extents(cName) as select c from cName c; for all obj in extents($className): for all Attr in localAttrs(refClass) obj.set (obj.Attr, valueOf(obj.refAttrName.Attr)) delete_attribute ($className, $refAttrName);} end template

Inline Template

Page 19: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

23

Flow of a Template

Template inline(className, refAttrName)

Type-check

className = Class Person, refAttrName = Attribute address

Bind-check

pass

Error to User

error

Instantiate

pass

syntax-check

inline(Person, address)

Execute

pass

error

error

Pure oql statements

Page 20: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

24

SERF Architecture

SERF Framework

OODB System

Query Engine

Schema Repository

Schema Evolution Manager

Object Repository

queries

queries

operates on

Template Processor

Instantiates and executes

Template Manager

UserInterface

SchemaViewer

TemplateEditor

uses

uses

uses uses

TemplateLibrary

uses

Page 21: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

25

SERF Architecture

Sc he m aRe p o sito ry

Sc he m aEvo lutio nM a na g e r

Q ue ryEng ine

O b je c tRe p o sito ry

q ue rie s tra nsfo rm s

use s e xe c ute s

Te m p la teEd ito r

Sc he m aVie we r

Te m p la teLib ra ry

Te m p la tePro c e sso r

Insta ntia to r

Ve rifie r

Te m p la teExe c uto r

sto re s &e d its te m p la te s

re trie ve ste m p la te s

e xe c ute s using

g ra p hic a ld isp la y o f sc he m a

ve rifie s

Page 22: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

26

Advantages of SERF Flexible

define semantics of choice Extensible

define new transformations Re-usable

define embedded transformations Portable

applicable to different ODMG systems

Page 23: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

28

Conclusions Identify limitation of fixed SE primitives Developed SERF solution

Transformations primitives + OQL + system update methods

Templates generalized, named and parameterized

transformations

Library of Templates (from Case Study) ODMG based SERF prototype

Page 24: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

29

Does it give us Consistency?

Person

name

Address

address street

city

state

Person

name

street

city

state

Department

Office

inline

Department

Office

Page 25: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

30

Consistency Responsibilities

Who should be responsible for guaranteeing consistency? SERF Templates or Core Schema Evolution Primitives or End User or Application Developer

Page 26: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

31

Concept of “Contracts” into Templates

Mechanism for declaratively expressing constraints of “schema” in template

System-enforced consistency guarantee Use for consistency of relationship evolution --> ER’2000 Use for software upgrade itself for better maintainability -->

TOMORROW

Page 27: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

32

Performance of such Complex Templates

Schema evolution are expensive methods Evolution is embedded within multiple query statements CHOP: ECOOP OODB workshop’2000 on an optimizer for

evolution sequences CHOP-OQL: optimizer for embedded oql queries [ MS thesis,

Ming Li, 2000 ]

Page 28: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

33

Supporting Other Applications

Page 29: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

34

Issues: Supporting Other Applications

DBMS

Structurechange

DBAs

Applications

View Evolutionchange ?

??

Current approaches flexibility in structure

of queries [Lieberherr94]

Adapting queries using available redundant information [EVE97]

Hiding change from other applications [TOSE98]

Versioning [Zdonik, Lauteman, Sommerville]

Views

Page 30: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

WPI

35

Schema Evolution

Examples: Add-attribute: capacity-augmenting (CA) Delete-attribute: capacity-reducing (CR) Change-class-name: capacity-equivalent

(CE)

Page 31: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

WPI

36

Application Interoperability Problem

I: Deleting DataApp1 App2

App1

Schema Data

Data Deleted

old

new

Page 32: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

WPI

37

Transparent Object Schema Evolution: View-based Solution

App1 App2

App1

ViewView

View

Schema Data

old

new

Data Removed

Page 33: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

WPI

38

Application Interoperability Problem:

II: Adding Data

App1 App2

Schema Data

Data Added

old

App2new

Page 34: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

WPI

39

TOSE for Capacity-Augmenting Changes

App1 App2

Schema Data

Data Added

old

App2new

View

ViewView

?

?

Page 35: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

WPI

40

Previous Solution:Transparent Schema Evolution For Capacity-Reducing/-Equivalent:

Derive target view schema from input view schema by e.g. hiding

For Capacity-Augmenting:Require capacity-augmenting views to store additional capacity

Capacity-Augmenting Views not (yet) Commercially Available!

Page 36: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

TOSE: Three-Pronged Process for Capacity-Augmenting Schema Changes

1. Base augmentation step (schema change)2. Target generation step (view)3. Original base restoration step (view)

BS BS’

VS VS’CA Schema Change

1

2

3

User-defined SC

In-Place SC

View Derivation

Requires only conventional view- and schema evolution tools!

Page 37: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

Degenerate Process for Capacity-Reducing/-Equivalent SCs

1. Base Augmentation Step (skip)2. Target Generation Step (from VS)3. Original Base Restoration Step (skip)

BS=BS’

VS VS’CR or CE SC

2User-defined SC

View Derivation

Page 38: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

WPI

43

Update Semantics Preservation(USP)

Given: Users work on view schemas instead of on shared database

Goal: Provide base-like working environment

Requires: Updates on views translated into updates on

base schema Back-propagation from base to views have same

effects as if updates directly performed on views

Page 39: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

44

USP Virtual Class Definition: Update on virtual class (VC)

has same effect as if VC were base class Mechanism: Create, delete, and set

update applied to VC is translated to its source class, and then propagates back to VC.

Example: Person’ is usp.

Person name

incomePerson’name

hide(income,Person)

(base class)(hide virtual class)

Page 40: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

45

USP Case Study

Use MultiView’s object algebra as basis

Identify set of algebra operators generating virtual classes sufficient for TOSE, namely: ident, hide, union, difference, intersect.

Prove this set of view derivation operators to be USP.

Page 41: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

46

USP Definitions

Definition: (VCi,VCj) is a usp pair, if (VCj,VCi) possesses typical update behavior as if they were base classes.

Definition: view schema VS is usp if every pair of classes in VS is a usp pair.

Definition: VD(Si) = Si, is usp derivation if Si is guaranteed to be usp whenever Si is usp.

Page 42: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

Dept

Root

StudentBS

Personname

income

Root

Person’

Student’

VS VS’Root

Person’’

Student’’

add-attr(birthday, Person’)

Capacity-Augmenting Schema Change Example

nameincome

name

birthdayincome

Page 43: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

Root

Dept*Person*

Student* BS’

Dept

Root

StudentBS

Personname

incomename

birthdayincome

Root

Person’

Student’

VS VS’Root

Person’’

Student’’

add-attr(birthday, Person’)

Step 1: Base Augmentation

nameincome

name

birthdayincome

1

Page 44: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

Root

Dept*Person*

Student* BS’

Dept

Root

StudentBS

Personname

incomename

birthdayincome

Root

Person’

Student’

VS VS’Root

Person’’

Student’’

add-attr(birthday, Person’)

Step 2: Target Generation

nameincome

name

birthdayincome

1

2

Page 45: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

Root

Dept*Person*

Student* BS’

Dept

Root

StudentBS

Personname

incomename

birthdayincome

Root

Person’

Student’

VS VS’Root

Person’’

Student’’

add-attr(birthday, Person’)

Step 3: Original Base Restoration

nameincome

name

birthdayincome

1

2

3

Page 46: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

Capacity-Reducing Schema Change Example

Root

Person’

Student’

VS VS’’

Root

Person’’’

Student’’’

del-attr(income, Person’)

nameincome nam

e

VS’’ = VDdel-attr(income,Person’)(VS(BS))

Page 47: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

Capacity-Reducing Schema Change Example

Root

Person’

Student’ VS’’

Root

Person’’’

Student’’’

del-attr(income, Person’)

nameincome nam

e

VS’’ = VDdel-attr(income,Person’)(VS(BS))

hide(income, Person’)

hide(income, Student’)

VS

Page 48: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

53

TOSE Transparency Theorem

Study complete taxonomy of schema evolution primitives

If capacity-reducing or -equivalent: give corresponding usp view derivation

If capacity-augmenting transformation a. give three steps of process b. prove their correctness

Page 49: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

54

Advantages of TOSE

Applications can interoperate on same set of objects flexibly

Shared schema can evolve without affecting existing applications (migration)

Data consistency is guaranteed, since there is only one copy of base objects

TOSE relies on standard tools only

Page 50: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

55

TOSE Implementation

SPARCstation 10 GemStone 4.0 SunOS 4.1.3

TOSE

MultiViewView System

MultiView Object Model

GemStone

Page 51: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

56

Related Work Banerjee et al.: regular in-place schema

changes Breche et al. / Ra et al.: rely on CA views Tresch et al.: tackle CR/CE changes only Lautemann: use versioning approach Kaushal et al.: handle schema changes from

base Claypool et al.: define complex change

operations using primitive change operations

Page 52: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

57

Conclusions

Development of three-step methodology to achieve schema changes transparently

Identification of update semantics preservation requirement of view schema & case study

Formalization of the TOSE methodology Development of TOSE algorithms for

standard taxonomy of SE primitives Feasibility demonstration via prototype

Page 53: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

58

Next Research Tasks

Propagate SERF Transformations through Views

Expand work to complex views (usp?) Apply TOSE to relational databases

(e.g., object-views by Oracle-8) Extend to distributed environments:

modeling management technology

Page 54: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

59

Research Direction

Model the “Mapping” between two “Systems” as First-Class Citizen

Model “Models” as First-Class Citizens

Support Operations on Mapping and Models themselves: Change Propagation is one such operator

Page 55: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

60

Issues: Execution of Change

DBMS

Structurechange

Constraintchange

ApplicationsViews

Base change - process somewhat understood.

View change - limited work done.

Web Pages

Page 56: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

61

Execution of View ChangeRelated Work

Schema change Object-preserving OO views [Rundensteiner98,

Bertino96]

Data change [Keller82, Scholl91, Bancilhon81 ,EBG97, Dayal89]

Other types of changes Meta-data [Sciore94]

Security [Rosenthal97]

Problem: Individual similar solutions for each change

type

Page 57: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

62

Change by Propagation Process

Common process: Find the “right” source Translate the change and apply it Re-write the view/propagate the change back

up

DB

V(DB)

V

U(V(DB)) = V(DB’)

T(U)T(U)(DB) = DB’

TV

U

Page 58: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

63

Goals Goal 1: Framework that is:

Extensible. Propagation of different information.

Flexible. Propagation algorithms can be altered easily.

Portable. Applicable for heterogeneous sources and views.

Goal 2: Apply for evolving object-generating views.

Page 59: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

64

PURE - Propagation Rule Framework

Rule-based Handle different information types Proposal: PR-OQL

rule extension for OQL

Object-oriented easier application to heterogeneous sources and

views Middle-layer

Propagate from source to view, view to source applicable in multi-tier environment

Page 60: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

65

PURE Architecture

IS 1 IS 2 IS n

De riva tio n Tre e

ED

De riva tio nBa se

Pro p a g a tio n-IDEve nt G e ne ra to r

RuleM a na g e r

RulePro c e sso r

Pro p a g a tio n-IDEve nt G e ne ra to r

Inte rna lEve nt G e ne ra to r

RuleC a ta lo g

V1 V2 Vn

SE o p e ra tio no p _1

SE o p e ra tio no p _2

O p _ 1 O p _ 2

Eve nt E1

E1, No d e

E1, No d e

R No d ei,

De riva tio nfro m IS

Vie w e xp o se d to use r

R , No d ei

M ultip leEve nts E2, E3..En

So urc e c ha ng e , o p _ 3

O p _ 3

Eve nt E3

Rule

Page 61: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

66

Goal 2 - Evolution of Object-Generating Views

Existing work [Rundensteiner98, Bertino96]

evolution of select-project views Complex applications (e-commerce,

design) [Abiteboul99, Peckham95] views are often join views

Evolution more complicated than select-project views!

Page 62: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

67

Join ExampleJOINCLASS JOINCLASS'AdvisorEmailstudentNamegrade

AdvisorEmailstudentNamegradedateOfGrad

Advisor StudentadvisorNameAdvisorEmailphoneNumber

studentNameadvisorNamegrade

J

Add-attribute(JOINCLASS, dateOfGrad, DATE, "1999")

select a.AdvisorEmail, s.studentName, s.gradefrom Advisor a, Student swhere a.advisorName = s.advisorName

Advisor Student'AdvisorNameAdvisorEmailphoneNumber

studentNameadvisorNamegradedateOfGrad

J

Add-attribute(Student, dateOfGrad, DATE, "1999")

Propagate

Translate/Apply

Derive/up propagation

Page 63: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

68

Model Management Goal 1: Framework that is:

Extensible. Propagation of different information.

Flexible. Propagation algorithms can be altered easily.

Portable. Applicable for heterogeneous sources and views.

Goal 2: Apply for evolving object-generating views.

Page 64: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

69

PURE - Propagation Rule Framework

Rule-based Handle different information types Proposal: PR-OQL

rule extension for OQL

Object-oriented easier application to heterogeneous sources and

views Middle-layer

Propagate from source to view, view to source applicable in multi-tier environment

Page 65: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

70

PURE Architecture

IS 1 IS 2 IS n

De riva tio n Tre e

ED

De riva tio nBa se

Pro p a g a tio n-IDEve nt G e ne ra to r

RuleM a na g e r

RulePro c e sso r

Pro p a g a tio n-IDEve nt G e ne ra to r

Inte rna lEve nt G e ne ra to r

RuleC a ta lo g

V1 V2 Vn

SE o p e ra tio no p _1

SE o p e ra tio no p _2

O p _ 1 O p _ 2

Eve nt E1

E1, No d e

E1, No d e

R No d ei,

De riva tio nfro m IS

Vie w e xp o se d to use r

R , No d ei

M ultip leEve nts E2, E3..En

So urc e c ha ng e , o p _ 3

O p _ 3

Eve nt E3

Rule

Page 66: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

71

Goal 2 - Evolution of Object-Generating Views

Existing work [Rundensteiner98, Bertino96]

evolution of select-project views Complex applications (e-commerce,

design) [Abiteboul99, Peckham95] views are often join views

Evolution more complicated than select-project views!

Page 67: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

72

Join ExampleJOINCLASS JOINCLASS'AdvisorEmailstudentNamegrade

AdvisorEmailstudentNamegradedateOfGrad

Advisor StudentadvisorNameAdvisorEmailphoneNumber

studentNameadvisorNamegrade

J

Add-attribute(JOINCLASS, dateOfGrad, DATE, "1999")

select a.AdvisorEmail, s.studentName, s.gradefrom Advisor a, Student swhere a.advisorName = s.advisorName

Advisor Student'AdvisorNameAdvisorEmailphoneNumber

studentNameadvisorNamegradedateOfGrad

J

Add-attribute(Student, dateOfGrad, DATE, "1999")

Propagate

Translate/Apply

Derive/up propagation

Page 68: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

73

Finding the “right” Source Left or right outer join Key “Meta-properties” [Rosenthal94]

allow user to specify dependencies at evolution time

E.G. dateOfGrad <functionalDependent> on studentName

Page 69: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

74

Relationship Example

V1

De p a rtm e ntd s

V2

De p a rtm e ntd f

ARA(V1, advisor, V2)

ARA( , advisor, ) (d,s) (d,f)

(d,s)ARA(s, advisor_1, f)ARA(s, advisor_2, d)

Example 2•Complex changes require

• finding the “right” source•finding the “right” view

•E.G: add a relationship between two views

Page 70: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

75

Tasks Propagation & Re-write algorithms Equivalence Model

rigid equivalence too strict user preference model for

equivalence with “threshold equivalence”

Cost Model to evaluate the choices

Page 71: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

76

Evaluation and Validation Implementation using PURE Validation

Show algorithms are: Equivalence preserving Confluent and terminating Update semantics preserving

Compare cost model with experimental results

Page 72: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

77

Applications of SERF

Page 73: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

78

Re-WEB - Re-Usable Templates for Web

View Generation and Restructuring

WIDM’98 Workshop, Kajal T. Claypool, Elke A. Rundensteiner, Li Chen, Bhupesh Kothari

WWW Journal 2000, Li Chen, Kajal Claypool, and Elke A. Rundensteiner

Funded in part by NSF, IBM and ODI Inc.

Dept. of Computer ScienceWorcester Polytechnic Institute

Page 74: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

79

Motivation Re-WEB - Re-Usable Templates for Web

View Generation and Restructuring

The Digital Age! web-sites - a large data repository

Generating web pages from database exploit database and optimization

technology easier to manage dynamic generation easier to propagate changes

Issue: Diverse web-site representations needed

Page 75: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

80

Re-WEB Approach: Internal Views

Html /XML

Html/ XML

View classes

SERFTemplates

WebMapping semantics

Page 76: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

81

Re-WEB Approach (contd.)

Advantages no new web language required for

restructure, since re-use OQL change management is reduced to

propagating changes to views apply SERF (re-usable re-structuring

templates)

Disadvantages define view schema in database

Page 77: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

82

Re-WEB: Re-using SERF

Original schema

SERFTemplate

New schema

Original Web Page View New Web Page View

Page 78: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

83Schema

Professor

nameemailaddressphoneNumcourses

Course

namedescriptionlocationtimetaughtBy

courses

taughtBy

Data

S.H. [email protected] 116999-9999

Tom [email protected] 117999-9999

Roberto [email protected] 118999-9999

Advanced DatabasesRequires Databases 101Fuller 3203:30-5:30pm

Advanced OSRequires OS 101Fuller 3201:30-3:30pm

AnimationRequires Graphics 101Fuller 3209:30-11:30pm

Obj4

Obj5

Obj6

Obj1

Obj2

Obj3

Original Schema Structure

Page 79: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

84

Original Web Pages

Course Name: Advanced Databases

Description: Requires Databases 101

Location: Fuller 320

Time: 3:30-5:30pm

TaughtBy:

S.H.Waverly Thomas Dooley

Name: S.H. Waverly

Email: [email protected]

Location: Fuller 116

Phone Num: 999-9999

Courses:

Advanced Databases

Advanced OS

Course Web Pages Professor Web Pages

Page 80: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

85

To Get Desired Web Pages ?

Course Name: Advanced Databases Description: Requires Databases 101

Location: Fuller 320

Time: 3:30-5:30pm

TaughtBy:

ProfName: S.H.Waverly

Email: [email protected]

Location: Fuller 116

Phone Num: 999-9999

ProfName: Thomas Dooley …

Page 81: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

86

Re-structuring of DB: Convert-To-Literal

Professor

nameemailphoneNumcourses

Course

namedescriptionlocationtimetaughtBy

courses

taughtByProfessorStruct

nameemailphoneNumcourses

CourseView

namedescriptionlocationtimestruct-taughtBy

Struct-taughtBy

create_view_class (Course, CourseView, “select c from Course c”);create_view_struct (Professor, ProfessorStruct);add_attribute(CourseView, struct-taughtBy, set, null);for all objects in taughtBy flatten to struct of type struct-taughtBy;delete_attribute(CourseView, taughtBy);

SERF Transformation

Page 82: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

88

Advantages of Re-WEB Web views are consistent with one another

change management is easier (database)

One step generation of web site structure

Easy to generate diversely restructured

web-sites

Re-usable library of Re-WEB templates

Uses standard technology: ODMG,OQL,XML

Page 83: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

89

More Information….

http://davis.wpi.edu/dsrg/OOSE/SERF

Page 84: Spring 2000 WPI 1 Generic Change Management: The SERF Project and Beyond Elke A. Rundensteiner In collaboration with students at WPI including: Kajal Claypool,

Spring 2000

WPI

90

Some project publications SERF Framework

(Claypool, Jin and Rundensteiner 1998) [CIKM], journal in submission

SERF Prototype (Rundensteiner, Claypool, Chen, Koenoki et al.) [SIGMOD99, CASCON’99]

Validation Web site re-structuring

(Rundensteiner, Claypool, Chen et al.) [SIGMOD00, WWWJournal00] Object model evolution

(Claypool, Rundensteiner and Heineman) [ER’2000, TR99-15, TR00-09]

Other Relevant Work Consistency management in SERF

(Claypool and Rundensteiner) [TR99-21] Optimization of schema evolution sequences

(Claypool, Natarajan and Rundensteiner) [ECOOP-WS’00]