Designing a Flexible Designing a Flexible Framework for a Table Framework for a Table

AbstractionAbstraction

H. Conrad Cunningham H. Conrad Cunningham 1 1 Yi Liu Yi Liu 2 2 Jingyi Wang Jingyi Wang 3 3

11 University of Mississippi University of Mississippi2 2 South Dakota State UniversitySouth Dakota State University

33Acxiom CorporationAcxiom Corporation

Hot Spots, Patterns, and Hot Spots, Patterns, and Contracts:Contracts:

An Exploration of Software An Exploration of Software Framework DesignFramework Design

H. Conrad Cunningham H. Conrad Cunningham 1 1 Yi Liu Yi Liu 2 2

11 University of Mississippi University of Mississippi2 2 South Dakota State UniversitySouth Dakota State University

26-Jan-2010 3

Hot Spots ReferencesHot Spots References Coplien, J., Hoffman, D., Weiss, D. Coplien, J., Hoffman, D., Weiss, D.

Commonality and variability in Commonality and variability in software engineering, software engineering, IEEE SoftwareIEEE Software, , vol. 15, no. 6, pp. 37-45, 1998.vol. 15, no. 6, pp. 37-45, 1998.

Schmid, H. A., Framework design by Schmid, H. A., Framework design by systematic generalization, In: Fayad, systematic generalization, In: Fayad, M. E., Schmidt, D. C., Johnson, R. E., M. E., Schmidt, D. C., Johnson, R. E., editors, editors, Building Application Building Application Frameworks: Object-Oriented Frameworks: Object-Oriented Foundations of Framework DesignFoundations of Framework Design, , Wiley, pp. 353-378, 1999Wiley, pp. 353-378, 1999..

26-Jan-2010 4

Patterns ReferencesPatterns References Gamma, R., Helm, R., Johnson, Gamma, R., Helm, R., Johnson,

R., Vlissides, J., R., Vlissides, J., Design Patterns: Design Patterns: Elements of Reusable Object-Elements of Reusable Object-Oriented SoftwareOriented Software, Addison , Addison Wesley, 1995.Wesley, 1995.

Buschmann, F., Meunier, R., Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., Stal, Rohnert, H., Sommerlad, P., Stal, M., M., Pattern-Oriented Software Pattern-Oriented Software Architecture: A System of Architecture: A System of PatternsPatterns, Wiley, 1996., Wiley, 1996.

26-Jan-2010 5

Contracts ReferencesContracts References

Meyer, B., Meyer, B., Object-Oriented Object-Oriented Software ConstructionSoftware Construction, 2nd , 2nd Edition, Prentice Hall PTR, 1997.Edition, Prentice Hall PTR, 1997.

26-Jan-2010 6

Source PapersSource Papers Cunningham, H. C., Liu, Y., Wang. J., Cunningham, H. C., Liu, Y., Wang. J.,

Designing a flexible framework for a table Designing a flexible framework for a table abstraction, Chapter 13 in: Chan, Y., abstraction, Chapter 13 in: Chan, Y., Talburt, J., Talley, T., editors, Talburt, J., Talley, T., editors, Data Data Engineering: Mining, Information, and Engineering: Mining, Information, and IntelligenceIntelligence, Springer, pp. 279-314, 2010. , Springer, pp. 279-314, 2010.

Cunningham, H. C., Wang, J., Building a Cunningham, H. C., Wang, J., Building a Layered framework for the table Layered framework for the table abstraction, In abstraction, In Proceedings of the 16th Proceedings of the 16th ACM Symposium on Applied Computing,ACM Symposium on Applied Computing, pp. 668-674, 2001.pp. 668-674, 2001.

26-Jan-2010 7

Software Interfaces Project Software Interfaces Project (1996-2000)(1996-2000)

Context: development of an instructional Context: development of an instructional data and file structures library (for Java data and file structures library (for Java in CS 3)in CS 3) artifacts for study of good design techniquesartifacts for study of good design techniques system for use, extension, and modificationsystem for use, extension, and modification

Motivation: study techniques forMotivation: study techniques for presenting important methods to students presenting important methods to students

(frameworks, software design patterns, (frameworks, software design patterns, design by contract, etc.)design by contract, etc.)

unifying related file and data structures in unifying related file and data structures in frameworkframework

26-Jan-2010 8

Table Abstract Data Type Table Abstract Data Type (ADT)(ADT)

Collection of recordsCollection of recordso each a finite sequence of data fieldseach a finite sequence of data fieldso keykey field value uniquely identifies record within field value uniquely identifies record within

collectioncollection Operations on collection and on record using keyOperations on collection and on record using key Several different implementations possibleSeveral different implementations possible

key1key1 data1data1

key2key2 data2data2

key3key3 data3data3

key4key4 data4data4

26-Jan-2010 9

Table ADT OperationsTable ADT Operations

Insert new recordInsert new record Delete existing record given keyDelete existing record given key Update existing recordUpdate existing record Retrieve existing record given keyRetrieve existing record given key Get number of recordsGet number of records Query whether contains given keyQuery whether contains given key Query whether emptyQuery whether empty Query whether fullQuery whether full

26-Jan-2010 10

Software FrameworkSoftware Framework

Generic application allowing Generic application allowing creation of members of family of creation of members of family of related programsrelated programs

Reusable design expressed as set of Reusable design expressed as set of abstract classes and way they abstract classes and way they collaboratecollaborate

Common and variable aspects Common and variable aspects known as known as frozen spotsfrozen spots and and hot spotshot spots

FrameworkFramework

library

User-supplied code(application specific)

Hot spots

Frozen spots

26-Jan-2010 11

Software Design PatternsSoftware Design Patterns

Describe recurring design problems Describe recurring design problems arising in specific contextsarising in specific contexts

Present well-proven generic solution Present well-proven generic solution

schemesschemes Describe solution’s components and Describe solution’s components and

their responsibilities and relationshipstheir responsibilities and relationships To use:To use:

o select pattern that fits problemselect pattern that fits problemo structure solution to follow patternstructure solution to follow pattern

26-Jan-2010 12

Software Design ContractsSoftware Design Contracts

PreconditionsPreconditions for correct use of operation for correct use of operation

PostconditionPostconditions for correct result of s for correct result of

operationoperation Invariant Invariant conditions for correct conditions for correct

implementation of classimplementation of classInsert record operation pre: record is valid and not already in table post: record now in table

Invariant for table all records are valid, no duplicate keys

26-Jan-2010 13

Initial Table Framework Initial Table Framework RequirementsRequirements

1.1. Provide Table ADT functionality for client-Provide Table ADT functionality for client-defined records and keysdefined records and keys

2.2. Support many implementations of access Support many implementations of access and storage mechanismsand storage mechanisms

3.3. Separate key-based access mechanism Separate key-based access mechanism from storage mechanismfrom storage mechanism

4.4. Present coherent abstractions with well-Present coherent abstractions with well-defined interfacesdefined interfaces

5.5. Use software design patterns and design Use software design patterns and design contractscontracts

26-Jan-2010 14

Hot Spot Analysis of Hot Spot Analysis of RequirementsRequirements

1.1. Provide Table ADT functionalityProvide Table ADT functionality for client-for client-defined records defined records andand keys keys

2.2. Support Support many implementations of access many implementations of access andand storage mechanisms storage mechanisms

3.3. Separate key-based access mechanism Separate key-based access mechanism from storage mechanismfrom storage mechanism

4.4. Present coherent abstractions with well-Present coherent abstractions with well-defined interfacesdefined interfaces

5.5. Use software design patterns and design Use software design patterns and design contractscontracts

26-Jan-2010 15

Table Hot Spot AnalysisTable Hot Spot Analysis

Frozen spotFrozen spot

1.1. Table ADT functionality Table ADT functionality

Hot spotsHot spots

1.1. Variability in keys (defined by clients)Variability in keys (defined by clients)

2.2. Variability in records (defined by clients)Variability in records (defined by clients)

3.3. Variability in external representation of record Variability in external representation of record statestate

4.4. Variability in access (i.e, indexing) mechanism Variability in access (i.e, indexing) mechanism

5.5. Variability in storage mechanismVariability in storage mechanism

Hot spots #1 and #2 not completely independentHot spots #1 and #2 not completely independent

Hot spot #3 quite subtleHot spot #3 quite subtle

Desire to separate concerns (i.e., the hot spotsDesire to separate concerns (i.e., the hot spots))

26-Jan-2010 16

Layered Architecture PatternLayered Architecture Pattern

Distinct groups of services Distinct groups of services Hierarchical arrangement of groups into layersHierarchical arrangement of groups into layers Layer implemented with services of layer belowLayer implemented with services of layer below Enables independent implementation of hot Enables independent implementation of hot

spots in layersspots in layers

Client Layer

Access Layer

Storage Layer

26-Jan-2010 17

Applying Layered Architecture Applying Layered Architecture PatternPattern

Client Layer – Hot spots #1 and #2Client Layer – Hot spots #1 and #2o includes client application programsincludes client application programso implements record and key abstractionsimplements record and key abstractionso uses layer below to store and retrieve recordsuses layer below to store and retrieve records

Access Layer – Frozen spot and Hot spot #4Access Layer – Frozen spot and Hot spot #4o includes table implementationsincludes table implementationso provides key-based access to records for layer aboveprovides key-based access to records for layer aboveo calls back to record and key implementations in level calls back to record and key implementations in level

aboveaboveo uses physical storage in layer belowuses physical storage in layer below

Storage Layer – Hot spot #5Storage Layer – Hot spot #5o storage managersstorage managerso provides physical storage for recordsprovides physical storage for recordso calls back to record implementations in layers abovecalls back to record implementations in layers above

26-Jan-2010 18

Client Layer of Client Layer of ArchitectureArchitecture

Encapsulate hot spots #1 and Encapsulate hot spots #1 and #2#2

1.1. Variability in keys (defined Variability in keys (defined by clients)by clients)

2.2. Variability in records Variability in records (defined by clients(defined by clients))

26-Jan-2010 19

Client Layer DesignClient Layer Design

ChallengesChallengeso enabling Access Layer to access enabling Access Layer to access

client-defined records and keysclient-defined records and keyso avoiding unnecessary programming avoiding unnecessary programming

to use common data typesto use common data types

PatternPatterno Interface – applied independently Interface – applied independently

for hot spots #1 (keys) and #2 for hot spots #1 (keys) and #2 (records)(records)

26-Jan-2010 20

Client Layer InterfacesClient Layer Interfaces(1 of 2)(1 of 2)

Hot spot #1 – Variability in keys Must be able to (at a minimum) compare with Must be able to (at a minimum) compare with

total orderingtotal ordering Choose to use Comparable interface from Java Choose to use Comparable interface from Java

API for convenienceAPI for convenience

Comparable Comparable interface for keysinterface for keyso int compareTo(Object key)int compareTo(Object key)

compares object with argumentcompares object with argument

26-Jan-2010 21

Client Layer InterfacesClient Layer Interfaces(2 of 2)(2 of 2)

Hot spot #2Hot spot #2 – – Variability in recordsVariability in records Must be able to extract keys for comparisonMust be able to extract keys for comparison Choose to define an appropriate new interfaceChoose to define an appropriate new interface

Keyed Keyed interface for recordsinterface for recordso Comparable getKey()Comparable getKey() extracts key extracts key

from recordfrom record

26-Jan-2010 22

Client Layer ModelClient Layer Model

Abstract predicates – Abstract predicates – for use in for use in specification of this and other layers specification of this and other layers (not necessarily implemented)(not necessarily implemented)

o boolean isValidKey(Object key) boolean isValidKey(Object key) yields true for valid key valuesyields true for valid key values

o boolean isValidRec(Object rec) boolean isValidRec(Object rec) yields true for valid record valuesyields true for valid record values

26-Jan-2010 23

Client Layer Design ContractsClient Layer Design Contracts

Hot spot #1Hot spot #1 Comparable Comparable int compareTo(Object key)int compareTo(Object key)

Pre:Pre: isValidKey(this) && isValidKey(key) isValidKey(this) && isValidKey(key) Post:Post: result == (if this < key then -1 result == (if this < key then -1 else if this == key then 0else if this == key then 0 else 1)else 1)

Hot spot #2Hot spot #2 Keyed KeyedComparable getKey()Comparable getKey() Pre: Pre: isValidRec(this)isValidRec(this) Post: (Post: (result == this.keyresult == this.key) )

&& isValidKey(result)&& isValidKey(result)

26-Jan-2010 24

Client Layer Challenges Met?Client Layer Challenges Met?

Enabling Access Layer to access Enabling Access Layer to access client-defined records and keysclient-defined records and keys

o records and keys implementations records and keys implementations encapsulated within objects that encapsulated within objects that implement implement KeyedKeyed and and ComparableComparable

Avoiding unnecessary programming Avoiding unnecessary programming to use common data types to use common data types

o reuse reuse ComparableComparable from Java API from Java API

26-Jan-2010 25

Access Layer of Access Layer of ArchitectureArchitecture

Provide frozen spotProvide frozen spot

Table ADT functionality Table ADT functionality

Encapsulate hot spot #4:Encapsulate hot spot #4:

Variability in access (i.e, Variability in access (i.e, indexing) mechanismindexing) mechanism

26-Jan-2010 26

Access Layer DesignAccess Layer Design

ChallengesChallengeso enabling use of client-defined keys and enabling use of client-defined keys and

recordsrecordso enabling diverse implementations of enabling diverse implementations of

indexing structureindexing structureo enabling use of diverse storage enabling use of diverse storage

structuresstructures

PatternPatterno Interface – applied for frozen spot (Table Interface – applied for frozen spot (Table

ADT) and hot spot #4 (access) ADT) and hot spot #4 (access)

26-Jan-2010 27

Access Layer Interface TableAccess Layer Interface Table

void insert(Keyed r) void insert(Keyed r) inserts inserts rr into table into tablevoid delete(Comparable key)void delete(Comparable key) removes record with removes record with keykey

void update(Keyed r) void update(Keyed r) changes record with same changes record with same keykey

Keyed retrieve(Comparable key)Keyed retrieve(Comparable key) returns record returns record with with keykey

int getSize()int getSize() returns size of table returns size of tableboolean containsKey(Comparable key)boolean containsKey(Comparable key) searches for searches for keykey

boolean isEmpty() boolean isEmpty() checks whether table is emptychecks whether table is emptyboolean isFull() boolean isFull() checks whether table is fullchecks whether table is full

o for unbounded, always returns falsefor unbounded, always returns false

26-Jan-2010 28

Access Layer ModelAccess Layer Model

Partial function Partial function table :: Comparable table :: Comparable Keyed Keyedo represents abstract table staterepresents abstract table stateo consider this: number of entries finite or infinite? If consider this: number of entries finite or infinite? If

finite, bounded or unbounded? If infinite, countable or finite, bounded or unbounded? If infinite, countable or not? not?

o #table#table in postcondition denotes table before operation in postcondition denotes table before operationo use as either function or set of key-record pairsuse as either function or set of key-record pairs

Abstract predicates (to capture environmental Abstract predicates (to capture environmental assumptions)assumptions)o isValidKey(Object)isValidKey(Object) and and isValidRec(Object)isValidRec(Object)

defined in Client Layer to identify valid keys and recordsdefined in Client Layer to identify valid keys and recordso isStorable(Object)isStorable(Object) defined in Storage Layer to defined in Storage Layer to

identify records that can be storedidentify records that can be stored

26-Jan-2010 29

Table Design Contract (1 of 5)Table Design Contract (1 of 5)

InvariantInvariant

(( k, r : r == table(k) : k, r : r == table(k) :

isValidRec(r) && isStorable(r) isValidRec(r) && isStorable(r)

&& k == r.getKey() )&& k == r.getKey() )

All stored keys and records in the All stored keys and records in the tabletable are valid and capable of being stored are valid and capable of being stored on the chosen external device, and the on the chosen external device, and the records can be assessed by their keys.records can be assessed by their keys.

26-Jan-2010 30

Table Design Contract (2 of 5)Table Design Contract (2 of 5)

void insert(Keyed r) void insert(Keyed r) inserts inserts rr into table into tablePre: Pre: isValidRec(r) && isStorable(r) && !isValidRec(r) && isStorable(r) && !containsKey(r.getKey())&& !isFull()containsKey(r.getKey())&& !isFull()

Post: Post: table == #table table == #table {(r.getKey(),r)} {(r.getKey(),r)}

void delete(Comparable key)void delete(Comparable key) removes record with removes record with key key from tablefrom tablePre:Pre: isValidKey(key) && containsKey(key) isValidKey(key) && containsKey(key) Post:Post: table == #table - {(key,#table(key))} table == #table - {(key,#table(key))}Question: Does this allow Question: Does this allow tabletable be initialized be initialized

nonempty?nonempty?

26-Jan-2010 31

Table Design Contract (3 of 5Table Design Contract (3 of 5))

void update(Keyed r)void update(Keyed r)changes record with same changes record with same keykey

Pre: Pre: isValidRec(r) && isStorable(r) && isValidRec(r) && isStorable(r) && containsKey(r.getKey())containsKey(r.getKey())

Post: Post: table == (#table - table == (#table - {(r.getKey(),#table(r.getKey()))} ) {(r.getKey(),#table(r.getKey()))} ) {(r.getKey(),r)}{(r.getKey(),r)}

Keyed retrieve(Comparable key)Keyed retrieve(Comparable key) returns record returns record with with keykey

Pre: Pre: isValidKey(key) && containsKey(key)isValidKey(key) && containsKey(key)

Post: Post: result == #table(r.getKey())result == #table(r.getKey())

26-Jan-2010 32

Table Design Contract (4 of 5)Table Design Contract (4 of 5)

int getSize()int getSize() returns size of table returns size of table

Pre: Pre: truetrue

Post: Post: result == cardinality(#table)result == cardinality(#table)

Note: If size not finite, need way to represent infinityNote: If size not finite, need way to represent infinity

boolean containsKey(Comparable key)boolean containsKey(Comparable key) searches searches table for table for keykey

Pre: Pre: isValidKey(key)isValidKey(key)

Post: Post: result == defined(#table(key))result == defined(#table(key))

26-Jan-2010 33

Table Design Contract (5 of 5)Table Design Contract (5 of 5)

boolean isEmpty()boolean isEmpty()checks whether table is emptychecks whether table is empty

Pre: Pre: truetrue

Post: Post: result == (#table = result == (#table = ))

boolean isFull()boolean isFull()checks whether table is fullchecks whether table is full

– – for unbounded, always returns for unbounded, always returns falsefalse

Pre : Pre : truetrue

Post: Post: result == (#tableresult == (#table implementation has implementation has

no free space to store no free space to store record)record)

26-Jan-2010 34

Client/Access Layer Client/Access Layer InteractionsInteractions

Client calls Access Layer class Client calls Access Layer class implementing implementing TableTable interface interface

Access calls back to Client Access calls back to Client implementations of implementations of KeyedKeyed and and ComparableComparable interfaces interfaces

26-Jan-2010 35

Access Layer Challenges Met?Access Layer Challenges Met?

Enabling use of client-defined keys and Enabling use of client-defined keys and recordsrecordso callbacks to callbacks to ComparableComparable and and KeyedKeyed abstractions abstractions

that hide the implementation detailsthat hide the implementation details

Enabling diverse implementations of the Enabling diverse implementations of the tabletableo careful design of table interface semantics using careful design of table interface semantics using

design by contractdesign by contract

Enabling use of diverse storage mechanismsEnabling use of diverse storage mechanismso calls to Storage Layer interfacescalls to Storage Layer interfaces

26-Jan-2010 36

Storage Layer of Storage Layer of ArchitectureArchitecture

Encapsulate hot spot #5:Encapsulate hot spot #5:Variability in storage Variability in storage

mechanismmechanism

26-Jan-2010 37

Storage Layer DesignStorage Layer Design

Challenges Challenges o supporting client-defined records supporting client-defined records o supporting diverse table implementations in supporting diverse table implementations in

Access Layer (simple indexes, hashing, Access Layer (simple indexes, hashing, balanced trees, etc.)balanced trees, etc.)

o allowing diverse physical media (in-memory, allowing diverse physical media (in-memory, on-disk, etc.)on-disk, etc.)

o enabling persistence of table, including access enabling persistence of table, including access layerlayer

o decoupling implementations as much as decoupling implementations as much as possiblepossible

PatternsPatternso BridgeBridgeo ProxyProxy

26-Jan-2010 38

Bridge PatternBridge Pattern Decouple “interface” from Decouple “interface” from

“implementation”“implementation”o table from storage in this casetable from storage in this case

Allow them to vary independentlyAllow them to vary independentlyo plug any storage mechanism into plug any storage mechanism into

tabletable Table RecordStore

Simple Indexed

File

Hashed File

Slotted File Store

Vector File

uses

26-Jan-2010 39

Proxy PatternProxy Pattern

Transparently manage services of Transparently manage services of target objecttarget objecto isolate isolate TableTable implementation implementation

from nature/location of record from nature/location of record slots in slots in RecordStoreRecordStore implementationimplementation

Introduce proxy object as surrogate Introduce proxy object as surrogate for targetfor target handle

TableRecordSlot

RecordStore

26-Jan-2010 40

Storage Layer InterfacesStorage Layer Interfaces

RecordStoreRecordStoreo operations to allocate and deallocate operations to allocate and deallocate

storage slotsstorage slots

RecordSlotRecordSloto operations to get and set records in operations to get and set records in

slots slots o operations to get handle and containingoperations to get handle and containing RecordStoreRecordStore

26-Jan-2010 41

Storage Layer ModelStorage Layer Model(1 of 2)(1 of 2)

Partial function Partial function store :: int store :: int Object Objecto represents abstract represents abstract RecordStoreRecordStore state state

Set Set Handles Handles int, NULLHANDLE int, NULLHANDLE Handles Handles

Set Set alloc alloc Handles Handles o represents set of allocated slot handlesrepresents set of allocated slot handles

SetSet unalloc == Handles - allocunalloc == Handles - alloc o represents set of unallocated slot handlesrepresents set of unallocated slot handles

Abstract predicateAbstract predicate isStorable(Object)isStorable(Object) o depends on storage mechanismdepends on storage mechanism

26-Jan-2010 42

Storage Layer ModelStorage Layer Model(2 of 2)(2 of 2)

Invariant: Invariant:

(( h,r : r == store(h) : isStorable(r)) h,r : r == store(h) : isStorable(r))

&&&&

(( h :: h h :: h alloc == defined(store(h))) alloc == defined(store(h)))

All records in the All records in the storestore are capable of are capable of being stored on the selected medium and being stored on the selected medium and the stored records can be access by their the stored records can be access by their handleshandles..

26-Jan-2010 43

RecordStore InterfaceRecordStore Interface

RecordSlot newSlot()RecordSlot newSlot()allocates a new record slotallocates a new record slot

RecordSlot getSlot(int handle)RecordSlot getSlot(int handle) rebuilds record slot using given rebuilds record slot using given handlehandle

void releaseSlot(RecordSlot slot)void releaseSlot(RecordSlot slot)deallocates recorddeallocates record slotslot

26-Jan-2010 44

RecordStore Design Contract RecordStore Design Contract (1 of 2)(1 of 2)

RecordSlot RecordSlot newSlotnewSlot()() allocates a new record slot, but it might be allocates a new record slot, but it might be done in a lazy fashiondone in a lazy fashion

Pre: Pre: truetrue

Post: Post: result.getContainer() == this && result.getRecord() result.getContainer() == this && result.getRecord() == == NULLRECORDNULLRECORD && && result.getHandle() result.getHandle() #alloc && #alloc && result.getHandle() result.getHandle() alloc alloc {NULLHANDLE} {NULLHANDLE}

Note: Note: NULLRECORDNULLRECORD built according to Null Object pattern built according to Null Object pattern

RecordSlot getSlot(int handle)RecordSlot getSlot(int handle) rebuilds record slot using given rebuilds record slot using given handlehandle

Pre: Pre: handle handle alloc alloc

Post: Post: result.getContainer() == this && result.getContainer() == this && result.getRecord() == #store(handle) &&result.getRecord() == #store(handle) && result.getHandle() == handleresult.getHandle() == handle

26-Jan-2010 45

RecordStore Design Contract RecordStore Design Contract (2 of 2)(2 of 2)

void releaseSlot(RecordSlot slot)void releaseSlot(RecordSlot slot) deallocates deallocates record record slotslot

Pre: Pre: slot.getHandle() slot.getHandle() alloc alloc {NULLHANDLE} {NULLHANDLE} && && slot.getContainer() == thisslot.getContainer() == this

Post: Post: alloc == #alloc - {slot.getHandle()} &&alloc == #alloc - {slot.getHandle()} &&

store == #store –store == #store –

{(slot.getHandle(),slot.getRecord())}{(slot.getHandle(),slot.getRecord())}

26-Jan-2010 46

RecordSlot InterfaceRecordSlot Interfacevoid setRecord(Object rec)void setRecord(Object rec) stores stores recrec in this slot in this slot

allocation of handle done here or already done by allocation of handle done here or already done by getSlotgetSlot

Object getRecord()Object getRecord() returns record stored in this returns record stored in this slotslot

int getHandle()int getHandle() returns handle of this slot returns handle of this slot

RecordStore getContainer()RecordStore getContainer() returns reference to returns reference to RecordStoreRecordStore holding this slot holding this slot

boolean isEmpty()boolean isEmpty() determines whether this slot determines whether this slot emptyempty

26-Jan-2010 47

RecordSlot ModelRecordSlot ModelEach slot has two attributes:Each slot has two attributes:

reference toreference to RecordStore RecordStore to which this to which this RecordSlotRecordSlot belongs, which is immutablebelongs, which is immutable

handlehandle for the associated physical storage slot in for the associated physical storage slot in the the RecordStore RecordStore (as constrained in(as constrained in RecordStore RecordStore))

Invariant: Invariant: getHandle() getHandle() alloc alloc {NULLHANDLE} {NULLHANDLE}

The handle of aThe handle of a RecordSlotRecordSlot object denotes a slot of object denotes a slot of thethe storestore that has been allocated unless it has the that has been allocated unless it has the valuevalue NULLHANDLE.NULLHANDLE.

26-Jan-2010 48

RecordSlot Design Contract RecordSlot Design Contract (1 of 3)(1 of 3)

void setRecord(Object rec)void setRecord(Object rec) stores stores recrec in in thisthis slot sloto allocation of handle done here or already done by allocation of handle done here or already done by getSlot()getSlot()

Pre: Pre: isStorable(rec)isStorable(rec)Post: Post:

Let Let h == getHandle()h == getHandle():: (h (h #alloc #alloc store == (#store - store == (#store -

{(h,#store(h))}) {(h,#store(h))}) {(h,rec)} ) && {(h,rec)} ) && (h = NULLHANDLE (h = NULLHANDLE

(( g : g g : g #unalloc : #unalloc : alloc == #alloc alloc == #alloc {g} && {g} && store = #store store = #store {(g,rec)} {(g,rec)}

))))

26-Jan-2010 49

RecordSlot Design Contract RecordSlot Design Contract (2 of 3)(2 of 3)

Object getRecord()Object getRecord() returns record stored in this returns record stored in this slotslot

Pre: Pre: truetrue

Post: Let Post: Let h == getHandle()h == getHandle(): :

((h h #alloc #alloc result == #store(h)) result == #store(h)) && &&

((h == NULLHANDLEh == NULLHANDLE result == NULLRECORDresult == NULLRECORD) )

int getHandle()int getHandle() returns handle of this slot returns handle of this slot

Pre: Pre: truetrue

Post: Post: result == this.handleresult == this.handle

26-Jan-2010 50

RecordSlot Design Contract RecordSlot Design Contract (3 of 3)(3 of 3)

RecordStore getContainer()RecordStore getContainer() returns reference to returns reference to RecordStoreRecordStore holding this slot holding this slotPre: Pre: truetruePost: Post: result == this.containerresult == this.container

boolean isEmpty()boolean isEmpty() determines whether this slot determines whether this slot emptyemptyPre: Pre: truetruePost: Post: result ==result ==

(getHandle() == NULLHANDLE || (getHandle() == NULLHANDLE || getRecord() == NULLRECORD) getRecord() == NULLRECORD)

26-Jan-2010 51

Storage Layer Challenges Met?Storage Layer Challenges Met?(1 of 2)(1 of 2)

Supporting client-defined recordsSupporting client-defined recordso deferred to Externalization Moduledeferred to Externalization Module

Supporting diverse table implementations Supporting diverse table implementations in Access Layerin Access Layero careful design of careful design of RecordStoreRecordStore and and RecordSlotRecordSlot

abstractions to have sufficient functionalityabstractions to have sufficient functionality

Allowing diverse physical mediaAllowing diverse physical mediao careful design of careful design of RecordStoreRecordStore abstraction to abstraction to

hide media, implementable in many ways, hide media, implementable in many ways, o partly deferred to Externalization Modulepartly deferred to Externalization Module

26-Jan-2010 52

Storage Layer Challenges Met?Storage Layer Challenges Met?(2 of 2)(2 of 2)

Decoupling implementations as much Decoupling implementations as much as possibleas possibleo use of use of RecordSlotRecordSlot, , handlehandle, and , and

Externalization ModuleExternalization Module

Enabling persistence of Table, Enabling persistence of Table, including Access Layerincluding Access Layero store store RecordStoreRecordStore identifier and identifier and handleshandles

26-Jan-2010 53

Access/Storage Layer Access/Storage Layer InteractionsInteractions

Access Layer Access Layer TableTable calls calls RecordStoreRecordStore in in Storage Layer to get Storage Layer to get RecordSlotRecordSlot object object

Access Layer callsAccess Layer calls RecordSlot RecordSlot to store and to store and retrieve its recordsretrieve its records

If needed, If needed, RecordSlotRecordSlot calls back to calls back to RecordRecord implementation in Access Layer implementation in Access Layer o interface interface RecordRecord defined in Externalization defined in Externalization

ModuleModule

26-Jan-2010 54

Externalization Module of Externalization Module of ArchitectureArchitecture

Encapsulate Hot Spot #3:Encapsulate Hot Spot #3:Variability in external Variability in external representation of record staterepresentation of record state

26-Jan-2010 55

Externalization Module DesignExternalization Module Design

Challenges (e.g., to support Storage Challenges (e.g., to support Storage Layer)Layer)

o supporting client-defined records supporting client-defined records o allowing diverse physical mediaallowing diverse physical mediao decoupling implementations decoupling implementations

Pattern: InterfacePattern: Interface

26-Jan-2010 56

Externalization Module Externalization Module InterfaceInterface

RecordRecordo void writeRecord(DataOutput out) void writeRecord(DataOutput out) writes writes

the client’s record to stream the client’s record to stream outout

o void readRecord(DataInput in)void readRecord(DataInput in) reads the reads the client’s record from stream client’s record from stream inin

o int getLength()int getLength() returns number of bytes returns number of bytes in external representation of client’s in external representation of client’s record (e.g., written by record (e.g., written by writeRecord)writeRecord)

26-Jan-2010 57

Externalization Module Externalization Module ContractContract(1 of 2)(1 of 2)

void writeRecord(DataOutput out) void writeRecord(DataOutput out) writes writes thisthis record to record to stream stream outoutPre:Pre: true truePost:Post: suffix of suffix of stream stream out == out ==

this this record’s state encoded as byte record’s state encoded as byte sequencesequence

void readRecord(DataInput in)void readRecord(DataInput in) reads reads thisthis record from record from stream stream ininPre: Pre: true true Post: Post: thisthis record’s state == record’s state ==

prefix of prefix of streamstream in in decoded fromdecoded from byte byte sequencesequence

26-Jan-2010 58

Externalization Module Externalization Module ContractContract(2 of 2)(2 of 2)

int getLength()int getLength() returns number of bytes in returns number of bytes in external representation of external representation of thisthis record (e.g., record (e.g., written by written by writeRecord)writeRecord)Pre:Pre: true truePost:Post: result == result == number of bytes in external number of bytes in external

representation ofrepresentation of this this recordrecord

State Restoration Property: State Restoration Property: If, for some record, If, for some record, writeRecord()writeRecord() is followed by is followed by readRecord()readRecord() of the same byte sequence, the of the same byte sequence, the observable state of the record will be observable state of the record will be unchanged.unchanged.

26-Jan-2010 59

Externalization Challenges Externalization Challenges Met?Met?

Supporting client-defined recordsSupporting client-defined recordso records needing to be externalized records needing to be externalized

implement implement RecordRecord interface interface

Allowing diverse physical mediaAllowing diverse physical mediao RecordRecord interface implementation provides interface implementation provides

low-level physical representation of record low-level physical representation of record storable on most mediastorable on most media

Decoupling implementationsDecoupling implementationso RecordRecord decouples use from implementation decouples use from implementation

26-Jan-2010 60

Externalization Module Externalization Module InteractionsInteractions

RecordRecord used used by Storage Layer may be by Storage Layer may be defined by either layer abovedefined by either layer aboveo hold one Client Layer hold one Client Layer KeyedKeyed record recordo hold multiple, or fractional, Client Layer hold multiple, or fractional, Client Layer

records managed by Access Layer (e.g, records managed by Access Layer (e.g, multiway tree nodes)multiway tree nodes)

Storage Layer calls back to Storage Layer calls back to Record Record implementation in layer aboveimplementation in layer aboveo implementation in Access Layer might call implementation in Access Layer might call

back to implementations in Client Layerback to implementations in Client Layer

26-Jan-2010 61

Abstraction Usage Abstraction Usage RelationshipsRelationships

Table

Keyed Comparable

RecordStore RecordSlot

RecordAccess Layer

Storage Layer

Client Layer Externalization Module

26-Jan-2010 62

Other Design Patterns UsedOther Design Patterns Used

Null Object (for Null Object (for NULLRECORDNULLRECORD in Storage in Storage Layer)Layer)

IteratorIteratoro extended Table operationsextended Table operationso query mechanismquery mechanismo utility classesutility classes

Template MethodTemplate Method DecoratorDecorator StrategyStrategy

26-Jan-2010 63

Evolving Frameworks PatternsEvolving Frameworks Patterns

Generalizing from three examplesGeneralizing from three examples Whitebox frameworkWhitebox framework Component libraryComponent library

o Wang prototype: two Wang prototype: two TableTables and three s and three RecordStoreRecordStoress

Hot spotsHot spots Pluggable objectsPluggable objects Fine-grained objectsFine-grained objects Blackbox frameworkBlackbox framework

26-Jan-2010 64

ConclusionsConclusions

Hot spot analysis reveals Hot spot analysis reveals generalization opportunitiesgeneralization opportunities

Design patterns offer systematic way Design patterns offer systematic way to discover reliable designsto discover reliable designs

Design contracts help make Design contracts help make specifications precisespecifications precise

Novel design separates access and Novel design separates access and storage mechanismsstorage mechanisms

Case study potentially useful for Case study potentially useful for educational purposeseducational purposes

26-Jan-2010 65

Future WorkFuture Work

Study more formal use of Schmid’s generalization Study more formal use of Schmid’s generalization methodologymethodology

Study use of function generalization methodology Study use of function generalization methodology Modify prototypes to match revised designModify prototypes to match revised design Redevelop prototypes to use Scala with generics Redevelop prototypes to use Scala with generics

instead of Java without genericsinstead of Java without generics Rethink Externalization, perhaps using Strategy Rethink Externalization, perhaps using Strategy

patternpattern Study hot spots and build finer-grained Study hot spots and build finer-grained

component librarycomponent library Adapt earlier work of students on AVL and B-Adapt earlier work of students on AVL and B-

Tree class librariesTree class libraries Integrate into Integrate into SoftwareInterfacesSoftwareInterfaces library library Develop instructional materialsDevelop instructional materials

26-Jan-2010 66

AcknowledgementsAcknowledgements

Jingyi Wang for her work on prototype Jingyi Wang for her work on prototype frameworkframework

Yi Liu and Jingyi Wang for their work on the Yi Liu and Jingyi Wang for their work on the paperspapers

Wei Feng, Jian Hu, and Deep Sharma for their Wei Feng, Jian Hu, and Deep Sharma for their work on earlier table-related librarieswork on earlier table-related libraries

Sudharshan Vazhkudai, Jennifer Jie Xu, Sudharshan Vazhkudai, Jennifer Jie Xu, Vandana Thomas, Cuihua Zhang, Xiaobin Pang, Vandana Thomas, Cuihua Zhang, Xiaobin Pang, and Ming Wei for work on other frameworksand Ming Wei for work on other frameworks

Pallavi Tadepalli for collaboration on function Pallavi Tadepalli for collaboration on function generalizationgeneralization

Various reviewers including Bob Cook, Jennifer Various reviewers including Bob Cook, Jennifer Jie Xu, Chuck Jenkins, and Pallavi TadepalliJie Xu, Chuck Jenkins, and Pallavi Tadepalli

Acxiom Corporation for its supportAcxiom Corporation for its support