Persistence

MSO 08/09, Chapter 11, WP

Persistence to persist = to last , to endure

In a business application you often need your data to last even if you sometimes have to shut down the application.

Simple: Save the data. When the application is up again, it re-load the data.

But this is both critical and challenging!

2

Challenges

Reliability my data should really be saved Data integrity (latter) Ability to rollback

Performance I want to persist terra bytes of data Fast query and update

Concurrency I have concurrent clients accessing the data

3

Objectives of chapter 11, learning:

several object-persistence formats about mapping objects to your persistence formats optimizing relational data bases

Design Implementation

4

How to 'persist' my data ?

Save them in files. Simple But a business app often requires more:

Querying specific parts of data Fast query Backup and roll back Access control

Save them in a database.

DBMS

5

Typical Architecture (of business app)

Presentation / User Interface

Application Logic / PD Layer

Persistence/DAM Layer

database

In our setup this layer is in OO.

Design issue: which persistence approach should we use?6

Recommended architecture in Dennis et al

7

Problem Domain (PD) Layer

Data Access Management (DAM) Layer

DBMS

DAM takes care interaction with DBMS (e.g. to save and load).

• Relieve PD classes from having to implement save and load themselves

• Makes PD independent of the underlying DBMS.

Person-------------Name Age

PersonDAM----------------------------------savedeleteload

Database ... traditionally is relational Means that you store data in relation forms, aka tables.

8

Basic elements of a table

attribute

row

column

primary keyforeign key

9

Referential Integrity Foreign and primary keys should be consistent.

10

RDBMS Relational Data Base Management System

Refers to an implementation of a relational database concept.

They come with strong features: Powerful query language (SQL) Access control Referential integrity High performance ACID (Atomicity, Consistency, Isolation, Durability) transactions Proven technology ...

11

So, how to persist objects in an RDB ? RDB has no notion of inheritance. Still, we can map a class diagram to an ER diagram (next

slide)

This will tell how to map our class structures to tables.

But ... when we save an object:

save(patient)

this is not the same as inserting a row. We may potentially have to save the object substructures its induced relations.

Same goes with load and delete.12

Mapping ER diagram

Person--------------------Name : StringAge : int

Person--------------------Name : StringAge : int

Patient------------------ID : StringInsurance : String

Symptom----------------Code : StringName : String

suffers

0..*

0..*

Patient--------------------IDInsurance

SubObjectOf

0..1

1

Symptom--------------------CodeName

Suffers

0..*

0..*

13

So, how do you implement : save (Patient p) Patient load()

Various DB persistence alternatives

Persisting in RDBMS you have to build the DAM yourself

Persisting in ORDBMS support for inheritance thinner DAM.

Persisting in OODBMS in principle no DAM needed.

Persisting in RDMBS + ORM (Object-Relation mapping) gives a virtual OODBMS

14

ORDB(MS)

It's a classical RDB extended with some OO concepts User Defined Type class REF allow object navigation, e.g. x.attr1.attr SET (of REFs) allows a more direct representation of

one-to-many relation. Inheritance

SQL 1999 comes with all these extensions. Many vendors of traditional RDBMS already support SQL 99

(DB2, Oracle, PostgreSQL, Microsoft)

15

ORDB, example with SQL 99

16

CREATE TYPE Person AS OBJECT ( Name CHAR(20) Age INT)

CREATE TYPE Symptom AS OBJECT ( Code CHAR(10) Name CHAR(20))

CREATE TYPE Patient UNDER Person ( ID CHAR(12) Insurance ... Suffers SET (REF Symptom) )

ORDB, example with SQL 99

17

CREATE TABLE Persons OF TYPE PersonCREATE TABLE Symptoms OF TYPE SymptomCREATE TABLE Patients OF TYPE Patient

INSERT INTO Persons VALUES (Person("Octo", 50))...INSERT INTO Patients VALUES (Patient("Spons Bob",3,SET(1465)))

SELECT p.Name FROM Persons p

So, mapping OO-PD to ORDBMS ...

In Dennis et al: Rule 1 .. 9b p340 Assuming ORDBMS that does not support inheritance Read it yourself

I'll give you a simplified set of 'rules' We'll assume an ORDBMS that supports (single)

inheritance After all, SQL 99 already includes single inheritance

18

Simplified mapping rules

Map your class diagram to ER diagram (as before).

But no need to factor-out inheritance. This is already supported in our ORDB.

Map an entity to a table over a User Defined Type (UDT)UDT can express inheritance.

You have more options when implementing a relation via REF / OID via SET of REFs

19

Converting to ER diagram

Person--------------------Name : StringAge : int

Person--------------------Name : StringAge : int

Patient------------------ID : StringInsurance : String

Symptom----------------Code Name

suffers

0..*

0..*

Patient--------------------IDInsurance

Symptom--------------------CodeName

Suffers

0..*

0..*

20

From ER diagram to ORDB tables

Person--------------------Name : StringAge : int

Patient--------------------IDInsurance

Symptom--------------------CodeName

Suffers

0..*

0..*

21

CREATE TYPE Person AS OBJECT ( Name CHAR(20) Age INT)

CREATE TYPE Symptom AS OBJECT ( Code CHAR(10) Name CHAR(20))

CREATE TYPE Patient UNDER Person ( ID CHAR(12) Insurance ... Suffers SET (REF Symptom) )

But keep in mind you still have to build your DAM !!

OODB

Db4o for Java and .Net Available in GPL and as commercial Stable, and is just .5 MB !

ObjectContainer db = Db4o.openFile(Util.15DB4OFILENAME)

try { ... // do something with db4o }

finally { db.close(); }From db4o " Formula One Tutorial"

22

Examples of using db4o

Person p = new Person("Octo", 50)Person q = new Patient("Spons Bob", 3)Person r = new Patient("Patrick", 5)

db.set(p) ; db.set(q) ; db.set(r)

ObjectSet result1 =db.get(Patient.class)

ObjectSet result2 =db.get(new Person(null,5))

List <Person> result3 = db.query(

new Predicate<Person>() { public boolean match(Person p) { return p.age >= 5 ; } } )

r.suffers.add(PanicDisorder)

23

So ...

Notice that when using an OODB we don't need a DAM; we just call:

db.set(object1)

object2 = db.get(prototype) ;

24

Hibernate

Is an ORM (Object-Relation mapping) solution : Allow you to use ordinary RDB to persist Automate the back and forth transformation (from objects

to tables)

No DAM is needed either give you the illusion of using an OODB.

But you have to tell Hibernate how your classes are mapped to different tables.

Very popular choice at the moment.

25

Describe the mapping to tables

26

<?xml version="1.0"?><!DOCTYPE ... "... hibernate-mapping-3.0.dtd">

<hibernate-mapping> <class name="Person" table="PersonTab"> ... <property name="Name" column="Name"/> <property name="Age" column="Age"/>

<subclass name="Patient" table="PatientTab"> ... <set name="suffers" ...> < key column="patientId"> <one-to-many class="Symptom"/> </set> </subclass> </class></hibernate-mapping>

Then you can directly save and load

27

Session session = HibernateUtil.getSessionFactory().getCurrentSession()

session.beginTransaction() Patient p = ... session.save(p)

session.getTransaction().commit()

List result = session.createQuery("from Patient as r where r.insurance != null").list()

Patient q = (Patient) session.load(Patient.class, 101)

If MI / SI mismatch is an issue ...

Problem: my application uses a MI OO language, buy my persistence technology only supports SI.

Map MI to SI : By representing inheritance as association. Or by flattening

Your DAM will have to implement the back and forth mapping between MI PD and SI OODB.

28

But perhaps we should avoid this kind of mismatch, unless we can find a tool that can do the mapping automatically.

Example of approach 1

29

Person-------------Name Age

Patient---------------IDInsurance

Chat----------------Nickname

ChatBot

PersonODB-----------------Name Age

PatientODB---------------IDInsurance

ChatODB----------------Nickname

subObjOf

1..1

1..1

AutomatedPrg

ChatBotODB

AutomatedPrgODB

subObjOf

1..1

1..1

XOR

PD classes OODB classes

Example of approach 2

30

Person-------------Name Age

Patient---------------IDInsurance

Chat----------------Nickname

ChatBot

PersonODB-----------------Name Age

PatientODB---------------IDInsuranceNickname

AutomatedPrg

ChatBotODB--------------------Nickname

AutomatedPrgODB

PD classes OODB classes

Optimizing your RDB

31

duplicated information wasted space

Normalization

Decomposing your initial DB schemes into a new set of schemes: optimal e.g. no unnecessary duplication and no wasted

space we can still reconstruct the original schemes

You may want to recall the DB course. Here we'll discuss

1st Normal Form (1NF) 2NF 3NF

32

1NF

A table T is in 1NF if : No attribute a2 in T that actually is a duplication of attribute a1.

T has no empty cell.

33

OrderNr CustID Name State Tax PrdNr1 Desc1 ProdNr2 Desc2

239 1035 Black MD 0.05 555 Tray

241 1123 William CA 0.08 444 Wine

290 1123 William CA 0.08 555 Tray

237 2242 Berry DC 0.06 111 Guide 444 Wine

234 2242 Berry DC 0.06 555 Tray

This table is not 1NF

Table Order

34

OrderNr CustID Name State Tax PrdNr1 Desc1 ProdNr2 Desc2

239 1035 Black MD 0.05 555 Tray

241 1123 William CA 0.08 444 Wine

290 1123 William CA 0.08 555 Tray

237 2242 Berry DC 0.06 111 Guide 444 Wine

234 2242 Berry DC 0.06 555 Tray

OrderNr CustID Name State Tax

239 1035 Black MD 0.05

241 1123 William CA 0.08

290 1123 William CA 0.08

237 2242 Berry DC 0.06

234 2242 Berry DC 0.06

OrderNr PrdNr Desc

239 555 Tray

241 444 Wine

290 555 Tray

237 111 Guide

237 444 Wine

234 555 Tray

Org. Table Order

New Table Order Table ProductOrder

2NF A table T is in 2NF if

it is in 1NF no nonkey attribute in T depends on only a part of T's

primary key.

35

OrderNr PrdNr Desc

239 555 Tray

241 444 Wine

290 555 Tray

237 111 Guide

237 444 Wine

234 555 Tray

Old Table ProductOrder

PrdNr Desc

OrderNr PrdNr

239 555

241 444

290 555

237 111

237 444

234 555

PrdNr Desc

555 Tray

444 Wine

111 Guide

New ProductOrder Table Product

3NF transformation

36

OrderNr CustID Name State Tax

239 1035 Black MD 0.05

241 1123 William CA 0.08

290 1123 William CA 0.08

237 2242 Berry DC 0.06

234 2242 Berry DC 0.06

Old Table Order

OrderNr CustID

239 1035

241 1123

290 1123

237 2242

234 2242

CustID Name State Tax

1035 Black MD 0.05

1123 William CA 0.08

2242 Berry DC 0.06

Table Order

Table Customer

A table T is in 3NF if it is in 2NF every attribute in T

directly depends on T's primary key.

Transformation

Note that we do not ideally normalize/transform tables. We ideally apply normalization at the design phase on our DB schemes, which form the structural design of an RDB.

37

ProductOrder : (OrdNr, PrdNr, Desc)

ProductOrder : (OrdNr, PrdNr) Product : (PrdNr, Desc)

(OrdNr, PrdNr) is primary keyPrdNr Desc

3NF

38

CustID Name State Tax

1035 Black MD 0.05

1123 William CA 0.08

2242 Berry DC 0.06

5555 Spons DC 0.06

Old table Customer

CustID Name State

1035 Black MD

1123 William CA

2242 Berry DC

5555 Spons DC

State Tax

MD 0.05

CA 0.08

DC 0.06

Table Customer

table State

State Tax

So ... After 3NF we optimize space usage :

No wasted empty cells No unnecessarily duplicated information

But now information is scattered over multiple tables

We rely on joins

E.g. to know which customers order wine we have to query on:

will cost time!

39

Customer join Order join ProductOrder join Product

Optimizing data access speed Clustering see book Indexing below Denormalization

40

CustID Name State

1035 Black MD

1123 William CA

2242 Berry DC

5555 Spons DC

6666 Octo MD

Table Customer

State Pointer

MD

MD

CA

DC

DC

Customer-State Index

Denormalization

putting back some redundancy to make certain things faster.

41

OrderNr PrdNr Desc

239 555 Tray

241 444 Wine

290 555 Tray

237 111 Guide

237 444 Wine

234 555 Tray

Old Table ProductOrder

OrderNr PrdNr

239 555

241 444

290 555

237 111

237 444

234 555

PrdNr Desc

555 Tray

444 Wine

111 Guide

New ProductOrder Table Product

Remember we did this 2NF normalization:

denormalize

Estimating storage size Scheme of ProductOrder

OrdNr : INT(10)PrdNr : INT(10)Desc : VARCHAR(160)

Average row size 100 bytes #row now 106 growth: 100 new orders per day

This is a crude estimation.

42

Size now 100 MBAfter 3 years 110 MB