Using the Transformational Approach to Build a Safe and Generic Data Synchronizer

Pascal Molli, Gérald Oster, Hala Skaf-Molli, Abdessamad

ImineECOO and Cassis Project

Inria Lorraine

Context Mobile computing

Before disconnection : replication When reconnect : synchronization

Groupware Before insulation : replication before publishing : reconciliation

Synchronizers… File synchronizer

Unisson, IntelliSync, Microsoft Synchronizer Data Synchronizers

ActiveSync, Hotsync, I-Sync Merge tools and CM Tools

CVS, rcsmerge, Diff3, XyDiff, XmlDiff… Distributed System

CODA, Bayou, Ficus, IceCube Replicated database

Asymetric replication

Synchonization Copies are identical after

synchronization… But in which state ??

At which level of granularity ?? File system level, file content level

For which data type ?? XML, text, file system, …

Who is resolving conflict ? Synchronizer, user, admin

And when ? During synchro, after ??

Safety… At least, convergence… Not true for a lot of actual

synchronizer Propagate non conflictual update and

delegate the others to users Synchronizers stops with errors in

case of conflicts

Synchronizing All copies are identical after

synchronization Synchronizers have to solve

conflicts that occurs during synchronization…

A critical application… Has to be safe…

tata

>>> add >>> Thu Jan 01 00:00:00 CET 1970 : :

>>>> add >>> Mon Nov 10 00:00:00 CET 2003 : localhost.users.momo54:c51e0b4fa9fe407401df27be35416c45

>>zidane

>>abdou

>====== Mon Nov 10 15:02:18 CET 2003 : localhost.users.momo54:c520825fa9fe4074005ddc65434fbba3

>>melchior

>>balthazar

><<< add <<<

====== Thu Jan 01 00:59:59 CET 1970 : :

>>>> add >>> Mon Nov 10 00:00:00 CET 2003 : localhost.users.momo54:c51e0b4fa9fe407401df27be35416c45

>>zidane

>>abdou

>====== Mon Nov 10 15:02:35 CET 2003 : localhost.users.momo54:c52307b7a9fe4074007525ad12ae999b

>>riri

>>fifi

>>loulou

><<< add <<<

<<< add <<<

Objectives Define a correctness criteria for

synchronization (safety) Provide an unique algorithm for

synchronizing any kind of data (genericity)

Prove correctness of our new synchronizer

Problems Safety

Ensure convergence in all cases... Without loosing data…

Genericity/Granularity Reconcile at all level of granularity

for any kind of data…

Idea Transformational Approach with

real-time editors… A real-time Synchronizer:

Replication of data… Data can diverge due to network

latency (650ms)

Transformational approach Consistency defined!

convergence, causality, intention preservation

Safety… Integration algorithm is

independant from data type Granularity/Genericity

Real Time Editor/Synchronizer : Just the value of the delay

Operational Transformation n sites each site has a copy of shared

data One operation:

generated and executed on one site broadcasted to other sites received by other sites re-executed on other sites

efect

effect

efect

efects

Ins(5,s)Ins(2,f)

effecst effects

Ins(5,s) Ins(2,f)

Op1 Op2

S1 o Op1

State S1 State S1

efect

effect

efect

efects

Ins(5,s)Ins(2,f)

effects effects

Ins(6,s) Ins(2,f)

T(Ins(5,s),Ins(2,f))=

Ins(6,s)

Op1 Op2

Op’2

Site 1 : user 1 Site 2 : user 2

State S1

S1 o Op1

State S1

Correctness of transformation function Algorithms (Goto,SOCT2,3,4,

Adopted…) ensure CCI if… …Transformation functions ensure

at Least :

What we did… Rewrite SOCT4 algorithm for

synchronization needs Develop transformation functions

for the file system and text files… Others data type can be

integrated…

The SOCT4 Algorithm

• transposing forward opi with local operations• transposing forward local operations with opi’• putting opi at the right place

- backward transposition unnecessary- state vectors no needed

Beforeintegration

local operationswaiting for broadcast

opi-1

opi

op1

op2

. . . . . . . .

opL

opL m1

Afterintegration

Integrationof opi

☛ Sequential reception + Differed broadcast

opi

opop

iop

op1

op2

. . .. . . . .

opL

opL

opi-1

opim1

’ ’forward transposed

local operations

op1 m

op2

. . . . . . . .

opL

opL 1

opi’op

i-1

opi

Synchronization example

Logs after reconciliation (S3) Site1 Site2

A channel with 3 workspaces

T(opl1,op3)=opl’1T(op3,opl1)=op’3

Opl’1 opl2Op’3 T(opl2,op’3)=opl’2T(op’3,opl2)=op’’3

Opl’1 Opl’2 Op’’3

Site « Hala », Ns=2, Synchronize !

Execute(op’’3)Ns=Ns+1, getOp ?No more remote op

Send(opl’1)Send(opl’2)

opl1 opl2

Log[0]=opl1,log[1]=opl2,

op3getOp(Ns+1)=op3

What we have done now… Write transformation functions for

file system and text file content In case of conflict -> authorize

compensation… (no lost operation) Prove C1 on transformation

function Specified it in first order logic Use the SPIKE automatic theorem

prover to automatically prove it [ECSCW03]

riri

fifi

>>> add >>> Sun Nov 09 00:00:00 CET 2003 : localhost.users.momo54:c03328027f0000010152997671df9f2d

>melchior

>balthazar

====== Sun Nov 09 18:34:11 CET 2003 : localhost.users.momo54:c03251f47f000001012803bab735f091

>zidane

>abdou

<<< add <<<

loulou

Example for file system/ /

mf(1,0,a) md(1,0,a)

//a

//a

mf(1,0,a#1)

//a

/a#1

md(1,0,a);mv(1,0,a,1,0,a#1)

//a

/a#1

op1 op2

Example: état initial

Dir:0

a

1:gaspardmelchiorbalthazar

c

Scenario

synchronize

synchronize

synchronize

synchronize

synchronize

Ab(b,0,zidane)Db(b,2, melchior, balthazar)

Ab(a,3,abdou)Mf(b)Mv(/a,/b)

U3U2U1

État final

Dir:0

b1

1:gaspard>>melchiorbalthazar=abdou<<

2:

zidane

b c

http://www.libresource.org

Conclusion One synchronizer for all data types Convergence is achieved in all

cases It relies on a formal framework Transformation functions are

proven An original way to use the

tansformational approach

Commit Client...

Update Client...

Perspectives:Synchronization networks

Example scenario

Renaming for resolving conflicts How to compute new unique

name ?? Mf(id,pid,name,)

mf(1,0,a,{a}) md(2,0,b,{a,b}) mf(3,2,a,{a})

Max(s)+id is unique

File System Representation

mf(1,0,a) md(2,0,b) mf(3,2,a)

mv(2,4,b,0,c)