Boosting Locality in

Multi-version Partial Data

Replication

João A. Silva, João M. Lourenço, and

Hervé Paulino

Replicate only a subset of the system’s data at

each node

Explored by many transactional systems [Sou+01,

SSP06, Ser+07, SSP10]

Partial Data Replication

SAC'15, April 13-17 2015, Salamanca, Spain 2

Group2Group1

Node1 Node2

A B C D

Node3

A B

Node4

C D

Partial Data Replication

+ High scalability

potential

+ Better resource

utilization

+ (Parameterizable)

Fault tolerance

- Data partitioning

- More complex

coordination

- Remote data access

SAC'15, April 13-17 2015, Salamanca, Spain 3

Remote data cache

Approach

SAC'15, April 13-17 2015, Salamanca, Spain 4

Coming Next...

ConclusionsExperimental

ResultsOur Solution

SystemModel

SAC'15, April 13-17 2015, Salamanca, Spain 5

System Model

SAC'15, April 13-17 2015, Salamanca, Spain 6

Strong consistency

MVCC

Primary owner

T.tsS

T.tsC

Classical

asynchronous

system

mostRecentTSi

Comprised by 2 components:

Consistency algorithm

Ensures data consistency

Extension mechanism

Maximizes cache effectiveness

Our Solution

SAC'15, April 13-17 2015, Salamanca, Spain 8

< 𝑘, 𝑣𝑎𝑙, 𝑣𝑒𝑟𝑠𝑖𝑜𝑛, 𝑣𝑎𝑙𝑖𝑑𝑖𝑡𝑦 >

Ensuring Data Consistency

SAC'15, April 13-17 2015, Salamanca, Spain 9

version: 2 version: 4 version: 9

validity: 4 validity: 9 validity: mostRecentTS

When reading data item D inside transaction T:

Ensuring Data Consistency

SAC'15, April 13-17 2015, Salamanca, Spain 10

Is D a local data item?

Is D in cache?

Do remoteread

Is there anyvisible version

V of D?

Do remoteread

T.tsS < V.validity?

Do remoteread

Return cachedvalue

Read locallyYes

No

Yes

No

Yes

No

Yes

No

Extension mechanism

Modified Set (mSet) – set of ids of all data

items updated since the last time an mSet

was built and sent

Maximizing Cache

Effectiveness

SAC'15, April 13-17 2015, Salamanca, Spain 11

...version: 9validity: 10

...version: 9validity: 10

mostRecentTS = 12...version: 9validity: 12

mostRecentTS = 12

Extension strategies:

Eager – transaction committed

Batch – periodically

Lazy – remote read request

Extension Mechanism

SAC'15, April 13-17 2015, Salamanca, Spain 12

Extension message:

[𝑚𝑆𝑒𝑡,𝑚𝑜𝑠𝑡𝑅𝑒𝑐𝑒𝑛𝑡]

∀𝑥 ∶ 𝑥 ∈ 𝑐𝑎𝑐ℎ𝑒 ∧𝑚𝑜𝑠𝑡𝑅𝑒𝑐𝑒𝑛𝑡𝑉𝑒𝑟𝑠𝑖𝑜𝑛 𝑥 ∧

𝑥 ∉ 𝑚𝑆𝑒𝑡,𝑚𝑜𝑠𝑡𝑅𝑒𝑐𝑒𝑛𝑡𝑉𝑒𝑟𝑠𝑖𝑜𝑛 𝑥 . 𝑣𝑎𝑙𝑖𝑑𝑖𝑡𝑦 = 𝑚𝑜𝑠𝑡𝑅𝑒𝑐𝑒𝑛𝑡

Extension Mechanism

SAC'15, April 13-17 2015, Salamanca, Spain 13

Shared validity:

𝑚𝑜𝑠𝑡𝑅𝑒𝑐𝑒𝑛𝑡𝑉𝑎𝑙𝑖𝑑𝑖𝑡𝑦[𝑛𝑜𝑑𝑒𝐼𝑑]

Associated with all the versions from nodeId

that are known to be the most recent ones

Extension Mechanism

SAC'15, April 13-17 2015, Salamanca, Spain 14

Extension process:

1. ∀𝑥 ∶ 𝑥 ∈ 𝑚𝑆𝑒𝑡, detach 𝑥 from shared validity

2. 𝑚𝑜𝑠𝑡𝑅𝑒𝑐𝑒𝑛𝑡𝑉𝑎𝑙𝑖𝑑𝑖𝑡𝑦 𝑛𝑜𝑑𝑒𝐼𝑑 = 𝑚𝑜𝑠𝑡𝑅𝑒𝑐𝑒𝑛𝑡

Extension Mechanism

SAC'15, April 13-17 2015, Salamanca, Spain 15

Experimental Results

System

8 node cluster

DTM framework

SCORe [PRQ12]

Benchmarks

Red-Black Tree

SAC'15, April 13-17 2015, Salamanca, Spain 16

What is the impact of the cache mechanism in the:

1. System’s overall throughput?

2. Amount of remote read operations?

Experimental Results

SAC'15, April 13-17 2015, Salamanca, Spain 17

0,4

0,6

0,8

1

1,2

1,4

2 4 6 8

Tra

nsactions

x10

5/s

econ

d

Number of nodes

RBT 0% updates

noCache Batch Eager Lazy

Throughput

Experimental Results

SAC'15, April 13-17 2015, Salamanca, Spain 18

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5

2 4 6 8

Perc

en

tage

Number of nodes

RBT 0% updates

noCache Batch Eager Lazy

Remote Reads

Experimental Results

SAC'15, April 13-17 2015, Salamanca, Spain 19

0,8

0,9

1

1,1

1,2

1,3

1,4

1,5

1,6

2 4 6 8

Tra

nsactions

x10

4/s

econ

d

Number of nodes

RBT 10% updates

noCache Batch Eager Lazy

Throughput

Experimental Results

SAC'15, April 13-17 2015, Salamanca, Spain 20

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5

2 4 6 8

Perc

en

tage

Number of nodes

RBT 10% updates

noCache Batch Eager Lazy

Remote Reads

Experimental Results

SAC'15, April 13-17 2015, Salamanca, Spain 21

Eager – Long read-only transactions, big mSets

Batch – Tradeoff between reduced bandwidth

and delays by batching updates

Lazy – Communication intensive workloads,

small mSets

Experimental Results

SAC'15, April 13-17 2015, Salamanca, Spain 22

Partial replication is severely affected by the

inefficient data placement

Generic caching mechanism

Improvements in the system’s throughput

Reduction of remote read operations

Conclusions

SAC'15, April 13-17 2015, Salamanca, Spain 23

Garbage collection mechanism for old

cached data

Extensive experimental study using several

benchmarks with different workloads and

data access patterns

Future Work

SAC'15, April 13-17 2015, Salamanca, Spain 24

Thank you!Questions?

References[Sou+01] A. Sousa, R.Oliveira, F . Moura, and F. Pedone. “Partial Replication

in the Database State Machine”. NCA 2001.

[SSP06] N. Schiper, R. Schmidt, and

F. Pedone. “Optimistic Algorithms for Partial Database Replication”. Principles

of Distributed Systems 2006.

[Ser+07] D. Serrano, M. Patiño-Martinez, R. Jimenez-Peris, and

B. Kemme. “Boosting Database Replication Scalability through Partial

Replication and 1-Copy-Snapshot-Isolation”. PRDC 2007.

[SSP10] N. Schiper, P. Sutra, and F. Pedone. “P-Store: Genuine Partial

Replication in Wide Area Networks”. SRDS 2010.

[PRQ12] S. Peluso, P. Romano, and F. Quaglia. “SCORe: A Scalable One-

Copy Serializable Partial Replication Protocol”. Middleware 2012.

SAC'15, April 13-17 2015, Salamanca, Spain 26