Analysis of Replicated Data with Repair Dependency

1

Analysis of Replicated Analysis of Replicated Data withData with

Repair DependencyRepair Dependency

ING-RAY CHEN AND DING-CHAU WANG

Presented By Weiping He and Li-Yuan Kuan

2

INTRODUCTION

Pessimistic control algorithms for replicated data permit only one partition to perform update operations at any given time so as to ensure mutual exclusion of the replicated data object.

3

INTRODUCTION(contd)

Existing availability modelling and analyses of pessimistic control algorithms for replicated data management are constrained to either site-failure-only or link-failure-only models, but not both.

4

INTRODUCTION(contd)

Using stochastic Petri nets, this paper investigates the effect of repair dependency which occurs when many sites and links may have to share the same repairman due to repair constraints.

5

Four repairman models in the paper

(a) independent repairman with one repairman assigned to each link and each node;

(b) dependent repairman with FIFO servicing discipline;

(c) dependent repairman with linear-order servicing discipline;

(d) dependent repairman with best servicing discipline.

6

SYSTEM MODEL AND ASSUMPTIONS

Background Dynamic voting uses current up to

date copies at any time for deriving its dynamic quorum set. the system can dynamically adjust its quorum set in response to state changes and it results in an increase in availability.

7

Background (contd)

Suppose a file f is replicated to n copies, stored at sites S1, S2, ….Sn.

The essence of dynamic voting is that it must keep track of the number of up to date copies involved in the last update, and also which copies are up to date.

8

Background (contd)

Each copy is associated with 3 variables version number (VN): to tell if the local copy is current site cardinality(SC): the number of current

copies that participated in the last update

distinguished site (DS): stores the ID of the highest linearly ordered site among all sites that presently store current copies.

9

Background (contd)

(VNi, SCi , DSi) be the set associated with the copy stored in site Si For all Si, initial states are Vni= 0,

Sci =n Dsi= n. Dsi is initialized to n because Sn

is the highest linearly ordered site among all initially.

10

Background (contd)

Process When updating, a site C (coordinator)

requests all sites that have a copy of f to send their values of (VNi, SCi , DSi).

P : the set consisting of C and all subordinates that responded to the request; each site in P locks its copy of the data item f during the process.

11

Background (contd)

Site C then inspects the largest version number VNp in set P .

I : the subset containing a copy with version number VNp.

SCp , DSp : the value of SC and DS in set I

12

Background (contd)

site C is in the major partition if(1) the cardinality of I is larger than one half of SCp;

(2) the cardinality of I is exactly equal to one half of SCp and set I contains site DSp.

If site C is in the major partition , it can commit the update locally.

13

Background (contd)

(VN, SC, DS) is set to (VNp+1, the cardinality of set P , the ID of the highest linearly ordered site found in P ).

14

Assumptions

(1) there are n sites connected by a topology to be specified. Each site is assigned a single vote and a unique site ID, numbered 1,2 …n. where n is the total number of sites in the system. site n is the highest linearly ordered site.

15

Assumptions (contd)

(2) sites and links have independent failure rate λs and λl.

(3) A repairman repairs a failed site with rate μs and a failed link with rate μl .

(4) times between these events are exponentially distributed.

16

Independent repairman model

Site subnet A site can be in

one of four states up and

current(upcci) up and out-of-date

(upoci) down and current

(downcci) down and out-of-

date (downoci)

17


Suppose site i initially is in the state of up and current: #.upcci = i, #.upoci = 0, #.downcci = 0

and #.downoci =0 an update occurs and site i is not in

the major partition. upcci upoci #.upcci = 0, #.upoci = i, #.downcci = 0

and #.downoci =0

18


upcci upoci: (t5i ; 3; gi ) t5i is the name

of the transition, 3 is the priority of transition current, gi is the enabling function

19


a transition is enabled when (1) input places contains tokens

>= the multiplicity of the input arc.

(2) enabling function returns TRUE.

the transition with the highest priority level will fire first.

20


When readyi =1, upcci = i, the enabling function return TRUE:

transition t5i will be fired , tokens in readyi and upcci will be removed and i tokens will be added to upoci. i is now in the state of up and out of date.

21


When an update arrives and a major partition exists, a token will be put into readyi. only one out of the six transitions can fire,all have the same priority level

22


t0i will fire if site i is in down and current. #i Downccidownoci, an update occurs and site i is in down and current, new state will be down and out of date.

Site i is down can’t attend in the update.

23


t1i will fire if site i is in down and out of date.

Site i remains in down and out of date..

24


t2i will fire if site i is in up and out of date,.

upoci upcci, new state will be up and current,

t2i’s enabling function gi returns TRUE if site i is in the major partition.

25


t3i will fire if site i is in up and current.

upcci upcci, new state will be up and current.

enabling function gi

26


t4i will fire if site i is in up and out of date, and not in the major partition

upoci upoci, new state will be up and out of date.


27


t5i will fire if site i is in up and current, and not in the major partition.

upcci upoci, new state will be up out of date.


28


all transitions in are immediate transitions.

29

System subnet

At the bottom of Figure 2, each of the boxes labeled site i is the SPN subset shown in Figure 1.

30

System subnet

Transitions tf and tfbar are given the highest priority levels (5 and 4).

When an update event arrives, a token will be put in place update event.

31

System subnet

tf is evaluated prior to tfbar since it has the highest priority.

If a major partition exists, tf will fire, remove the token in update_event and place one token in all Is.

32

System subnet

If a major partition does not exist, tfbar will fire, remove the token in update_event.

33

System subnet

The next highest priority level (i.e. 3) is assigned to each of the boxes.

34

System subnet

After all sites are evaluated and each site's status are updated, tds and tsc which have lower priority levels will execute.

35

System subnet

tsc updates the site cardinality.

tsc sc, multiplicity is the number of sites with mark (upcc) >0 in the major partition.

If update changes the system status, the site cardinality is stored as the number of tokens in place sc.

36

System subnet

tds updates the distinguished site.

tds ds, multiplicity is maximum #(upcc) value among all the sites in the major partition.

After update, the site ID of the new distinguished site will be stored as the number of tokens in place ds.

37

Site failure/repair subnets

This figure describes the effect of site i’s failure and repair on the system state, for the independent repairman model.

38


site i can only be in one state at a time, only one transition out of these two subnets is possible at a time.

39


failure events: upcci downcci

tccfi λs or upoci downoci

tocfi λs

repair events: Downcci upcci

tccri μs downoci upoci

Tocri μs

40


system will update its status upon every failure or repair event.

It ensures that the system state is updated before another failure or repair event occurs.

41

Link failure/repair subnets

subscript ij refer to the link between i, j.

failure events: uplinkij dwlinkij

tlinkfi λl

repair events: dwlinkij uplinkij

tlinkri μl

42

Meanings of places.

43

Arc multiplicity functions.

44

Enabling functions.

45

FIFO repairman model

we can make use of the independent repairman model and modify the repair rates to account for repair dependencies.

repair rate now becoming a function of the total number of failed sites and links.

46


If a state has two failed sites and one failed link, independent

repairman model, repair rates are μs, μs and μl,

FIFO repairman model, repair rates are μs/3, μs/3 and μl/3,

47


In general, if a state has M failed sites and links, then the respective repair rates of these failed entities in that state are “deflated” by a factor of M to account for the effect of repair resource sharing.

48

Linear-order repairman model

there is a prespecified order among failed entities.Because of selecting the distinguished site (DS), it is naturally to follow that order for repair.

a higher linearly ordered site will become a distinguished site in the event that the number of sites within the major partition is an even number. Therefore, giving a higher repair priority to a higher linearly ordered site increases the chance of finding the majority partition.

49


Example: a state in which there are only two

sites D and E left in the major partition with E being the distinguished site. Suppose sites D and E subsequently fail. Then, repairing D would not result in a major partition being found while repairing E would.

50


By making use of the independent repairman model, create a new enabling function associated with the transition of each site or link repair event as in the right table

51


Only one enabling function at any state shall return TRUE based on the prespecified linear order and all others shall return FALSE.

52


Example:

a 5-site ring topology linear order is (sites 5,4,3,2,1; links 54,51,43,32,21).

sites 4, 2 and link 51 are down

53


enabling functions associated with sites 4, 2 and link 51 will return TRUE, FALSE and FALSE, meaning 4 will be repaired next over site 2 and link 51.

54

What is system

availability?

What is site

availability?

System availability: The steady-state probability that a major partition exists.

Site availability: The probability that an update arriving at an arbitrary site will succeed.

55

Best-first repairman model

Linear-order repair order is fixed at run time.

Best-first repairman repair order is not fixed.

Preference given to the site or link which can most improve the site availability of the system after its repair with respect to the current state.

56

Best-first repairman model- Repair Strategy

If the repair of a distinct failed site or link would lead to the existence of a major partition, then the distinct failed site or link will be selected to be repaired next.

If more than one failed site or link whose repair would lead to the existence of a major partition, then a tie-breaker rule will be applied to select one distinct member of the group to be repaired next.

major partition: the number of current copies is the largest among all possible repair choices in that state

57

Repair StrategyTie-breaker rules

(Both failed sites and links) if μs/λs>=μl/ λl, preference is given to the failed sites, otherwise given to the failed links.

(Failed sites) preference is given to the highest linearly ordered in the group.

Preference is given to the failed entity which can be repaired fastest.

A higher linearly ordered site is more likely to become the distinguished site.

58

How to increase the probability of

uniting more sites in the major

partition?

Repair sites or links Repair sites or links closer tocloser to

the current distinguished sitethe current distinguished site

59

Tie-breaker rules-Example

Example: five-site ring system major partition contains only

the distinguished site, site4. all other sites and links have

failed. repairing any of the failed

sites or links yields the same availability improvement.

60

better to unite

sites 4 & 5

No benefit

Failed site only

61

Repair StrategyTie-breaker rules (contd)

(Failed links) preference is given to the link that connects the highest linearly ordered sites in the group.

if μs/λs<μl/ λl, repairing links takes precedence over repairing sites.

Repair links around current distinguished.

62

Higher linearly ordered site has a higher probability to become the distinguished site

Repair a link connecting higher linearly ordered site

Put the system in a better position to unite more sites

63

better to unite

sites 4 & 5

No benefit

Failed link only

Example

64

Best-first repairman model

Similar to linear-ordered repairman model.

Associate an enabling function with the repair transaction of each site or link.

Only one enabling function can return TRUE at any state.

65

Implementations

Modeling techniques was tested with five-site ring topology.

Four repairman models: Independent repairman model Dependent model with FIFO Dependent model with linear-order Dependent model with best-first

Using SPNP on SUN Ultra1

66

Implementations - Models

IndependentDependent

FIFODependent

linear-orderDependent best-first

Figures 1, 2, 3, 4 1, 2, 3, 4 1, 2, 3, 4 1, 2, 3, 4

Tables 1, 2, 3, 4 1, 2, 4, 5 1, 2, 3, 4, 6 1, 2, 3, 4, 7

Markov chain states

8674 8674 5429 3821

67

Implementations - System and Site Availability metrics

Definition Reward Assignment

System

The steady-state probability that a major partition exists.

Reward rate = 1:

For those states in which enabling function f() is evaluated to TRUE.

Reward rate = 0: otherwise

Site

The probability that an update arriving at an arbitrary site will succeed.

Reward rate = 1*k/n:

For those states in which enabling function f() is evaluated to TRUE.

Reward rate = 0: otherwise

k: # of ‘up’ site in the major partition in a particular state; n: total site # in a system

68

Results (Indep.)

l:l siteA s:s siteA Increase rates of siteA

slow down as l:l or s:s

Either only fallible links (or fallible sites, not both) will overestimate the site availability metric unrealistically.

MTTF:MTTR

[2]

?

[2] Jajodia, S. and Mutchler, D. (1990) Dynamic voting algorithms for maintaining the consistency of

replicated database. ACM Trans. Database Systems, 15, 230-280.

69

Results (dep.)

Site availability with independent repair is higher than those with dependent repair.

Only when l:l is relatively high can the availability of best-first repairman model approximate that of the independent repairman model.

70

Results (dep., contd)

System availability is higher than than the corresponding site availability under the same l:l and s:s ratio.

Independent > Best-first > Linear-Order > FIFO for system and site availability under the same l:l and s:s

ratio.

71

Results - Three Dependent Repairman Models

Best-first repairman model can always provide a better site availability than the other two.

Avoid unfavorable states resulting from ineffective repair activities (major partition not exist).

Unfavorable states occur when any of the following occurs:

(a) One of more up but out-of-date copies.

(b) One up and current copy (not the distinguished copy) + (a). Avoid these states by repairing the distinguished sites first: a major

partition always results. (assumption: frequent updates.)

72

Results - Three Dependent Repairman Models (contd)

Best-first repairman model can always provide a better site availability than the other two.

Choose a failed site or link to repair such that the size of the major partition after repair will become the largest among all possible choices. (major partition exists).

Eliminate ineffective repair.

73

Results - Three Dependent Repairman Models (contd)

Two caseseffectivebest-first (size 4)

ineffectivelinear-order/FIFO (size 3)

Repairing any member of a group results in the same availability improvement, best-first will select the one with the shortest repair time. So that most of time the system can stay in the in states with high availability.

Case 1

Case 2

74

Results - Difference in Availability

Assumption: s = l = ; s = l =

75

Results - Difference in Availability

When ratio / >20: different among these three model is small.

Sites/links can be repaired much faster than they could fail.

When ratio / is 5~15: difference becomes larger.

the effective repairs have more prominent impact.

When ratio / <1: different among these three model is small.

Sites/links can be failed much faster than they could repair. The effective repairs become less significant.

76

Conclusions

a significant difference in availability exists between two systems with independent and dependent repairman models, except when the repair rate is much higher than the failure rate (i.e. when / >20)

when several sites and links have to share the same repairman, the best-first repairman model can always provide a better availability than the FIFO and linear-order repairman models

77

Conclusions (contd)

The difference in availability becomes more pronounced as the site/link failure rate increases relative to the site/link repair rate (i.e. / );

An availability model that considers only site or link failures/repairs, but not both, can give a very unrealistic, overestimated value of the availability metric.

Ignoring concurrent site/link failure modes or repair dependency can overestimate the availability of replicated data.

78

Future Work

Extend and apply the modeling techniques to analyzing other pessimistic algorithms.

Develop modeling techniques to study the trade-off between the reduction in data processing overheads and the sacrifice in availability for database environments where a combined performance/availability design goal must be met.

79

Thank you!

Analysis of Replicated Data with Repair Dependency

Documents

Transcript of Analysis of Replicated Data with Repair Dependency