Information Processing Letters 80 (2001) 105–111

Impossibility of scalar clock-basedcommunication-induced checkpointing protocols

ensuring the RDT property

Roberto Baldonia, Jean-Michel Hélaryb,∗, Achour Mostéfaouib, Michel Raynalba DIS, University La Sapienza, Roma, Italy

b IRISA, Campus de Beaulieu, 35042 Rennes-cedex, France

Received 7 February 2000; received in revised form 20 September 2000Communicated by J.L. Fiadeiro

Abstract

Communication-induced checkpointing protocols constitute an interesting approach to the on-line determination ofcheckpoint and communication patterns enjoying desirable properties such as domino-effect freedom. They do not add controlmessages to the computation, but instead may attach control information to computation messages. Among these protocols,scalar clock-based protocols are particularly attractive as they use a single integer as control information.

An interesting property of checkpoint and communication patterns is Rollback-Dependency Trackability, which ensures thatall local checkpoint dependencies are on-the-fly trackable. So, it would be nice to design scalar clock-based communication-induced checkpointing protocols providing the RDT property, a previously open question. This paper shows that the design ofsuch protocols is impossible. 2001 Elsevier Science B.V. All rights reserved.

Keywords:Distributed computing; Checkpoint; Communication-induced checkpointing protocol; Domino-effect freedom;Rollback-dependency trackability; Scalar clock

1. Introduction

A local checkpointis a snapshot of a local stateof a process, aglobal checkpointis a set of localcheckpoints, one from each process, and aconsistentglobal checkpointis a global checkpoint in which noone happens-beforeanother [11]. The determinationof consistent global checkpoints is a fundamentalproblem in distributed computing. When processes

* Corresponding author.E-mail addresses:baldoni@dis.uniroma1.it (R. Baldoni),

helary@irisa.fr (J.-M. Hélary), mostefaoui@irisa.fr(A. Mostéfaoui), raynal@irisa.fr (M. Raynal).

independently take local checkpoints there is a riskthat no consistent global checkpoint can ever beformed (but the first trivial global checkpoint which ismade up of the process initial states). This is caused bythe well-knownunbounded domino effect[15]. Evenif consistent global checkpoints can be formed, it isstill possible that some local checkpoints may never beincluded in a consistent global checkpoint; such localcheckpoints are calleduseless.

As consistency of global checkpoints depends onthe flow of messages exchanged during a computation,on-the-fly determination of consistent global check-points requires some coordination between processes

0020-0190/01/$ – see front matter 2001 Elsevier Science B.V. All rights reserved.PII: S0020-0190(01)00145-4

106 R. Baldoni et al. / Information Processing Letters 80 (2001) 105–111

when they take their local checkpoints [5,9]. This co-ordination, executed bycheckpointing protocols, re-sults in two kinds of local checkpoints. A local statedefined as a local checkpoint by a process on its owninitiative is called abasic checkpoint. If the defini-tion of a local checkpoint is due to the coordination,it is called aforced checkpoint. The execution of adistributed computation, extended with a checkpoint-ing protocol, produces aCheckpoint and Communica-tion Pattern (CCP) [6], which consists of the set oflocal checkpoints plus a dependency relation on thosecheckpoints. Checkpointing protocols have been de-signed in order that the CCPs produced by their ex-ecutions satisfy some desirable properties. The “min-imal” property requires that the CCP has no uselesscheckpoint. At the conceptual level this is equivalentto the absence of cycles in a dependency graph on lo-cal checkpoints. That is why this property is calledNoZ-Cycle(or NZC). A stronger property, introduced byWang [20], who called itRollback-Dependency Track-ability (RDT), has also been considered. RDT requiresthat all dependencies between local checkpoints becausal (i.e., be on-the-fly trackable). In CCPs satis-fying the RDT property, one can calculate very ef-ficiently the minimum and the maximum consistentglobal checkpoints that contain a given set of localcheckpoints. This allows to develop simple and dis-tributed solutions to garbage collection, recovery linecomputation and output commits using a checkpoint-based rollback-recovery technique (see [20] for moredetails on the benefits of the RDT property).

This paper considers the family ofCommunication-Induced Checkpointing(CIC) protocols. This familyis characterized by the fact that no control messageis added to computations. Instead, protocols of thisfamily use application messages to piggyback controlinformation. When a process receives a message ituses this control information to know whether it has totake a forced checkpoint before delivering the message[1,2,7–9,13,20].

The CIC protocols maintain control information onthe past CCP in order to ensure the NZC or RDTproperty. Among them, there is a particular family ofprotocols that we callscalar clock-basedprotocols. Inthis family, the only control information maintainedby processes and carried by messages is a Lamport-like (scalar) clock. This protocol family is particularlyinteresting because it has a low overhead in terms of

control information size (one integer per message andper process).

“Scalar clock-based” CIC protocols [1,4,8,13,19]and “Non-scalar clock-based” CIC protocols [3,8,7]ensuring the NZC property have been designed. But,while “non-scalar clock-based” CIC protocols havebeen designed to ensure the RDT property (e.g., theprotocols described in [2,16,20]), so far no scalarclock-based CIC protocol ensuring the RDT propertyhas been designed. This observation sets the followingquestion: “Is it possible to design scalar clock-basedCIC protocols that ensure the RDT property?”. Thispaper shows that the answer is negative.

The rest of the paper is organized as follows.Section 2 presents the model of computation and afew checkpoint-related definitions. Section 3 states theNZC and RDT properties. Section 4 defines the twoclasses of CIC protocols (scalar clock-based protocolsversus non-scalar clock-based protocols). Section 5states and proves the impossibility result. Finally,Section 6 concludes the paper.

2. Distributed computations, checkpoints anddependencies

2.1. Distributed computations

A distributed computation consists of a finite setP of n processes{P1,P2, . . . ,Pn} that communicateand synchronize only by exchanging messages. We as-sume that each ordered pair of processes is connectedby an asynchronous directed logical channel whosetransmission delays are unpredictable. Processes donot share a common memory, and there is no boundon their relative speeds.

A process executes internal, send, and deliver state-ments. Executions of an internal (respectively send ordeliver) statement is modeled by an internal (respec-tively send or deliver) event. Each processPi is se-quential. In other words,Pi produces a (finite or infi-nite) sequenceof eventsei,1 . . . ei,s . . . . Every processPi has an initial local state denotedσi,0. The local stateσi,s (s > 0) results from the execution of the sequenceei,1 . . . ei,s applied to the initial stateσi,0.

Let H be the set of all the events produced by a dis-tributed computation. This computation is represented

by the partially ordered setH = (H,hb→), where

hb→

R. Baldoni et al. / Information Processing Letters 80 (2001) 105–111 107

denotes the well-known Lamport’shappened-beforerelation [11].

2.2. Local and global checkpoints

2.2.1. Local checkpointsA local checkpointC is a recorded state (snapshot)

of a process. Not every local state is necessarilyrecorded as a local checkpoint, so the set of localcheckpoints is only a subset of the set of local states.

Definition 2.1 [6,7]. A checkpoint and communica-tion pattern(CCP in brief) is a pair(H ,CH ) whereH

is a distributed computation andCH is a set of localcheckpoints defined onH .

Ci,x represents thexth local checkpoint of processPi . The local checkpointCi,x corresponds to somelocal stateσi,s with x � s. Fig. 1 shows an example ofa checkpoint and communication pattern. (This figureuses the usual space-time diagram. Local checkpointsare indicated by black rectangular boxes; the otherlocal states are not explicitly indicated.) We assumethat each processPi takes an initial local checkpointCi,0 (corresponding toσi,0), and after each event acheckpoint will eventually be taken.

A messagem sent by processPi to processPj iscalledorphanwith respect to the ordered pair of localcheckpoints(Ci,x,Cj,y ) if the delivery ofm belongsto Cj,y but its sending event does not belong toCi,x .An ordered pair of local checkpoints isconsistentifthere are no orphan messages with respect to this pair.For example, Fig. 1 shows that the pair(Ck,1,Cj,1)

is consistent, while the pair(Ci,2,Cj,2) is inconsistent(because of the orphan messagem5).

Fig. 1. A checkpoint and communication pattern.

2.2.2. Global checkpointsA global checkpointis a set of local checkpoints,

one from each process. For example,{Ci,1,Cj,1,Ck,1}and {Ci,2,Cj,2,Ck,1} are two global checkpoints de-picted in the Fig. 1.

Definition 2.2. A global checkpoint isconsistentif allits pairs of local checkpoints are consistent.

For example, Fig. 1 shows that{Ci,1,Cj,1,Ck,1}is a consistent global checkpoint, and due to theinconsistent pair(Ci,2,Cj,2), the global checkpoint{Ci,2,Cj,2,Ck,1} is not consistent.

2.3. Checkpoint dependencies

Netzer and Xu [14] have stated that two localcheckpoints cannot belong to a same consistent globalcheckpoint if and only if they are related by some de-pendency relation. More precisely, there are two kindsof dependencies:causal dependency(C-dependency)

and zigzag dependency(Z-dependency), denotedC→

andZ→, respectively.

Definition 2.3. Ci,xC→ Cj,y if the last event occurring

on Pj beforeCj,y causally depends on the first eventoccurring onPi afterCi,x .

In other words,Ci,xC→ Cj,y if and only if i = j

and x < y or there exists a sequence of messagesm1, . . . ,mk such thatsend(m1) occurs onPi after

Ci,x , send(m1)hb→ deliver(mk), and deliver(mk) oc-

curs onPj beforeCj,y . Remark thatC→ is a strict par-

tial order onCH , and thus for any local checkpointC,

we have¬(CC→ C). For example, in Fig. 1 we have

Ci,0C→ Ck,2.

Definition 2.4. Ci,xZ→ Cj,y if there exists a sequence

of local checkpointsCk1,z1, . . . ,Ckα,iα (α � 0) suchthat:

Ci,xC→ Ck1,z1 ∧ Ck1,z1−1

C→ Ck2,z2

∧ · · · ∧ Ckα,zα−1C→ Cj,y .

108 R. Baldoni et al. / Information Processing Letters 80 (2001) 105–111

Remark that

Ci,xC→ Cj,y ⇒ Ci,x

Z→ Cj,y .

A Z-dependencywhich is not a C-dependency is calleda “hidden” dependency, because it cannot be tracked

on-line. For example, in Fig. 1 we haveCk,0Z→ Ci,2

but¬(Ck,0C→ Ci,2). 1

Definition 2.5. A Z-cycle is a Z-dependency from a

local checkpointCi,x to itself:Ci,xZ→ Ci,x .

We can observe that a Z-cycle fromCi,x to itselfnecessarily involves at least another checkpointCj,y

with i �= j ,

Ci,xZ→ Cj,y and Cj,y−1

Z→ Ci,x .

For example, in Fig. 1,Ci,2Z→ Ci,2 andCk,2

Z→ Ck,2

are two Z-cycles

Ci,2Z→ Cj,2 ∧ Cj,1

Z→ Ci,2

and

Ck,2Z→ Ci,3 ∧ Ci,2

Z→ Ck,2.

The following result characterizes useless check-points (i.e., local checkpoints that cannot belong to aconsistent global checkpoint) in terms of Z-cycles.

Theorem 2.6 [10,14,20].A local checkpointCi,x is

uselessiff Ci,xZ→ Ci,x .

3. The NZC and RDT properties

3.1. The “No Z-Cycle” property

Let (H ,CH ) be a checkpoint and communicationpattern. The “No Z-Cycle” property (in brief NZC)states that “Every local checkpoint belongs to at leastone consistent global checkpoint”. When we consider

1 Let us note that Netzer and Xu have characterized the notionof Z-dependency by using the notion ofZigzag Path[14], whichgeneralizes the notion of a causal path of messages.

the Z-dependency relation, this property is equivalentto: “the Z-dependency graph has no cycle”.

Definition 3.1. (H ,CH ) satisfies the NZC property ifit has no useless checkpoint, i.e.,

∀C ∈ CH :¬(C

Z→ C).

The following result, stated and proved in [7],constitutes an operational characterization of the NZCproperty. Let us consider a mechanism that associateswith each checkpointCi,x a scalar timestamp denotedCi,x .t . We have the following property:

Theorem 3.2 [7]. (∀Ci,x,Cj,y : (Ci,xZ→ Cj,y ⇒

Ci,x .t < Cj,y .t)) ⇔ there is no Z-cycle.

3.2. The “Rollback-Dependency Trackability”property

Let us consider two local checkpointsCi,x and

Cj,y such thatCi,xZ→ Cj,y but ¬(Ci,x

C→ Cj,y).This means that the dependency betweenCi,x andCj,y cannot be tracked on the fly. In particular,it is not possible to determine on-the-fly thefirst(also calledminimal) consistent global checkpoint(if any) including Ci,x . The Rollback-DependencyTrackability property (RDT) states that the CCP doesnot include such “hidden” dependencies and hence allcheckpoint dependencies are on-the-fly trackable. Thisproperty is ensured if and only if the Z-dependencygraph and the C-dependency graph are identical.

Definition 3.3. (H ,CH ) satisfies the RDT property if:C→⇔ Z→.

As the C-dependency graph cannot have cycles, itresults from these definitions that every CCP satisfy-ing the RDT property also satisfies the NZC property.At the operational level, Wang [20] has given a char-acterization of the RDT property based on TransitiveDependency Vectors (vector timestamps that exactlytrack the C-dependency relation on local checkpoints).

Fig. 2 shows three examples of CCPs: the firstone violates NZC (and hence, RDT), the second onesatisfies NZC but not RDT, and the third one satisfiesRDT (and hence, NZC).

R. Baldoni et al. / Information Processing Letters 80 (2001) 105–111 109

Fig. 2. Examples of checkpoint and communication patterns.

4. Communication-induced checkpointingprotocols

As noted in the Introduction, CIC protocols add nocontrol message to the computation, but applicationmessages are allowed to piggyback control informa-tion. When a process receives a message it uses thiscontrol information to know whether it has to take aforced checkpoint before delivering the message in or-der that the CCP keeps on satisfying the desired rel-evant (NZC or RDT) property. We say a CIC proto-col ensures NZC (respectively RDT) if, applied to anyrun, it produces a CCP satisfying the NZC (respec-tively RDT) property.

4.1. Scalar clock-based CIC protocols

We have presented in the Introduction the particularfamily of scalar clock-basedCIC protocols (SC_CICprotocols). In these protocols, the only control infor-mation maintained by processes and carried by mes-sages is a Lamport-like (scalar) clock. More precisely,a SC_CIC protocol is defined as follows:• Each processPi manages a local scalar clocklci

(integer variable). This variable can be updated byPi at any time.

• When a messagem is sent byPi , an integer valuedepending on the current value oflci is attached tothe message. Letm.lc denote this value.

• When a messagem is received byPi , this processevaluates a conditionC(lci ,m.lc) and takes a forcedcheckpoint if and only if this condition evaluates totrue. Then, the message is delivered toPi .

Table 1Families of CIC protocols

SC_CIC NSC_CIC

RDT – [2,16,20]

NZC [1,4,8,13,19] [3,8,7]

In the following, we assume thatC is not thetautology.2

At any time, no control information other thanm.lcandlci is used by an SC_CIC protocol.

4.2. Non-scalar clock-based CIC protocols

Let us call NSC_CIC protocol, any CIC protocolthat is not an SC_CIC protocol. In an NSC_CICprotocol, processes manage local control information,and messages may carry control information, thatcannot be reduced to a scalar clock.

4.3. The problem

Table 1 provides a classification of CIC protocols.This classification considers the (strongest) property(namely, NZC or RDT) ensured by the concernedprotocol. It indicates if this protocol is SC_CIC orNSC_CIC.

As the table does not show any SC_CIC protocolensuring the RDT property, a natural question is “Doesthere exist an SC_CIC protocol that ensures the RDT

2 The protocol based on the tautology (called CBR — CheckpointBefore Receive) described in [12,20]) ensures the RDT property.However, its condition does not make use of scalar clocks, and thuswe do not consider it as a SC_CIC protocol.

110 R. Baldoni et al. / Information Processing Letters 80 (2001) 105–111

Fig. 3. Proof of Theorem 5.1.

property?”. The next section proves that the answer tothis question is negative.

5. An impossibility result

Let P(C) denote an SC_CIC protocol using theconditionC upon the arrival of messages.

Theorem 5.1. There is no SC_CIC protocol thatensures the RDT property.

Proof. The proof is by contradiction. Suppose thatP(C) is an SC_CIC protocol ensuring the RDT prop-erty. In particular,P(C) ensures the NZC property. Itfollows from Theorem 3.2 thatP(C) cannot decreasethe timestamp values assigned to messages or to lo-cal scalar clocks when they are updated. As the con-dition C is not the tautology, there exist runs where,for at least one messagem (received by a process,say Pi ), the conditionC(m.lc, lci ) evaluates to falseand thusPi does not take a forced checkpoint be-fore deliveringm. Let us consider one of these runs.SinceP(C) is scalar-clock based, all the informationon the causal past of the eventdeliver(m) is encodedin the valuesm.lc and lci . Moreover, as timestampvalues cannot decrease, no “reset” operation (e.g., re-set to zero) can be performed when a checkpoint istaken, or when an event occurs. So, the processPi

is unable to distinguish the runs 1, 2 and 3 depictedin Fig. 3. More precisely, neither the occurrence ofthe messagem′ nor the local ordering of the check-point Ci,x with respect to the eventsend(m′) (if itexists), can be deduced from the values ofm.lc andof lci upon the arrival ofm. But, the checkpoint andcommunication pattern produced by run 3 andP(C)

does not satisfy the RDT property (due tom′, m andto the ordering ofCi,x with respect tosend(m′), we

haveCj,yZ→ Ck,z but ¬(Cj,y

C→ Ck,z)). Thus, thereis a run such that the CCP produced by this run andthe protocolP(C) does not satisfy the RDT property.Consequently,P(C) does not ensure RDT, a contra-diction. ✷

6. Conclusion

CIC protocols allow to build on-line checkpoint andcommunication patterns satisfying desirable proper-ties. The NZC property eliminates the domino effect.The RDT property is stronger: it additionally elimi-nates all hidden dependencies between local check-points. Most CIC protocols ensuring NZC are scalarclock-based. Those protocols are particularly interest-ing because they have a low overhead in term of thecontrol information size (one integer per message andper process). This paper has shown that it is not possi-ble to design scalar clock-based CIC protocols ensur-ing the RDT property, a previously open question.

Previous results by Tsai, Wang and Kuo have shownthat it is impossible to design optimal (in terms of thenumber of forced checkpoints) CIC protocols ensuringthe NZC property [18] or the RDT property [17].Considering the RDT property, this paper has providedanother impossibility result concerning the family ofscalar clock-based CIC protocols.

References

[1] D. Briatico, A. Ciufoletti, L. Simoncini, A distributed domino-effect free recovery algorithm, in: Proc. 4th IEEE Symposium

R. Baldoni et al. / Information Processing Letters 80 (2001) 105–111 111

on Reliability in Distributed Software and Database Systems,IEEE Society Press, Silver Spring, MD, 1984, pp. 207–215.

[2] R. Baldoni, J.M. Hélary, A. Mostefaoui, M. Raynal, A com-munication-induced checkpointing protocol that ensures roll-back-dependency trackability, in: Proc. 27th IEEE Internat.Symposium on Fault Tolerant Computing (FTCS’97), Seattle,WA, IEEE Society Press, Silver Spring, MD, 1997, pp. 68–77.

[3] R. Baldoni, F. Quaglia, B. Ciciani, A VP-accordant check-pointing protocol preventing useless checkpoints, in: Proc.17th IEEE Internat. Symposium on Reliable Distributed Sys-tems (SRDS’98), West-Lafayette, IN, IEEE Society Press, Sil-ver Spring, MD, 1998, pp. 61–67.

[4] R. Baldoni, F. Quaglia, P. Fornara, An index-based check-pointing algorithm for autonomous distributed systems, IEEETrans. Parallel Distributed Systems 10 (2) (1999) 181–192.

[5] K.M. Chandy, L. Lamport, Distributed snapshots: Determin-ing global states of distributed systems, ACM Trans. Comput.Systems 3 (1) (1985) 63–75.

[6] E.N. Elnozahy, D.B. Johnson, Y.M. Wang, A survey ofrollback-recovery protocols in message-passing systems,Technical Report CMU-CS-96-181, Carnegie-Mellon Univer-sity, Pittsburgh, PA, 1996.

[7] J.-M. Hélary, A. Mostefaoui, R.H.B. Netzer, M. Raynal,Communication-based prevention of useless checkpoints indistributed computation, Distributed Comput. 13 (1) (2000)29–43.

[8] J.-M. Hélary, A. Mostefaoui, M. Raynal, Virtual prece-dence in asynchronous systems: Concept and applications,in: Proc. 11th Internat. Workshop on Distributed Algorithms(WDAG’97), Lecture Notes in Comput. Sci., Vol. 1320,Springer, Berlin, 1997, pp. 170–184.

[9] J.-M. Hélary, A. Mostefaoui, M. Raynal, Communication-induced determination of consistent snapshots, IEEE Trans.Parallel Distributed Systems 10 (9) (1999) 865–877.

[10] J.-M. Hélary, R.H.B. Netzer, M. Raynal, Consistency issues indistributed checkpointing, IEEE Trans. Software Engrg. 25 (2)(1999) 274–281.

[11] L. Lamport, Time, clocks and the ordering of events in adistributed system, Comm. ACM 21 (7) (1978) 558–565.

[12] D. Manivannan, M. Singhal, Quasi-synchronous checkpoint-ing: Models, characterization and classification, IEEE Trans.Parallel Distributed Systems 10 (7) (1999) 703–713.

[13] D. Manivannan, M. Singhal, A low overhead recovery tech-nique using quasi-synchronous checkpointing, in: Proc. 16thIEEE Internat. Conference on Distributed Computing Sys-tems, Hong Kong, IEEE Society Press, Silver Spring, MD,1996, pp. 100–107.

[14] R.H.B. Netzer, J. Xu, Necessary and sufficient conditions forconsistent global snapshots, IEEE Trans. Parallel DistributedSystems 6 (2) (1995) 165–169.

[15] B. Randell, System structure for software fault-tolerance,IEEE Trans. Software Engrg. 1 (2) (1975) 220–232.

[16] D.L. Russell, State restoration in systems of communicatingprocesses, IEEE Trans. Software Engrg. 6 (2) (1980) 183–194.

[17] J. Tsai, S.Y. Kuo, Y.M. Wang, Theoretical analysis forcommunication-induced checkpointing protocols with roll-back-dependency trackability, IEEE Trans. Parallel Distrib-uted Systems 9 (10) (1998) 963–971.

[18] J. Tsai, Y.-M. Wang, S.-Y. Kuo, Evaluations of domino-free communication-induced checkpointing protocols, Inform.Process. Lett. 69 (1999) 31–37.

[19] K. Vankatesh, T. Radakrishanan, H.L. Li, Optimal checkpoint-ing and local recording for domino-free rollback-recovery, In-form. Process. Lett. 25 (1987) 295–303.

[20] Y.-M. Wang, Consistent global checkpoints that contain agiven set of local checkpoints, IEEE Trans. Comput. 46 (4)(1997) 456–468.