Idempotent Transactional Workflow (POPL 2013) G. Ramalingam Kapil Vaswani Microsoft Research India.
-
Upload
claude-bruce -
Category
Documents
-
view
213 -
download
0
Transcript of Idempotent Transactional Workflow (POPL 2013) G. Ramalingam Kapil Vaswani Microsoft Research India.
Idempotent Transactional Workflow
(POPL 2013)
G. RamalingamKapil Vaswani
Microsoft Research India
Application
The Problem
Partitioned Data
scale-out
Can we simplify
writing suchapplications?
Transfer (amt, acct1, acct2) {Debit amt from acct1;Credit amt to acct2;
}
Transfer (amt, acct1, acct2) atomic {
Debit amt from acct1; Credit amt to acct2;
}
ACID Transaction+ Strong consistency− Distributed
transaction
Transfer (amt, acct1, acct2) atomic {Debit …}; atomic {Credit …};
Workflow− Weaker consistency
− No isolation+ No distributed transaction
What about process failure? Claim: Workflows are common in applications over partitioned data
Storage Layer
Application Logic
Stopping (non-byzantine) failure
The Problem
(failures handled by storage layer)
Goal• Fault-tolerance in application• A transactional workflow
engine• decentralized!
Modern Cloud Platforms
request response
Making Workflows Fault-Tolerant
Request or response
may be lost!
Taking a step back …
Resending messages
is a critical elementof fault-tolerance
Must be Idempotent!
(tolerate duplicatemessages)
Transfer (amt, acct1, acct2) {Debit amt from acct1;Credit amt to acct2;
}
Goal:Idempotent Fault-Tolerance
• (Idempotent Workflow)• A program is said to be idempotent & fault-
tolerant iff– its behavior is unaffected by process failures– its behavior is unaffected by duplicate input
requests
• Behavioral equivalence:– duplicate output responses allowed– progress (liveness) conditions
• slightly weakened
request response
Making WorkflowsIdempotent & Fault-Tolerant
request response
Making Computations Idempotent
Make every effectful step idempotent:1. Associate unique id with every step2. Modify step to log execution of step3. Modify step to check if it has already
executedAll must be done atomically !
AutomatedIdempotent Fault-Tolerance
• As a library– In C# & F#– Technically, a monad
• As a compiler
• As a programming-language construct
Formal Results
Theorem. A well-typed monadic program isidempotent and fault-tolerant.
Theorem. compile[e] is an idempotent and fault-tolerant realization of e.
Any (well-typed) program e can be automatically translated (compiled) into a program compile[e]
Idempotence: A Language Construct
• “idworkflow uid e’’
transfer (uid, amt, acct1, acct2) { idworkflow uid {
atomic T1 Debit amt from acct1 atomic T2 Credit amt to acct2}}
}
Extensions
• Compensating actions– Undo earlier actions when later actions
encounter logical failure
• Automatic retry– Detect process failures & restart
• Checkpointing– Restart at most recent checkpoint
Questions?
Fault-Tolerance & Idempotence: Simpler Together
Storage Layer
Application Logic
client
service
partitioneddata
Problem Setting