Recovery
description
Transcript of Recovery
03/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Integrity or correctness of data
Would like data to be “accurate” or “correct” at all times
EMP
Name
WhiteGreenGray
Age
523421
1
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Integrity or consistency constraints
Predicates data must satisfy Examples:
- x is key of relation R- x y holds in R- Domain(x) = {Red, Blue, Green}is valid index for attribute x of R- no employee should make more than twice the average
salary
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Definition:
Consistent state: satisfies all constraints Consistent DB: DB in consistent state
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Observation: DB cannot be consistent always!
Example: a1 + a2 +…. an = TOT (constraint)
Deposit $100 in a2: a2 a2 + 100
TOT TOT + 100
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
a2
TOT
.
.50
.
.1000
.
.150
.
.1000
.
.150
.
.1100
Example: a1 + a2 +…. an = TOT (constraint)Deposit $100 in a2: a2 a2 + 100
TOT TOT + 100
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Transaction: collection of actions that preserve consistency
Consistent DB Consistent DB’T
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
How can we prevent/fix violations?
cc: due to data sharing only recovery: due to failures only
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Will not consider:
How to write correct transactions How to write correct DBMS Constraint checking & repair
That is, solutions studied here do not need to know constraints
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Storage hierarchy
Memory Disk
x x
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Operations:
Input (x): block with x memory Output (x): block with x disk
Read (x,t): do input(x) if necessary t value of x in blockWrite (x,t): do input(x) if necessary value of x in block t
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Key problem Unfinished transaction
Example Constraint: A=B
T1: A A 2
B B 2
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
T1: Read (A,t); t t2Write (A,t);Read (B,t); t t2Write (B,t);Output (A);Output (B);
A: 8B: 8
A: 8B: 8
memory disk
1616
16
failure!
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
T1: Read (A,t); t t2 A=BWrite (A,t);Read (B,t); t t2Write (B,t);Output (A);Output (B);
A:8B:8
A:8B:8
memory disk log
Undo logging
1616
<T1, start><T1, A, 8>
<T1, commit>16 <T1, B, 8>
16
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
One “complication” Log is first written in memory Not written to disk on every action
memory
DB
LogA: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8>
A: 8B: 8
16BAD STATE
# 1
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
One “complication” Log is first written in memory Not written to disk on every action
memory
DB
LogA: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8><T1, commit>
A: 8B: 8
16BAD STATE
# 2
<T1, B, 8><T1, commit>
...
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Undo logging rules
(1) For every action generate undo log record (containing old value)
(2) Before x is modified on disk, log records pertaining to x must be on disk (write ahead logging: WAL)
(3) Before commit is flushed to log, all writes of transaction must be reflected on disk
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery rules: Undo logging
(1) Let S = set of transactions with <Ti, start> in log, but no
<Ti, commit> (or <Ti, abort>) record in log
(2) For each <Ti, X, v> in log,
in reverse order (latest earliest) do:
- if Ti S then - write (X, v)
- output (X)
(3) For each Ti S do
- write <Ti, abort> to log
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
What if failure during recovery?
No problem! Undo idempotent
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
To discuss: Redo logging Undo/redo logging, why both? Checkpoints
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Redo logging (deferred modification)
T1: Read(A,t); t t2; write (A,t);
Read(B,t); t t2; write (B,t);
Output(A); Output(B)
A: 8B: 8
A: 8B: 8
memory DB LOG
1616
<T1, start><T1, A, 16><T1, B, 16>
<T1, commit>
output16
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Redo logging rules
(1) For every action, generate redo log record (containing new value)
(2) Before anything is modified on disk (DB), all log records for transaction that modify things (including commit) must be on disk
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
(1) Let S = set of transactions with <Ti, commit> in log
(2) For each <Ti, X, v> in log, in forward
order (earliest latest) do:
- if Ti S then Write(X, v) Output(X)
Recovery rules: Redo logging
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery is very, very SLOW !
Redo log:
First T1 wrote A,B Last
Record Committed a year ago Record
(1 year ago) --> STILL, Need to redo after crash!!
... ... ...
Crash
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Undo Recovery – Better?
The first record without commit or abort should not be too far away from crash
But immediate update itself is expensive
... ... ...
first record without commit or abort
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Solution: Checkpoint (simple version)
Periodically:
(1) Do not accept new transactions
(2) Wait until all transactions finish
(3) Flush all log records to disk (log)(4) Flush all buffers to disk (DB) (do not discard buffers)
(5) Write “checkpoint” record on disk (log)
(6) Resume transaction processing
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Redo log (disk):
<T1
,A,1
6>
<T1
,com
mit
>
Ch
eck
poin
t
<T2
,B,1
7>
<T2
,com
mit
>
<T3
,C,2
1>
Crash... ... ... ...
...
...
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Undo log (disk):
<T1
,A,8
>
<T1
,com
mit
>
Ch
eck
poin
t
<T2
,B,1
7>
<T2
,com
mit
>
<T3
,C,2
1>
Crash... ... ... ...
...
...
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Nonquiescent Checkingpointing
“Quiescent” checkpointing stalls DB “Nonquiescent” checkpointing admits new transactions
while checkpointing
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Nonquiescent Checkpoint Rules - undo
Write and flush a log record <START CKPT(T1, …, Tk)> where T1,…,Tk are active transactions
When all T1, …, Tk have completed, write and flush a log record <END CKPT>
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Undo log (disk):
<T1
,A,8
>
<C
KPT(T
1,T
2)>
<T3
,B,1
7>
<T1
,com
mit
>
<T3
,C,2
1>
Crash... ... ...
...
<E
ND C
KPT>
<T2
,B,1
2>
<T2
,com
mit
>
Crash after end of checkpoint
Only need to undo all the incomplete transactions started after <START CKPT>
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Undo log (disk):
<T1
,A,8
>
<C
KPT(T
1,T
2)>
<T3
,B,1
7>
<T1
,com
mit
>
<T3
,C,2
1>
Crash... ... ...
... …
<T2
,B,1
2>
<T2
,com
mit
>
Crash before end of checkpoint
Only need to undo all incomplete transactions started after <START CKPT> and those in <CKPT (…)>
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Nonquiescent Checkpoint Rules - redo
Write and flush a log record <START CKPT(T1, …, Tk) where T1,…,Tk are active transactions
Flush all the updates of the transactions committed before <START CKPT(T1,…,Tk)>
Write and flush a log record <END CKPT>
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Redo log (disk):
<T1
,A,8
>
<C
KPT(T
1,T
2)>
<T3
,B,1
7>
<T1
,com
mit
>
<T3
,C,2
1>
Crash... ... ...
...
<E
ND C
KPT>
<T2
,B,1
2>
<T2
,com
mit
>
Crash after end of checkpoint
Only need to redo all the transactions committed after the latest <START CKPT>
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Redo log (disk):
<T1
,A,8
>
<C
KPT(T
1,T
2)>
<T3
,B,1
7>
<T1
,com
mit
>
<T3
,C,2
1>
Crash... ... ...
... …
<T2
,B,1
2>
<T2
,com
mit
>
Crash before end of checkpoint
Only need to redo all the transactions committed after the latest successful <START CKPT>
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Key drawbacks:
Undo logging: need to update immediately, increasing I/O Redo logging: need to keep all modified blocks in memory until
commit
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Solution: undo/redo logging!
Update <Ti, Xid, New X val, Old X val>
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Rules
Page X can be flushed before or after Ti commit Log record flushed before corresponding updated
page (WAL) Flush log at commit
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Non-quiesce checkpoint
LOG
for undo
Start-ckptactive TR:
Ti,T2,...
endckpt
.........
... Flush
dirty buffers
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Examples what to do at recovery time?
no T1 commit
LOG
T1,-a
...CkptT1
...Ckptend
...T1-b
...
Undo T1 (undo a,b)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example
LOG
...T1
a... ...
T1
b... ...
T1
c...
T1
cmt...
Ckptend
ckpt-sT1
Redo T1: (redo b,c)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Examples what to do at recovery time?
no T1 commit
LOG
T1,-a
...CkptT1
... ...T1-b
...
Undo T1 (all the actions)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example
LOG
...T1
a... ...
T1
b... ...
T1
c...
T1
cmt...
ckpt-sT1
Redo T1: (redo b,c, a and the actions after last successful ckp-s)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery process:
Backwards pass (end of log -> latest checkpoint start) construct set S of committed transactions undo actions of transactions not in S
Undo pending transactions follow undo chains for transactions in
(checkpoint active list) - S Forward pass (latest successful checkpoint start -> end of log)
redo actions of S transactions
backward pass
forward passstart
check-point
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery with Checkpoint Summary
Logging still obey the logging rules undo, redo, undo-redo
Recovery still obey rules Undo: undo all incomplete transactions Redo: redo all completed transactions Undo-redo: combined
Checkpoint provides a way to limit the transactions needing undo or redo
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Summary
undo redo Undo-redo
commit commit means all updates on disk
Commit means some updates may start to migrate to disk
Commit does not signal disk update
recovery Undo all incomplete transactions
Redo all committed transactions
Undo all incomplete transactions and redo all committed transactions
Nonquiescent ckpt
<end ckpt(T)> means every tran started before the matching <start ckpt (T)> is on disk
<end ckpt(T)> means every tran committed before the matching <start ckpt(T)> is on disk
<end ckpt(T)> means every update before the matching <start ckpt(T)> is on disk
WAL
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Summary
Consistency of data One source of problems: failures
- WAL