Database Transactions Database Transactions and Transaction and Transaction

ManagementManagement

Svetlin NakovSvetlin NakovNational Academy for Software National Academy for Software DevelopmentDevelopment

academy.devbg.org

AgendaAgendaAgendaAgenda

• What is a Transaction?What is a Transaction?

• ACID TransactionsACID Transactions

• Concurrency ProblemsConcurrency Problems

• Concurrency Control TechniquesConcurrency Control Techniques

• Locking StrategiesLocking Strategies

• Optimistic vs. Pessimistic LockingOptimistic vs. Pessimistic Locking

• DeadlocksDeadlocks

• Transactions and RecoveryTransactions and Recovery

• What is a Transaction?What is a Transaction?

• ACID TransactionsACID Transactions

• Concurrency ProblemsConcurrency Problems

• Concurrency Control TechniquesConcurrency Control Techniques

• Locking StrategiesLocking Strategies

• Optimistic vs. Pessimistic LockingOptimistic vs. Pessimistic Locking

• DeadlocksDeadlocks

• Transactions and RecoveryTransactions and Recovery

Agenda (2)Agenda (2)Agenda (2)Agenda (2)

• Transactions and SQL LanguageTransactions and SQL Language

• Transaction Isolation LevelsTransaction Isolation Levels

• When and How to Use Transactions?When and How to Use Transactions?

• Transactions and SQL LanguageTransactions and SQL Language

• Transaction Isolation LevelsTransaction Isolation Levels

• When and How to Use Transactions?When and How to Use Transactions?

What is a What is a Transaction?Transaction?

TransactionsTransactionsTransactionsTransactions

• TransactionsTransactions are a sequence of actions are a sequence of actions ((database operationsdatabase operations)) which are which are executed as a wholeexecuted as a whole::

• Either all of them execute successfullyEither all of them execute successfully

• Or none of the themOr none of the them

• ExampleExample::

• A bank transfer from one account into A bank transfer from one account into anotheranother ( (withdrawalwithdrawal + + depositdeposit))

• If either the withdrawal or the deposit If either the withdrawal or the deposit failsfails the whole operation is cancelledthe whole operation is cancelled

• TransactionsTransactions are a sequence of actions are a sequence of actions ((database operationsdatabase operations)) which are which are executed as a wholeexecuted as a whole::

• Either all of them execute successfullyEither all of them execute successfully

• Or none of the themOr none of the them

• ExampleExample::

• A bank transfer from one account into A bank transfer from one account into anotheranother ( (withdrawalwithdrawal + + depositdeposit))

• If either the withdrawal or the deposit If either the withdrawal or the deposit failsfails the whole operation is cancelledthe whole operation is cancelled

RollbackRollback

CommitCommit

A TransactionA TransactionA TransactionA Transaction

ReadRead WriteWrite

WriteWriteDurable Durable starting starting

statestate

Durable,Durable,consistent,consistent,

ending stateending state

Collection of Collection of reads and reads and

writeswrites

Transactions BehaviorTransactions BehaviorTransactions BehaviorTransactions Behavior

• Transactions guarantee the consistency Transactions guarantee the consistency and the integrity of the databaseand the integrity of the database• All changes in a transaction are All changes in a transaction are

temporarytemporary

• Changes become final when COMMIT is Changes become final when COMMIT is executedexecuted

• At any time all changes can be canceled At any time all changes can be canceled by ROLLBACKby ROLLBACK

• All of the operations are executed as a All of the operations are executed as a whole, either all of them or none of themwhole, either all of them or none of them

• Transactions guarantee the consistency Transactions guarantee the consistency and the integrity of the databaseand the integrity of the database• All changes in a transaction are All changes in a transaction are

temporarytemporary

• Changes become final when COMMIT is Changes become final when COMMIT is executedexecuted

• At any time all changes can be canceled At any time all changes can be canceled by ROLLBACKby ROLLBACK

• All of the operations are executed as a All of the operations are executed as a whole, either all of them or none of themwhole, either all of them or none of them

Transactions: ExamplesTransactions: ExamplesTransactions: ExamplesTransactions: Examples

Withdraw $100Withdraw $100Withdraw $100Withdraw $100

1.1. Read savingsRead savings

2.2. New savings =New savings =current - 100current - 100

3.3. Read checkingRead checking

4.4. New checking =New checking =current + 100current + 100

5.5. Write savingsWrite savings

6.6. Write checkingWrite checking

1.1. Read current Read current balancebalance

2.2. New balance = New balance = current - 100current - 100

3.3. Write new Write new balancebalance

4.4. Dispense cashDispense cash

Transfer $100Transfer $100Transfer $100Transfer $100

What Can Go Wrong?What Can Go Wrong?What Can Go Wrong?What Can Go Wrong?

• Some actions fail to completeSome actions fail to complete

• For example, the application software or For example, the application software or database server crashesdatabase server crashes

• Interference from another transactionInterference from another transaction

• What will happen if several transfers run What will happen if several transfers run for the same account in the same time?for the same account in the same time?

• Some data lost after actions completeSome data lost after actions complete

• Database crashes after withdraw is Database crashes after withdraw is complete and all other actions are lostcomplete and all other actions are lost

• Some actions fail to completeSome actions fail to complete

• For example, the application software or For example, the application software or database server crashesdatabase server crashes

• Interference from another transactionInterference from another transaction

• What will happen if several transfers run What will happen if several transfers run for the same account in the same time?for the same account in the same time?

• Some data lost after actions completeSome data lost after actions complete

• Database crashes after withdraw is Database crashes after withdraw is complete and all other actions are lostcomplete and all other actions are lost

ACID TransactionsACID Transactions

Transactions PropertiesTransactions PropertiesTransactions PropertiesTransactions Properties

• DBMS servers have built-in transaction DBMS servers have built-in transaction supportsupport

• Contemporary databases implement Contemporary databases implement “ACID” transactions“ACID” transactions

• ACID means:ACID means:

• AAtomicitytomicity

• CConsistencyonsistency

• IIsolationsolation

• DDurabilityurability

• DBMS servers have built-in transaction DBMS servers have built-in transaction supportsupport

• Contemporary databases implement Contemporary databases implement “ACID” transactions“ACID” transactions

• ACID means:ACID means:

• AAtomicitytomicity

• CConsistencyonsistency

• IIsolationsolation

• DDurabilityurability

AtomicityAtomicityAtomicityAtomicity

• AtomicityAtomicity means that means that

• Transactions execute as a whole Transactions execute as a whole

• DBMS to guarantee that either all of the DBMS to guarantee that either all of the tasks of a transaction are performed or tasks of a transaction are performed or none of them arenone of them are

• Atomicity example:Atomicity example:

• Transfer funds between bank accountsTransfer funds between bank accounts

• Either withdraw and deposit both execute Either withdraw and deposit both execute successfully or none of themsuccessfully or none of them

• In case of failure DB stays unchangedIn case of failure DB stays unchanged

• AtomicityAtomicity means that means that

• Transactions execute as a whole Transactions execute as a whole

• DBMS to guarantee that either all of the DBMS to guarantee that either all of the tasks of a transaction are performed or tasks of a transaction are performed or none of them arenone of them are

• Atomicity example:Atomicity example:

• Transfer funds between bank accountsTransfer funds between bank accounts

• Either withdraw and deposit both execute Either withdraw and deposit both execute successfully or none of themsuccessfully or none of them

• In case of failure DB stays unchangedIn case of failure DB stays unchanged

ConsistencyConsistencyConsistencyConsistency

• ConsistencyConsistency means thatmeans that• The The database database is is in a legal state when the in a legal state when the

transaction begins and when it endstransaction begins and when it ends

• OOnly valid data will be written to the nly valid data will be written to the databasedatabase

• Transaction cannot break the rules of the Transaction cannot break the rules of the database, e.g. integrity constraintsdatabase, e.g. integrity constraints• Primary, foreign, alternate keysPrimary, foreign, alternate keys

• Consistency exampleConsistency example• Transaction cannot end with a duplicate Transaction cannot end with a duplicate

primary key in a tableprimary key in a table

• ConsistencyConsistency means thatmeans that• The The database database is is in a legal state when the in a legal state when the

transaction begins and when it endstransaction begins and when it ends

• OOnly valid data will be written to the nly valid data will be written to the databasedatabase

• Transaction cannot break the rules of the Transaction cannot break the rules of the database, e.g. integrity constraintsdatabase, e.g. integrity constraints• Primary, foreign, alternate keysPrimary, foreign, alternate keys

• Consistency exampleConsistency example• Transaction cannot end with a duplicate Transaction cannot end with a duplicate

primary key in a tableprimary key in a table

IsolationIsolationIsolationIsolation

• IsolationIsolation means thatmeans that• MultipleMultiple transactions transactions runningrunning at the same at the same

time not impact each other’s executiontime not impact each other’s execution• Transactions don’t see other transaction’s Transactions don’t see other transaction’s

uncommitted changesuncommitted changes• Isolation level defines how deep Isolation level defines how deep

transactions isolate from one anothertransactions isolate from one another• Read committed, read uncommitted, Read committed, read uncommitted,

repeatable read, serializable, etc.repeatable read, serializable, etc.

• Isolation example:Isolation example:• Manager can see the transferred funds on Manager can see the transferred funds on

one account or the other, but never on bothone account or the other, but never on both

• IsolationIsolation means thatmeans that• MultipleMultiple transactions transactions runningrunning at the same at the same

time not impact each other’s executiontime not impact each other’s execution• Transactions don’t see other transaction’s Transactions don’t see other transaction’s

uncommitted changesuncommitted changes• Isolation level defines how deep Isolation level defines how deep

transactions isolate from one anothertransactions isolate from one another• Read committed, read uncommitted, Read committed, read uncommitted,

repeatable read, serializable, etc.repeatable read, serializable, etc.

• Isolation example:Isolation example:• Manager can see the transferred funds on Manager can see the transferred funds on

one account or the other, but never on bothone account or the other, but never on both

DurabilityDurabilityDurabilityDurability

• DurabilityDurability means thatmeans that

• If a transaction is confirmed it become If a transaction is confirmed it become persistentpersistent

• Cannot be lost or undoneCannot be lost or undone

• EEnsured through the use of database nsured through the use of database backups and transaction logsbackups and transaction logs

• Durability example:Durability example:

• After transfer funds and commit the After transfer funds and commit the power supply is lostpower supply is lost

• Transaction stays persistentTransaction stays persistent

• DurabilityDurability means thatmeans that

• If a transaction is confirmed it become If a transaction is confirmed it become persistentpersistent

• Cannot be lost or undoneCannot be lost or undone

• EEnsured through the use of database nsured through the use of database backups and transaction logsbackups and transaction logs

• Durability example:Durability example:

• After transfer funds and commit the After transfer funds and commit the power supply is lostpower supply is lost

• Transaction stays persistentTransaction stays persistent

ACID Transactions and ACID Transactions and RDBMS ServersRDBMS ServersACID Transactions and ACID Transactions and RDBMS ServersRDBMS Servers

• PopularPopular RDBMS servers are transactionalRDBMS servers are transactional::

• Oracle DatabaseOracle Database

• Microsoft SQL ServerMicrosoft SQL Server

• IBM DB2IBM DB2

• PostgreSQLPostgreSQL

• Borland InterBase / FirebirdBorland InterBase / Firebird

• All of the above servers support ACID All of the above servers support ACID transactionstransactions

• MySQL can also run in ACID modeMySQL can also run in ACID mode

• PopularPopular RDBMS servers are transactionalRDBMS servers are transactional::

• Oracle DatabaseOracle Database

• Microsoft SQL ServerMicrosoft SQL Server

• IBM DB2IBM DB2

• PostgreSQLPostgreSQL

• Borland InterBase / FirebirdBorland InterBase / Firebird

• All of the above servers support ACID All of the above servers support ACID transactionstransactions

• MySQL can also run in ACID modeMySQL can also run in ACID mode

Concurrency Concurrency ProblemsProblems

Scheduling TransactionsScheduling TransactionsScheduling TransactionsScheduling Transactions

• Serial schedule – the ideal caseSerial schedule – the ideal case

• An ordering of operations of the An ordering of operations of the transactions so with no interleavingtransactions so with no interleaving

• Problem: Doesn’t allow for as much Problem: Doesn’t allow for as much concurrency as we’d likeconcurrency as we’d like

• Conflicting operationsConflicting operations

• Two operations conflict if theyTwo operations conflict if they

1) are from different transactions1) are from different transactions

2) access the same item, and2) access the same item, and

3) at least one of the transactions does a 3) at least one of the transactions does a write operation to that itemwrite operation to that item

• Serial schedule – the ideal caseSerial schedule – the ideal case

• An ordering of operations of the An ordering of operations of the transactions so with no interleavingtransactions so with no interleaving

• Problem: Doesn’t allow for as much Problem: Doesn’t allow for as much concurrency as we’d likeconcurrency as we’d like

• Conflicting operationsConflicting operations

• Two operations conflict if theyTwo operations conflict if they

1) are from different transactions1) are from different transactions

2) access the same item, and2) access the same item, and

3) at least one of the transactions does a 3) at least one of the transactions does a write operation to that itemwrite operation to that item

Serial Schedule – Example Serial Schedule – Example Serial Schedule – Example Serial Schedule – Example

• T1:T1: Adds 50 to the balanceAdds 50 to the balance

• T2: Subtracts 25 from the balanceT2: Subtracts 25 from the balance

• T1 completes before T2 begins: no T1 completes before T2 begins: no concurrency problemsconcurrency problems

• T1:T1: Adds 50 to the balanceAdds 50 to the balance

• T2: Subtracts 25 from the balanceT2: Subtracts 25 from the balance

• T1 completes before T2 begins: no T1 completes before T2 begins: no concurrency problemsconcurrency problems

TimeTime Trans.Trans. StepStep ValueValue

66 TT22 Write Write balancebalance 12512555 TT2244 TT22 Read Read balancebalance 15015033 TT11 Write Write balancebalance 15015022 TT11 balancebalance = 100 + 50= 100 + 5011 TT11 Read Read balancebalance 100100

balancebalance = 150 - 25= 150 - 25

Serializable TransactionsSerializable TransactionsSerializable TransactionsSerializable Transactions

• SerializabilitySerializability

• Want to get the effect of serial schedules, but Want to get the effect of serial schedules, but allow for more concurrencyallow for more concurrency

• Serializable schedulesSerializable schedules

• Equivalent to serial schedulesEquivalent to serial schedules

• Produce same final result as serial scheduleProduce same final result as serial schedule

• Locking mechanisms can ensure Locking mechanisms can ensure serializabilityserializability

• Serializability is too expensiveSerializability is too expensive

• Optimistic locking allows better concurrencyOptimistic locking allows better concurrency

• SerializabilitySerializability

• Want to get the effect of serial schedules, but Want to get the effect of serial schedules, but allow for more concurrencyallow for more concurrency

• Serializable schedulesSerializable schedules

• Equivalent to serial schedulesEquivalent to serial schedules

• Produce same final result as serial scheduleProduce same final result as serial schedule

• Locking mechanisms can ensure Locking mechanisms can ensure serializabilityserializability

• Serializability is too expensiveSerializability is too expensive

• Optimistic locking allows better concurrencyOptimistic locking allows better concurrency

Concurrency ProblemsConcurrency ProblemsConcurrency ProblemsConcurrency Problems

• Problems from conflicting operations:Problems from conflicting operations:

• Dirty ReadDirty Read (Temporary Update) (Temporary Update)

• A transaction updates an item, then failsA transaction updates an item, then fails

• The item is accessed by another transaction The item is accessed by another transaction before rollbackbefore rollback

• Non-Repeatable ReadNon-Repeatable Read

• A transactions reads an item twice and gets A transactions reads an item twice and gets different values because of concurrent changedifferent values because of concurrent change

• Phantom ReadPhantom Read

• A transaction executes a query twice, and obtains A transaction executes a query twice, and obtains a different a different numbers of rows because anumbers of rows because another nother transaction inserted new rows transaction inserted new rows meantimemeantime

• Problems from conflicting operations:Problems from conflicting operations:

• Dirty ReadDirty Read (Temporary Update) (Temporary Update)

• A transaction updates an item, then failsA transaction updates an item, then fails

• The item is accessed by another transaction The item is accessed by another transaction before rollbackbefore rollback

• Non-Repeatable ReadNon-Repeatable Read

• A transactions reads an item twice and gets A transactions reads an item twice and gets different values because of concurrent changedifferent values because of concurrent change

• Phantom ReadPhantom Read

• A transaction executes a query twice, and obtains A transaction executes a query twice, and obtains a different a different numbers of rows because anumbers of rows because another nother transaction inserted new rows transaction inserted new rows meantimemeantime

Concurrency Problems (2)Concurrency Problems (2)Concurrency Problems (2)Concurrency Problems (2)

• Problems from conflicting operations:Problems from conflicting operations:

• Lost UpdateLost Update

• Two transactions update the same itemTwo transactions update the same item

• Second update overwrites the first (last wins)Second update overwrites the first (last wins)

• Incorrect SummaryIncorrect Summary

• One transaction is calculating an aggregate One transaction is calculating an aggregate function on some records while another function on some records while another transaction is updating themtransaction is updating them

• The aggregate function calculate some values The aggregate function calculate some values before updating and some afterbefore updating and some after

• Problems from conflicting operations:Problems from conflicting operations:

• Lost UpdateLost Update

• Two transactions update the same itemTwo transactions update the same item

• Second update overwrites the first (last wins)Second update overwrites the first (last wins)

• Incorrect SummaryIncorrect Summary

• One transaction is calculating an aggregate One transaction is calculating an aggregate function on some records while another function on some records while another transaction is updating themtransaction is updating them

• The aggregate function calculate some values The aggregate function calculate some values before updating and some afterbefore updating and some after

Dirty Read (Read Dirty Read (Read Uncommitted) – Example Uncommitted) – Example Dirty Read (Read Dirty Read (Read Uncommitted) – Example Uncommitted) – Example

T2 writes T2 writes incorrect incorrect balancebalance

• Update from T1 was rolled back, but Update from T1 was rolled back, but T2 doesn’t know about it, so finally T2 doesn’t know about it, so finally the balance is incorrect.the balance is incorrect.

TimeTime Trans.Trans. StepStep ValueValue

66 TT11 RollbackRollback

125125

55 TT2244 TT22 Read balanceRead balance33 TT1122 TT1111 TT11 Read Read balancebalance 100100

balancebalance = 150 - 25= 150 - 25150150

balancebalance = 100 + 50= 100 + 50Write balanceWrite balance 115500

77 TT22 Write balanceWrite balance

Uncommitted

Undoes T1

Lost Update – ExampleLost Update – ExampleLost Update – ExampleLost Update – Example

Lost update!!

• Update from T1 is lost because T2 Update from T1 is lost because T2 reads balance before T1 was completereads balance before T1 was complete

TimeTime Trans.Trans. StepStep ValueValue

66 TT22 Write Write balancebalance 775555 TT1144 TT22 balancebalance = balance - 25= balance - 2533 TT11 balancebalance = balance + 50= balance + 5022 TT22 Read balanceRead balance11 TT11 Read Read balancebalance 100100

Write balanceWrite balance

100100

150150

Concurrency Control Concurrency Control TechniquesTechniques

Concurrency ControlConcurrency ControlConcurrency ControlConcurrency Control

• The problemThe problem

• Conflicting operations in simultaneous Conflicting operations in simultaneous transactions may produce an incorrect transactions may produce an incorrect resultsresults

• What is concurrency control?What is concurrency control?

• Managing simultaneous operations on Managing simultaneous operations on the database without having them the database without having them interfere with one anotherinterfere with one another

• Prevents conflicts when two or more Prevents conflicts when two or more users access database simultaneously users access database simultaneously

• The problemThe problem

• Conflicting operations in simultaneous Conflicting operations in simultaneous transactions may produce an incorrect transactions may produce an incorrect resultsresults

• What is concurrency control?What is concurrency control?

• Managing simultaneous operations on Managing simultaneous operations on the database without having them the database without having them interfere with one anotherinterfere with one another

• Prevents conflicts when two or more Prevents conflicts when two or more users access database simultaneously users access database simultaneously

Concurrency Control Concurrency Control TechniquesTechniquesConcurrency Control Concurrency Control TechniquesTechniques

• Two basic concurrency control techniques:Two basic concurrency control techniques:

• LockingLocking

• Used in most RDBMS servers, e.g. Oracle, Used in most RDBMS servers, e.g. Oracle, SQL Server, etc.SQL Server, etc.

• TimestampingTimestamping

• Both are conservative (pessimistic) Both are conservative (pessimistic) approaches: delay transactions in case approaches: delay transactions in case they conflict with other transactionsthey conflict with other transactions

• Optimistic methods assume conflict is rare Optimistic methods assume conflict is rare and only check for conflicts at commitand only check for conflicts at commit

• Two basic concurrency control techniques:Two basic concurrency control techniques:

• LockingLocking

• Used in most RDBMS servers, e.g. Oracle, Used in most RDBMS servers, e.g. Oracle, SQL Server, etc.SQL Server, etc.

• TimestampingTimestamping

• Both are conservative (pessimistic) Both are conservative (pessimistic) approaches: delay transactions in case approaches: delay transactions in case they conflict with other transactionsthey conflict with other transactions

• Optimistic methods assume conflict is rare Optimistic methods assume conflict is rare and only check for conflicts at commitand only check for conflicts at commit

LockingLockingLockingLocking

• Transaction uses locks to deny access to Transaction uses locks to deny access to shared data by the other transactionsshared data by the other transactions

• Most widely used approach to ensure Most widely used approach to ensure serializabilityserializability

• Generally, a transaction must claim a read Generally, a transaction must claim a read (shared) or write (exclusive) lock on a data (shared) or write (exclusive) lock on a data item before read or writeitem before read or write

• Lock prevents another transaction from Lock prevents another transaction from modifying item or even reading it, in the modifying item or even reading it, in the case of a write lockcase of a write lock

• Deadlock is possibleDeadlock is possible

• Transaction uses locks to deny access to Transaction uses locks to deny access to shared data by the other transactionsshared data by the other transactions

• Most widely used approach to ensure Most widely used approach to ensure serializabilityserializability

• Generally, a transaction must claim a read Generally, a transaction must claim a read (shared) or write (exclusive) lock on a data (shared) or write (exclusive) lock on a data item before read or writeitem before read or write

• Lock prevents another transaction from Lock prevents another transaction from modifying item or even reading it, in the modifying item or even reading it, in the case of a write lockcase of a write lock

• Deadlock is possibleDeadlock is possible

TimestampingTimestampingTimestampingTimestamping

• A unique identifierA unique identifier

• Created by the DBMSCreated by the DBMS

• Indicates relative starting time of a Indicates relative starting time of a transactiontransaction

• Transactions ordered globallyTransactions ordered globally

• Older transactions (earlier timestamps) get Older transactions (earlier timestamps) get priority in the event of conflictpriority in the event of conflict

• Conflict is resolved by rolling back and Conflict is resolved by rolling back and restarting transactionrestarting transaction

• No locks so no deadlockNo locks so no deadlock

• A unique identifierA unique identifier

• Created by the DBMSCreated by the DBMS

• Indicates relative starting time of a Indicates relative starting time of a transactiontransaction

• Transactions ordered globallyTransactions ordered globally

• Older transactions (earlier timestamps) get Older transactions (earlier timestamps) get priority in the event of conflictpriority in the event of conflict

• Conflict is resolved by rolling back and Conflict is resolved by rolling back and restarting transactionrestarting transaction

• No locks so no deadlockNo locks so no deadlock

Locking StrategiesLocking Strategies

Locking StrategiesLocking StrategiesLocking StrategiesLocking Strategies

• Optimistic lockingOptimistic locking

• Locks are not usedLocks are not used

• Conflicts are possible but are resolved Conflicts are possible but are resolved before commitbefore commit

• High concurrency – scale wellHigh concurrency – scale well

• Pessimistic lockingPessimistic locking

• Use exclusive and shared locksUse exclusive and shared locks

• Transactions wait for each otherTransactions wait for each other

• Low concurrency – does not scaleLow concurrency – does not scale

• Optimistic lockingOptimistic locking

• Locks are not usedLocks are not used

• Conflicts are possible but are resolved Conflicts are possible but are resolved before commitbefore commit

• High concurrency – scale wellHigh concurrency – scale well

• Pessimistic lockingPessimistic locking

• Use exclusive and shared locksUse exclusive and shared locks

• Transactions wait for each otherTransactions wait for each other

• Low concurrency – does not scaleLow concurrency – does not scale

Optimistic LockingOptimistic LockingOptimistic LockingOptimistic Locking

• Optimistic lockingOptimistic locking means no locking means no locking

• Based on assumption that conflicts are rareBased on assumption that conflicts are rare

• It is more efficient to let transactions proceed It is more efficient to let transactions proceed without delays to ensure serializabilitywithout delays to ensure serializability

• At commit, check is made to determine At commit, check is made to determine whether conflict has occurredwhether conflict has occurred

• If there is a conflict, transaction must be rolled If there is a conflict, transaction must be rolled back and restartedback and restarted

• Allows greater concurrency than pessimistic Allows greater concurrency than pessimistic lockinglocking

• Optimistic lockingOptimistic locking means no locking means no locking

• Based on assumption that conflicts are rareBased on assumption that conflicts are rare

• It is more efficient to let transactions proceed It is more efficient to let transactions proceed without delays to ensure serializabilitywithout delays to ensure serializability

• At commit, check is made to determine At commit, check is made to determine whether conflict has occurredwhether conflict has occurred

• If there is a conflict, transaction must be rolled If there is a conflict, transaction must be rolled back and restartedback and restarted

• Allows greater concurrency than pessimistic Allows greater concurrency than pessimistic lockinglocking

Optimistic Locking PhasesOptimistic Locking PhasesOptimistic Locking PhasesOptimistic Locking Phases

• Three phasesThree phases• ReadRead

• Transaction reads the DB, does computations, Transaction reads the DB, does computations, then makes updates to a then makes updates to a privateprivate copy of the DB copy of the DB (e.g. in the memory)(e.g. in the memory)

• ValidationValidation• Make sure that transaction doesn’t cause any Make sure that transaction doesn’t cause any

integrity/consistency problemsintegrity/consistency problems• If no problems, transaction goes to write phaseIf no problems, transaction goes to write phase• If problems, changes are discarded and If problems, changes are discarded and

transaction is restartedtransaction is restarted

• WriteWrite• Changes are made persistent to DBChanges are made persistent to DB

• Three phasesThree phases• ReadRead

• Transaction reads the DB, does computations, Transaction reads the DB, does computations, then makes updates to a then makes updates to a privateprivate copy of the DB copy of the DB (e.g. in the memory)(e.g. in the memory)

• ValidationValidation• Make sure that transaction doesn’t cause any Make sure that transaction doesn’t cause any

integrity/consistency problemsintegrity/consistency problems• If no problems, transaction goes to write phaseIf no problems, transaction goes to write phase• If problems, changes are discarded and If problems, changes are discarded and

transaction is restartedtransaction is restarted

• WriteWrite• Changes are made persistent to DBChanges are made persistent to DB

Pessimistic LockingPessimistic LockingPessimistic LockingPessimistic Locking

• AAssume conflicts are likelyssume conflicts are likely

• Lock shared data to avoid conflictsLock shared data to avoid conflicts

• Transactions wait each other – does not Transactions wait each other – does not scale wellscale well

• UseUse shared and exclusive locksshared and exclusive locks

• Transactions must claim a read (shared) or Transactions must claim a read (shared) or write (exclusive) lock on a data item before write (exclusive) lock on a data item before read or writeread or write

• Locks prevents another transaction from Locks prevents another transaction from modifying item or even reading it, in the modifying item or even reading it, in the case of a write lockcase of a write lock

• AAssume conflicts are likelyssume conflicts are likely

• Lock shared data to avoid conflictsLock shared data to avoid conflicts

• Transactions wait each other – does not Transactions wait each other – does not scale wellscale well

• UseUse shared and exclusive locksshared and exclusive locks

• Transactions must claim a read (shared) or Transactions must claim a read (shared) or write (exclusive) lock on a data item before write (exclusive) lock on a data item before read or writeread or write

• Locks prevents another transaction from Locks prevents another transaction from modifying item or even reading it, in the modifying item or even reading it, in the case of a write lockcase of a write lock

Locking – Basic RulesLocking – Basic RulesLocking – Basic RulesLocking – Basic Rules

• If transaction has read lock on an item, the If transaction has read lock on an item, the item can be read but not modifieditem can be read but not modified

• If transaction has write lock on an item, the If transaction has write lock on an item, the item can be both read and modifieditem can be both read and modified

• Reads cannot conflict, so multiple transactions Reads cannot conflict, so multiple transactions can hold read locks simultaneously on the can hold read locks simultaneously on the same itemsame item

• Write lock gives one transaction exclusive Write lock gives one transaction exclusive access to an itemaccess to an item

• Transaction can upgrade a read lock to a write Transaction can upgrade a read lock to a write lock, or downgrade a write lock to a read locklock, or downgrade a write lock to a read lock

• Commits or rollbacks release the locksCommits or rollbacks release the locks

• If transaction has read lock on an item, the If transaction has read lock on an item, the item can be read but not modifieditem can be read but not modified

• If transaction has write lock on an item, the If transaction has write lock on an item, the item can be both read and modifieditem can be both read and modified

• Reads cannot conflict, so multiple transactions Reads cannot conflict, so multiple transactions can hold read locks simultaneously on the can hold read locks simultaneously on the same itemsame item

• Write lock gives one transaction exclusive Write lock gives one transaction exclusive access to an itemaccess to an item

• Transaction can upgrade a read lock to a write Transaction can upgrade a read lock to a write lock, or downgrade a write lock to a read locklock, or downgrade a write lock to a read lock

• Commits or rollbacks release the locksCommits or rollbacks release the locks

DeadlockDeadlockDeadlockDeadlock

• What is deadlock?What is deadlock?

• When two (or more) transactions are When two (or more) transactions are each waiting for locks held by the other each waiting for locks held by the other to be releasedto be released

• Breaking a deadlockBreaking a deadlock

• Only one way to break deadlock: abort Only one way to break deadlock: abort one or more of the transactionsone or more of the transactions

• What is deadlock?What is deadlock?

• When two (or more) transactions are When two (or more) transactions are each waiting for locks held by the other each waiting for locks held by the other to be releasedto be released

• Breaking a deadlockBreaking a deadlock

• Only one way to break deadlock: abort Only one way to break deadlock: abort one or more of the transactionsone or more of the transactions

Dealing with DeadlockDealing with DeadlockDealing with DeadlockDealing with Deadlock

• Deadlock preventionDeadlock prevention

• Transaction can’t obtain a new lock if the Transaction can’t obtain a new lock if the possibility of a deadlock existspossibility of a deadlock exists

• Deadlock avoidanceDeadlock avoidance

• Transaction must obtain all the locks it Transaction must obtain all the locks it needs before it startsneeds before it starts

• Deadlock detection and recoveryDeadlock detection and recovery

• DB checks for possible deadlocksDB checks for possible deadlocks

• If deadlock is detected, one of the If deadlock is detected, one of the transactions is killed, then restartedtransactions is killed, then restarted

• Deadlock preventionDeadlock prevention

• Transaction can’t obtain a new lock if the Transaction can’t obtain a new lock if the possibility of a deadlock existspossibility of a deadlock exists

• Deadlock avoidanceDeadlock avoidance

• Transaction must obtain all the locks it Transaction must obtain all the locks it needs before it startsneeds before it starts

• Deadlock detection and recoveryDeadlock detection and recovery

• DB checks for possible deadlocksDB checks for possible deadlocks

• If deadlock is detected, one of the If deadlock is detected, one of the transactions is killed, then restartedtransactions is killed, then restarted

Lock ManagementLock Management

• Lock and unlock requests are handled by the Lock and unlock requests are handled by the lock manager, stored in the “lock table”lock manager, stored in the “lock table”

• Lock table entries store:Lock table entries store:

• Number of transactions currently holding a lockNumber of transactions currently holding a lock

• Type of lock held (shared or exclusive)Type of lock held (shared or exclusive)

• Pointer to queue of lock requestsPointer to queue of lock requests

• Locking and unlocking have to be atomic Locking and unlocking have to be atomic operationsoperations

• Lock upgrade: transaction that holds a shared Lock upgrade: transaction that holds a shared lock can be upgraded to exclusive locklock can be upgraded to exclusive lock

Locking GranularityLocking GranularityLocking GranularityLocking Granularity

• Size of data items chosen as unit of Size of data items chosen as unit of protection by concurrency controlprotection by concurrency control

• Ranging from coarse to fine:Ranging from coarse to fine:

• Entire databaseEntire database

• FileFile

• Page (block)Page (block)

• RecordRecord

• Field value of a recordField value of a record

• Size of data items chosen as unit of Size of data items chosen as unit of protection by concurrency controlprotection by concurrency control

• Ranging from coarse to fine:Ranging from coarse to fine:

• Entire databaseEntire database

• FileFile

• Page (block)Page (block)

• RecordRecord

• Field value of a recordField value of a record

Coarse vs. Fine GranularityCoarse vs. Fine GranularityCoarse vs. Fine GranularityCoarse vs. Fine Granularity

• GGranularity is a measure of the amount of ranularity is a measure of the amount of data the lock is protectingdata the lock is protecting

• Coarse granularityCoarse granularity• Small number of locks protecting large Small number of locks protecting large

segments of data, e.g. DB, file, page lockssegments of data, e.g. DB, file, page locks

• Small overhead, small concurrencySmall overhead, small concurrency

• Fine granularityFine granularity

• Large number of locks over small areas of Large number of locks over small areas of data, e.g. table row of field in a rowdata, e.g. table row of field in a row

• More overhead, more concurrencyMore overhead, more concurrency• DBMS servers are “smart” and use bothDBMS servers are “smart” and use both

• GGranularity is a measure of the amount of ranularity is a measure of the amount of data the lock is protectingdata the lock is protecting

• Coarse granularityCoarse granularity• Small number of locks protecting large Small number of locks protecting large

segments of data, e.g. DB, file, page lockssegments of data, e.g. DB, file, page locks

• Small overhead, small concurrencySmall overhead, small concurrency

• Fine granularityFine granularity

• Large number of locks over small areas of Large number of locks over small areas of data, e.g. table row of field in a rowdata, e.g. table row of field in a row

• More overhead, more concurrencyMore overhead, more concurrency• DBMS servers are “smart” and use bothDBMS servers are “smart” and use both

Transactions and Transactions and RecoveryRecovery

Transactions and RecoveryTransactions and Recovery

• Transactions represent basic unit of recoveryTransactions represent basic unit of recovery

• Recovery manager responsible for atomicity Recovery manager responsible for atomicity and durabilityand durability

• What happens at failure?What happens at failure?

• If transaction had not committed at failure time, If transaction had not committed at failure time, recovery manager has to recovery manager has to undoundo ( (rollbackrollback) any ) any effects of that transaction for atomicityeffects of that transaction for atomicity

• If failure occurs between commit and database If failure occurs between commit and database buffers being flushed to secondary storage, buffers being flushed to secondary storage, recovery manager has to recovery manager has to redoredo ( (rollforwardrollforward) ) transaction's updatestransaction's updates

Crash Before Completion – Crash Before Completion – Sample ScenarioSample ScenarioCrash Before Completion – Crash Before Completion – Sample ScenarioSample Scenario

• Application tries to transfer $100Application tries to transfer $100

• Read savingsRead savings

• new savings = current - 100new savings = current - 100

• Read checkingRead checking

• new checking = current + 100new checking = current + 100

• Write savings to DBWrite savings to DB

• System crash before write of new System crash before write of new checking balancechecking balance

• Application tries to transfer $100Application tries to transfer $100

• Read savingsRead savings

• new savings = current - 100new savings = current - 100

• Read checkingRead checking

• new checking = current + 100new checking = current + 100

• Write savings to DBWrite savings to DB

• System crash before write of new System crash before write of new checking balancechecking balance

Recovery from CrashRecovery from CrashRecovery from CrashRecovery from Crash

• RollbackRollback

• Recover to the starting state:Recover to the starting state:

• Take snapshot (checkpoint) of starting stateTake snapshot (checkpoint) of starting state

• E.g., initial bank balance (and all other states)E.g., initial bank balance (and all other states)

• And keep a “redo” logAnd keep a “redo” log

• Alternative: keep an “undo” logAlternative: keep an “undo” log

• E.g., bank balance changed: old value was xE.g., bank balance changed: old value was x

• Resume (if recoverable)Resume (if recoverable)

• Redo all committed actions (since last Redo all committed actions (since last checkpoint)checkpoint)

• Or undo all uncommitted actionsOr undo all uncommitted actions

• RollbackRollback

• Recover to the starting state:Recover to the starting state:

• Take snapshot (checkpoint) of starting stateTake snapshot (checkpoint) of starting state

• E.g., initial bank balance (and all other states)E.g., initial bank balance (and all other states)

• And keep a “redo” logAnd keep a “redo” log

• Alternative: keep an “undo” logAlternative: keep an “undo” log

• E.g., bank balance changed: old value was xE.g., bank balance changed: old value was x

• Resume (if recoverable)Resume (if recoverable)

• Redo all committed actions (since last Redo all committed actions (since last checkpoint)checkpoint)

• Or undo all uncommitted actionsOr undo all uncommitted actions

Creating REDO LogCreating REDO LogCreating REDO LogCreating REDO Log

• Keep a log of all database writes ON DISK (so Keep a log of all database writes ON DISK (so that it is still available after crash)that it is still available after crash)

• <transaction ID>; <data item>; <new value><transaction ID>; <data item>; <new value>

• (Tj; x=125) (Ti; y=56)(Tj; x=125) (Ti; y=56)

• Actions must be idempotent (redoable)Actions must be idempotent (redoable)

• NOT x = x + 100NOT x = x + 100

• But don't write to the database yetBut don't write to the database yet

• At the end of transaction executionAt the end of transaction execution

• Add "commit <transaction ID>" to the logAdd "commit <transaction ID>" to the log

• Do all the writes to the databaseDo all the writes to the database

• Add "complete <transaction ID>" to the logAdd "complete <transaction ID>" to the log

• Keep a log of all database writes ON DISK (so Keep a log of all database writes ON DISK (so that it is still available after crash)that it is still available after crash)

• <transaction ID>; <data item>; <new value><transaction ID>; <data item>; <new value>

• (Tj; x=125) (Ti; y=56)(Tj; x=125) (Ti; y=56)

• Actions must be idempotent (redoable)Actions must be idempotent (redoable)

• NOT x = x + 100NOT x = x + 100

• But don't write to the database yetBut don't write to the database yet

• At the end of transaction executionAt the end of transaction execution

• Add "commit <transaction ID>" to the logAdd "commit <transaction ID>" to the log

• Do all the writes to the databaseDo all the writes to the database

• Add "complete <transaction ID>" to the logAdd "complete <transaction ID>" to the log

Sample REDO Log FileSample REDO Log File

Recovering From a CrashRecovering From a Crash

• There are 3 phases in the recovery algorithm:There are 3 phases in the recovery algorithm:

• Analysis Analysis – scan the log forward to identify all – scan the log forward to identify all transactions that were active, and all dirty pages transactions that were active, and all dirty pages in the buffer pool at the time of the crashin the buffer pool at the time of the crash

• RedoRedo – redoes all updates to dirty pages in the – redoes all updates to dirty pages in the buffer pool, as needed, to ensure that all logged buffer pool, as needed, to ensure that all logged updates are in fact carried out and written to diskupdates are in fact carried out and written to disk

• UndoUndo – all transactions that were active at the – all transactions that were active at the crash are undone, working backwards in the logcrash are undone, working backwards in the log

• Some care must be taken to handle the case of a Some care must be taken to handle the case of a crash occurring during the recovery process!crash occurring during the recovery process!

Transactions and Transactions and SQL LanguageSQL Language

Transactions and SQLTransactions and SQLTransactions and SQLTransactions and SQL

• Start a transactionStart a transaction

• BEGIN TRANSACTIONBEGIN TRANSACTION

• Some databases assume implicit startSome databases assume implicit start

• E.g. OracleE.g. Oracle

• Ending a transactionEnding a transaction

• COMMITCOMMIT

• Used to end a successful transaction and make Used to end a successful transaction and make changes “permanent”changes “permanent”

• ROLLBACKROLLBACK

• ““Undo” changes from an aborted transactionUndo” changes from an aborted transaction

• May be done automatically when failure occursMay be done automatically when failure occurs

• Start a transactionStart a transaction

• BEGIN TRANSACTIONBEGIN TRANSACTION

• Some databases assume implicit startSome databases assume implicit start

• E.g. OracleE.g. Oracle

• Ending a transactionEnding a transaction

• COMMITCOMMIT

• Used to end a successful transaction and make Used to end a successful transaction and make changes “permanent”changes “permanent”

• ROLLBACKROLLBACK

• ““Undo” changes from an aborted transactionUndo” changes from an aborted transaction

• May be done automatically when failure occursMay be done automatically when failure occurs

Transactions in SQL Transactions in SQL Server: ExampleServer: ExampleTransactions in SQL Transactions in SQL Server: ExampleServer: Example

• We have a table with bank accountsWe have a table with bank accounts::

• We use a transaction to transfer money We use a transaction to transfer money from one account into anotherfrom one account into another

• We have a table with bank accountsWe have a table with bank accounts::

• We use a transaction to transfer money We use a transaction to transfer money from one account into anotherfrom one account into another

CREATE TABLE CREATE TABLE ACCOUNTACCOUNT(( id int NOT NULL,id int NOT NULL, balancebalance decimaldecimal NOT NULL) NOT NULL)

CREATE OR REPLACE PROCEDURE sp_Transfer_Funds(CREATE OR REPLACE PROCEDURE sp_Transfer_Funds( from_account IN INT,from_account IN INT, to_account IN INT, to_account IN INT, ammount IN ammount IN NUMBERNUMBER) IS) ISBEGINBEGIN BEGIN TRANBEGIN TRAN

((example continuesexample continues))

Transactions in SQL Transactions in SQL Server: Example (2)Server: Example (2)Transactions in SQL Transactions in SQL Server: Example (2)Server: Example (2)

UPDATE ACCOUNT set balance = balance - ammountUPDATE ACCOUNT set balance = balance - ammount WHERE id = from_account;WHERE id = from_account; IF SQL%ROWCOUNT IF SQL%ROWCOUNT <><> 11 THEN THEN ROLLBACK;ROLLBACK; RAISE_APPLICATION_ERROR(-20001, 'Invalid src account!');RAISE_APPLICATION_ERROR(-20001, 'Invalid src account!'); END IF;END IF;

UPDATE ACCOUNT set balance = balance + ammountUPDATE ACCOUNT set balance = balance + ammount WHERE id = to_account;WHERE id = to_account; IF SQL%ROWCOUNT IF SQL%ROWCOUNT <><> 11 THEN THEN ROLLBACK;ROLLBACK; RAISE_APPLICATION_ERROR(-20002, 'Invalid dst account!');RAISE_APPLICATION_ERROR(-20002, 'Invalid dst account!'); END IF;END IF; COMMIT;COMMIT;END;END;

Transaction Transaction Isolation LevelsIsolation Levels

Transactions and isolationTransactions and isolationTransactions and isolationTransactions and isolation

• Transactions can define different Transactions can define different isolation levels for themselvesisolation levels for themselves

• Stronger isolation ensures better Stronger isolation ensures better consistency but has less concurrency consistency but has less concurrency and the data is locked longerand the data is locked longer

• Transactions can define different Transactions can define different isolation levels for themselvesisolation levels for themselves

• Stronger isolation ensures better Stronger isolation ensures better consistency but has less concurrency consistency but has less concurrency and the data is locked longerand the data is locked longer

Level of isolationLevel of isolation Dirty readsDirty reads Repeatable Repeatable readsreads

Phantom Phantom readsreads

Read uncommittedRead uncommitted yesyes yesyes yesyes

Read committedRead committed nono yesyes yesyes

Repeatable readRepeatable read nono nono yesyes

SerializableSerializable nono nono nono

Isolation levelsIsolation levelsIsolation levelsIsolation levels

• Uncommitted ReadUncommitted Read

• Reads everything, even data not committed by Reads everything, even data not committed by some other transactionsome other transaction

• No data is lockedNo data is locked

• Not commonly usedNot commonly used

• Read CommittedRead Committed

• Current transaction sees only committed dataCurrent transaction sees only committed data

• Records retrieved by a query are not prevented Records retrieved by a query are not prevented from modification by some other transactionfrom modification by some other transaction

• Default behavior in most databasesDefault behavior in most databases

• Uncommitted ReadUncommitted Read

• Reads everything, even data not committed by Reads everything, even data not committed by some other transactionsome other transaction

• No data is lockedNo data is locked

• Not commonly usedNot commonly used

• Read CommittedRead Committed

• Current transaction sees only committed dataCurrent transaction sees only committed data

• Records retrieved by a query are not prevented Records retrieved by a query are not prevented from modification by some other transactionfrom modification by some other transaction

• Default behavior in most databasesDefault behavior in most databases

Isolation levelsIsolation levelsIsolation levelsIsolation levels

• Repeatable ReadRepeatable Read

• Records retrieved cannot be changed from Records retrieved cannot be changed from outsideoutside

• The transaction acquires read locks on all The transaction acquires read locks on all retrieved data, but does not acquire range locks retrieved data, but does not acquire range locks (phantom reads may occur)(phantom reads may occur)

• Deadlocks can occurDeadlocks can occur

• SerializableSerializable

• Acquires a range lock on the dataAcquires a range lock on the data

• Simultaneous transactions are actually Simultaneous transactions are actually executed one after anotherexecuted one after another

• Repeatable ReadRepeatable Read

• Records retrieved cannot be changed from Records retrieved cannot be changed from outsideoutside

• The transaction acquires read locks on all The transaction acquires read locks on all retrieved data, but does not acquire range locks retrieved data, but does not acquire range locks (phantom reads may occur)(phantom reads may occur)

• Deadlocks can occurDeadlocks can occur

• SerializableSerializable

• Acquires a range lock on the dataAcquires a range lock on the data

• Simultaneous transactions are actually Simultaneous transactions are actually executed one after anotherexecuted one after another

When and How to When and How to Use Transactions?Use Transactions?

Transactions UsageTransactions UsageTransactions UsageTransactions Usage

• When force using transactions?When force using transactions?• Always when a business operation Always when a business operation

modifies more than one table (atomicity)modifies more than one table (atomicity)

• When you don’t want conflicting updates When you don’t want conflicting updates (isolation)(isolation)

• How to choose isolation level?How to choose isolation level?• Use read committed, unless you need Use read committed, unless you need

more strong isolationmore strong isolation

• Keep transactions small in timeKeep transactions small in time• Never keep transactions opened for longNever keep transactions opened for long

• When force using transactions?When force using transactions?• Always when a business operation Always when a business operation

modifies more than one table (atomicity)modifies more than one table (atomicity)

• When you don’t want conflicting updates When you don’t want conflicting updates (isolation)(isolation)

• How to choose isolation level?How to choose isolation level?• Use read committed, unless you need Use read committed, unless you need

more strong isolationmore strong isolation

• Keep transactions small in timeKeep transactions small in time• Never keep transactions opened for longNever keep transactions opened for long

Transactions Usage – Transactions Usage – Examples Examples Transactions Usage – Transactions Usage – Examples Examples

• Transfer money from one account to Transfer money from one account to anotheranother

• Either both withdraw and deposit succeed Either both withdraw and deposit succeed or neither of themor neither of them

• At the pay desk of a storeAt the pay desk of a store: : we buy a cart of we buy a cart of productsproducts as a wholeas a whole

• We either buy all of them and pay or we buy We either buy all of them and pay or we buy nothing and give no moneynothing and give no money

• If any of the operations fails we cancel the If any of the operations fails we cancel the transactiontransaction ( (the entire purchasethe entire purchase))

• Transfer money from one account to Transfer money from one account to anotheranother

• Either both withdraw and deposit succeed Either both withdraw and deposit succeed or neither of themor neither of them

• At the pay desk of a storeAt the pay desk of a store: : we buy a cart of we buy a cart of productsproducts as a wholeas a whole

• We either buy all of them and pay or we buy We either buy all of them and pay or we buy nothing and give no moneynothing and give no money

• If any of the operations fails we cancel the If any of the operations fails we cancel the transactiontransaction ( (the entire purchasethe entire purchase))

Database Transactions and Database Transactions and Transaction ManagementTransaction ManagementDatabase Transactions and Database Transactions and Transaction ManagementTransaction Management

QuestionsQuestions??