Bringing Transactional Guarantees to MongoDB
-
Upload
paul-robinson -
Category
Software
-
view
195 -
download
0
description
Transcript of Bringing Transactional Guarantees to MongoDB
Agenda
•ACID Transactions
•Compensating Transactions
•Code Example
•Today and Planned
Transactions with RDBS
Update balance and create an order atomically
id username item size
0 0 Stumpjumper L
id username email voucher
0 paul.robinson paul… 3000
Invoices
Users
id username item size
0 0 Stumpjumper L
1 0 Zesty L
id username email voucher
0 paul.robinson paul… 200
Invoices
Users
{ id: "<ObjectID1>", username: "paul.robinson", email: "[email protected]" voucher: 3000, invoices: { {"Stumpjumper", "L"}, } }
Transactions with Document Stores
Update balance and create an order atomically
{ id: "<ObjectID1>", username: "paul.robinson", email: "[email protected]" voucher: 200, invoices: { {"Stumpjumper", "L"}, {"Zesty", "L"} } }
But, sometimes this isn’t possible…
Change Multiple Documents{ user: ‘Paul’ balance: 1000 }
{ user: ‘Fred’ balance: 0 }
{ user: ‘Paul’ balance: 700 }
{ user: ‘Fred’ balance: 300 }
E.g. Money Transfer, audited delete, integration
Why doesn’t MongoDB support multi-document transactions?
Scaling MongoDB
Router (mongos)
Shard
Master
D1
Slave 1
D1
Slave 2
D1
Shard
Master
D2
Slave 1
D2
Slave 2
D2
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
D1
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
Client D1?
Client
Multi-Document Transactions
ClientD2
Shard
D1Router
(mongos)Shard
D2
Client D1?
Client
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
Client D1?
D1Client
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
Done
ClientD1
(Multi-document) ACID doesn’t scale …
Don’t throw out transactions altogether!
Extended Transactions
• Umbrella term
• Alternatives to ACID
Guarantees
ACIDNone Extended
ACID Vs Compensating Transactions
• ACID 1. Lock each resource 2. Commit/rollback each resource !
• Compensating 1. Commit each resource 2. Confirm/compensate each resource
Compensating Transactions
• Eventually consistent
• Relaxed isolation
• Can move locks to application logic
• Support Long running tasks
Transaction OptionsAtomic Atomic Batch Traditional
ACIDCompensating!Transactions
Single Doc ✓ ✓ ✓ ✓Multi Doc O ✓ ✓ ✓Sharding ✓ O O ✓
Multi stores O O ✓ ✓ACID ✓ ✓ ✓ O
App Unaware ✓ ✓ ✓ O
Existing Patterns
• Compensating Transaction
• Just a pattern
• Needs implementing
• Recovery is hard!
MongoDB
Application (incl transaction and recovery code)
Narayana’s Solution in WildFly 8
• Middleware
• Transaction Manager driven
• Annotation-based API
• Based on Sagas
MongoDB
Narayana
Application
Protocol Details
Starting State{ user: ‘A’ bal: 1000 }
{ user: ‘B’ bal: 0 }
Log Compensation Handler 1{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
<txid> pending <comp1>
Update Document 1{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
<txid> pending <comp1>
<txid> pending <comp1>
Log Compensation Handler 2{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
Update Document 2{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
Update Transaction Log{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
Confirm Document 1{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
Confirm Document 2{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
Completed{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 }
{ user: ‘B’ bal: 300 !}
Code Example
Money Transfer
• Each user has a balance
• Move money between users
• Update both documents atomically
public class BankingService { ! @Inject AccountManager accountManager; ! @Compensatable public void transferMoney(String fromAccount, String toAccount, Integer amount) { ! accountManager.debitAccount(fromAccount, amount); accountManager.creditAccount(toAccount, amount); }
Service
public class AccountManager { ! @Inject AccountDAO accountDAO; @Inject private CompensationManager compensationManager; @Inject CreditData creditData; ! @TxCompensate(UndoCredit.class) public void creditAccount(String account, Integer amount) { ! //High value transfers (over 500) are not allowed with this service if (amount > 500) compensationManager.setCompensateOnly() && return; creditData.setToAccount(account); creditData.setAmount(amount); accountDAO.incrementBalance(account, amount); } …
Account Manager (credit part)
… @Inject DebitData debitData; ! @TxCompensate(UndoDebit.class) public void debitAccount(String account, Integer amount) { ! debitData.setFromAccount(account); debitData.setAmount(amount); ! accountDAO.incrementBalance(account, -1 * amount); } }
Account Manager (debit part)
@CompensationScoped public class CreditData implements Serializable { ! private String toAccount; private Integer amount; … } !@CompensationScoped public class DebitData implements Serializable { ! private String fromAccount; private Integer amount; … }
Compensation Data
public class UndoCredit implements CompensationHandler { ! @Inject CreditData creditData; @Inject AccountDAO accountDAO; ! public void compensate() { ! if (creditData.getToAccount() != null) { accountDAO.incrementBalance( creditData.getToAccount(), -1 * creditData.getAmount()); } } }
Compensation Handler (Credit)
What we have today
• Compensating-Transactions API
• MongoDB Example
• Other Quickstarts
• Blog post series
• Ships with WildFly 8.Final+
What’s Planned
• MongoDB RM
• Compensating state Recovery
• Throughput/Scalability study
• RDBMS integration
• Other language support
Getting Started• Explore the quickstarts http://github.com/jbosstm/quickstart/
• Give feedback (forum) http://community.jboss.org/en/jbosstm
• Track issues http://issues.jboss.org/browse/JBTM
• Subscribe to Blog http://jbossts.blogspot.co.uk/
• Contact me [email protected], @pfrobinson