Post on 04-Jan-2016
11
Transaction reordering
Speaker :Po-Hung LaiSpeaker :Po-Hung Lai
Adviser : Chih-Hung LinAdviser : Chih-Hung Lin
Date :2011.1.4Date :2011.1.4
22
OutlineOutline
IntroductionIntroduction Transaction reordering frameworkTransaction reordering framework MethodMethod ExperimentExperiment conclusionconclusion
33
IntroductionIntroduction
Traditional Traditional :: allows a certain number of complex queriesallows a certain number of complex queries
a crasha crash
focus on the current system statusfocus on the current system status
Our Our :: information about the interaction information about the interaction
wait runwait run
current system statuscurrent system status
44
IntroductionIntroduction Why transaction reordering be effectiveWhy transaction reordering be effective
– system independent system independent
eliminates some intrinsic lock conflictseliminates some intrinsic lock conflicts– system dependentsystem dependent
55
IntroductionIntroduction Advantage/ flawAdvantage/ flaw
insideinside outsideoutside
Advantage : Advantage :
current state of the systemcurrent state of the system
synchronized scanssynchronized scans more opportunities for more opportunities for
reorderingreordering
Don’t change database engineDon’t change database engine
Flaw :Flaw :Need set system and reorder Need set system and reorder
modelmodelSystem will be miss transaction System will be miss transaction
an additional parsing of each tran additional parsing of each transaction ansaction
66
IntroductionIntroduction
general transaction reordering frameworkgeneral transaction reordering framework– current system status current system status – queued and running transactions queued and running transactions
77
IntroductionIntroduction
reordering algorithms reordering algorithms – Run all the transactions Run all the transactions
– Analyze and record transactions Analyze and record transactions
88
IntroductionIntroduction
Our reordering algorithms Our reordering algorithms – Luck conflictsLuck conflicts– buffer pool buffer pool
99
Transaction reordering frameworkTransaction reordering framework
Two method to increase the transaction throughput: Two method to increase the transaction throughput:
– increasing concurrency preventing conflicting transactions preventing conflicting transactions
– sharing resources No deadlock No deadlock
PurposePurpose– improve RDBMS performance
1010
Our methodOur method typical operational data warehouse typical operational data warehouse [16][16]
– OLTP data warehouseOLTP data warehouse(Operational Load Transaction Process)(Operational Load Transaction Process)
– Acknowledge client input somethingAcknowledge client input something– Ensuring data’s durabilityEnsuring data’s durability
Fig :operational data warehouseFig :operational data warehouse
1111
Our methodOur method
loading data continuously into an RDBMSloading data continuously into an RDBMS
Files Files 、 、 message message 、 、 queues …queues …
1212
Our methodOur method
Continuous load utilities Continuous load utilities definitiondefinition– ([20]Oracle([20]Oracle ,, [6]Teradata[6]Teradata))
– like RDBMS like RDBMS series of transactions series of transactions – free to arbitrarily groupfree to arbitrarily group
free to arbitrarily group Characteristic– It’s not anomalies
1313
Our methodOur method
applications tolerate such anomalies applications tolerate such anomalies
Ex Ex 刪除記錄後 紀錄表刪除記錄後 紀錄表 尚未更新記錄前尚未更新記錄前 交易隨即移動走交易隨即移動走 使其發生之異常使其發生之異常
1414
Our methodOur method
application ensures not allow such anomalieapplication ensures not allow such anomaliess
ex ex 在新增交易在新增交易資料完成之前資料完成之前不允許做任何不允許做任何動作動作
1515
Our methodOur method
Increase concurrencyIncrease concurrency
Files Files 、 、 message message 、 、 queues …queues …
1616
Our methodOur method
Materialized view maintenanceMaterialized view maintenance– Important class of Materialized view call join vieImportant class of Materialized view call join vie
ww
Deadluck occurDeadluck occur– becausebecause
allow data load into multiple relations allow data load into multiple relations RDBMS’s data as up-tpdateRDBMS’s data as up-tpdate JV is defined on multiple base relations JV is defined on multiple base relations
1717
Our methodOur method
Deadluck probabilityDeadluck probability– k > 1 concurrent transactik > 1 concurrent transacti
ons ons – n modification operations n modification operations – p = A’s probabilityp = A’s probability– P-1 = B’s probabilityP-1 = B’s probability– S = totally S = totally S >> knS >> kn
)2/()1)(1( 2 snkpp
1818
Our methodOur method
ExampleExample– p = 50%p = 50% ,, k = 8k = 8 ,, n = 32 and s = 10000n = 32 and s = 10000
– Deadluck probability = 9%Deadluck probability = 9%
– n = 64 n = 64 , , Deadluck probability = 36%Deadluck probability = 36%
1919
Our methodOur method
Solution follow as ruleSolution follow as rule– data can only be loaded into one base relation data can only be loaded into one base relation
of JV of JV – Modification operations Modification operations – uses a high concurrency locking protocol uses a high concurrency locking protocol
2020
Our methodOur method
we schedule transactions as follows we schedule transactions as follows – Action 1 Action 1
2222
Our methodOur method
– Action 3Action 3– schedule a transaction to the RDBMS schedule a transaction to the RDBMS
desirable transaction T or first transactions desirable transaction T or first transactions
starvation problemstarvation problem– Use timestamp(TS) to solutionUse timestamp(TS) to solution– give pointer r ; r = 0 ; give pointer r ; r = 0 ; (0<=r<=k)(0<=r<=k)
2323
Our methodOur method
Situation Situation – Qm Qm (1<=m<=k)(1<=m<=k) is empty is empty ,, r = 0r = 0– Anytime r =0; transaction coming Qm Anytime r =0; transaction coming Qm r = m r = m– r = m ; first transaction leave ; r = r+1r = m ; first transaction leave ; r = r+1
if (m = kif (m = k , , r =1)r =1)
v = r last number ; TS is pre-defined v = r last number ; TS is pre-defined – If v > ts then Qv transaction become firstIf v > ts then Qv transaction become first
2525
Our methodOur method
buffer pool analysis buffer pool analysis – buffer management methods cannot utilize the sbuffer management methods cannot utilize the s
ynchronized scan technique efficiently ynchronized scan technique efficiently
– solution to this problem solution to this problem synchronized scan techniquesynchronized scan technique
2727
Our methodOur method lower RDBMS’s capability problemlower RDBMS’s capability problem
– involve full table scans involve full table scans
2828
Our methodOur method
Solution : synchronized scan techniqueSolution : synchronized scan technique
2929
Our methodOur method
Our methodOur method– addition addition K1 buffer; K1 buffer;
Hold recent Hold recent resourceresource
– DS ;DS ; Record T1 need all resourceRecord T1 need all resource
– fast scan; slow scanfast scan; slow scan
3232
Our methodOur method
DefectDefect– t1 doing transactiont1 doing transaction– t2 transaction end t2 transaction end – New transaction can’t enter share resourceNew transaction can’t enter share resource
3333
Our methodOur method
ImproveImprove– Applying transaction reorderingApplying transaction reordering
Technique1 maintain Technique1 maintain hash tablehash table (HT) and find (HT) and find desirable transactiondesirable transaction
Technique2 new transaction , check DSTechnique2 new transaction , check DS
ProposePropose– improve the throughput improve the throughput – economize I/O resource economize I/O resource
3434
experimentexperiment
IBM DB2IBM DB2– ConcurrencyConcurrency
deadlock probability and throughput testing resultsdeadlock probability and throughput testing results
– deadlock probability. Fig. 5 and 6deadlock probability. Fig. 5 and 6
P =50% ; s=10000 ; n=1 to 128 ; K =2 or 4 or 6 P =50% ; s=10000 ; n=1 to 128 ; K =2 or 4 or 6 or 8 or 8
3535
experimentexperiment
ExEx
P =50% ; s=10000 ; n=64 ; K =2P =50% ; s=10000 ; n=64 ; K =2
Deadluck probability = 5% Deadluck probability = 5%
3636
experimentexperiment
Predicted deadlock probability Measured deadlock probability of the naive
of the naive method method
3838
experimentexperiment
throughput throughput
Naive’Naive’
probabilityprobability
Naive’Naive’
throughputthroughput
ReorderReorder
probabilityprobability
ReorderReorder
throughputthroughput
deadluckdeadluck badbad BadBad GoodGood goodgood
No deadlNo deadluckuck
SimilerSimiler
ReorderReorder
SimilerSimiler
ReorderReorder
SimilerSimiler
NaiveNaive
SimilerSimiler
NaiveNaive
3939
experimentexperiment
Data ratio test results Data ratio test results – K=16; n=64; p=50%; s= 5000 to 20000K=16; n=64; p=50%; s= 5000 to 20000– Fig10Fig10
4040
experimentexperiment
Data ratio test resultsData ratio test results– throughput of Reprdering and naivethroughput of Reprdering and naive