A New Cache Management Approach for Transaction Processing on Flash-based Database

Da Zhou

2008-4-23

2

Outline

• Introduction of flash memory/magnetic disk• Traditional IO mechanism on magnetic disk• Problem statement• Our method and Experiments

– Committed transactions – Uncommitted transactions– Log-based write

3

Architecture of Hardware

4

Difference on Physical Characteristics

1. Read/Write/Erase

2. Asymmetric speed of read/write/erase

3. Erase before rewrite

4. Limited erase times

5. Read/write page, erase block, 1 block=64 pages

1. Read/Write

2. The same speed of read/write

3. Rewrite in-place

4. No limited write times

5. Read/write sectorNote: one flash page contains 2048 bytes

5

Outline

• Introduction to flash memory/magnetic disk• Traditional IO mechanism on magnetic disk

– Stealing frame– Not forcing pages

• Problem statement• Our method and Experiments

6

Introduction to “Stealing Frame” Algorithm

T1 T2---------------------R(A)W(A) R(B)

T1 reads A and updates A, then T2 read B. At this time, although T1 has not committed, the memory has no more capacity to load B. At this scenario, the updated A is written to disk, and B is loaded.

7

Introduction to “Not Forcing Pages” AlgorithmT1 T2

------------------------

R(A)

W(A)

Commit

R(A)

W(A)

T1 reads A and updates A, then T1 commits. Although T1 has committed, memory still has large free space, so the updated A is still in memory. Another case: After T1 committed, then next transaction T2 need to read the result of the T1. At this case, the updated A will not be written to disk.

8

Outline

• Introduction to flash memory/magnetic disk• Traditional IO mechanism on magnetic disk• Problem statement

– “Stealing frame” on flash memory– “Not forcing pages” on flash memory– LRU on flash memory

• Our method and Experiments

9

Basic Difference

The characteristics of the Server :1. In-place update

2. High lateness on I/O

3. Large memory

The characteristics of the embedded computer:1. Out-of-place update

2. High read/write speed

3. Limited memory

10

Stealing Frame (Magnetic Disk)

• IO Cost: 2 IOs

Magnetic DiskRAM

data

Temporal data

Initial data

Time 1

Time 2

11

Stealing Frame (Embedded Computer)

• Out-place-update– Series update

– Garbage collectionClean block

Erase block

IO: depth

garbage: depth*page

IO:depth write

depth/64 erase

IO Cost

12

Not Forcing Pages (Magnetic Disk)• Large memory

• High cost of IO160G 7200rpm

Average Seek Time 9ms

Configuration options

HP 9000 Superdome 16 slots

HP 9000 Superdome 32 slots

HP 9000 Superdome 64 slots

Min./ Max. Memory

8 GB/512 GB 8 GB/1 TB 24 GB/2 TB

13

Not Forcing Pages (Flash Memory)

Flash memory Magnetic disk

Read 25μs 9ms (Rand.)

Write 200μs 9ms (Rand.)

Erase 1.5ms N/A

• Limited memory

• Low cost of IO

HP iPAQ rw6818 Multimedia

HP iPAQ 112

Memory 64MB 64MB SDRAM

14

LRU on Disk

Tim

e

Tran.T1

t1

t5t4t3t2

t7t6

T5T3

commitcommitVisit GVisit F

Visit E

Visit DVisit CVisit B

Visit A

t9t8

t10 Visit A

According to LRU, A will be firstly outputted to disk. If A is the next data to be read, series update and garbage collection will be unavoidable.

15

Outline

• Introduction to flash memory/magnetic disk• Traditional IO mechanism on magnetic disk• Problem statement• Our method and Experiments

– Committed Transactions– Uncommitted Transactions– Log-based write

16

Committed Transactions

Tim

e

Trans.T1

t1

t5t4t3t2

t7t6

T3T2

commitcommit

Access GAccess F

Access E

Access DAccess CAccess B

Access A

t9t8

Although T2 accesses F later than T3 accesses D, T2 will be outputted from memory earlier than T3 because T2 commits earlier than T3. According to our design, T3 is outputted earlier than T1.

17

Committed Transactions

(a)Commi t ted Trans. Perf ormance

02000400060008000

100001200014000160001800020000

ARI ES(HDD) ARI ES(SSD) Our Method

ReadWri teI O

768 transactions are running in the RAM. When 256 transactions are committed, each of other 512 uncommitted transactions only loads half of the data.

According to the result, our method reduces by 63% IO time compared with the ARIES on SSD, even more on the write time.

18

Uncommitted Transactions

Tim

e

Trans.

T1

t1

t5t4t3t2

t7t6

T3T2

Access D

Access CAccess B

Access A

Access G

Access F

Access E

The AAT of T1, T2, T3 is 6.5, 3.5, 4, then the output order is T2, T3, T1. So our method reflects the real situation of the transaction replacement mechanism of the RAM.

nAiAATn

i

/1

Average Access Time

19

Uncommitted Transactions

(b)Uncommi t ted Trans. Perf ormance

0

5000

10000

15000

20000

25000

30000

35000

40000

ARI ES(HDD) ARI ES(SSD) Our Method

ReadWri teI O

1024 transactions run in the RAM at the same time. Every transaction loads a page of data in turn.

Our AAT-based method can enhance the replace right ratio, reduce the temporary data, and then improve by 11%

20

Log-based Write

T1(A)

Log(T2)

Log(T1)

T2(H)

T2(G)

T2(F)

T1(D)

T1(C)

T1(B)

T2(E)

RAM

Flash memory

When the transaction is outputted from the RAM, the modifications of transaction are organized as logs. The logs are written to the flash memory instead of data.

21

Log-based Write

T1(A)

Log(T1)

T1(C)

T1(B)

T1(A)

T1(D)

T1(C)

T1(B)

RAM

Flash memory

A

B

C

D

If the transaction needs to read the data again, we must to read the log and the original data, merge the data and its log to get the newest data.

When the transaction is outputted from RAM, the logs are written to flash memory, and the data is not deleted in RAM. Only when new data is loaded into RAM, the data will be deleted.

22

Log-based Write

( c)Log- based Wri te Perf ormance

0

5000

10000

15000

20000

25000

Data- based Wri te Log- based Wri te

ReadWri teI O

1024 transactions run in the RAM at the same time. Every transaction loads a page of data in turn.

Although the performance of read increases by 25%, the total performance enhances by 34% as write performance improves by 75%.

23

Total Design

Tran. Num.

RA

M

N1 N2 N3

Committed Trans. Uncommitted Trans. With Stealing pagesUncommitted Trans.

The uncommitted transactions have higher priority than the committed transactions in memory when transaction must be outputted to flash memory. As shown in figure 3, the number of committed transactions reduces and that of uncommitted transactions with stealing pages increases when the number of total transactions increases. When only uncommitted transactions are in memory, AAT is used to decide which transaction will be outputted to flash memory. When an uncommitted transaction is outputted, the modifications are formed as logs which are written to flash memory instead of the data.

Thank You