Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big...
-
Upload
roy-summers -
Category
Documents
-
view
215 -
download
0
Transcript of Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big...
![Page 1: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/1.jpg)
Flashing Up the Storage Layer
I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008
Shimin ChenBig Data Reading Group
![Page 2: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/2.jpg)
Motivation: Flash Disks: 64GB – 128GB SSDs available as
of Feb’08 Intel announced 80GB SSDs
Flash disks vs. magnetic disks Same I/O interface: logical 512B sectors No mechanical latency, I/O asymmetry, erase-before-
write: Random reads 10X faster than magnetic disks Random writes 10X slower than magnetic disks, esp
MLC
Exploit flash disks for storage?
![Page 3: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/3.jpg)
Architecture
Flash disk as a cache for magnetic disk? Suboptimal for database workloads
because of write inefficiency
Flash disk and magnetic disk on the same level (This Paper)
![Page 4: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/4.jpg)
ProblemStatement
Page migrations (Storage Manager)
Workload prediction Self-tuning
Page replacement (Buffer Manager)
![Page 5: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/5.jpg)
Outline
Introduction Page placement Page replacement Experimental study Conclusion
![Page 6: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/6.jpg)
Model
Random read/write costs of flash and magnetic disks Page migration decision is always made when a page
is in buffer pool Migration cost == write cost
The ideas are not new. The novel thing here is that logical I/Os are served by buffer pool. Only part of them are seen physically.
![Page 7: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/7.jpg)
r, w: the cost of the current disk; r’, w’: the cost of the other disk
pg.C: a counter per page – the accumulated cost difference
![Page 8: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/8.jpg)
Conservativeness Migration operation only after the cost
of migrating to and back Only physical operations on pages
3-competitive to optimal offline algorithm
![Page 9: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/9.jpg)
![Page 10: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/10.jpg)
Properties
Not conservative on migrations Based on logical operations
![Page 11: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/11.jpg)
Hybrid Algorithm Idea:
Consider both physical and logical operations
More weight on physical ones
If a file has n pages, and b pages are cached in the buffer pool, then Prob_miss = 1 – b/n
![Page 12: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/12.jpg)
![Page 13: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/13.jpg)
Outline
Introduction Page placement Page replacement Experimental study Conclusion
![Page 14: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/14.jpg)
Eviction Cost
Evicting a page: Dirty page incurs write cost Fetching a page back in the future
incurs read cost Cost:
![Page 15: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/15.jpg)
Buffer Pool Organization
Sorted on timestamp
Sorted on cost of eviction
LRU
![Page 16: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/16.jpg)
![Page 17: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/17.jpg)
![Page 18: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/18.jpg)
Impact of λ As λ increases:
Time segment decreases Cost segment increases Disk pages increases, flash pages decreases
Flash pages are evicted first, typically only found in time segment
Let Hm be the increase of disk hit rate, Mf be the increase of flash miss rate So we want
![Page 19: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/19.jpg)
Outline
Introduction Page placement Page replacement Experimental study Conclusion
![Page 20: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/20.jpg)
Experimental Setup Implementation:
Buffer manager, storage manager, B+trees for storing data
Machine: 2.26GHz Pentium4, 1.5GB RAM Debian linux, kernel 2.6.21 Two magnetic disks (300GB Maxtor
DiamondMax) 1 SSD (Samsung MLC 32GB) Data is stored on 1 disk + 1 SSD (both raw
devices)
![Page 21: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/21.jpg)
Experimental Setup Cont’d Capacity of either disk is enough to hold all
data Metadata for files, pages, page mappings, and
free space are not modeled
B+tree is 140MB large, scattered across 1.4GB address space
Buffer pool is 20MB large
![Page 22: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/22.jpg)
Raw Performance: 1 million 4KB random accesses
![Page 23: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/23.jpg)
Impact of Using Both Disks
Conservative + LRU Query mix: read-only, write-only,
read/write Each set of queries executed 15
times
![Page 24: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/24.jpg)
Read-Only
![Page 25: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/25.jpg)
Write-Only
![Page 26: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/26.jpg)
Mixed
![Page 27: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/27.jpg)
Page Placement Algorithms
Infrequently changing workload
![Page 28: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/28.jpg)
Frequently changing workload
![Page 29: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/29.jpg)
Buffer Pool Replacement
![Page 30: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/30.jpg)
Conclusion
Flash disk vs. magnetic disk Page migration and placement Page replacement Can be applied to databases and
file systems (?)
![Page 31: Flashing Up the Storage Layer I. Koltsidas, S. D. Viglas (U of Edinburgh), VLDB 2008 Shimin Chen Big Data Reading Group.](https://reader030.fdocuments.in/reader030/viewer/2022013012/56649e4f5503460f94b45e1c/html5/thumbnails/31.jpg)
Outline
Introduction Page placement Page replacement Experimental study Conclusion