Enhanced Performance Scenarios Using DBMS+OSDMySQL Architecture 17 Some Detail of MySQL Storage...
Transcript of Enhanced Performance Scenarios Using DBMS+OSDMySQL Architecture 17 Some Detail of MySQL Storage...
Biplob Debnath
Mohamed Khalefa
Alex Marchenko
Mohamed Mokbel
David Lilja
Enhanced Performance Scenarios Using
DBMS+OSD
Outline
! Overview of OSD
! Overview of SUN Solaris OSD Implementation
! Overview of DBMS Storage Manager
! Why DBMS + OSD?
! Overview of MySQL Architecture
! Integration Detail
! Current Status
2
Object-based Storage Device (OSD)
Object
! An object is a logical unit– File-like access methods
– Attributes describes the characteristics
– Security policies authorizes access
– Variable size and can store any type of data
– Application can decide what should go into object
– Dynamically shrink and grow4
Object Hierarchy
! The Root object: The OSD itself
! Partition object: Containers for user objects and collections that share
common security and space management characteristics, such as
quotas and keys
! User object: Created by SCSI commands from the application or client
! Collection objects: Index over user objects
5
OSD T10 Commands
6
Traditional File System and OSD
7
A Pictorial View
8
OSD Vs Block-Based Storage Devices
Main Functions Provided OSD
! Data Storage
! Intelligent Layout and Caching
! Metadata Management
! Security
SUN OSD Implementation
11
Database Storage Manger
! The interface between
database systems and disk
storage is in terms of disk
pages (blocks) of
predetermined size (e.g., 8K,
16K, or 32K)
! The data layout on disk is the
N-ary storage model (NSM)
where full records are stored
consecutively within the disk
page.
New Applications
! Peta-scale data size– Data administration becomes challenging
! High-dimensional tables (e.g., 500 attributes in asingle table is common)– NSM data layout is completely inefficient
! A radical change in the query workload as manyqueries ask only about very small part of the table(e.g., decision support queries)– Need a radical change in the data layout
OSD + DBMS : Why?
! Over the last three decades annually
– CPU speed has increased by 60%
– while magnetic disks access time improved by 8%
! Caching + Prefetching is used to reduce this gap
! Block-based devices does not have co-relations knowledge among
the stored blocks
! OSD has explicit block co-relations knowledge
– Very useful to make caching + prefetching decisions
! DBMS query optimizer can specify the access pattern to OSD
through shared attributes
– Nested-loop join, SORT, HASH
! Faster Log flushing in DBMS
Nested Loop Query
SORT with OSD
MySQL Architecture
17
Some Detail of MySQL Storage Engines
! MySQL provides a layer of abstraction that permits different
storage engines to access their tables using the same API
(handler).
! CREATE Table Table1 ….
– MySQL creates files named Table1.MYD ("MySQL Data"),
Table1.MYI ("MySQL Index"), and Table1.frm ("Format")
! Each file will be mapped to a single object
! Our interest is in:
– File Create
– File READ
– FILE WRITE
18
Current Status
! Our plan
– Modify mysiam storage manager
– Run TPC-H
– Compared OSD and traditional disks
! Integration effort is on going
– OSD
– MySQL
Thank You!
20
Comments / Questions?
References
! http://opensolaris.org/os/project/osd/
! http://developers.sun.com/solaris/articles/osd.html
! http://developers.sun.com/solaris/articles/osd_solaris.html
! http://forge.mysql.com/wiki/MySQL_Internals_Custom_Engine
! SNIA Tutorial: Object-based Storage (OSD) Architecture and
Systems, Eric Riedel, April 2006.
22