Primal Ladis

download Primal Ladis

of 30

Transcript of Primal Ladis

  • 7/31/2019 Primal Ladis

    1/30

    DISTRIBUTED INDEXING AND LOCKING :IN SEARCH OF SCALABLE CONSISTENCY

    Mahdi Tayarani Najaran, Primal Wijesekera, AndrewWarfield and Norman C. Hutchinson

    Networks, Systems,andSecurity Lab (NSS)

  • 7/31/2019 Primal Ladis

    2/30

    Why isnt there just one instance ofgiant SQL Database ?

    2

  • 7/31/2019 Primal Ladis

    3/30

    SQL

    3

  • 7/31/2019 Primal Ladis

    4/30

    Alternative - 01

    Data Partitioning

    4

  • 7/31/2019 Primal Ladis

    5/30

    Alternative - 02

    NOSQLKV StorageSystems

    5

  • 7/31/2019 Primal Ladis

    6/30

    What about SQL

    Highly Optimized Sound mathematical base

    Highly adopted across domains Very rich query model

    6

  • 7/31/2019 Primal Ladis

    7/30

    What is wrong with SQL

    Why we think that we can do better

    Our Solution

    Evaluation

    What is next

    7

  • 7/31/2019 Primal Ladis

    8/30

    We think

    SQL is just great but

    Developers arent

    8

  • 7/31/2019 Primal Ladis

    9/30

    We have a great platform

    SinfoniaEx: A Distributed Transactional Memory Service An open source implementation of Sinfonia

    9

  • 7/31/2019 Primal Ladis

    10/30

    Minitransaction

    A given minitransaction has A read set A compare set A conditional write set

    All updates occur atomically only if all comparisonssucceed

    Locks are held only for a very small time

    10

  • 7/31/2019 Primal Ladis

    11/30

    What is wrong with SQL

    Why we think that we can do better

    Our Solution

    Evaluation

    What is next

    11

  • 7/31/2019 Primal Ladis

    12/30

    Replacethe single-nodestoragelayer of an off-the-shelf DBMS engine(eg. MySQL) with a distributed storage

    layer

    12

  • 7/31/2019 Primal Ladis

    13/30

    The Design

    13

  • 7/31/2019 Primal Ladis

    14/30

    The Design contd.

    14

  • 7/31/2019 Primal Ladis

    15/30

    Why it is hard to do so

    Distributed Indexing With high expressibility

    Distributed Locking With complex range locking

    ( Distributed Storage )

    Fundamental Requirements Distributed Scalability even in the presence of write heavy workloads Fault tolerance Strict consistency guarantees

    15

  • 7/31/2019 Primal Ladis

    16/30

    Getting Expressibility

    B-Tree Single-attribute range queries Suffers for write-intense workloads Balanced tree

    Spatial Partitioning Tree Multi-attribute range queries Scales for all types of workloads Un-balanced tree

    We picked Spatial Partitioning Tree (SPT) over B-Tree

    Store data only in leaf nodes to avoid contention SPTs increase partitioning granularity by re-partitioningleaves when required

    16

  • 7/31/2019 Primal Ladis

    17/30

    3-D Spatial Partitioning Tree

    src : wikipedia.org

    17

    18

  • 7/31/2019 Primal Ladis

    18/30

    SPT for Indexing

    ALTER TABLE orders

    ADD CONSTRAINTpkey_ordersPRIMARY KEY(o_w_id, o_d_id, o_id);

    18

    19

  • 7/31/2019 Primal Ladis

    19/30

    SPT for Locking Manager

    Modified SPT from Index Manager Key is a range. Range insert => lock acquiring Range removal => lock releasing

    Supports both records locking and complex rangelocking.

    All other locking semantics are intact.

    Read and write locks.

    19

    20

  • 7/31/2019 Primal Ladis

    20/30

    SinExTree

    Is the SPT implementation on top of SinfoniaEx Gives a transactionaly consistent and scalable approach Fulfills all the fundamental requirements

    20

    21

  • 7/31/2019 Primal Ladis

    21/30

    Minitransactions in SPT

    Writing a value to Node L

    In this instance, Compare set = {A, C, E, H, J, L} Conditional write set = {L}

    21

    22

  • 7/31/2019 Primal Ladis

    22/30

    Final Architectural Setup

    22

    23

  • 7/31/2019 Primal Ladis

    23/30

    What is wrong with SQL

    Why we think that we can do better

    Our Solution

    Evaluation

    What is next

    23

    24

  • 7/31/2019 Primal Ladis

    24/30

    Evaluation Setup

    The main objective is to measure the throughput of the indexunder maximum stress at different scales

    Log and replay Logged the index operations of TPC-C benchmark Replayed logs on the distributed index

    Used Amazon EC2 using 72 High CPU Extra Large instances

    Varied the number of index hosts from 1 to 64 with 8 fixed

    memnodes

    24

    25

  • 7/31/2019 Primal Ladis

    25/30

    Evaluation ResultsThroughput

    25

    26

  • 7/31/2019 Primal Ladis

    26/30

    Evaluation ResultsThroughput

    26

  • 7/31/2019 Primal Ladis

    27/30

    28

  • 7/31/2019 Primal Ladis

    28/30

    What is wrong with SQL

    Why we think that we can do better

    Our Solution

    Evaluation

    What is next

    28

    29

  • 7/31/2019 Primal Ladis

    29/30

    Next

    Future Work Integrate all the components into one system Use this setup for a DBMS as a Service

    Future Challenges Making sure the correctness is preserved through out all layers Implementing Security measures to guarantee privacy Define a pricing model with a emphasize on fluctuating workloads

    29

    30

  • 7/31/2019 Primal Ladis

    30/30

    Thank You !!!

    30