Post on 14-Apr-2017
@cockroachdb
PostgreSQL meetup, November 2015CockroachDB
presented by Peter Mattis / Co-Founder
@cockroachdb
1.Overview of CockroachDB2.SQL Data Model3.Logical Data Storage4.Online/Concurrent Schema Change
Agenda
@cockroachdb
What is CockroachDB?
■Scale out SQL■Distributed■Survivable■Consistent■Open source
@cockroachdb
CockroachDB: Architecture
■Layered abstractions■SQL is starting point■Distributes at map■Replicates at physical layer
SQLTransactional KVMonolithic Map
Raft
@cockroachdb
CockroachDB: Architecture
■Layered abstractions■SQL is starting point■Distributes at map■Replicates at physical layer
Transactional KVMonolithic Map
Raft
GraphSQL
@cockroachdb
CockroachDB: Architecture
■Layered abstractions■SQL is starting point■Distributes at map■Replicates at physical layer
Transactional KVMonolithic Map
Raft
SQL Graph Doc
@cockroachdb
CockroachDB: Architecture
■Layered abstractions■SQL is starting point■Distributes at map■Replicates at physical layer
Transactional KVMonolithic Map
Raft
SQL
@cockroachdb
CockroachDB: Architecture
■Layered abstractions■SQL is starting point■Distributes at map■Replicates at physical layer
Transactional KVMonolithic Map
Raft
SQL
Physical
@cockroachdb
SQL Data Model
@cockroachdb
■Tables
SQL Data Model
@cockroachdb
■Tables
SQL Data Model
Inventory
@cockroachdb
■Tables■Rows
SQL Data Model
Inventory
@cockroachdb
■Tables■Rows■Columns
SQL Data Model
InventoryID Name Price
1 Glove 1.11
2 Ball 2.22
3 Shirt 3.33
4 Shorts 4.44
5 Bat 5.55
6 Shoes 6.66
@cockroachdb
■Tables■Rows■Columns■Indexes
SQL Data Model
InventoryID Name Price
1 Glove 1.11
2 Ball 2.22
3 Shirt 3.33
4 Shorts 4.44
5 Bat 5.55
6 Shoes 6.66
Name
Ball
Bat
Glove
Shirt
Shoes
Shorts
Name_Idx
@cockroachdb
PostgreSQL: Logical Data Storage
@cockroachdb
■Rows are stored in an unordered heap■Indexes are btrees■Primary key is a unique index
PostgreSQL: Data Storage
@cockroachdb
CREATE TABLE test (
id INTEGER PRIMARY KEY,
name VARCHAR,
price FLOAT,
);
PostgreSQL: Example Table
@cockroachdb
INSERT INTO test VALUES (1, “ball”, 3.33);
PostgreSQL: Logical Data Storage
@cockroachdb
INSERT INTO test VALUES (1, “ball”, 3.33);
PostgreSQL: Logical Data Storage
Tuple ID (Page# / Item#) Row
(0, 1) (1, “ball”, 3.33)
test (heap)
@cockroachdb
INSERT INTO test VALUES (1, “ball”, 3.33);
PostgreSQL: Logical Data Storage
Tuple ID (Page# / Item#) Row
(0, 1) (1, “ball”, 3.33)
Index Key Tuple ID
1 (0, 1)
test (heap)test_pkey (btree)
@cockroachdb
INSERT INTO test VALUES (1, “ball”, 3.33);INSERT INTO test VALUES (2, “glove”, 4.44);
PostgreSQL: Logical Data Storage
Tuple ID (Page# / Item#) Row
(0, 1) (1, “ball”, 3.33)
(0, 2) (2, “glove”, 4.44)
Index Key Tuple ID
1 (0, 1)
2 (0, 2)
test (heap)test_pkey (btree)
@cockroachdb
CockroachDB: Logical Data Storage
@cockroachdb
■Keys and values are strings■Monolithic, sorted map
CockroachDB: KV
@cockroachdb
Get(key)
Put(key, value)
ConditionalPut(key, value, expValue)
Scan(startKey, endKey)
CockroachDB: KV Primitives
@cockroachdb
Get(key)
Put(key, value)
ConditionalPut(key, value, expValue)
Scan(startKey, endKey)
Del(key)
CockroachDB: KV Primitives
@cockroachdb
■All tables have a primary key■One key/value pair per column
CockroachDB: Row Storage
@cockroachdb
■All tables have a primary key■One key/value pair per column■Key anatomy:/<table>/<index>/<pkey>/<column>
CockroachDB: Row Storage
@cockroachdb
CREATE TABLE test (
id INTEGER PRIMARY KEY,
name VARCHAR,
price FLOAT,
);
CockroachDB: Example Table
@cockroachdb
INSERT INTO test VALUES (1, “ball”, 2.22);
CockroachDB: Key Anatomy
Key: /<table>/<index>/<key>/<column>
Value
/test/primary/1/name “ball”
/test/primary/1/price 2.22
@cockroachdb
INSERT INTO test VALUES (1, “ball”, 2.22);INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Key Anatomy
Key: /<table>/<index>/<key>/<column>
Value
/test/primary/1/name “ball”
/test/primary/1/price 2.22
/test/primary/2/name “glove”
/test/primary/2/price 3.33
@cockroachdb
INSERT INTO test VALUES (1, “ball”, 2.22);INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Key Anatomy
Key: /<table>/<index>/<key>/<column>
Value
/test/primary/1/name “ball”
.../price 2.22
.../2/name “glove”
.../price 3.33
@cockroachdb
INSERT INTO test VALUES (1, “ball”, 2.22);INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Key Anatomy
Key: /<table>/<index>/<key>/<column>
Value
/1000/1/1/1 “ball”
.../2 2.22
.../2/1 “glove”
.../2 3.33
@cockroachdb
■Key encoding■NULL column values■Unique indexes■Non-unique indexes
CockroachDB: The Details
@cockroachdb
■Keys and values are strings■Columns are typed data■???
CockroachDB: Key Encoding
@cockroachdb
■NULL indicates value does not exist■NULL is weird: NULL != NULL
CockroachDB: NULL Column Values
@cockroachdb
■NULL indicates value does not exist■NULL is weird: NULL != NULL■CockroachDB: NULL values are not explicitly
stored
CockroachDB: NULL Column Values
@cockroachdb
INSERT INTO test VALUES (1, “ball”, NULL);
CockroachDB: NULL Column Values
Key: /<table>/<index>/<key>/<column>
Value
/test/primary/1/name “ball”
@cockroachdb
INSERT INTO test VALUES (1, “ball”, NULL);
INSERT INTO test VALUES (2, NULL, NULL);
CockroachDB: NULL Column Values
Key: /<table>/<index>/<key>/<column>
Value
/test/primary/1/name “ball”
??? ???
@cockroachdb
INSERT INTO test VALUES (1, “ball”, NULL);
INSERT INTO test VALUES (2, NULL, NULL);
CockroachDB: NULL Column Values
Key: /<table>/<index>/<key>[/<column>]
Value
/test/primary/1 Ø
/test/primary/1/name “ball”
/test/primary/2 Ø
@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
■Multiple table rows with equal indexed values are not allowed
CockroachDB: Unique Indexes
@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);
CockroachDB: Unique Indexes
Key: /<table>/<index>/<key> Value
/test/bar/”ball” 1
@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Unique Indexes
Key: /<table>/<index>/<key> Value
/test/bar/”ball” 1
/test/bar/”glove” 2
@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);INSERT INTO test VALUES (2, “glove”, 3.33);INSERT INTO test VALUES (3, “glove”, 4.44);
CockroachDB: Unique Indexes
Key: /<table>/<index>/<key> Value
/test/bar/”ball” 1
/test/bar/”glove” 2
/test/bar/”glove” 3
@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
■NULL is weird: NULL != NULL
CockroachDB: Unique Indexes (NULL Values)
@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (3, NULL, NULL);
CockroachDB: Unique Indexes (NULL Values)
Key: /<table>/<index>/<key> Value
/test/bar/NULL 3
@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (3, NULL, NULL);INSERT INTO test VALUES (4, NULL, NULL);
CockroachDB: Unique Indexes (NULL Values)
Key: /<table>/<index>/<key> Value
/test/bar/NULL 3
/test/bar/NULL 4
@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (3, NULL, NULL);
CockroachDB: Unique Indexes (NULL Values)
Key: /<table>/<index>/<key>[/<pkey>]
Value
/test/bar/NULL/3 Ø
@cockroachdb
CREATE UNIQUE INDEX bar ON test (name);
INSERT INTO test VALUES (3, NULL, NULL);INSERT INTO test VALUES (4, NULL, NULL);
CockroachDB: Unique Indexes (NULL Values)
Key: /<table>/<index>/<key>[/<pkey>]
Value
/test/bar/NULL/3 Ø
/test/bar/NULL/4 Ø
@cockroachdb
CREATE INDEX foo ON test (name);
■Multiple table rows with equal indexed values are allowed
CockroachDB: Non-Unique Indexes
@cockroachdb
CREATE INDEX foo ON test (name);
■Multiple table rows with equal indexed values are allowed
■Primary key is a unique index
CockroachDB: Non-Unique Indexes
@cockroachdb
CREATE INDEX foo ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);
CockroachDB: Non-Unique Indexes
Key: /<table>/<index>/<key>/<pkey>
Value
/test/foo/”ball”/1 Ø
@cockroachdb
CREATE INDEX foo ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);INSERT INTO test VALUES (2, “glove”, 3.33);
CockroachDB: Non-Unique Indexes
Key: /<table>/<index>/<key>/<pkey>
Value
/test/foo/”ball”/1 Ø
/test/foo/”glove”/2 Ø
@cockroachdb
CREATE INDEX foo ON test (name);
INSERT INTO test VALUES (1, “ball”, 2.22);INSERT INTO test VALUES (2, “glove”, 3.33);INSERT INTO test VALUES (3, “glove”, 4.44);
CockroachDB: Non-Unique Indexes
Key: /<table>/<index>/<key>/<pkey>
Value
/test/foo/”ball”/1 Ø
/test/foo/”glove”/2 Ø
/test/foo/”glove”/3 Ø
@cockroachdb
■Keys and values are strings■NULL column values■Unique indexes■Non-unique indexes
CockroachDB: Logical Data Storage
@cockroachdb
Logical Data Storage
PostgreSQL CockroachDBKeys are composite structures Keys are strings
Heap storage for rows Required primary key
Per-table heap/indexes Monolithic map
@cockroachdb
Online Schema Change
@cockroachdb
Schema Change Operations
CREATE INDEX foo ON test (col1, col2, …);
ALTER TABLE test DROP col1;
ALTER TABLE test ADD col3 INTEGER;
...
@cockroachdb
Schema Change (the easy way)
1. Lock table2. Adjust table data (add column, populate index,
etc.)3. Unlock table
@cockroachdb
Schema Change (the easy way)
1. Apologize for down time2. Lock table3. Adjust table data (add column, populate index,
etc.)4. Unlock table
@cockroachdb
Schema Change (the MySQL way)
1. Create new table with altered schema2. Capture changes from source to the new table3. Copy rows from the source to the new table4. Synchronize source and new table5. Swap/rename source and new table
@cockroachdb
Schema Change (the PostgreSQL way)
1. CREATE INDEX CONCURRENTLY
@cockroachdb
CockroachDB: Schema Change
■TableDescriptor contains table schema■TableDescriptor replicated on every node■Distributed atomic updates are difficult■Distributed locking is difficult■The easy way isn’t feasible
@cockroachdb
CockroachDB: CREATE INDEX
CREATE INDEX foo ON TEST
1. Backfill index entries2. Add index to TableDescriptor
@cockroachdb
CockroachDB: CREATE INDEX
T1 T2
CREATE INDEX foo ON test… INSERT INTO test…
@cockroachdb
CockroachDB: CREATE INDEX
CREATE INDEX foo ON TEST
1. Add index to TableDescriptor as write-only2. Backfill index entries3. Mark index as read-write
@cockroachdb
CockroachDB: CREATE INDEX
T1 T2
CREATE INDEX foo ON test… INSERT INTO test… orUPDATE test… orDELETE FROM test…
@cockroachdb
CockroachDB: CREATE INDEX
CREATE INDEX foo ON TEST
1. Add index to TableDescriptor as delete-only2. Wait for descriptor propagation3. Mark index as write-only4. Wait for descriptor propagation5. Backfill index entries6. Mark index as read-write
@cockroachdb
Online Schema Change
Online schema change is difficultThe database should do the heavy lifting
@cockroachdb
The End
SQL databases are KV stores on steroids
@cockroachdb
github.com/cockroachdb/cockroach
CockroachLabs.com
@cockroachdb
Thank You