Pipe[r] Dreams: Making NoSQL Great · Making NoSQL Great Again Kathryn Dahlgren DisorderlyLabs@UCSC...
Transcript of Pipe[r] Dreams: Making NoSQL Great · Making NoSQL Great Again Kathryn Dahlgren DisorderlyLabs@UCSC...
Pipe[r] Dreams: Making NoSQL Great
Again
Kathryn Dahlgren DisorderlyLabs@UCSC
HPTS, 10Oct2017
TL;DR• NoSQL is great, but could be better.
• Problem: The k-implementation of database management functionality across the total set of instances of
NoSQL products around the world is sub-optimal.
• Goals: 1. Motivation? 2. Articulate solution qualities. 3. Present Piper as a model.
2
BACKGROUND+
MOTIVATION
4
Ancestral Database Systems
5
Ancestral Database Systems
6
Ancestral Database Systems
7
Ancestral Database Systems
8
Database Management Systems
91980s-1990s
101980s-1990s
1990s11
Early 2000s12
• Revolution!• Coincides w/ rise of Internet + code repos (SourceForge, Github, …)• 200+ NoSQL tools in existence (http://nosql-database.org/index.html)
2000s13
• Flexibility
• Customizability
What makes NoSQL Great?
2010s14
15
16
17
Problem
• Raw NoSQL systems are Database Systems (in the common case)
• Transforming a NoSQL product into a Database Management System necessitates engaging in a k-implementation nightmare with all the other NoSQL users in the world.
Valid Solution Qualities
Thou Shalt Not :1. impede flexibility
2. jeopardize customizability
3. hamper dissemination
18
19
Piper
• Developers working with NoSQL systems can: 1. install management packages from the index. 2. publish management packages to the index.
• Usage standards impose strict regulations on the ease of package installation + deinstallation.
• Inspired by • Package indexes (PyPI, NPM, …) • Source code repositories (Github, BitBucket, …)
20
Piper in action!
aggsPack
SuperApp
piper_pickledb
Devyn
MongoDB
21
Piper in action!piper_pickledb
Piper SuperApp
piper_mongodb
MongoDB
Devyn
aggsPack
22
Piper in action!piper_pickledb
Piper SuperApp
piper_mongodb
simpleJoinMongoDB
Devyn
aggsPack
piper_mongodb
MongoDBsimpleJoin
23
Piper in action!piper_pickledb
aggsPack
Piper SuperApp
Devyn
piper_mongodb
MongoDBsimpleJoin
24
Piper in action!piper_pickledb
aggsPack
Piper SuperApp
Devyn
25
Piper in action!piper_mongodb piper_pickledb
simpleJoin
Piper SuperApp
piper_mongodb
MongoDB
Devyn
simpleJoin
aggsPack
26
Piper in action!piper_mongodb piper_pickledb
simpleJoin
SuperDuperApp Piper SuperApp
piper_mongodb
Eunice
MongoDBPickleDB
Devyn
simpleJoin
aggsPack
27
Piper in action!piper_mongodb piper_pickledb
simpleJoin
Piper SuperDuperApp
simpleJoin
Piper SuperApp
piper_mongodb
Eunice
MongoDBPickleDB
piper_pickledb
Devyn
simpleJoin
aggsPack
28
Piper in action!piper_mongodb piper_pickledb
simpleJoin
PickleDB
Piper SuperDuperApp
MongoDB
Piper SuperApp
piper_mongodb
simpleJoin
piper_pickledb
Eunice Devyn
simpleJoin
aggsPack
Devyn
29
Piper in action!piper_mongodb piper_pickledb
simpleJoin
PyMongo
Piper SuperApp
piper_mongodb
MongoDB
Piper SuperDuperApp
simpleJoin
Eunice
PickleDBsimpleJoin
piper_pickledb
aggsPack
Valid Solution!
30
WISH LIST :1. Promotes flexibility
2. Champions customizability
3. Eases solution dissemination
Valid Solution!
31
WISH LIST :1. Promotes flexibility
2. Champions customizability
3. Eases solution dissemination
Valid Solution!
32
WISH LIST :1. Promotes flexibility
2. Champions customizability
3. Eases solution dissemination
Valid Solution!
33
WISH LIST :1. Promotes flexibility
2. Champions customizability
3. Eases solution dissemination
Project Status• Bare-bones proof-of-concept.
• Supports two NoSQL DBSs : • MongoDB (document store) • PickleDB (key-value store)
• Contains two packages : • aggsPack • simpleJoin
https://github.com/PiperProject
34
Research Directions???
Research Directions?
36
Research Directions?
37
Automatic adapter
generation?
Research Directions?
38
Automatic package optimization per target environment?
Automatic adapter
generation?
Research Directions?
39
Automatic package optimization per target environment?
Automatic adapter
generation?
Generalizability standards?
Conclusion• NoSQL is here to stay • No rules != totally awesome • k-implementation is sub-optimal • Piper is a proof-of-concept tool modeling a valid
solution
40
Conclusion• NoSQL is here to stay • No rules != totally awesome • k-implementation is sub-optimal • Piper is a proof-of-concept tool modeling a valid
solution
Making NoSQL great :-
Being stronger together!
41
Thanks!
42