Augmenting MySQL with NoSQL options - Data Lifecycles
-
Upload
david-murphy -
Category
Documents
-
view
83 -
download
1
Transcript of Augmenting MySQL with NoSQL options - Data Lifecycles
![Page 1: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/1.jpg)
Augmenting MySQL with Big
Data and NoSQL options
The Data Lifecycle
![Page 2: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/2.jpg)
Lead DBA @ Data Services / ObjectRocket by Rackspace
15+ years in data and information systems, ranging from application develop,
data architecture, system design, and more.
Primary focus – Helping business focus on using data not managing
and storing it.
David Murphy
@davidmurphy_data
www.linkedin.com/in/davidbmurphy/
![Page 3: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/3.jpg)
True genius resides in the capacity for
evaluation of uncertain, hazardous,
and conflicting information.
- Winston Churchill
EVERYONE’S GOT TO HAVE A
GREAT DATA QUOTE RIGHT?!
![Page 4: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/4.jpg)
Lifecycle, say
what
Where are
the
technologies
Why One
isn't enough
How to fit
them
together
OutcomesWe want you to leave here understanding:
![Page 5: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/5.jpg)
This
is
NOT…
a deep dive on any technology
a comprehensive list
a roadmap discussion
the end of the journey
What We Will Cover
![Page 6: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/6.jpg)
What We’ll CoverConcepts What are the lifecycle stages
How to classify your workloads
Terminology
Actions What technologies are there
When to use them
Fitting them together
Why is this better
![Page 7: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/7.jpg)
What are the lifecycle stages
Transient
• Sessions
• Logins
• Shop Cart
Short - Medium
• Feeds
• E-Commerce
• Video Game Stats
Analytics
• Reports
• Summary Data
• Dash boards
Archival
• Cold Storage
• Seldom Access
• Governances
L i f e C y c l e
![Page 8: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/8.jpg)
What are the lifecycle stages
Transient
• Sessions
• Logins
• Shop Cart
Short - Medium
• Feeds
• E-Commerce
• Video Game Stats
Analytics
• Reports
• Summary Data
• Dash boards
Archival
• Cold Storage
• Seldom Access
• Governances
L i f e C y c l e
![Page 9: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/9.jpg)
What are the lifecycle stages
Transient
• Sessions
• Logins
• Shop Cart
Short - Medium
• Feeds
• E-Commerce
• Video Game Stats
Analytics
• Reports
• Summary Data
• Dash boards
Archival
• Cold Storage
• Seldom Access
• Governances
L i f e C y c l e
![Page 10: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/10.jpg)
What are the lifecycle stages
Transient
• Sessions
• Logins
• Shop Cart
Short - Medium
• Feeds
• E-Commerce
• Video Game Stats
Analytics
• Reports
• Summary Data
• Dash boards
Archival
• Cold Storage
• Seldom Access
• Governances
L i f e C y c l e
![Page 11: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/11.jpg)
Updated frequently
Ultra fast retrieval
If missing is OK
IS IS NOT
Workloads - Transient
Rich Query-able
Durable
Point of truth
![Page 12: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/12.jpg)
Some to many updates
Rich Query-able
Durable + Point of Truth
IS IS NOT
Workloads - Short to Medium
Built for short term
99% Write 1% Reads
Heavy Aggregations
![Page 13: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/13.jpg)
Heavy Aggregations
More Latency
Massive Parallelized
IS IS NOT
Workloads - Analytics
Rich Query-able
Good for many updates
Point of truth
![Page 14: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/14.jpg)
High / Extreme Latency
Ultra Cheap
Built for Retention
IS IS NOT
Workloads - Archival
Rich Query-able
Updateable
Short Term Storage
![Page 15: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/15.jpg)
Terminology:
Documents Rows
![Page 16: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/16.jpg)
Terminology:
Documents
Columns
Rows
![Page 17: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/17.jpg)
Terminology:
Documents
Columns
Rows
Partition
s
![Page 18: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/18.jpg)
Terminology:
Documents
Columns
Rows
Partition
s
![Page 19: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/19.jpg)
Terminology:
Documents
Columns
Rows
Partition
sGeo & DR
![Page 20: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/20.jpg)
Terminology:
Documents
Columns
Rows
Partition
s
Scaling
Geo & DR
![Page 21: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/21.jpg)
Terminology:
Documents
Columns
Rows
Backups
Partition
s
Scaling
Geo & DR
![Page 22: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/22.jpg)
Terminology:
Documents
Columns
Rows
Backups
Partition
s
Scaling
Geo & DR
The dreaded polyglot persistence
![Page 23: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/23.jpg)
Transient
• Memcache
• CouchBase
• Redis
• SQLite
Medium
• MySQL
• Maria
• PostgreSQL
• Mongo DB
• XtraCluster
• NDB
Analytics
• Hadoop
• InfoBright
• Cassandra
• Teradata
Archival
•Hadoop +
External
•Hadoop
Snapshots
•Cassandra
using S3
Technologies
![Page 24: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/24.jpg)
Fitting it together
• What is the fewest technologies we can use
• What will for new requests
• Do I have plans to handle each stage of data?
• If not can the technologies do a decent job on the
odd case?
• Have talent now? Can I get a service or person easily?
![Page 25: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/25.jpg)
Fitting it together - tools
Build a matrix with
• Features needs ( Transactions, Persistent , Geo,…)
• Importance ( 1- 5)
• Current or Attainable Talent ( 1 -5 )
• Does its Licensing work for this project ( 0 or 1)
(Features * Importance * Talent * License) = Combined Rank
![Page 26: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/26.jpg)
Klout’s great example, but it’s polyglot!
![Page 27: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/27.jpg)
Appboy getting better!
![Page 28: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/28.jpg)
How it should be…
![Page 29: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/29.jpg)
How to scale – focus on what you know
You scale your app by letting someone else
• Build the hardware
• Know the Ops side for the technology
• Make the technologies pass data as its ages vs duplicating
the data
• Be the experts
• You just focus on the features of your app and make $$$
![Page 30: Augmenting MySQL with NoSQL options - Data Lifecycles](https://reader030.fdocuments.in/reader030/viewer/2022020123/55ad9f241a28abe72d8b46e1/html5/thumbnails/30.jpg)
Questions?
WE ARE HIRING! ( DBA, DevOps, and more)
https://rackertalent.com
https://www.objectrocket.com/careers
Twitter: @dmurphy_data @rackspace @objectrocket
Email: [email protected]
Github: https://github.com/dbmurphy
SlideDeck: https://github.com/dbmurphy/presentations