Post on 02-Dec-2014
description
MagnetoDBScaling Your Data Horizontally on OpenStack
August 19, 2014
Keith NewstadtTechnical Director, Symantec Corporation@knewstadt
tesora.com 2
Management Platform
Identity Notifications Storage Data Sync Events
Norton Family
8/19/14
Infrastructure
tesora.com 3
Symantec Cloud Platform Engineering
8/19/14
Infrastructure as a Service
Infrastructure
Platform Services
Servers Networks Disks Images Ops Identity
Storage Databases Big Data Monitoring Deployment
tesora.com 4
What Kinds of Databases Do We Need?
8/19/14
Relational
NoSQLGood for relational dataIn teams’ comfort zone
Can be difficult to operate
Scales well horizontallyLearning curve
Can be difficult to operate
tesora.com 5
What Kinds of Models Do We Need?
8/19/14
Customer Operated As a
ServicePlatform provides
operational services
Platform provides the database itself
tesora.com 6
MagnetoDB – NoSQL + as a Service
8/19/14
• Open source, OpenStack standards
• Large data sets• Throughput, availability, latency• Secure multi-tenant• Integrated with Keystone
TablesCreate
Load
Index
Query Scan
Configurable Consistency
Data Types (maps, sets,
etc.)
Conditional Operations
MagnetoDB
tesora.com 7
MagnetoDB – A Simple Architecture
8/19/14
API Proxy Layer
Database Layer
MagnetoDB Driver Layer
• Simple ReST (ish) Web service API• Modeled after DynamoDB, plus…
• We’re using Cassandra• HBase also in the works
• Supports different backend databases
tesora.com 8
Some Symantec Use Cases
8/19/14
MagnetoDB
User Profile Data
Searchable Swift Object Metadata
Import from Big Data Processes
Real-time Metrics
DemoUsing MagnetoDB8/19/14
tesora.com 10
Demo – Top 100 Music Charts
8/19/14
Songs
Key Schema Secondary Index
Artist Artist
Track High
Charts
Key Schema
Year
Week-Position
http://bullfrogspond.com/
MagnetoDB
• Tables• Queries
tesora.com 11
Local VM
Environment
8/19/14
MagnetoDB
https://wiki.openstack.org/wiki/MagnetoDB/QA/Tests_on_env_with_devstack
Q&AKeith Newstadt@knewstadtkeith_newstadt@symantec.com
8/19/14