C* Summit 2013: Cassandra on Cloud Foundry by Renat Khasanshyn and Cornelia Davis
-
Upload
planet-cassandra -
Category
Technology
-
view
3.381 -
download
3
description
Transcript of C* Summit 2013: Cassandra on Cloud Foundry by Renat Khasanshyn and Cornelia Davis
Your own Cassandra as a Service
with Cloud Foundry
2
About Altoros
Services
Cloud Foundry implementation services
Hadoop/NoSQL - Benchmarking, Deployment and Customization
Cluster management and optimization in private/hybrid clouds
Customers
• Infrastructure software vendors
• IaaS service providers
• Information-heavy enterprises
Focus
• Platform as a Service (Cloud Foundry implementation services)
• Data science enablement (Hadoop/NoSQL Deployment services)
• Cluster management and optimization in private/hybrid clouds
(RightScale/Chef/Puppet cloud automation template services)
Number of
Engineers/Consultants250
Headquarters Sunnyvale, CA
Offices Location
3
About Pivotal
Cloud Storage
Virtualization
Data &
Analytics
Platform
Application
Development
Platform
Rapid
Application
Development
4
About Pivotal
Cloud Storage
Virtualization
PIVOTALData &
Analytics
Platform
Application
Development
Platform
Rapid
Application
Development
5
Why Bother? TCO of stand-alone Cassandra
vs Cassandra as a Service with CloudFoundry
Stand-alone CaaS
Provisioning Days or weeks Minutes
Scalability App by app basis Provided by CaaS. Easily
add new nodes
Development
productivity
Server setup, sysops
requests, etc
Use a simple CLI:
$ cf bind service
API None. In-house? Restfull API
6
What is Cloud Foundry?
7
Cloud Foundry Architecture
8
Why BOSH?
• Open source tool chain for release engineering, deployment and
lifecycle management of large scale distributed services
• IaaS provider independency (VMware vSphere, vCloud Director,
Amazon Web Services EC2, or OpenStack)
• Fast PaaS flexible deployment (on top of private, public or hybrid IaaS
configurations) and scalability
9
BOSH architecture and topology overview
10
Cassandra Service Overview
• As many nodes as
you need
• CaaS will be aware
of provisioning
capacity, status,
availability, servers
where Cassandra
runs, etc
11
Demo Screencast
• Initial BOSH deploy of the system, including Cassandra
• Pushing an app that binds to the Cassandra service
• Updating the BOSH manifest to add another node to the Cassandra
cluster
12
Binding to Cassandra Service
OS
DEA
OS
Cassandra
Node
OS
Cassandra
Node
OS
Cassandra
Node
OS
CassandraGateway
OS
Cloud Controller
OS
Cloud Controller
OS
NATS (messaging)
OS
Router
OS
DEA
AppApp
13
BOSH Managed Cassandra Cluster
_node
_nodecassandra_gateway
NATS
10.13.219.151
10.13.219.152
"credentials":{"host":"10.13.219.152","port":9160,"name":"e55bbca6-a2b5-4c94-a595-
b9aa1ad180b8"}
14
BOSH Managed Cassandra Cluster
_node
_nodecassandra_gateway
NATS
10.13.219.151
10.13.219.152
"credentials":{"host":"10.13.219.152","port":9160,"name":"e55bbca6-a2b5-4c94-a595-
b9aa1ad180b8"}
_node
10.13.219.153
15
...
properties:
...
cassandra:
seeds: 10.13.219.151
port: 9160
...
...
/var/vcap/packages/cassandra/bin/cassandra \
-p $PIDFILE \
-Dcassandra.rpc_port=<%=properties.cassandra.port%> \
>>$LOG_DIR/cassandra.stdout.log \
2>>$LOG_DIR/cassandra.stderr.log
...
...
rpc_address: <%= spec.networks.default.ip %>
rpc_port: <%= properties.cassandra &&
properties.cassandra.port || 9160 %>
...
seed_provider:
- class_name:
org.apache.cassandra.locator.SimpleSeedProvider
parameters:
- seeds: <%=properties.cassandra.seeds%>
deployment-manifest.yml
cassandra_ctl (
cassandra.yaml.erb
jobs
cassandra
packages
blobs
templates
16
Cloud Foundry Cassandra Service Use Case
• SaaS solutions built with xCP for the cloud
• Cassandra • Metadata lookup
• Which xDB server holds the metadata?
• Configuration
• Shared knowledge for all stateless data servers
• Scale out stateless data servers as CloudFoundry applications• Common security across all data services
• Multi-tenant provisioning
• Scale out xCP applications as CloudFoundryapplications
17
CloudFoundry Services
Clo
ud
Foundry
Apps
CloudFoundry Router
xCP Client
xDB cluster (metadata)
xDBServer
xDBServer
xDBServer
DataServer
DataServer
DataServer
DataServer
Cassandra cluster
Cass. Server
Cass.Server
Cloud Blob Stores
BLOBBLOBBLOB
xCP Application xCP Application xCP Application
Base scale-out architecture
18
History of Cassandra Service for CF
• V1 Services - Present
• Temporary V2 Services without Cassandra (June-Aug 2013)
• V3 Services – Aug-Sept 2013
19
Coming up –Multi-node Cassandra on V2 Cloud Foundry, with V3
Services on AWS
20
Stand-alone Cassandra vs Cassandra as a Service with CF
• Days of installation and administration knowledge and research
capitalized in single and stable service deployments.
• Free up developers from dealing with scalability and high availability
issues
• Guarantee consistent deployments among several environment
(development, staging and production) without misconfiguration risks
that would explode in production.
• Easy to scale, vertically and horizontally
21
Stand-alone Cassandra vs Cassandra as a Service with CF
• Manage different configurations (Services Plans) depending on your
needs.
• Made a bad sizing decision? Just reconfigure your app to use a
bigger/smaller service plan
• Provisioning: from days and weeks to a couple of minutes for a tested
and stable Cassandra
22
Extending CloudFoundry: Buildpacks
• Cloudfoundry provides a convenient way of packaging framework
and/or runtime support for your application with Buildpacks
• Based on Heroku open source buildpacks model
• Can’t find one that fits? You can extend and customize one to fit your
needs.
23
Thank you!Reach out to our teams for expert help on automated deployment, consulting
and integration of Cloud Foundry and Cassandra:
Altoros Pivotal
@renatkhasanshyn
(650) 395-7002
@cdavisafc
(805) 560-9039