2014 Ceph NYLUG Talk

2014 New York, NYCeph @ NYLUG

Copyright © 2014 by Inktank | Private and Confidential

WHO?

2

Patrick McGarry

Director, Community – Red Hat

/. -> ALU -> P4 -> Inktank

scuttlemonkey

Lies and misinformation!


AGENDA

3

INDUSTRY MUSINGS

INTRO TO CEPH

ARCHITECTURE

COMMUNITY

INKTANK CEPH ENTERPRISE


THE FORECAST

By 2020over 15 ZB of data will be stored.1.5 ZB are stored today.

4


THE PROBLEM

Existing systems don’t scale

Increasing cost and complexity

Need to invest in new platforms ahead of time

2010 2020

IT Storage Budget

Growth of data

5


THE SOLUTION

PAST: SCALE UP

FUTURE: SCALE OUT

6

INTRO TO CEPH


HISTORICAL TIMELINE

8

RHEL-OSP & RHEV Support FEB 2014

MAY 2012Launch of Inktank

OpenStack Integration 2011

2010Mainline Linux Kernel

Open Source 2006

2004 Project Starts at UCSC

Production Ready Ceph SEPT 2012

2012CloudStack Integration

OCT 2013Inktank Ceph Enterprise Launch

Xen Integration 2013

A STORAGE REVOLUTION

PROPRIETARY HARDWARE

PROPRIETARY SOFTWARE

SUPPORT & MAINTENANCE

COMPUTER

DISKCOMPUTE

RDISK

COMPUTER

DISK

STANDARDHARDWARE

OPEN SOURCE SOFTWARE

ENTERPRISEPRODUCTS &

SERVICES

COMPUTER

DISKCOMPUTE

RDISK

COMPUTER

DISK


ARCHITECTURE


ARCHITECTURAL COMPONENTS

11

RGWA web services

gateway for object storage, compatible

with S3 and Swift

LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby,

PHP)

RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors

RBDA reliable, fully-distributed block device with cloud

platform integration

CEPHFSA distributed file

system with POSIX semantics and

scale-out metadata management

APP HOST/VM CLIENT



12

RGWA web services


with S3 and Swift


PHP)







APP HOST/VM CLIENT

OBJECT STORAGE DAEMONS

13

FS

DISK

OSD

DISK

OSD

FS

DISK

OSD

FS

DISK

OSD

FS

btrfsxfsext4

M

M

M

RADOS CLUSTER

14

APPLICATION

M M

M M

M

RADOS CLUSTER

RADOS COMPONENTS

15

OSDs: 10s to 10000s in a cluster One per disk (or one per SSD, RAID

group…) Serve stored objects to clients Intelligently peer for replication & recovery

Monitors: Maintain cluster membership and state Provide consensus for distributed decision-

making Small, odd number These do not serve stored objects to

clients

M

WHERE DO OBJECTS LIVE?

16

??APPLICATION

M

M

M

OBJECT

A METADATA SERVER?

17

1

APPLICATION

M

M

M

2

CALCULATED PLACEMENT

18

FAPPLICATION

M

M

MA-G

H-N

O-T

U-Z

EVEN BETTER: CRUSH!

19

RADOS CLUSTER

OBJECT

10

01

01

10

10

01

11

01

10

01

01

10

10

01 11

01

1001

0110 10 01

11

01

CRUSH IS A QUICK CALCULATION

20

RADOS CLUSTER

OBJECT

10

01

01

10

10

01 11

01

1001

0110 10 01

11

01

CRUSH: DYNAMIC DATA PLACEMENT

21

CRUSH: Pseudo-random placement algorithm

Fast calculation, no lookup Repeatable, deterministic

Statistically uniform distribution Stable mapping

Limited data migration on change Rule-based configuration

Infrastructure topology aware Adjustable replication Weighting



22

RGWA web services


with S3 and Swift


PHP)







APP HOST/VM CLIENT

ACCESSING A RADOS CLUSTER

23

APPLICATION

M M

M

RADOS CLUSTER

LIBRADOS

OBJECT

socket

L

LIBRADOS: RADOS ACCESS FOR APPS

24

LIBRADOS: Direct access to RADOS for applications C, C++, Python, PHP, Java, Erlang Direct access to storage nodes No HTTP overhead



25

RGWA web services


with S3 and Swift


PHP)







APP HOST/VM CLIENT

THE RADOS GATEWAY

26

M M

M

RADOS CLUSTER

RADOSGWLIBRADOS

socket

RADOSGWLIBRADOS

APPLICATION APPLICATION

REST

RADOSGW MAKES RADOS WEBBY

27

RADOSGW: REST-based object storage proxy Uses RADOS to store objects API supports buckets, accounts Usage accounting for billing Compatible with S3 and Swift applications



28

RGWA web services


with S3 and Swift


PHP)







APP HOST/VM CLIENT

STORING VIRTUAL DISKS

29

M M

RADOS CLUSTER

HYPERVISORLIBRBD

VM

SEPARATE COMPUTE FROM STORAGE

30

M M

RADOS CLUSTER

HYPERVISORLIBRB

D

VM HYPERVISORLIBRB

D

KERNEL MODULE FOR MAX FLEXIBLE!

31

M M

RADOS CLUSTER

LINUX HOSTKRBD

RBD STORES VIRTUAL DISKS

32

RADOS BLOCK DEVICE: Storage of disk images in RADOS Decouples VMs from host Images are striped across the cluster

(pool) Snapshots Copy-on-write clones Support in:

Mainline Linux Kernel (2.6.39+) Qemu/KVM, native Xen coming soon OpenStack, CloudStack, Nebula,

Proxmox



33

RGWA web services


with S3 and Swift


PHP)







APP HOST/VM CLIENT

SEPARATE METADATA SERVER

34

LINUX HOST

M M

M

RADOS CLUSTER

KERNEL MODULE

datametadata 0110

SCALABLE METADATA SERVERS

35

METADATA SERVER Manages metadata for a POSIX-compliant

shared filesystem Directory hierarchy File metadata (owner, timestamps,

mode, etc.) Stores metadata in RADOS Does not serve file data to clients Only required for shared filesystem

CEPH AND OPENSTACK

36

RADOSGWLIBRADOS

M M

RADOS CLUSTER

OPENSTACK

KEYSTONE CINDER GLANCE

NOVASWIFTLIBRB

DLIBRB

D

HYPER-

VISORLIBRBD

Ceph Developer Summit

38

• Recent: “Giant”

• March 04-05• wiki.ceph.com • Virtual

(irc, hangout, pad, blueprint, youtube)

• 2 days (soon to be 3?)

• Discuss all work

• Recruit for your projects!

New Contribute Page

39

• http://ceph.com/community/Contribute

• Source tree• Issues• Share

experiences• Standups• One-stop shop

New Ceph Wiki

40

Accepted as a mentoring organization 8 mentors from Inktank & Community http://ceph.com/gsoc2014/ 2 student proposals accepted Hope to turn this into academic outreach

Google Summer of Code 2014

41

http://ceph.com/gsoc2014/

Ceph Days

42

• inktank.com/cephdays

• Recently:London, Frankfurt, NYC, Santa Clara

• Aggressive program

• Upcoming:Sunnyvale, Austin, Boston, Kuala Lumpur

Meetups

43

• Community organized

• World wide• Wiki• Ceph-

community• Goodies

available• Logistical

support• Drinkup to

tradeshow

We haven’t forgotten! Looking for potential founding members Especially important to keep the IP clean

Ceph Foundation

44

Coordinated Efforts

45

• Always need help

• CentOS SIG• OCP• Xen• Hadoop• OpenStack• CloudStack• Ganetti• Many more!


http://metrics.ceph.com

46

THE PRODUCT

Copyright © 2014 by Inktank | Private and Confidential48

INKTANK CEPH ENTERPRISEWHAT’S INSIDE?

Ceph Object and Ceph Block

Calamari

Enterprise Plugins (2014)

Support Services

Subscription-based

Priced on capacity

Single price for all protocols


ROADMAPINKTANK CEPH ENTERPRISE

50

1.2 2.0

CEPH

CALAMARI

PLUGINS

Erasure Coding

RHEL7 Support

Cache Tiering

User Quotas

RADOS Management

Analytics Hosted/SaaS

SNMP, Hyper-V

Ceph 0.77 Firefly Ceph 0.87 “H-Release”

April 2014 September 2014

CephFS

Ubuntu 14.04 Support

VMware

HDFS Support

iSCSI

Intelligent Objects

QoS

2015


Emperor Giant

H

I

J

Inktank Ceph Enterprise v1.1 (Dumpling LTS until May 2015)

Inktank Ceph Enterprise v1.2 (Firefly LTS until November 2015)

RELEASE SCHEDULE

51

2013 2014 2015

Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2

FireflyDumpling


Read about the latest version of Ceph. The latest stuff is always at http://ceph.com/get

Deploy a test cluster using ceph-deploy. Read the quick-start guide at http://ceph.com/qsg

Read the rest of the docs! Find docs for the latest release at http://ceph.com/docs

Ask for help when you get stuck! Community volunteers are waiting for you at

http://ceph.com/help

GETTING STARTED WITH CEPH

52

http://ceph.com/get

http://ceph.com/get

http://ceph.com/qsg

http://ceph.com/qsg

http://ceph.com/qsg

http://ceph.com/qsg

http://ceph.com/docs

http://ceph.com/docs

http://ceph.com/help

THANK YOU!

Patrick McGarryDirector, CommunityRed Hat

[email protected]

@scuttlemonkey

YOUR PICTURE HERE

2014 Ceph NYLUG Talk

Software

Transcript of 2014 Ceph NYLUG Talk