Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar

download Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar

of 27

  • date post

  • Category


  • view

  • download


Embed Size (px)


This was co-presented at the OpenStack Summit 2013 in Portland by Kamesh Pemmaraju, Product Manager from Dell and Neil Levine Inktank. Inktank Ceph is a transformational open source storage solution fully integrated into OpenStack providing scalable object and block storage (via Cinder) using commodity servers. The Ceph solution is resilient to failures, uses storage efficiently, and performs well under a variety of VM Workloads. Dell Crowbar is an open source software framework that can automatically deploy Ceph and OpenStack on bare metal servers in a matter of hours. The Ceph team worked with Dell to create a Ceph barclamp (a crowbar extention) that integrates Glance, Cinder, and Nova-Volume. As a result, it is lot faster and easier to install, configure, and manage a sizable OpenStack and Ceph cluster that is tightly integrated and cost- optimized. Hear how OpenStack users can address their storage deployment challenges: Considerations when selecting a cloud storage system Overview of the Ceph architecture with unique features and benefits Overview of Dell Crowbar and how it can automate and simplify Ceph/OpenStack deployments Best practices in deploying cloud storage with Ceph and OpenStack

Transcript of Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar

  • 1. Wicked Easy Ceph BlockStorage & OpenStackDeployment withCrowbarKamesh Pemmaraju, DellNeil Levine, Inktank

2. Outline Introduction Dell OpenStack-Powered Cloud Solution Ceph Crowbar Why Crowbar + Ceph in OpenStack? OpenStack Block Storage gap Automation, scale What we have done to enable things? Customer benefits, drivers an example Whats coming next? 3. Clouds demand significant operationaland process controls Operational decisions drive hardwareand software decisions We are finding ways to productizeoperations into best practicesHWSWOPSClouds Require an Operational Focus 4. OPSSWHWSWOPSCrowbarCloudOpsSoftwareServices &ConsultingReferenceArchitectureDell OpenStack-Powered Cloud Solution 5. Dell OpenStack-Powered Cloud SolutionHW + SW + ServicesHardwareHW referencearchitecture C6220, C6105,R720, R720XD servers Storage and compute Force 10 S60 and PowerConnectConfiguration Min 6 nodes. Max 60 nodesSoftwareSoftware OpenStack Installer and continuous integration (Crowbar) OpenStack cloud SW Ceph for distributed storageOperating system Ubuntu 12.04 (host) Windows (guests) Linux (guests)Hypervisor KVMServicesDeployment andconsulting Hardware integration onsite or merge center Software installation onsite OpenStack consulting services Assessment, design and installation Delivered via partnerships with Mirantis and CanonicalSupport HW/Crowbar: Dell ProSupport OpenStack support via Canonical (coming soon) 6. OBJECTS VIRTUAL DISKS FILES & DIRECTORIESCEPHFILE SYSTEMA distributed, scale-outfilesystem with POSIXsemantics that providesstorage for a legacy andmodern applicationsCEPHOBJECT STORAGEA powerful S3- and Swift-compatible gateway thatbrings the power of theCeph Object Store tomodern applicationsCEPHBLOCK STORAGEA distributed virtual blockdevice that delivers high-performance, cost-effectivestorage for virtual machinesand legacy applicationsCeph Unified Storage Software 7. Ceph Key Differentiators Unified storage platform (Object + Block + File) Ceph Intelligent Placement (CRUSH) Ceph Intelligent Devices (Self-healing, P2P) Ceph Intelligent Objects (Embedded Software Classes) Ceph Integration (Linux Kernel, OpenStack, Cloudstack) 8. Ceph Storage Cluster (RADOS)A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodesLIBRADOSA library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHPCeph BlockDevice(RBD)A reliable and fully-distributed blockdevice, with a Linuxkernel client and aQEMU/KVM driverCeph FileSystem(CephFS)A POSIX-compliantdistributed filesystem, with a Linuxkernel client andsupport for FUSECeph ObjectGateway(RGW)A bucket-based RESTgateway, compatiblewith S3 and SwiftAPP APP HOST/VM CLIENTCeph Architecture 9. Ceph Monitor Nodes: Maintain cluster map Provide consensus fordistributed decision-making Must have an odd number These do not serve storedobjects to clientsM Ceph OSD Nodes: One Ceph OSD daemon per disk(recommended) At least three nodes in a cluster Serve stored objects to clients Intelligently peer to performreplication tasks Supports object classesCeph RADOS Components 10. DISKOSDFSbtrfsxfsext4MMMCephStorageCluster(RADOS)CephOSDNodeDISKOSDFSDISKOSDFSDISKOSDFSDISKOSDFS 11. Mission: A Zero Touch Cloud InstallerServers in boxes to full function cloud in under 2 hours Fast & Flexible Bare metal install including BIOS & RAID config Users can choose how their system is configured (barclamps) DevOps Embracing Ongoing Operations Model (DevOps for Clouds) Leverages & Wraps Opscode Chef Open Not specific to OpenStack Dell using for other Apps Not restricted to Dell hardware Apache 2 licensedWhat is Crowbar? 12. Images vs. Layers: OverviewImages: Single Unit Layers: Stacked PiecesIntegrations +Applications +Utilities + OperatingSystemConfigurationOperating SystemUtilitiesApplication BarApplication FooIntegrationsConfiguration 13. Images vs. Layers: LifecycleImages: Replacement Layers: UpgradeI+A+U+O/SConfigOSUBar v1FooIConfigI+A+U+O/SConfigI+A+U+O/SConfigI+A+U+O/SConfigOSUBar v2FooIConfigBar v2 14. Crowbar Uses the Layer Model for DeploymentDellCrowbarOpsManagementCore Components &Operating SystemsCloudInfrastructure &Dell IP ExtensionsPhysical ResourcesAPIs, User Access,& EcosystemPartners* Crowbar has potential to service other programs beyond OpenStack Includes all the components required to implement anentire cloud infrastructure including ecosystems partners Pluggable components deploy cloud infrastructure Allow for addition of Dell IP and expansion by thecommunity services and customers Can integrate with Dell existing products Delivers basic data center services and required cloudinfrastructure. Provision bare-metal servers from box to cloud WITHOUTuser intervention (other than racking/cabling and someminimal configuration questions) NOT EXCLUSIVE TO DELLHARDWARE 15. ServerHow Does Crowbar Work?DellCrowbarOpsManagementOperating SystemNetworkConfigChefServerNetworkConfigAppsDHCPCrowbarMagicHappens!OrchestrationState MachineRun ListConfig.Hardware Config(uses Sledgehammer)Apps 16. DellCrowbarOpsManagementWhat is a Barclamp?BarclampChef RecipesO/S AppPackagesCrowbar API &Partial UIComponents &Scripts Crowbar API & Partial UI Adds states and transitions to Orchestration Customizes UI specific to barclamp Visual extensions to nodes/networks/utils Chef Recipes DevOps description of application Enforced repeatable configuration O/A App Packages Operating system dependencies (Debs & RPMs) Components & Scripts Ad hoc configuration not covered by Chef 17. Modular Design: Barclamps The Configuration YouWant to DeployDellCrowbarOpsManagementCore Components &Operating SystemsCloud Infrastructure& Dell IP ExtensionsPhysical ResourcesAPIs, User Access, &Ecosystem PartnersCrowbarDeployerProvisionerNetwork RAIDBIOS IPMINTPDNS LoggingNova Swift GlanceNagios GangliaenStratus RightScaleDashboardCephHorizonKeystoneForce10 18. We should dothings the newway!Use ObjectStorageApp cant bere-architected!Use local storage!No SPOFsplease!Use this expensiveSAN!How do the Web2.0 folks do it?Use distributedstorage on utilityhardware!The 4 Stages of Cloud Storage Thinking 19. Ceph Block Storage for Dell OpenstackIntegrated: with Cinder and Nova (via qemu-kvm)Compatible: with GlanceFoundations: no SPOFs, self-healing, CRUSHFeatures: Copy-on-Write, Snapshots, CloningIntegrated: with Crowbar for automation 20. If you cant automate it, you cant scale itOSUNovaNodeIConfigOSUNovaAPIIConfigOSUNovaNodeIConfigOSUNovaNodeIConfigOSUNovaNodeIConfigOSUNovaNodeIConfigOSUNovaNodeIConfigOSUNovaDBIConfigOSUNovaCntrlIConfigOSUDashBoardIConfigOSUNagiosIConfigOSUCrowbarChefIConfigOSUCeph MonNodeIConfigOSUCeph MonNodeIConfigOSUCeph MonNodeIConfigOSUCeph OSDNodeIConfig OSUCeph OSDNodeIConfigOSUCeph OSDNodeIConfigOSUCinderAPIIConfigAPI &ServicesComputeNodesStorageNodesScale-Out Operations With OpenStack 21. Ceph Related Barclamps for Crowbar https://github.com/ceph/barclamp-ceph ceph.com packages into the Crowbar Admin Node Ceph OSD roles (ceph-store) Ceph Mon roles (ceph-mon) https://github.com/crowbar/barclamp-nova Hypervisor with RBD (ceph-client) 22. Dell and Inktank have partnered to bring a complete solution forOpenStack + Ceph + Automated deployment with Crowbar The joint solution provides: Crowbar barclamp to Deploy Ceph clusters automatically in a very short time Connect those clusters to OpenStack Professional Services, Support, and Training Collaborative Support for Dell hardware customers Joint Solution Validated against Dell Reference Architectures via theTechnology Partner programWhat Have We Done to Enable? 23. Customer:University to Deploy OpenStack + Ceph Using CrowbarSituation University employs close to 900 researchers and receiveshundreds of million dollars in Grants as a Top 10 ResearchInstitution primarily working on Cancer and GenomicProjects. The University has a need to provide a centralizeddata repository for Researchers in order to insurecompliances concerning retention of data. The intent of the Data Repository is to provide 2TB of freestorage space to each Researcher, with opportunity topurchase from the university more capacity at a veryreasonable cost.Decision Drivers University investigated using an traditional SAN storagesolution which was very expenses on a per TB basis. Theyalso investigated using public cloud storage options whichalso proved to be too expensive. Finally, they looked atHadoop for the project but found that it was not a good fit forthis use case. In the end, Universitychose Dell/Inktank toarchitect a platformthat would be veryscalable and providelost costs per TB andwas the best of allworlds that providecompute and storageon the samehardware. 24. Whats Next? May Ceph Cuttlefish Summer 2013 Dell Grizzly support Ceph Dumpling (August) https://github.com/crowbar/crowbar/wiki/Crowbar-2.0 Crowbar v2.0 work is happening in the open. Check it out andparticipate! 25. Dont Miss the OpenStack Summit SessionsHavana Sessions: Planning the Ceph Roadmap for OpenstackWednesday April 17, 20131:50pm - 2:30pm in Room - B113 Features for Ceph with Cinder and BeyondThursday, April 18, 20139:50 10:30am in Rooms C120+121+122 26. Contact InformationReach Kamesh and Neil for additional information:Dell.com/OpenStackDell.com/CrowbarInktank.com/DellKamesh_Pemmaraju@Dell.com@kpemmarajuNeil.Levine@Inktank.com@neilwlevineVisit the Dell and Inktank booths in the OpenStack Summit Expo Hall