Post on 11-May-2015
description
Mark R. Hinkle VP of Community Cloud.com Twitter: @clouddotcom @mrhinkle Email: mrhinkle@cloud.com mrhinkle@gmail.com
Open Source Toolchains to Manage Cloud Infrastructure
2
Introduction • Responsible for Driving Adoption of
CloudStack Open Source Cloud Computing Software
• Former manager of Zenoss Open Source project 100,000 users, 1.5 million downloads
• Former Linux Desktop Advocate (Zealot?) • Former LinuxWorld Magazine Editor-in-Chief • Open Management Consortium Conspirator • Open Desktop Consortium Instigator • Author - “Windows to Linux Business Desktop
Migration” - Thomson • NetDirector Project - Open Source
Con!guration Management Project • Sometimes Author and Blogger at
SocializedSoftware.com/NetworkWorld • Start-up junkie, Glutton for punishment
3
Agenda
• Introductions • Cloud & Systems Management Overview • Open Source Tools for Cloud
Management • Creating Open Source Toolchains • Questions (and Hopefully Answers)
Cloud Computing & Systems Management Overview
5
Jevon’s Paradox and the Cloud
1865, the English economist William Stanley Jevons
…the efficiency with which a resource is used tends to increase (rather than decrease) the rate of consumption of that resource.
…in cloud computing increased consumption can introduces signi!cant management overhead despite higher hardware utilization and other bene!ts
6
Balancing the Equation: Creating the Cloud
+ =
7
Balancing the Equation: Managing the Cloud
+ = ?
8 Automation Balances the Equation • MeatCloud, Can’t Keep up with
Cloud Computing • Devops & Agile IT Philosophy • Script Repetitive Tasks • Integrate Tools • Automate, Automate, Automate
9 DevOps – Agile Management for the Cloud
• Agile, fast moving • Smaller incremental change • Increased coordination across
groups (development/operations) • Eye towards Automation
10
Management Toolchains
Con!guration
Patching and Provisioning
Monitoring Toolchain (n): A set of tools where the output of one tool becomes the input of another tool
11
The Myth of the Nines Availability % Downtime per
Year Downtime per Month
Downtime per Week
99.9% (three nines) 8.76 hours 43.2 minutes 10.1 minutes
99.95% 4.38 hours 21.56 minutes 5.04 minutes
99.99% (four nines) 52.6 minutes 4.32 minutes 1.01 minutes
99.999% (!ve nines) 5.26 minutes 25.9 seconds 6.05 seconds
99.9999% (six nines) 31.5 seconds 2.59 seconds .0605 seconds
Average polling interval for monitoring? 5 minutes? Even superhuman operations people can’t be alerted and take action in under 5 minutes.
One outage per year could drop service level to three nines or worse.
Open Source Tools to Manage the Cloud
13
2011 CLOUD COMPUTING OUTLOOK SURVEY http://www.cloud.com/cloud-computing-outlook
14
4 Types of Management Tools
Provisioning Installation of operating systems and other software
Configuration Management Sets the parameters for servers, can specify installation parameters
Orchestration/Automation Automate tasks across systems
Monitoring Records errors and health of IT infrastructure
15
Cloudy Tools
• Network Capable • Cloud “Aware” • Easy-to-Integrate • Adhere to Open Standards • Lend Themselves to Automation
16 Open Source Provisioning Tools
Year Started Language License Installation Targets
Cobbler 2007 Python GPL Red Hat, OpenSUSE Fedora, Debian, Ubuntu
Fully Automatic Installation (FAI)
2000 Perl GPL Debian
Kickstart ? Python GPL Most .dep and RPM based Linux distros
Spacewalk 2008 Perl, Python, Java GPL Fedora, Centos
Crowbar/Chef 2011 Ruby Apache ?
17 Open Source Configuration Management Tools
Year Started Language License Client/Server
Bcfg2 2003 Python BSD Yes
Cfengine 1993 C Apache Yes
Chef 2009 Ruby Apache Chef Solo – No Chef Server - Yes
Puppet 2004 Ruby GPL yes
18 Open Source Monitoring Tools
Year Started License Language Type of Monitoring
Collection Methods
Cacti (RRDTool) 2001 GPL PHP Performance SNMP, syslog
Nagios 1999 GPL C/PHP Availability SNMP,TCP, ICMP, IPMI, syslog
OpenNMS 2000 GPL Java Availability/Performance
SNMP,
Zabbix 2001 GPL C/PHP Availability/ Performance and more
SNMP, TCP/ICMP, IPMI, Synthetic Transactions
Zenoss 2005 GPL Python Availability, Performance, Event Management
SNMP, ICMP, SSH, syslog, WMI
19 Open Source Automation/Orchestration Tools
Year Started
Language License Client/Server
Support Organization
Capistrano 2006 Ruby MIT Yes None
RunDeck 2010 Java Apache Yes DTO Solutions
Func 2007 Python GPL Yes Fedora Project
MCollective 2009 Ruby Apache Yes PuppetLabs
20 Cloud APIs Aren’t Created Equal Open Source Abstractions
API Language
jclouds java
deltacloud python
Lib cloud python
Fog Ruby
Open Source Management Toolchains
22
Toolchain Concept
Source:http://code.google.com/p/devops-toolchain/wiki/ArchitectureComponents
23
Automated Toolchain
Cloud Image Launch
OS Install
System Con!guration
Application Service Orchestration
Capistrano RunDeck Fabric Func
BCFG2 Cfengine Chef Puppet
Cloud: Eucalyptus OpenStack CloudStack Abiquo
OS Install: Kickstart Cobbler Spacewalk
Command and Control
Con!guration
Bootstrapping Prov
isio
ning
Act
ivity
24
Example: GeekNet
Hundreds of servers, serving web, databases, and other infrastructure for some of the world’s most highly trafficked websites – over 40 million visitors per month.
• Servers are automatically built using con!guration management software (Puppet)
• Discovery tool (Zenoss) !nds infrastructure and populates a CMDB then spits out information to scripts that translate information to BIND con!gurations for DNS
• Monitoring tool adds hosts to polling tool (Nagios) to check servers for availability
• As infrastructure changes systems are updated automatically
• Servers can be spun up and managed in minutes, not hours automatically with little or no human interaction
25 Example Toolchain: Online Gaming Studio
Questions?
27
Contact Me Professional: mrhinkle@cloud.com Personal: mrhinkle@gmail.com
Professional: mrhinkle@cloud.com Personal: mrhinkle@gmail.com
Professional: http://open.cloud.com Personal: http://socializedsoftware.com
Twitter: @clouddotcom @mrhinkle
Mark R. Hinkle
VP of Community Open Source Enthusiast
Open Source Toolchains to Manage Cloud Computing
http://www.slideshare.net/socializedsoftware
by Mark R. Hinkle is licensed under a Creative Commons Attribution-ShareAlike 3.0 United
States License.