My CloudConnect 2013 Talk: Infrastructure Automation in the Cloud
-
Upload
clogeny-technologies -
Category
Technology
-
view
6.639 -
download
0
description
Transcript of My CloudConnect 2013 Talk: Infrastructure Automation in the Cloud
Innova'on → Execu'on → Solu'on → Delivered
Infrastructure Automation for the Cloud
Chirag Jog, CTO Clogeny [email protected]
Clogeny Confidential
http://www.clogeny.com © 2013 Clogeny Technologies
About Me
• Strong Experience in DevOps, Infrastructure Automation
• Built cutting edge products: • Hybrid Cloud Migration Appliances • Enterprise Java PaaS • Multi-cloud single pane of glass Dashboard
• Experience building highly scalable automated cloud applications
• Leading engineering teams
Co-founder/CTO at Clogeny
http://www.clogeny.com © 2013 Clogeny Technologies
What is my talk about?
Infrastructure Automation for the Cloud
http://www.clogeny.com © 2013 Clogeny Technologies
What is my talk about?
Infrastructure Automation for the Cloud
http://www.clogeny.com © 2013 Clogeny Technologies
What is Infrastructure?
App LBs
App Servers
NoSQL
DB slaves
Cache
DB Cache
DBs
http://www.clogeny.com © 2013 Clogeny Technologies
What is Infrastructure?
DC1
DC3
DC2
http://www.clogeny.com © 2013 Clogeny Technologies
Problems in Managing Infrastructure
" Configuration Drift
" Changes causes Outages
" Wall of Confusion – Dev and Ops
" Non Functional Requirements: • Backups, security, monitoring, synchronization,
performance, capacity planning
" Dev, Test and Prod – All Different Environments
" Mystery Machine: On But don’t know why?
" Code Complete != Production Ready
http://www.clogeny.com © 2013 Clogeny Technologies
Infrastructure is Hard !!
http://www.clogeny.com © 2013 Clogeny Technologies
Old Problem: Infrastructure is Hard
http://www.clogeny.com © 2013 Clogeny Technologies
Old Problem: Infrastructure is Hard
" Proprietary Solutions
http://www.clogeny.com © 2013 Clogeny Technologies
Old Problem: Infrastructure is Hard
" Solve very little of the problem " Enterprise Focused " Large Professional Services Bills " On-premise " Acquired by companies with large consulting
organizations " Google, Amazon, Microsoft built their own tools
but its secret sauce
" Proprietary Solutions
http://www.clogeny.com © 2013 Clogeny Technologies
Infrastructure is Changing !!
http://www.clogeny.com © 2013 Clogeny Technologies
Cloud Computing
http://www.clogeny.com © 2013 Clogeny Technologies
New Problem: Infrastructure is Changing
" Easier to get .. but harder to manage
" Demand is dynamic
" Developers are crucial to Operations
" Web/Cloud services are proliferating
" Manual Configuration no longer a crutch
" Few tools solve all the problems
" No Shrink Wrap software
http://www.clogeny.com © 2013 Clogeny Technologies
Infrastructure is Code
" API Driven Abstractions " Infrastructure is now an Application
App LBs
App Servers
NoSQL
DB slaves
Cache
DB Cache
DBs
http://www.clogeny.com © 2013 Clogeny Technologies
Agile Infrastructure: The Final Frontier for Agile
http://www.clogeny.com © 2013 Clogeny Technologies
Agile Infrastructure
" Infrastructure is Code: Apply the lessons learnt from Software Development
" Build more Flexible systems and faster feedback
" Beyond Continuous Integration: Continuous Delivery
" 10+ deploys per Day – Flickr, 2009
http://www.clogeny.com © 2013 Clogeny Technologies
Amazon.com
" Mean time between deployments: 11.6s
" Max # of deploys per hour : 1079
" Mean # of host simultaneously receiving a
deployment: 10,000
" Max # of hosts receiving a deployment: 30,000
http://www.clogeny.com © 2013 Clogeny Technologies
What is my talk about?
Infrastructure Automation for the Cloud
http://www.clogeny.com © 2013 Clogeny Technologies
Agile Infrastructure Techniques
u Configuration Management
u Version Control Everything
u Build from source
u One step deploy
u Monitoring
u Continuous Delivery
u Work Together – Dev and Ops
http://www.clogeny.com © 2013 Clogeny Technologies
What is Infrastructure (again) ?
• Nodes • Networking • Files • Directories • Symlinks • Mounts • DB Schema
Infrastructure is a collection of resources:
• Routes • Users • Groups • Tasks • Packages • Software • Services • Configurations • Stuff
http://www.clogeny.com © 2013 Clogeny Technologies
Infrastructure as Code (really?)
package "ntp" do ! action :install !end !
service "ntpd" do ! action [:enable,:start] !end !
template "/etc/ntpd.conf" do ! source "ntpd.conf.erb" ! owner "root" ! group "root" ! mode 0644 ! action :create ! variables(:time_server => “time.example.com”) ! notifies :restart, “service[ntpd]”!end !
http://www.clogeny.com © 2013 Clogeny Technologies
Configuration Management
" Infrastructure as Code " Put systems into a known state " Audit and enforce consistency " Manage server lifecycle " Reason about services, instead of systems " Apply dev-test-prod cycle to infrastructure
http://www.clogeny.com © 2013 Clogeny Technologies
Configuration Management Tools
http://www.clogeny.com © 2013 Clogeny Technologies
Version Control Everything
" Network Configurations " System Configurations " Application Configurations " Application Code " Database Schema " Documentation " Anything else that matters!
http://www.clogeny.com © 2013 Clogeny Technologies
Build from Source
" Build Application and Infrastructure from source
" Dev, Test and Prod in sync
" Test from a known state
" Scaling
" Disaster Recovery
http://www.clogeny.com © 2013 Clogeny Technologies
One Step Deploy
" One automated process from version control to live services
" Lower the time and cost to deploy
http://www.clogeny.com © 2013 Clogeny Technologies
Monitoring
" Define ‘Normal’ Conditions " Monitor Baselines, Charts, trends " Monitor, Monitor and Monitor !
Ref: http://consultingblogs.emc.com/blogs/gracemollison/image_thumb_123A5775.png
http://www.clogeny.com © 2013 Clogeny Technologies
Continuous Integration
" Check in regularly " Create an automated and comprehensive test
suite. " Test New builds " Assert Services are running " Deploy early and often " No manual steps! " Never go home on a broken build " Build a pipeline!
Poll Master and
Compile
Run Unit Tests
Package
Deploy
Run Smoke/
Integration Tests
Release
http://www.clogeny.com © 2013 Clogeny Technologies
Continuous Delivery Pipeline
SCM Build CI Analyze Deploy Config
Private Clouds
Public Clouds
Repo Infrastructure
Continuous Integration
Continuous Deployment
http://www.clogeny.com © 2013 Clogeny Technologies
Work Together – Dev and Ops
" Dev and Ops see the same thing, in the same place
" Share Metrics " Share the Repository • Configs in sync with Application Code
" Minimize Surprise " Always ship trunk/master
http://www.clogeny.com © 2013 Clogeny Technologies
How often do we see this?
http://www.clogeny.com © 2013 Clogeny Technologies
Work Together – Dev and Ops
" Devs need to understand the infrastructure " Ops needs to understand the Application " Manage Flows – Avoid Fires " Build a culture – Avoid the blame game " Break the silos and work together! " Collaboration: not only when things go wrong " No Rock star mentality
http://www.clogeny.com © 2013 Clogeny Technologies
What is DevOps ?
DevOps: is a software development method that stresses communication, collaboration and integration between Dev and Ops.
http://www.clogeny.com © 2013 Clogeny Technologies
What is DevOps ?
DevOps really is: " Processes " Tools " Culture
http://www.clogeny.com © 2013 Clogeny Technologies
DevOps
" Processes and Tools • Automated infrastructure • Shared version control • One step build and deploy • Shared metrics
" Culture • Respect • Trust • Healthy attitude about failure • Avoiding Blame • Superhero Culture
http://www.clogeny.com © 2013 Clogeny Technologies
Conclusion
ü Agile Infrastructure ü Embrace DevOps ü Business Agility ü Improved Time to Market ü Ahead of the competition
http://www.clogeny.com © 2013 Clogeny Technologies
Questions ?
http://www.clogeny.com © 2013 Clogeny Technologies
References
" Dev2Ops - http://dev2ops.org
" Flickr’s 10 deploys per day - http://www.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-cooperation-at-flickr
" Agile Infrastructure - http://www.slideshare.net/littleidea/agile-infrastructure-agile-2009
" DevOps – The war is over - http://www.slideshare.net/jedi4ever/devops-the-war-is-over-if-you-want-it
" DevOps, the future is here – http://www.slideshare.net/KrisBuytaert/devops-the-future-is-here-its-just-not-evenly-distributed-yet
" Jesse Robbins’s DevOps talk - http://www.slideshare.net/jesserobbins/devops-interop-las-vegas-jesse-robbins-opscode