Cloudy Open Source and DevOpsMatt O’Keefe

@mattokeefehttp://www.slideshare.net/mokeefe

About Me

• Architect at Morningstar, Inc.– Liaison Between Development and Operations

• Meetup Organizer– Morningstar Tech Talks– Cassandra Chicago– DevOps Chicago

• Contributor at DevOps.com

Why Cloud Computing?

ScalabilityThe ability of something, especially a computer

system, to adapt to increased demands

Scalability at Morningstar26 countries

3,165 employees

7.4 mil individual investors served245,000 financial advisors served4,200 institutional clients served

70,000 print subscribers

18 data centers1,126 physical servers2,706 virtual servers

4,104 vCPUs383 TB SAN raw storage

491 TB direct attached raw storage27 Systems Engineers

Physical Server to Engineer Ratio

Morningstar (B.C. (Before Cloud)) – 42:1Amazon EC2 – 266:1Microsoft – 1,000:1Google – 10,000:1

Cloud Computing

http://www.bitcurrent.com/future-of-computing-forecast-calls-for-partly-cloudy/

What is Cloud Computing?

NIST Special Publication 800-145

A Simple Definition

Cloud Computing is a paradigm shift toward automation and utility computing. It features

on-demand, self-service, pay-as-you-go, seemingly infinite capacity of computing

resources. Anyone with an Internet connection can obtain anything-as-a-service.

Types of Service Offerings

http://www.oracle.com/us/technologies/036500.pdf

Ecosystem

http://cloud.com/

What is not Cloud Computing?

Cloudwashing

http://www.dilbert.com/strips/comic/2011-01-07/

Cloud Economics

• Jevons Paradox– technological progress that increases the efficiency

with which a resource is used tends to increase (rather than decrease) the rate of consumption of that resource

• HT @swardley

• On Demand Servers can be relatively expensive– Large EC2 instance: $0.34/hr = $248/mo = $2977/yr

Why Cloud Computing?

Automation

http://www.flickr.com/photos/tensafefrogs/2536062254/

“We own just four computers… our laptops” - @jedberg – Reddit.com

Elasticity

Increased Accuracy and Speed

Data Production15 hrs

QA3 hrs

Production3 hrs

QA15 hrs

Production3 hrs

QA3 hrs

Open Source IaaS

Cloud.com Architecture

Cloud.com CloudStack IaaS demo

APIsAgile Operations, Infrastructure as Code

http://ldsphilosopher.wordpress.com/2008/09/15/legos-make-better-societies/

DevOps is About CAMS

CultureAutomationMeasurementSharing

http://www.opscode.com/blog/2010/07/16/what-devops-means-to-me/

Culture

http://dev2ops.org/blog/2010/2/22/what-is-devops.html

Tools

http://dev2ops.org/blog/2010/2/22/what-is-devops.html

Deployments

http://dev2ops.org/blog/2010/2/22/what-is-devops.html

Results

http://dev2ops.org/blog/2010/2/22/what-is-devops.html

Business Agility

http://dev2ops.org/blog/2010/2/22/what-is-devops.html

You Build It, You Run It

Corollary

“For practice DevOps I recommend first follow cloud expert and devops expert on Twitter. Next step is automate bulls shit out of everything.”- @DEVOPS_BORAT

http://devops.com/2011/03/22/exclusive-interview-with-devops_borat/

DevOps is About CAMS

CultureAutomationMeasurementSharing

http://www.opscode.com/blog/2010/07/16/what-devops-means-to-me/

DevOps Toolchain

Continuous Delivery

Don’t Do This

http://gcaptain.com/ital-florida-container-casualties

Do This

http://www.wallpaperbase.com/wallpapers/photography/speedboat/speed_boat_1.jpg

Don’t Be a Cowboy

http://ilovepwnage.com/video.php?v=ODA3NQ==

Be a Luger

http://gazetteolympics.freedomblogging.com/files/2009/12/brian-martin-photo.jpg

DevOps is About CAMS

CultureAutomationMeasurementSharing

http://www.opscode.com/blog/2010/07/16/what-devops-means-to-me/

Anticipate Failure

Monitoring as an Enabler

http://codeascraft.etsy.com/2010/12/08/track-every-release/

Open Source Monitoring

Instrumentation• ERMA – Extremely Reusable Monitoring API

https://github.com/erma/erma

Monitors

Using EventMonitors

protected void doValidate(RequestContext context, Object formObject, Errors errors) throws Exception {

super.doValidate(context, formObject, errors);

if (errors.hasErrors()) { EventMonitor validationMonitor = new EventMonitor("ValidationErrors"); validationMonitor.set("errors", errors.getAllErrors()); validationMonitor.fire(); }}

CompositeMonitors

Using TransactionMonitors

public Hotel findHotelById(Long id) { TransactionMonitor monitor = new TransactionMonitor(getClass(), "findHotelById"); monitor.set(“id”, id); try { Hotel hotel = em.find(Hotel.class, id); monitor.succeeded(); return hotel; } catch (RuntimeException e) { monitor.failedDueTo(e); throw e; } finally { monitor.done(); }}

Annotations

@Monitoredpublic interface UnderpantsGnomes { void collectUnderpants(); void ?(); void profit(); }

Spring/AspectJ

<aop:config> <aop:aspect id="transactionMonitorActionAspect" ref="transactionMonitorActionAdvice"> <aop:pointcut id="transactionMonitorActionPointcut“ expression="target(org.springframework.webflow.execution.Action) and args(context)"/> <aop:around pointcut-ref="transactionMonitorActionPointcut“ method="invoke"/> </aop:aspect></aop:config>

<bean id="transactionMonitorActionAdvice" class= "c.o.webframework.aop.aspectj.TransactionMonitorActionAdvice"/>

Event Patternswl|httpIn_/shop/airsearch/search/air/pageView_airResults|13180 wl|RoundTripAirSearchAction.resolveRoundTripAirLocations|136 wl|jiniOut_LocationFinderService_findAirports|84 tbs-shop-13.31|jiniIn_LocationFinderService_findAirports|31 tbs-shop-13.31|jiniOut_AirportLookupService_findLocationByIATACode|9 market-8.4|jiniIn_AirportLookupService_findLocationByIATACode|3 tbs-shop-13.31|jiniOut_AirportLookupService_findLocationByIATACode|15 market-8.4|jiniIn_AirportLookupService_findLocationByIATACode|10 wl|jiniOut_LocationFinderService_findAirports|48 tbs-shop-13.31|jiniIn_LocationFinderService_findAirports|16 tbs-shop-13.31|jiniOut_AirportLookupService_findLocationByIATACode|14 market-8.4|jiniIn_AirportLookupService_findLocationByIATACode|10 wl|AirSearchExecuteAction.search|10422 wl|jiniOut_ShopService_createResultSet|9798 tbs-shop-13.31|jiniIn_ShopService_createResultSet|9601 tbs-shop-13.31|com.orbitz.tbs.host.shop.ShopServiceImpl.createResultSet.AIR|9361 tbs-shop-13.31|com.orbitz.tbs.spi.SpiShopService.createResultSet.AIR|9333 tbs-shop-13.31|jiniOut_LowFareSearchService_execute|9175 air-search-7.2.1|jiniIn_LowFareSearchService_execute|9094 air-search-7.2.1|LowFareSearchRequest|9048 air-search-7.2.1|com.orbitz.afo.lib.search.service.LowFareSearchServiceImpl|9038 air-search-7.2.1|com.orbitz.afo.lib.search.service.LowFareSearchServiceImpl.execute|9037 wl|jiniOut_ShopService_viewResultSet|607 tbs-shop-13.31|jiniIn_ShopService_viewResultSet|486 wl|pageView_airResults wl|jsp.render.air200.page|2475

DevOps is About CAMS

CultureAutomationMeasurementSharing

http://www.opscode.com/blog/2010/07/16/what-devops-means-to-me/

Feedback Loop

Graphite demo

Fast Data Analytics using Rocksteady

http://code.google.com/p/rocksteady/

Thanks!

Stay tuned for more at