© 2011 VMware Inc. All rights reserved

Nicholas Weaver - Cloud Automation ArchitectVMware vCloud Hybrid Service

VMware vCHS, Puppet, and Project Zombie

2

About me

3

About Nick

Nicholas Weaver

• VMware – Cloud Automation Architect, vCHS

• (formerly) EMC – Office of the CTO, Global Alliances, vSpecialist, Demo-builder

• (formerly) VMware/EMC Architect: Financial, Retail

• Co-creator Project Razor

• Twitter: @lynxbat

• Blog: nickapedia.com

• LinkedIn: www.linkedin.com/in/nicholasweaver

4

What is vCloud Hybrid Service?

5

vCloud Hybrid Service: Extending To The Cloud

VMware vCloud Hybrid Service

Your Data Center

Any Application… No Changes

Software-Defined Data Center

VMware vSphere & vCloud Suite

Existing & New Apps

Seamless Networking

Common management

One Support call

IaaS cloud owned and operated by VMware based on VMware software

6

What do I do?

7

Automation

8

9

10

11

Effort Evolution

12

Why is automation important for vCHS?

13

What is it really doing?

14

15

16

The problem with success is…

17

18

Nothing stays the same

19

20

21

We are not the first

22

23

24

25

26

We need automation special sauce

27

?

28

project zombie

29

Key principles

30

Scale

31

Extensibility

32

Simplicity

33

Resiliency

34

35

36

What is Project Zombie?

37

38

39

40

41

42

Rez

43

Distributed Resource Management

Stuff

• Resources

• State

• Configuration

• Location

• Distributed Locking

• Ownership

• Event History

44

Distributed Resource Management

Stuff

Razor

Razor

StorageManager

StorageManager

Compute Storage

45

Engine

46

Zombie Engine DSL - ZED

47

Zombie Engine DSL - ZED

PuppetApply

MCollective

MCollective

OVFTool

PuppetApply

MCollective

API Query

48

Zombie Engine DSL - ZED

Reserve Compute

Install ESXi

Add To Cluster

Add Compute

Reserve Compute

Install ESXi

Add To ClusterReplace Compute

Un-Reserve Compute

Remove From Cluster

49

Distributed and location-aware execution

Work to doConcurrent

WorkConcurrentWorkConcurrent

Work

ConcurrentWorkConcurrent

WorkConcurrentWork

P

P

P

B

B B

BB

Datacenter: Dallas

Datacenter: Singapore

Work to do

50

Why we use Puppet

• Greater investment in building modules than scripting

• Much greater return on investment with evolution of the product

• 13 in-house Puppet Modules so far (small example)

• vCloud Director

• vShield Networking

• vSphere

• We use a total of 47 modules for everything

• We have Puppet modules for installing Zombie in Production, Integration, and Development (including Vagrant + Puppet use for laptops)

• Project Zombie itself uses Puppet to do work, and is installed and upgraded using Puppet

51

Why we use Puppet

Puppet 1st Rule

52

Cool things we do with Puppet

• Engine can stage changes using MCollective to an environment

• We can update that environment throughout job execution

• We can apply the environment against a target at any time

• We have the ability to load balance/scale both Engine execution and Puppet execution

• Processes that would be complex to order in Puppet can be staged much easier with Zombie Engine + Puppet

• ZED allows for iteration using configuration data into execution concurrency and queuing. Very cool layer on top of Puppet.

53

Does it work?

54

55

Cloud Build A

Maximum capacity per week:

Man hours: 72

2

Time to delivery: 6 days

56

Maximum capacity per week:

Man hours: 1.5

336 x Zombie Engine instance count

Time to delivery: 2.5 hours*

Cloud Build A

57

Details (per job):

• ~120 individual tasks (plugin calls)

• ~2700 discrete configuration points

• ~1400 managed resources

• Dynamically sized (pick the # of compute and storage)

• Controls: vCloud Director, vCenter, ESXi, EMC VNX, Razor, vShield Manager, vShield Edge, Linux and Windows operating systems.

Cloud Build A

58

Cloud Build A

59

So yeah… it works

60

What’s next?

61

Bring Your Own Licenses

Bring Your Own VMs

Bring Your Own Tools

Customer Components

VMware vCloud Hybrid Service

Hybrid Service Delivery Components

Infrastructure Hardware & Facilities

Compute StorageNetworking &

Security

OS Catalogs

Application Catalogs

Infrastructure Management

Web Console vCloud API