VMworld 2015: Build and Run Cloud Native Apps in your Software Defined Data Center

Build and Run Cloud-Native Appsin Your Software-Defined Data Center

Kit Colbert, VMware, Inc@KitColbert

CNA6649-S

#CNA6649

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Disclaimer

CONFIDENTIAL 2

John Deere

CONFIDENTIAL 3

Sonos AMP

4CONFIDENTIAL

IT Drives These ChangesCompared to peers across the industry, high-performing IT organizations experience:

5

Source: https://puppetlabs.com/2015-devops-report

60xfewer failures

168xfaster failurerecovery time

30xmore frequentdeployments

200xshorter

lead times

CONFIDENTIAL

https://puppetlabs.com/2015-devops-report

https://puppetlabs.com/2015-devops-report

How to Get There

6

ApplicationCode

RunningApplication

DevOpsCulture, practice

Automation, frequent updates

Secure, resilient, scalable

CONFIDENTIAL

VMware Cloud-Native Apps Stack

7

vSphere

vSphere Integrated Containers

vRealize Automation, vRealize Operations,vRealize Log Insight

PivotalCloud Foundry

VMware NSX, VMware Virtual SAN

vRealize Code Stream

Chef,Ansible

JFrogArtifactory

Repository Config. Mgmt.

Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop

Development Lifecycle

Production Stack

Vagrant,Panamax

VMwareAppCatalyst

Photon Platform vCloud Air

Docker, Kubernetes,Mesos, Lattice

CONFIDENTIAL

vSphere





Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop


Production Stack

Vagrant,Panamax



VMwareAppCatalyst

8


CONFIDENTIAL


VMware AppCatalyst

9

Download Technology Preview Now! http://getappcatalyst.com

Built for DevelopersAppCatalyst is REST API- and CLI-driven for seamless integration with container-

and microservices-based workflows.

Free to UseAppCatalyst is available at no cost to the

user, and ready for download today.

Ready for Cloud NativeAppCatalyst ships with Photon OS and

Vagrant, and supports Docker containers out of the box.

CONFIDENTIAL

VMware AppCatalyst

10

Developer Desktop

VMware AppCatalystC

LIR

ES

T A

PI

Network and Storage Abstractions (coming soon)

DockerMachine

Container Container Container


Vagrant

or Bring Your Own Linux

PHOTON OS

10CONFIDENTIAL

Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

vSphere




Developer Laptop

Production Stack

Vagrant,Panamax



VMwareAppCatalyst

11




CONFIDENTIAL


12

DevOps Enables High Business Performance

Source: State of DevOps Report, 2013 and 2014

High performers are more agile 30x

Faster in shipping code

8,000xFaster in completing

deployments

High performers are more reliable 2x

The change success rate

12xFaster meant time to recover

(MTTR)

High performers win in the marketplace

2xMore likely to exceed profitability, market share & productivity goals

50%Higher market capitalization

growth over 3 years

CONFIDENTIAL

13

The Challenges of Continuous DeliveryRelease Process with Agile Development

Production

Agile Software Development with

Continuous Integration

Development

Test StageUATLoad Test

DevOps

Frequent release of small sets of changes

Manual process and inconsistent configurations

result in lengthy delivery and poor quality

Dev wants to push quickly into production

Operations wants stability

CONFIDENTIAL

14

vRealize Code Stream – Accelerate Application Delivery

• Eliminate the costs and errors associated with manual tasks and hand-offs$$$

• Ensure a consistent, repeatable & predictable software release process

• Leverage the value from all of the tools in your software development release chain

CUSTOM

CONFIDENTIAL

vSphere





Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop


Vagrant,Panamax

VMwareAppCatalyst



15


Production Stack


CONFIDENTIAL

16

Application Design is ChangingProperties of a Microservice• Small code base

• Easy to scale, deploy and throw away

• Autonomous

• Resilient

Benefits of a Microservices Architecture• A highly resilient, scalable and resource efficient

application

• Enables smaller development teams

• Teams free to use the right languages and tools for the job

• Rapid application development

Monolithic/Layered Micro Services

CONFIDENTIAL

17

PaaS = Platform-as-a-Service

Source: http://wikibon.com/cloud-native-application-platforms-structured-and-unstructured/

Structured• Pre-integrated and tested solution• Out-of-the-box functionality• “Just works”, but may not offer specifics

you want• Examples: Cloud Foundry, MS Azure,

Heroku, Google App Engine

Unstructured• DIY combination of cloud-provided

services & homegrown tools• Likely container-based• Maximum flexibility, but can be expensive

to build

PaaS Platform - Visible to Developers

Message Bus / Queuing / Routing Service Brokers

Capacity Planning Logging Monitoring

Application Staging / Application Services

Application SchedulingContainer Scheduling

Service DiscoveryContainer Cluster Management

Container Networking

Container RuntimeContainer OS

Container RuntimeContainer OS

Physical Host (or VM) Physical Host (or VM)DevOps Tools


Con

figur

atio

n M

anag

emen

t

Mar

ketp

lace

/ Im

age

Man

agem

ent

Sec

urity


Two Ways to Implement

CONFIDENTIAL

http://wikibon.com/cloud-native-application-platforms-structured-and-unstructured/

http://wikibon.com/cloud-native-application-platforms-structured-and-unstructured/


vSphere





Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop


Vagrant,Panamax

VMwareAppCatalyst



Production Stack


18CONFIDENTIAL

19

Pivotal Cloud FoundryCloud Native Application Platform for Public and Private Cloud

Continuously Deliver Applications with Speed Scale Confidence

VMware vSphere VMware vCloud Air

VMware Integrated OpenStack

CONFIDENTIAL

PCF: Core Tenets

21

Continuous DeliveryContinuous Development

Micro ServicesContainers

• Auto-detect frameworks• “Push and it works”

• .WAR• Dockerfile• .NET

• Simple service bonding• Agile micro-services

Extensivle Framework“Buildpack” Architecture

• Instant dynamic routing• Log streams & aggregation• Access controls & policies• APM and auto-scaling• 4 layers of High Availability

• App-instance• Availability zone• Process• Virtual machine

Extensible Service Broker Architecture

Deploy, Operate, Update, Scale Platform on Any Iaas

Simple, Developer-Friendly Commands and API

Operational Benefits forEvery Application

Built-in and EcosystemServices

• Elastic Hadoop• Mobile

• Push, Sync, API• MySQL HA• Redis• Rabbit MQ

• ElasticSearch• MongoDB• Cassandra• Jenkins (CI)• PHD• And more…

CONFIDENTIAL

PCF Architecture

22CONFIDENTIAL 22

vSphere





Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop


Vagrant,Panamax

VMwareAppCatalyst


23


Production Stack


CONFIDENTIAL


24

Hardware

OS Kernel

OS File system

Use

rspa

ce

Container

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

App

pro

cess

Container

App

pro

cess

App

pro

cess

Linux Containers

24

OS-level Isolation• Isolation at individual kernel subsystem

level (e.g. filesystem, process table, etc)

• User-level process (LXC, libcontainer) orchestrates these subsystems to create a container

Existed for Many YearsSolaris Zones, FreeBSD Jails, OpenVZ

Why?• Process isolation

• Reproducible environment

• Enables management at scale

CONFIDENTIAL

is a “Shipping Container” for Code

Developers because …

• Frictionless deployment and maximum portability

On developer laptop:

Then on server:

That’s it!!

• A natural fit for 3rd Platform, 12 factor, microservices

• It makes DevOps much, much easier25

~# docker build my_app~# docker push my_app~#

~# docker pull my_app~# docker run my_app~#

CONFIDENTIAL

26

Growing Ecosystem for Unstructured

CONFIDENTIAL





Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop


Vagrant,Panamax

VMwareAppCatalyst


27


Production Stack

vSphere


CONFIDENTIAL


Cloud-Native Identity & Access Management

Identity, Authentication and Authorization Server

LDAP, Kerberos, SAML, OAuth2.0, x.509

Scalable ArchitectureMulti-master state-based replication

Multi-data center replication

Multi-TenantMultiple independent forests

Open Source

CONFIDENTIAL 28

Secure Container Runtime

Container Optimized Linux OSDocker, rkt and Garden (Pivotal) support

Minimal footprint to run containers

vSphere IntegrationPart of your vSphere install

Hypervisor-optimized container runtime

Updates from VMwareEnterprise support

Security and update patches from VMware

Open Source

CONFIDENTIAL 29

30

Uniting Technologies – Introducing vSphere Integrated Containers

• EXTENDING an existing vSphere environment

• Containers become a first-class citizen

• Industry-leading capabilities• DRS, vMotion, HA/DR• Storage and Network Integration (VSAN and NSX)• NO rebuilding or re-architecture required

• Full compatibility with existing tools

• Broadest ecosystem• Cloud-Native developer tools, application services,

and hardware platforms

“Cloud-Native Platform”

VMware vSphere


NSX

VSAN

vRealize

Instant Clone,Project Bonneville, Photon OS

Leverage Your Existing Investments andEnable On-Ramp To Cloud-Native

CONFIDENTIAL

31

vSphere Integrated Containers Before & After

Docker API

VMware ESXPhotonOS

VMware ESXPhotonOS

Shared Datastores

C1

Layer1

Layer2

Layer3Image1

VM create, start, stop,

delete

docker rundocker stopdocker rm

VMware ESXVMware ESX

Shared Datastores

docker-machine1

VM createdw/ Docker Engine2

Docker API

docker run3

DE

C1 C2

DE’

VI admin creates Virtual Container Host

1

C2

3

2Virtual

Container Host

(Resoure Pool) and

Docker Engine uVM

created

uVM

uVM

uVM

vCenter

Docker containers are created inside the VM

4 Instant Clone!

4Docker containers created in uVMs via Instant Clone

VMware ESXPhotonOS

vCenter

CONFIDENTIAL

vSphere






Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop


Vagrant,Panamax

VMwareAppCatalyst vCloud Air


32


Production Stack

Photon Platform

CONFIDENTIAL

33

Select Customers Need an Optimized Approach for Cloud Scale

Focused Feature-setPurpose-built for cloud native, it provides just the features needed to provide a secure and SLA-capable infrastructure for next-gen apps.

API-first ModelBuilt on clean, development-friendly APIs, enabling consumption and operations to automate heavily to handle massive scale.

Scale-out Control PlaneA scale-out control plane optimizes for the creation of 1000s of simultaneous new workloads while also delivering active-active availability of system APIs.

New EconomicsEnables new “pay for what you need, when you need it” consumption model that is better suited for large-scale deployments with elastic and variable workload requirements.

CONFIDENTIAL

34

A Different Approach – Introducing the VMware Photon Platform

Redesign Your Infrastructure To Deliver Cloud-Native

New Platform Built from Ground Up andOptimized for Containers• Rich API Set• Distributed management• Streamlined and optimized hypervisor

Differentiating Benefits• Speed – Spin-up in seconds• Scale – Supports hundreds of thousands of

containers• Movement – Fluid and dynamic

Core Components• Photon Controller – Distributed Management,

Scheduling, Orchestration, Project Lightwave• Photon Machine – Photon OS, Microvisor

“Cloud-Native Platform”

VMware Photon Platform

Photon Controller

Photon Machine

CONFIDENTIAL

35

What’s in Photon Platform

Cloud Foundry Docker Hadoop … Deep integration with modern, open source

frameworks and application and data platforms

Photon ControllerHost controller and schedulerDistributed, multi-tenant control plane,includes Project LightwaveSingle API endpoint

Photon MachineCompute hostMicrovisor” based on ESX, includes Photon OSSimple, stackable, replaceable hosts

API

@cloudnativeapps #vmwcna CONFIDENTIAL

36

Photon Platform Architecture

Photon MachinePhotonOS



Photon API

Photon Controller Clustered design delivers massive scale and high-

availability.

Combination of local and/or shared

Photon Machine datastores.

Photon Controller #2

Photon Controller #3

Photon MachineCombination of core ESX with

PhotonOS

Cloud Foundry API Kubernetes API

Create CF clusterCreate Kubernetes clustercf push

cf scalekubectl createkubectl get pods

Photon Controller #1Cloud Foundry Cluster Kubernetes Cluster

CONFIDENTIAL

Photon Platform

vSphere





Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop


Vagrant,Panamax


37


Production Stack


CONFIDENTIAL


38


Containers Run Inside of VMs• One VM per server per security domain

• Containers often behind NAT

• No container level networking

Does This Make Sense? • It actually does…

Enterprise Model Today

VM

Con

tain

er

Con

tain

er

Con

tain

er

Con

tain

er

Hypervisor

VM

Con

tain

er

Con

tain

er

Con

tain

er

Con

tain

er

vSwitch

VM VM

CONFIDENTIAL

39


Two Levels of vSwitch

• First layer vSwitch inside the container VM

• Second layer vSwitch inside the Hypervisor

• Container level networking

In the Future, Container Level Visibility

VM

Con

tain

er

Con

tain

er

Con

tain

er

Con

tain

er

Hypervisor

VM

Con

tain

er

Con

tain

er

Con

tain

er

Con

tain

er

vSwitch

VM VM

vSwitch vSwitch

CONFIDENTIAL

Physical Network Infrastructure

Internet

ServerC

onta

iner

Con

tain

er

Con

tain

er

Con

tain

er

vSwitch

Server

Con

tain

er

Con

tain

er

Con

tain

er

Con

tain

er

vSwitch

Server

Con

tain

er

Con

tain

er

Con

tain

er

Con

tain

er

vSwitch

Containers – Do We Still Need a Hypervisor?

40

Without a Hypervisor Attackers Can Spread

Without Hypervisor…Attacker compromises containerPrivilege escalation to get root

access on container hostNow has direct access to the

physical networkCan compromise other physical

hosts

CONFIDENTIAL

41

NSX and ContainersNSX Networking and Security Services for VMs and Containers

Virtual Machine

Container/Docker

Firewall

Virtual Network (Subnet)

Virtual Network (Subnet)

Tenant Logical Router

Provider Router

WAN / Internet

Con

tain

er

1C

onta

iner

2

Con

tain

er

3

VM

1

VM

2

Con

tain

er

4C

onta

iner

5

VM

3

VM

4

VM

5

Security Group

Load Balancer

CONFIDENTIAL

Photon Platform

vSphere





Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop


Vagrant,Panamax


42


Production Stack


CONFIDENTIAL


Containerized Storage Apps are Rapidly Increasing

Thousands of DB appsMillions of downloads

Container Data Volumes• Usage

– Contains persistent data for local containers– Appears as folder within host’s file system

(e.g. “/mount/yourdata/”)– Can locate on host or external storage

• Benefits– Manage and preserve your stateful data– Utilize storage platform data services

But…how do you preserve data when moving apps between hosts?

Host

Container

Storage Platform

Container

DataVolumes

Announcing vSphere Driver for Flocker 1.0!• Run containerized stateful apps on your current vSphere deployment

using open-source Flocker software• Move containers and attached data volumes between ESX VMs• Compatible with ALL vSphere storage (VSAN, VVOL, VMFS, NFS)• Straightforward install/configure/deploy process• Free! • Available at https://github.com/vmware/vsphere-flocker-driver

https://github.com/vmware/vsphere-flocker-driver

Native Docker on vSphere

CONFIDENTIAL

ESX VM2

Container DB App

ESX VM1

Container DB App

VMDK1+ Container Volume

VMDK2

ESX VM2

Container DB App

ESX VM1

Container DB App

vSphere + Flocker

MoveContainer

MoveContainer

VMDK1 VMDK2Container VolumeVMDK

When container moves, data volume stays on host VMDK. Database starts on new

VM without any of its data.

Data Volume stored on separate VMDK. When container moves, VMDK moves with

it. Database keeps its data!

Summary


48

vSphere






Chef,Ansible

JFrogArtifactory


Git,Perforce

Code Repo

Jenkins,Bamboo

CI/CD

Developer Laptop


Production Stack

Vagrant,Panamax

VMwareAppCatalyst



CONFIDENTIAL

49

Connect with Us @ VMworld

CNA Track• 10 Breakout Sessions

DevOps Day• VMware AppCatalyst Workshop

Booth Demos• vSphere Integrated Containers• VMware Photon Platform• VMware AppCatalyst• Project Cell

Hand on Labs• HOL-SDC-1630, 2 workshop slots

Hang space• VMware Video Game Container System

(VCS)

Social Media• @cloudnativeapps | #vmwcna• blogs.vmware.com/cloudnative• [email protected]• vmware.github.io

CNA Show Guide Graphichttp://www.vmware.com/go/cnaguide

CONFIDENTIAL

http://www.vmware.com/go/cnaguide

Thank You!

@cloudnativeapps#vmwcna

vmware.github.ioblogs.vmware.com/cloudnative

[email protected]

CONFIDENTIAL51

VMworld 2015: Build and Run Cloud Native Apps in your Software Defined Data Center

Technology

Transcript of VMworld 2015: Build and Run Cloud Native Apps in your Software Defined Data Center