Architecting Container or distribution - RainFocus...Pivotal Cloud Foundry 101 6 war Availability...

CNA2006BU

#VMworld #CNA2006BU

Deep Dive: Architecting Container Services with VMware & Pivotal Developer-Ready Infrastructure

VMworld 2017 Content: Not fo

r publication or distri

bution

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Disclaimer

#CNA2006BU CONFIDENTIAL 2



bution

Merlin Glynn (VMware)Ramiro Salas (Pivotal)

CNA2006BU

#VMworld #CNA2006BU

Deep Dive: Architecting Container Services with VMware & Pivotal Developer-Ready Infrastructure



bution

Agenda

1 Pivotal Cloud Foundry 101Why do my Developers want it?

2 Kubernetes 101Why do my Developers want it?

3 Ops: Architecture for Containers 101

4 Ops: Network & Security Controls

5 Ops: Monitoring & Logging

6 Ops: Platform as Code{}

7 Ops: PCF+PKS

4#CNA2006BU CONFIDENTIAL



bution

Pivotal Cloud Foundry 101Why do my Developers want It?



bution

Pivotal Cloud Foundry 101

6

war

Availability Zone 1 Availability Zone 2 Availability Zone 3

Staging

Root

FS

Build

Pack

war

`cf push`

Drop

let

A

I

A

Imyapp.foo.com

*.foo.com = NSX Edge Vip

NSX Edge

PCF Routing PCF Routing PCF Routing

LB Pool Members

“Here is my source code

Run it on the cloud for me

I do not care how”

URL Request:

myapp.foo.com

Developer

#CNA2006BU CONFIDENTIAL



bution

Kubernetes 101Why do my Developers want It?



bution

Kubernetes 101

8

K8s Cluster

Worker

`kubectl apply –f myapp.yml`

Developer

Worker

kube-proxyMaster

etcd

kube-proxy

Service: nodeport | ingress

POD POD

Load Balancer

URL Request:

myapp.foo.com/k8siscool

Docker

Registry




bution

Architecting for Containers 101



bution

DRI … Architect for Agility

Virtual Data Center

• Architect the right Abstractions

• Automate Everything

• Build for Failure

Control Agility

vSphere NSX vSAN

Pivotal Cloud Foundry

PCF

PKS

BOSH powered Kubernetes

BO

SH

Platform

Operator

Developer

Wavefront

Self Service

• Automation

• Day 2 Operations

• Control

• Application Services or Container Services

• Application Logging & Monitoring

vRLI (Dev)

vRops

vRLI (Ops)

vRNI




bution

Ops: Architecting for Availability & Scale



bution

vSphere Fundamentals for PCF

12

BOSH

AZ1 AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)

cc uaa brain cc uaa brain cc uaa brain

mysql mysql mysql

Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2

go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr

loggre

gator

loggre

gator

loggre

gator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

Developer

PCF OrgPCF Space

App App

Architecting for Availability & Scale

Virtual Data Center




bution

Physical Fault Domains

13

AZ1 AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)


mysql mysql mysql



loggre

gator

loggre

gator

loggre

gator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

Developer

PCF OrgPCF Space

App App

Virtual Data Center

Cluster Design Best Practices

• Enable vSphere HA

• Enable & Tune BOSH HealthMonitor Resurrection

vSphere HA

vSphere HA

BOSH Agent(s)

BOSH

BOSH Hlth

Monitor




bution

Physical Fault Domains

14

AZ1 AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)


mysql mysql mysql



loggre

gator

loggre

gator

loggre

gator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

Developer

PCF OrgPCF Space

App App

Virtual Data Center




• Plan For Singletons

– Externalize

– DR (vDP, Image, Snapshot, pgdump)

BOSH Agent(s)

BOSH

webdav

(blob)

BOSH

S3 Compat

Storage

PCF BlobStore

DR

DR DR




bution

IaaS Multi Tenancy

15

AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)

vSAN/NFS/VMFS

Resource Pool

AZ1 Foundation 1

ESX Cluster

vSAN/NFS/VMFS

ESX Cluster

vSAN/NFS/VMFS

ESX Cluster

Virtual Data Center





– Externalize

– DR (vDP, Image, Snapshot, pgdump)

• Use Resource Pools & Scale Clusters as needed

BOSH

Resource Pool

AZ2 Foundation 1

Resource Pool

AZ3 Foundation 1

Resource Pool

AZ1 Foundation 2Resource Pool

AZ2 Foundation 2

Resource Pool

AZ3 Foundation 2

Dev|Test|UAT

Foundation

Prod

Foundation

C

P

I

C

P

I

CPI Acct 1 Assigned

vCenter PermsPool Limits & Shares

CPI Acct 2 Assigned

vCenter PermsPool Limits & Shares

AC

L

Quota




bution

Recovering the Platform

16

AZ1 AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)


mysql mysql mysql



loggre

gator

loggre

gator

loggre

gator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

PCF OrgPCF Space

App App

BC/DR Best Practices

• Platform as Code{}

BOSH Agent(s)

BOSH

BOSH

S3 Compat

Storage

PCF BlobStore




bution

Backup Job


17

AZ1 AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)


mysql mysql mysql



loggre

gator

loggre

gator

loggre

gator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

PCF OrgPCF Space

App App



• Backup Services for Platform Persistent Data

• Backup Services for App Service Persistent Data

– Don’t Forget External App Data not managed by PCF

BOSH Agent(s)

BOSH

BOSH

S3 Compat

Storage

PCF BlobStore

MySql PCF Service

Tile

mysql mysql mysql




bution

Backup Job


18

AZ1 AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)


mysql mysql mysql



loggre

gator

loggre

gator

loggre

gator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

PCF OrgPCF Space

App App





– Don’t Forget External App Data not managed by PCF

• VMotion (Yes)

• SVMotion (NO)

BOSH Agent(s)

BOSH

BOSH

S3 Compat

Storage

PCF BlobStore

*

*

vmdk




bution

{}

NSXEdge LTM

Multi-Site Platforms

19

AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster


• Business Continuity w/ Multi Site

– GSLB

BOSH

GSLB

NSXEdge LTM

Health Checks

Health Checks

{}{}

Common ServiceMeshData




bution

20

VMware PKS

Kubernetes on BOSH (Kubo)

BOSH

NSX

Analytics Automation

SecurityOperations

Monitoring

GCP

Service Brokermasteretcd worker

Logging

vSANvSphere

masteretcd workerContainer

Registry

(PKS)




bution

What about PKS?

21

BOSH Deploys KUBO

• Same BOSH Availability Zone Constructs are available

• Spread Core K8S Jobs across BOSH Availability Zones

– Master

– ETCD

– Workers

• Multi Site can be GSLB in much the same way as PCF

• BOSH Makes Kubernetes Day 1 & Day 2 easy.

• Does NOT require PCF




bution

Architecting the Platform

22





• Business Continuity w/ Multi Site



• Enable & Tune BOSH HealthMonitorResurrection


• Use Resource Pools & Scale Clusters as needed

• VMotion (Yes)

• SVMotion (NO)

DEVELOPER-READY

INFRASTRUCTURE

Deliver innovation faster

to customers

Architectural Resource(s) Link(s)

VMware VVD (Validated Design) In Progress

Pivotal ’Lite’ Reference Architecture https://docs.pivotal.io/pivotalcf/1-11/refarch/vsphere/vsphere_ref_arch.html




bution

https://docs.pivotal.io/pivotalcf/1-11/refarch/vsphere/vsphere_ref_arch.html

Ops: Network & Security Controls



bution

http://myapp.default-apps.foo.com

Network Fundamentals for PCF

24

Network Design Best Practices

• Get Wildcard Certs & DNS Approved

DNS:

*.sys.pcf.foo.com

*.default-apps.foo.com

Single PCF ‘Foundation’

PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge

LS: ERTLS: Services #

LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *

AP

PS

Logical Routing (DLR)

IaaS: vSphere Security Zone A (Hub)

Cell

Cell

SS

HTC

P

TC

P

SS

H

AP

PS

A

I

A

I

External Services

Internal Apps

LS: OSPF

app.public-apps.foo.com

CF

ASG

{}




bution

DNS:

*.sys.pcf.foo.com



PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge


LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *

AP

PS



Cell

Cell

SS

HTC

P

TC

P

SS

H

AP

PS

A

I

A

I

External Services

Internal Apps

LS: OSPF

CF

ASG

Network Security & Controls

25



• Use Multiple NSX Logical Switches & Subnets, 1 per Deployment (PCF Tile) allow Subnet to Service Level ACLs

– On Demand: Developer trigger VM provision

– Pre-Provisioned: Ops triggers VM provision

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster




bution

DNS:

*.sys.pcf.foo.com



PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge


LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *

AP

PS



Cell

Cell

SS

HTC

P

TC

P

SS

H

AP

PS

A

I

A

I

External Services

Internal Apps

LS: OSPF

CF

ASG


26






• Use Application Security Groups (ASGs), App level egress firewall to PCF & external IP ranges

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster




bution

DNS:

*.sys.pcf.foo.com



PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge


LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *

AP

PS



Cell

Cell

SS

HTC

P

TC

P

SS

H

AP

PS

A

I

A

I

External Services

Internal Apps

LS: OSPF

CF

ASG


27







• Use NSX Edge for Load Balancing, SSL Termination, & Perimeter FW ACLs

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster




bution

DNS:

*.sys.pcf.foo.com



PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge


LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *

LS: Isolation_A

PCF

PCF Isolation Segment

GO

RTRCELL CELL CELL

/22

Public Apps

AP

PS

DNS:

*.public-apps.foo.com


IaaS: vSphere Security Zone A (Hub) IaaS: vSphere Security Zone B (Spoke)

Cell

Cell

SS

HTC

P

ISO

ISO

TC

P

SS

H

AP

PS

A

I

A

I

CF

ASG

External Services

Internal Apps

LS: OSPF


28








• Use NSX DLR for PCF Org & Space level segmentation

– Multiple Isolation Segments

– Isolation segments allow Operators to group Diego cells and attach to multiple Logical Swicthes.

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster




bution

DNS:

*.sys.pcf.foo.com



PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge


LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *

LS: Isolation_A

PCF


GO

RTRCELL CELL CELL

/22

Public Apps

AP

PS

DNS:




Cell

Cell

SS

HTC

P

ISO

ISO

TC

P

SS

H

AP

PS

A

I

A

I

CF

ASG

External Services

Internal Apps

LS: OSPF


29

Network Design Best Practices …

• Use NSX Security Groups for dynamic security principals

– BOSH Integrated NSX (Dynamic Membership)

– Ingress & Egress PCF Org/Space Specific FW

– Dynamic LB Pool Membership

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

BOSH




bution

DNS:

*.sys.pcf.foo.com



PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge


LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *

LS: Isolation_A

PCF


GO

RTRCELL CELL CELL

/22

Public Apps

AP

PS

DNS:




Cell

Cell

SS

HTC

P

ISO

ISO

TC

P

SS

H

AP

PS

A

I

A

I

CF

ASG

External Services

Internal Apps

LS: OSPF


30






• Use Distributed Firewall Policy

– Leverage PCF Integrated Dynamic Security Groups

– Control East+West from single policy engine

– Control App to App at the Org/Space level with Isolation Segments

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

BOSH

{}




bution

DNS:

*.sys.pcf.foo.com



PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge


LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *

LS: Isolation_A

PCF


GO

RTRCELL CELL CELL

/22

Public Apps

AP

PS

DNS:




Cell

Cell

SS

HTC

P

ISO

ISO

TC

P

SS

H

AP

PS

A

I

A

I

CF

ASG

External Services

Internal Apps

LS: OSPF


31

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

{}







– Leverage PCF Integrated Dynamic Security Groups



• Use RFC 1918 for Repeatability




bution


32

Platform

Operator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster


• Platform as Code{} to automate Day 1 & Day 2 ops

DNS:

*.sys.pcf.foo.com



PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge


LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *A

PP

S



Cell

Cell

SS

HTC

P

TC

P

SS

H

AP

PS

A

I

A

I

External Services

Internal Apps

LS: OSPF

CF

ASG

DNS:

*.sys.pcf.foo.com



PCF

Elastic Runtime

** *

LS: Services #

LS: Infra

NSX Edge


LS: Services #

CF Control

Plane

Ops

Mgr

(PCF)

BOSH GO

RTR

Diego

Brain

TCP

RTR

/26 /22/24(s)

PCF

…

PCF

Rabbit

PCF

Mysql

* * *

AP

PS



Cell

Cell

SS

HTC

P

TC

P

SS

H

AP

PS

A

I

A

I

External Services

Internal Apps

LS: OSPF

CF

ASG




bution


PCF OrgPCF Space

AppA AppB AppC

cf create-security-group SECURITY-GROUP PATH-TO-RULES-FILE cf create-security-group dev-mssql mssql.json

PCF Application Security Groups (ASG):

– Uses iptables in the Diego Cell Server

– Controls Egress only at the container source level

– Can control any IP address as the target

• Operator Declares in the Platform

[ {

"protocol": "tcp",

"destination": "10.0.11.0/24",

"ports": "1-65535"

},

{

"protocol": "udp",

"destination": "10.0.11.0/24",

"ports": "1-65535"

} ]

Platform

Operator

Prod Mssql

192.168.11.10

Prod Mssql

10.0.11.10




bution


PCF OrgPCF Space

AppA AppB AppC

cf allow-access SOURCE-APP DESTINATION-APP --protocol PROTOCOL --port PORT• cf allow-access “AppA” “Appc” --protocol TCP --port 443

Developer

PCF Container to Container Networking:

– Creates and Overlay (VXLAN)

– Controls ingress & egress between Ais(containers)

– Uses CNI

• Today Flannel

• Tomorrow NSX-T

– Developer can Declare in CI/CD




bution

What about PKS?

35

KUBO Networking is less Complex

• Typically multiple smaller K8s Deployments

• The core Kubernetes components need to route to each other

• Services Deployed on an Overlay Network

– NSX-T

• Enterprise Security Policy

• Enterprise Tools & Logging

• Common Ingress Paths:

– kube-proxy running on external gateway

– Load Balance to kube-proxy

Image source: https://github.com/cloudfoundry-incubator/kubo-deployment/blob/master/docs/images/kubo-network.png

Load Balancer

EXTERNAL

SVC

Request

External

Service Gateway

kubeproxy




bution


36

DEVELOPER-READY

INFRASTRUCTURE


to customers

Resource(s) Link(s)

KUBO Git Repo https://github.com/cloudfoundry-incubator/kubo-deployment

VMware PCF & NSX Design Guide Coming Soon






• Use Container to Container Networking to allow developer to define fine grained App level security

• Use RFC 1918 Repeatability

• Platform as Code{} to automate Day 1 & Day 2 ops





• Use NSX DLR for PCF Org & Space level segmentation




bution

Ops: PCF Monitoring & Logging



bution

Monitoring & Logging

38

Developer

Virtual Data Center

– I need to keep my apps healthy

– I need self service to my Apps Log’s

– I need to instrument my Apps (APM)

Platform

Operator

– I need to keep the Platform healthy

– I need to plan capacity

– I need to watch & Alert on KPIs

– I need to audit




bution


39

Developer

Virtual Data Center




PCF Metrics

`cf logs appA`

https://metrics.sys.pcf-foundation.io

Nozzle

vRLIhttps://vrli.pcf-foundation.io

Developer Log Access Routes

– `cf logs`: streams single app’s log events for dev to redirect where needed

– PCF Metrics: PCF app correlating App logs, and container Metrics, ~2week retention

– vRLI: Longer term scalable log storage and indexing, dashboards, & alerts




bution

Agents Added to Buildpacks

Future !!!


40

Developer

Virtual Data Center




App & App execution specific Metrics

• tc_server: jdbc_query_failed

• custom_app_metric: transaction_response_time

Platform

Operator

Exposed to developers via CF Service Broker

`cf create service my-apm-endpoint`




bution


41

Platform

Operator




– I need to audit

vRops

vRops Nozzle

Cloud Foundry Metrics (KPIs)

vSphere & NSX Metrics (KPIs)




bution


42

Platform

Operator




– I need to audit

vRLI

vRops

vRops Nozzle



Syslog Nozzle

vSphere & NSX Events

CF Platform Events

Thre

shold

s

Ale

rts

Da

sh

bo

ard

s




bution


43

Platform

Operator




– I need to audit

vRLI

vRops

vRops Nozzle



Syslog Nozzle

vSphere & NSX Events

CF Platform Events

Thre

shold

s

Ale

rts

Da

sh

bo

ard

s

All App Events




bution

Deamon

Set

Deamon

Set

What about PKS?

POD vRLI

POD

vRLI

• App Logging

• System Logging

– OS & Processes not run in Containers

App Logging

• Per App Only

Sidecar

• App Logging @ Pod level

POD

Daemon

Set

(PODs)

vRLI

POD

LOGGER

DOCKERDDOCKERD

vRLI

DaemonSet

• App Logging @ Cluster level

• Cluster Logging

Dockerd

• App Logging @ Cluster level

• Cluster Logging

• Not handled in K8s API

SyslogD

Platform

Operator

Developer




bution

What about PKS?

K8s Monitoring Integration w/ Wavefront by VMware

Wavefront Integration can be deployed as containers within the K8s Cluster

– Proxy

– Heapster

• Comprehensive Dashboards

– SaaS

• APM for the Developer

• Cluster KPIs for the Operator

• Integrated with PKS

Image source: https://www.wavefront.com/surf-container-wave-join-wavefront-container-world-santa-clara/

Platform

Operator

Developer




bution

46

Platform

Operator

What about PKS?

vRealize Operations & K8s

• Operator KPIs

• Single Pane for SDDC & K8s clusters monitoring

• vRLI Integrated

• Alert on K8s KPIs

• Entity Relationship

• Capacity Planning

• Integrated with PKS




bution

Ops: Monitoring & Logging

47

DEVELOPER-READY

INFRASTRUCTURE


to customers

Resource(s) Link(s)

Wavefront: KUBO Integration https://community.wavefront.com/docs/DOC-1204

Blue Medora : vRops MP https://marketplace.vmware.com/vsx/solutions/blue-medora-mp-for-pivotal-cloud-foundry

Blue Medora : vRLI Pack https://marketplace.vmware.com/vsx/solutions/content-pack-for-pivotal-cloud-foundry

Developer

Virtual Data Center




Platform

Operator




– I need to audit




bution

Ops: Platform as Code{}



bution

BOSH 101

49

• Built for Platform Operators

• Deploys Complex Distributed Systems

– PCF

– Kubo

• Day 1 & Day 2 Ops

– Initial Deployment

– Updates/Patches

– Maintains Health

Platform

Operator




bution

NSX_Config:

edge_vip_1:3

nsxmgr_endpoint: nsxmgr.vmware.io

lswicth_ert_cidr: 192.168.10.0/22

50

AZ1 AZ2 AZ3

Platform

Operator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

Drives NSX-V

NSX-V (Edge - Load Balancing – Logical Switch – Firewall Services)

Platform as Code{}

• Declarative

Day 1 & Day 2

YAML




bution

Ert_config:

diego_database_instances:3

diego_brain_instances: 3

diego_cell_instances: 9

51

AZ1 AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)


mysql mysql mysql



loggre

gator

loggre

gator

loggre

gator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

PCF OrgPCF Space

App App

BOSH

Drives NSX-V


Platform as Code{}

• Declarative

Day 1 & Day 2

YAML




bution

Ert_config:

diego_database_instances:3

diego_brain_instances: 3

diego_cell_instances: 12

52

AZ1 AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)


mysql mysql mysql



loggre

gator

loggre

gator

loggre

gator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

PCF OrgPCF Space

App App

BOSH

Drives NSX-V


Platform as Code{}

• Declarative

Day 1 & Day 2

YAML

Cell_3 Cell_3 Cell_3




bution

53

AZ1 AZ2 AZ3

Platform

Operator

Ops Manager

(OVA)


mysql mysql mysql



loggre

gator

loggre

gator

loggre

gator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

PCF OrgPCF Space

App App

BOSH

Drives NSX-V


Platform as Code{}

• Declarative

• Change Controlled

• Archived

• Audited

Day 1 & Day 2

Cell_3 Cell_3 Cell_3




bution

54

Platform

Operator

Ops Manager

(OVA) vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

BOSH

Drives NSX-V

Platform as Code{}

• Declarative


• Archived

• Audited

Day 1 & Day 2

NSXEdge LTM

NSXEdge LTM

• Repeat

– Scale

• Repair

– Recovery

• Repave

– Rotate Creds




bution

55

Platform

Operator

CVE & Update Patching

The New Stack

• Patch at ANY Layer of the Application Stack

• Address CVE in minutes/hours versus days/weeks

• Simply re-stage all apps when any layer is patched


Day 1 & Day 2

Developer

PCF ERT Tile

PCF Stemcells

PCF OrgPCF Space

App App

CVE in Root File

System of Container

CVE Exec Layer: TC

Server

CVE on the Container

Host OS

Vulnerability in

Code{}

Restage Applications

PCF BuildPack




bution

56

Platform

Operator


The New Stack

• Stemcells still there …

• Harbor Scans Images for Vulnerability (Clair)



Developer

Stemcells

CVE in Root File

System of Container

CVE Exec Layer: TC

Server

CVE on the Container

Host OS

Vulnerability in

Code{}

Restage Applications

What about PKS?

Docker

Registry

CVE

FOUND

!!!

BOSH




bution

What about PKS?

KUBO Can scale …. A lot

BOSH allows for a repeatablepattern of K8S Clusters as well.

• Many Development teams

• Multiple Security Zones for Applications

• Multi Cluster HA within a DC

• CI/CD Pattern similar to PCF

Platform

Operator

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

vSAN/NFS/VMFS

Resource Pool

ESX Cluster

VCF

BOSH

Developer

Developer

A

BvRA

PKS




bution

Ops: Platform As Code{}

58

DEVELOPER-READY

INFRASTRUCTURE


to customers

Resource(s) Link(s)

Pivotal NSX + PCF Pipeline https://github.com/cf-platform-eng/nsx-ci-pipeline

Pivotal Generic PCF Install & Upgrade pipelines https://github.com/pivotal-cf/pcf-pipelines

Virtual Data Center


The New Stack

• Patch at ANY Layer of the Application Stack


• Simply re-stage all apps when any layer is patched


Day 1 & Day 2

• Declarative


• Archived

• Audited

• Repeat

– Scale

• Repair

– Recovery

• Repave

– Rotate Creds




bution

Wrapping It up …



bution

Developer Ready Infrastructure

vSphere NSX vSAN

Pivotal Cloud Foundry

PCF

PKS

BOSH powered KubernetesB

OS

H

Platform

Operator

Developer

Wavefront

Self Service

• Automation

• Day 2 Operations

• Control

• Application Services or Container Services

• Application Logging & Monitoring

Solves for DevOps Reqs …

vRLI (Dev)

vRops

vRLI (Ops)

vRNI




bution

61

VMworld US Key Focus Description

CNA1509BU DRI Developer-Ready Infrastructure from VMware & Pivotal

CNA1612BU PCF & KuboUse Cases: Deploying real-world workloads on Kubernetes and Pivotal Cloud

Foundry

CNA2006BU DRIDeep Dive: Architecting Container Services with VMware and Pivotal

Developer Ready Infrastructure

CNA2080BU Kubo Deep Dive: How to Deploy and Operationalize Kubernetes

CNA3429BU KuboBasics of Kubernetes on BOSH: Run Production-grade Kubernetes on the

SDDC

CNA3430BU PCFYour Enterprise Cloud-Native App Platform: An Introduction to Pivotal Cloud

Foundry

MGT2871BUPCF & vRops,

vRLI

Bridging the Operations Gap Between the Software-Defined Data Center

and Pivotal CF for VMware Deployments

NET1523BU PCF & NSX Integrating NSX and Cloud Foundry

PAR4411PU DRIEmerging Technologies with VMware and Pivotal - presented jointly by

VMware, Pivotal and Special Guest Speakers from Cognizant and WWT

Developer Ready Infrastructure @ VMworld




bution



bution

Architecting Container or distribution - RainFocus...Pivotal Cloud Foundry 101 6 war Availability...

Documents

Transcript of Architecting Container or distribution - RainFocus...Pivotal Cloud Foundry 101 6 war Availability...