Cloud Native Networking with FD.io/VPPbos.itdks.com/23e6f3d8cfbf4316a1ec5177ac087b0d.pdf•...
Transcript of Cloud Native Networking with FD.io/VPPbos.itdks.com/23e6f3d8cfbf4316a1ec5177ac087b0d.pdf•...
Cloud Native Networking with FD.io/VPP
Frank Brockners
Distinguished Engineer, Chief Technology and Architecture Office, Cisco
March 17, 2018
1
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
The way Applications are developed and deployed has changed…..
2
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Microservices & Containers have changed many things…Applications are being developed and deployed very differently today
• Microservices allow you to split an application into many modular pieces, the network is how you stitch the pieces back together.
• The interconnection of the pieces results in a more complex application network which consumes lots of resources
• The performance of the cloud native network is crucial to the behavior of the overall application.
3
Pod
Pod
PodPod
Pod
Pod
Pod
It’s crucial we get ”Container Networking” right! Lets not get “Openstacked”
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
What Container Network Stacks Provide Today:
• Lifecycle management for application Containers
• Overlay connectivity for application Containers:
• NAT communication with external world
• Policy controlled overlay, may extend policy control to DC fabric in some cases
• Network policy addresses security/connectivity
• Designed for Data Center applications / use cases
Good start, but not sufficient for NFV use cases (Firewalls, Customs VNFs)
Container
Orchestration, Scheduling
Container
Networking Control
Container
Data-Plane
4
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
What Container Networks Stacks Lack for NFV Use Cases: • NVF-specific policy APIs (e.g. QoS, placement
considering network resources, etc.)
• Networking:
• NAT is a bottleneck, not suitable for NFV use cases
• VNFs often require more than 1 interface / IP address
• No support for high-speed wiring of NFVs:• To the outside world; To application containers;
Between NFV containers; Creation of Service Function Chains (mixed physical & virtual)
• Management/Control:
• Containerized NVFs not really in the data plane (except for the vswitch)
• No support for cloud-native, high-performance NVFs
• Forwarding:
• Kernel or OVS used for forwarding
Container
Orchestration,
Scheduling
Flexible & High-Speed
Container
Networking
High-Performance/Scale
Container Data-Plane
Flexible
Container
“wiring” and
control
5
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cloud-Native Network Function Needs: Summary
• Container-based
• Container stack lifecycle (same as application containers)
• 12-factor app design for management/control
• High-performance forwarding
• High-performance networking
• Seamless virtual/physical world interworking
• Common policy control with application containers (as much as possible)
• Must solve 3 key problems:
• Lifecycle management
• High-Performance Networking (wiring/config, operations)
• Easy installation, operation – policy, security
Container
Orchestration,
Scheduling
Container
Networking
Control
High-Performance/Scale
Container Data-Plane
Flexible
Container
“wiring” and
Control
LIGATO
6
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Container Networking moving from Kernel to Userspace
• Userspace enables rapid upgradability, highly available (doesn't bring down node), no system call overhead, no dependency on Linux kernel networking community for features, higher performance and scale
• FD.io (dataplane), DPDK (network), SPDK (Storage) are examples
• Cloud Native apps are all connected by the network – lots of network end points to be managed, userspace offers lower overhead and higher performance
• Meltdown/Spectre bugs add a new tax for kernel networking
7
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Kubernetes Networking – Key Communication patterns
4 distinct communication patterns:
8
Highly-coupled container-to-
container communications
Pod concept and
localhost communications
Pod-to-Pod communications Network Plugins
Pod-to-Service
communications
Kubernetes services
concept
External-to-Service
communications
Kubernetes services
concept
Kubernetes, Rocket…
Container Network Interface
Plugins
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Kubernetes Network Plugins
• Kubernetes assumes seamless connectivity between pods, wherever it decides to place them. A networking plugin is needed to abstract the network
• Kubernetes fundamental requirements for any networking implementation
• all containers can communicate with all other containers without NAT
• all nodes can communicate with all containers (and vice-versa) without NAT
• the IP that a container sees itself as is the same IP that others see it as
• Network Plugins provide various levels of sophistication – from simple to highly feature rich and performant
9
Examples
• Cilium
• Contiv
• Contrail
• Flannel
• Google Compute Engine (GCE)
• Kube-router
• L2 networks and linux bridging
• Multus
• NSX-T
• Nuage Networks VCS
• OpenVSwitch
• OVN (Open Virtual Networking)
• Project Calico
• Romana
• Weave Net from Weaveworks
• CNI-Genie from Huawei
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Container Networking Control: Contiv-VPPhttps://github.com/contiv/vpp
• Contiv is a networking plugin for Kubernetes that:
• Allocates IP addresses to Pods (IPAM)
• Programs the underlying infrastructure it uses (Linux TCP/IP stack, OVS, VPP, …) to connect the Pod’s to other Pods in the cluster and/or to the external world.
• Implements K8s network policies that define which pods can talk to each other.
• Implements K8s services; a service exposes one or more (physical) service instances implemented as K8s pods to the other pods in the cluster and/or to external clients as a virtual instance (e.g. as a virtual “service” IP address).
• Contiv is a user-space based, high-performance, high-density networking plugin for Kubernetes. Contiv-VPP leverages FD.io/VPP as the industry’s highest performance data plane
10
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Ligato – Accelerate Cloud-Native DevelopmentPlatform and code samples for development of cloud native VNFs
LIGATO provides a mechanism for delivering and managing agents for cloud-native Network Functions to enable them to become part of the application service topology - all in user-space.
Components:
• Cloud-Native Infrastructure – a Golang platform for building cloud-native microservices
• VPP-agent - Golang implementation of a control/management plane for FD.io/VPP based cloud-native Virtual Network Functions (VNFs)
• SFC-Controller - Service Function Chain (SFC) Controller for stitching virtual and physical networks
• BGP-Agent - BGP Agent is a BGP information provider
https://ligato.github.io/
11
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Network Functions can now be delivered as containers
• Take advantage of the low overhead of containers to deliver higher performance.
• Using cloud native technologies to build the Network Functions so they run in the same network and user-space as the applications.
• Network functions become part of the service topology
• Network Functions are truly just another service and can be developed/deployed using the same tools as the applications with the same velocity.
• Leverage cloud-native development & deployment velocity – #devployfaster
• Eliminates the overlay tax for services
12
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Putting it all togetherEnabling Production-Grade Native Cloud Network Services at Scale
13
Contiv-VPP NetmasterCalico
SFC
Controller
Kubernetes API Proxies
Service Policy Service Topology Lifecycle
KubeletCNI
CRIContiv-VPP NetpluginCalico
Production-Grade Container Orchestration
Network Function and Network Topology Orchestration
Containerized Network Data Plane
Container
Network FunctionCNF
Agent Agent Agent
FD.io VPP
Container Switch
Agent
Container Networking
Networking Plugin
CNF
LIGATO
Contiv
Production-Grade
Container Orchestration
Cloud-native NF Orchestration
Cloud-native NF Agent platform
Containerized Fast
Data Input/ Output
Performance-Centric
Container Networking
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
GoAGENT
Containerized
Network Functions
GoAGENT
Containerized
Network Functions
SFC ControllerSFC Controller
Configuration
Operational State
Message Bus
SFC ControllerSFC ControllerApplications Tools (e.g.agentctl)
clientv1
ContainersLifecycle
Orchestration
clientv1
Data Store
GoAGENT
Configuration
Notifications
Control and Management Plane
Inter-Process Communication
Containerized
Switch
Containerized Network Data Plane
<<-- microservices -->>
40GE 100GE
CNF CNFcSwitch
Contiv-VPP Netmaster
Cloud-native Network Micro-Services Putting It All Together Now – The System Design
Functional Layered Diagram System Components
Contiv-VPP NetmasterCalico
SFC
Controller
Kubernetes API Proxies
Service Policy Service Topology Lifecycle
KubeletCNI
CRIContiv-VPP NetpluginCalico
Production-Grade Container Orchestration
Network Function and Network Topology Orchestration
Containerized Network Data Plane
Container
Network FunctionCNF
Agent Agent Agent
FD.io VPP
Container Switch
Agent
Container Networking
Networking Plugin
CNF
SFC Controller
clientv1
14
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Host
VPP Vswitch
CNF
VPP
10.1.0.127
…
CNF1
VPP
…
CNF2
VPP
…
…Server
Vswitch VPP
CNF
VPP
…
CNF
VPP
…
CNF3
VPP
…
…
NF1 NF2 NF3
Overlay Tunnel
Logical Representation
Physical Representation
Ingress Network
Ingress Classifier
Egress Network
Egress Classifier
TopologyPlacement
(K8s)Rendering
IngressRouter
EgressRouter
Overlay Tunnel Overlay Tunnel
Ingress Classifier Egress Classifier
Network Micro-Service Use Case:Service Function Chaining with Cloud-Native NFs
15
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Ligato
Controller
Data Plane Network 1
Data Plane Network 2
Cloud (Overlay) Network
…
KubernetesEtcd
Contiv
Netmaster
(opt)
Physical
Device
Physical
Device
Physical
Device
Physical
Device…
Cloud tools
& services
CNF CNF CNF CNF CNF
Example: Distributed, multi-cloud CUPS architecture enabling Cloud Native Network Functions
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Host
VPPvswitch
VNF
CN AppCN App
CN App
Host
VPPvswitch
VNF
CN-VNFCN-VNF
CN-VNF
Host
VPPvswitch
VNF
CN AppCN App
CN AppHost
VPPvswitch
VNF
CN AppCN App
CN App
Host
VPPvswitch
VNF
CN-VNFCN-VNF
CN-VNFHost
VPPvswitch
VNF
CN-VNFCN-VNF
CN-VNF Cloud
Network
Kubernetes
Cloud tools & services
Cloud-Native
Control Plane
Cloud-Native
Data Plane
VPP VPP
LIGATO Controller
CPE
CPECPE
CPE
CPE
CPE
CPE
…
…
Use Case: Cloud-Native CUPS ArchitectureEdge compute application and network service orchestration
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
A glimpse at performance: Example topology
x86 Server
Host NW Stack
SFCController
etcd
Kubernetes
+K8s NW
Plugin
VPP VPPVPP
VPP cSwitch
CNF CNFCNF
vint1
vint1
vint2
vint2
I2xconn
NIC interface1 NIC interface2 Mgmt interface
veth
veth
veth
kafkaveth
I2bd
Topology: Containerized Switch with one or more Containerized Network Functions
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
memif: Shared Memory Packet Interface for VPP
• Packet based shared memory interface for user-mode application
• Container friendly (no privileged containers needed)
• Polling and interrupt mode operation:
• Interrupts simulated with linux eventfd infrastructure
• Support for interrupt masking in polling mode
• 3rd-party library - allows easy creation of applications which communicate over memif
• vpp-to-vpp, vpp-to-3rd-party and 3rd-party-to-3rd-party operation
• Support for multiple queues (incl. asymmetric configurations)
• Jumbo frames support (chained buffers)
• Secure
19
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Example Benchmark: memif 10x faster than veth
0
1
2
3
4
5
6
7
8
9
10
0
2
4
6
8
10
12
14
16
1xCNF
cSwitch1core
2xCNF
cSwitch2cores
4xCNF
cSwitch4cores
Topology[1]:Memifvs.vEthBenchmarks64B Ethernetframesize
MemifMpps
vEthMpps*
MemifGbps
vEthGbps
PacketThroughput
[Mpps]
BandwidthThroughput
[Gbps]Memiffaster:10 times
64B Ethernet frame size – No Frame Loss
64B Ethernet frame size – No Frame Loss
Memif – cSwitch 1core
vEth – cSwitch 1core
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
One more thing…
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Application Driven Service Mesh to enable Policy enforcement & Security
• Agents deployed as part of the app code to enable policy and security enforcement at the app level
• Provides a way to express application security consistently to support mutli-cloud application deployments
• Enterprise can get back consistent Networking & control when leveraging cloud-native and multi-cloud technologies Security Controls
Consistent Network & Security Control across cloud environments
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Istio provides a dedicated infrastructure layer for handling service-to-service communication
• Responsible for the reliable delivery of requests through the complex topology of services that comprise a modern, cloud native application
• Deployed as a sidecar container with the application
• Platform agnostic, runs where ever the containers run
• Envoy agent in the sidecar provides, routing, load balancing, traffic steering, stats collection, flow data – leveraging VPP for high performance
Istio Service Mesh with Envoy Agent
• Intelligent routing & Load Balancing
• Resilience Across Languages and Platforms
• Fleet-Wide Policy Enforcement
• In-Depth Telemetry and Reporting
Istio & Envoy
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Network
Endpoint
Public Cloud & Serverless - Networking & Security controls are limited/non existent
On Prem
Service A
RULES
Serverless
Service D
Physical Network & SDN
Container
Networking
Container
Service C
Operator
programmed
RULES
Rules programmed by Network OperatorRules programmed
by DevOps
No controls
available
Network
Endpoint
Co-Lo
Service B
RULES
Enforcement
?
Multi-cloud environments are extremely challenging to secure
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Service Mesh
Network
Endpoint
Public Cloud & Serverless - Networking & Security controls are limited/non existent
On Prem
Service A
RULES
Serverless
Service D
Physical Network & SDN
Container
Networking
Container
Service C
Operator
programmed
RULES
Rules programmed by Network OperatorRules programmed
by DevOps
No controls
available
Network
Endpoint
Co-Lo
Service B
RULES
Enforcement
?
Application Driven Service Mesh
RULES
Agents deployed part of the app –Enterprise get back consistent Networking & Security Controls
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Istio CA
Kubernetes Pod
Service AEnvoy(Sidecar
Container)
Kubernetes Pod
Service BEnvoy(Sidecar
Container)
mTLS + Secure Naming
Issue & mount as
K8s secrets
Orchestrate Key & Certificate:
- Generation
- Deployment
- Rotation
- Revocation
spiffe://myorg.com/*
SAN:
“spiffe://myorg.com/ns/prod/sa/foo”
- Namespace: prod
- Service account: foo
SAN:
“spiffe://myorg.com/ns/prod/sa/bar:
- Namespace: prod
- Service account: bar
FD.io adds performance to the
Service Mesh K8s Master
Security at Scale with Istio
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Complete Cloud Native Network Stack
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OpenStack
ODL/ML2
Data Plane
Neutron
…
GBP
Neutron NB
VPP Renderer Topology Mgr.
VPP
Honeycomb
DPDK
Policy,
Lifecycle
ONAP/ OPNFV FastDataStacks
LIGATOContiv
Production-Grade
Container Orchestration
Cloud-native Network Function
Orchestration
Containerized Fast
Data Input/ Output
Performance-Centric
Container Networking
Enabling Production-Grade Native Cloud Network Services at Scale
Istio
Application Service
Mesh
Contiv NetmasterCalico
LIGATOController
Kubernetes API Proxies
Service Policy Service Topology Lifecycle
KubeletCNI
CRIContiv NetmasterCalico
Production-Grade Container Orchestration
Network Function and Network Topology Orchestration
Containerized Network Data Plane
Container
Network FunctionCNF
Agent Agent Agent
FD.io VPP
Container Switch
Agent
Container Networking
Networking Plugin
CNF
Istio Service Mesh
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• CNFs: Need to have cloud-native network functions in order to meet the
needs of cloud-native applications. Kernel networking is not appropriate for
cloud-native environments.
• Userspace Networking: Finish getting the kernel out of the way with
userspace host stacks
• Userspace/Cloud Native Storage: Storage needs to start down the
Userspace/Cloud Native road networking is walking: SPDK/Ceph/VPP
integration
• Unified IO: Networking/Storage: Start thinking of blocks/packets as
interchangeable units of IO with well integrated processing paths.
• Security Policy Orchestration: Integrate Application Driven security into the
development toolchain for DevOps.
What’s left to do ?
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Summary
LIGATO
Contiv
Production-Grade Container Workload
Scheduling and Orchestration
Cloud-native NF Orchestration
Cloud-native NF Agent platform
Data-Plane: Containerized
Fast Data Input/ Output
Performance-Centric
Container Networking
IstioSecure overlay network with granular
policy control using a “sidecar” (L7 proxy)
Towards an orchestrated multi-cloud
from edge to centralized
andon-Prem to Co-lo to
public cloud
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Thank you
31
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
References
• FD.io/VPP
• https://fd.io/
• memif:• https://docs.fd.io/vpp/17.10/libmemif_doc.html
• https://docs.google.com/presentation/d/1KrpOUUD7oHML6Plge_4E3-oq5YBun5T8pfoDiCBVk74
• Contiv-VPP
• https://github.com/contiv/vpp
• Ligato
• https://ligato.github.io/
32