The Data Center Network Evolution

Journey to the Programmable FabricThe Data Center Network Evolution

Robert ZalobinskiTechnical Solutions [email protected]

In partnership with:

Cisco Confidential 2© 2013-2014 Cisco and/or its affiliates. All rights reserved.

• Basics of SDN and Overlay Networks

• Application Centric Infrastructure (ACI)

• Virtual Topology System (VTS 2.0)

• Unified Open NX-OS


What is SDN

Software defined networking (SDN) is an approach to building computer networks that separates and abstracts elements of these systems

In other words…

In the SDN paradigm, not all processing happens inside the same device


Current Network ArchitectureTightly coupled Control and Data Planes

• One Control Plane per Device

• Each Device Managed Individually

• All Command Line Managed

Routing protocols (i.e. OSPF, IS-IS, BGP), Spanning Tree, SYSLOG, AAA

(Authentication Authorization Accounting), NDE (Netflow Data Export), CLI

(Command Line interface), SNMP

Layer 2 switching, Layer 3 (IPv4 | IPv6) switching, MPLS forwarding, VRF

Forwarding, QOS (Quality of Service) Marking, Classification, Policing, Netflow flow collection, Security Access Control

Lists

cpu

asic


The Promise of SDN

Overlay Protocol

Physical Network

10001101000110101

Control & Data PlaneDecoupled

NetworkVirtualization

DirectProgrammability

Centralized ManagementSimplification

Agility Programmatically ConfiguredDynamic

Automated


SDN Programming Models

Applications Control

DataAPI

ControllerAPI API Data

ControllerAPI API Contr

ol

Data

vSwitchAPI OverlayProtocol

Control

DataOverlay

Controller


Types of Overlay Edge Devices

• Virtual end-points only

• Single admin domain

• VXLAN, NVGRE, STT

• Physical and Virtual

• Resiliency + Scale

• Cross-organizations/Federation

• Open Standards

Network Overlays Integrated OverlaysHost Overlays

• Router/switch end-points

• Protocols for resiliency/loops

• Traditional VPNs

• OTV, VXLAN, VPLS, LISP

App

OS

App

OS

Virtual Physical

Fabric DB

VM

OS

VM

OS

Virtual Virtual

VM

OS

VM

OS

Physical Physical


VXLAN Overview

Outer MACDA

Outer MACSA

Outer 802.1Q

Outer IP DA

Outer IP SA

Outer UDP

VXLAN ID

(24 bits)

Inner MAC DA

InnerMACSA

Optional Inner

802.1Q

Original Ethernet Payload

CRC

VXLAN Encapsulation Original Ethernet Frame

CRC

Data

Plane

16 M Segments

Control

Informatio

n

Tunnel Endpoints DiscoveryHost Reachability Information

• Mac Address• IP address

Draft Data Plane Multicast based flood and learn


Eth Eth Eth

vEth vEth vEth vEth vEth vEth

Overlay Network Communications - VTEP

VXLAN utilizes a VTEP:• Virtual Tunnel End Point• IP address assigned• Layer-3 Transportable• IP/UDP Packets

10.10.10.101 10.10.10.211 172.18.22.12

Inter VXLAN communications

VTEP VTEP VTEP


Eth Eth Eth

vEth vEth vEth vEth vEth vEth

Overlay Network Communications

VXLAN 55110

VXLAN 45235

16m VXLANs

VXLAN requires a network gateway function:• VXLAN to VLAN Bridge• VXLAN to VLAN Router• VXLAN to VXLAN Router

VLANs


VXLAN54210

VXLAN Gateway Functions

VXLAN55110VXLAN45235 VLAN 235

VLAN 110

VXLAN55110 VXLAN45235

VXLAN55110 VLAN 235

VXLAN to VLAN Bridging (L2 Gateway)

VXLAN-to-VXLAN Routing (L3 Gateway)

VXLAN-to-VLAN Routing (L3 Gateway)

VXLAN45235


Programmable NetworkProgrammable FabricApplication Centric Infrastructure

DB DB

Web Web App Web App

VxLAN-BGP EVPN standard-based

3rd party controller support

Modern NX-OS with enhanced NX-APIs

Automation Ecosystem (Puppet, Chef, Ansible etc.)

Common NX-API across N2K-N9K

Turnkey integrated solution with security, centralized management,

compliance and scale

Automated application centric-policy model with embedded security

Broad and deep ecosystem

Cisco SDN: Providing Choice in Automation and Programmability

Mass Market (commercial, enterprises, public sector)

Service Providers Mega Scale Datacenters

VTS for software overlay provisioning and management

across N2K-N9K

14© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

Application Centric Infrastructure (ACI)


Two Types of Languages

Infrastructure Applications

HumanTranslator

• Application Tier Policy and Dependencies

• Security Requirements• Service Level Agreement• Application Performance• Compliance• Geo Dependencies

• VLAN• IP Address• Subnets• Firewalls • Quality of Service • Load Balancer• Access Lists

Cisco Confidential 16©2014 Cisco and/or its affiliates. All rights reserved.

Introducing: Application Centric Infrastructure

Apps + Infrastructure

Physical + VirtualOpen + Secure

On-Premises + Cloud

Application Oriented Policy = Operational Simplicity


Application Centric Infrastructure Components

Fabric

Centralized Policy ManagementOpen APIs, Open Source, Open Standards

Policy ControllerApplication Network Profile APIC

End Points

Physical Networking

Nexus 2K

Nexus 7K

Hypervisors and Virtual Networking

Compute L4–L7Services

Storage Multi DC WAN and Cloud

Integrated WAN Edge

VirtualPhysical

NorthboundManagement

IntegrationPartner Ecosystem

Automation

OVM

HypervisorManagement Monitoring

SystemsManagement

OrchestrationFramework


Typical Three Tier Application

Web Servers

Firewall

Server Load Balancer

Access Switch

Server

vSwitch

Firewall

Access Switch

App Servers

DatabaseServer

Application Requirements tightly coupled to the Network

Port Group, VLAN, IP Address, IP Mask

Interface, Trunk, VLAN, IP Subnets

Interface, Trunk, VLAN, IP Subnets

• Network Connectivity• Security Policies• Quality of Service• Layer 4 – 7 Application

Services• Storage Policies• Compute Policies• Hypervisor Policies


The Policy-based Datacenter

IP Fabric

• Single APIC Controller:• End-to-end Application

Profile• ACI IP Fabric encompasses

o Infrastructureo Physicalo Virtualo Services

• ANP Profile pushed to all components

• Full Workload Mobility, Replication and Instantiation Application Network Profile

Web Servers App Servers Database Server


ACI Benefit: Deep Telemetry — Application and Tenant

APIC

APP

TEN

AN

T

TenantTenant 1 Tenant 2

Tenant 3 Tenant 4


Application Approach To Networking

F/W DB DBDecouple Policy from Infrastructure

Simple & Scalable Stateless Infrastructure Optimized Forwarding & Mobility

Abstracted Policies for definition of Applications & Connectivity

Open REST APIsCentralized ManagementOpen Source APIC

Application Network Profile

F/W F/W F/W

STORAGE STORAGE

WEB DBAPP

Highest Performance & ReliabilityLowest Power Consumption


Virtual Topology System (VTS)


Programmable FabricNX-API, VXLAN BGP EVPN Fabric, and Virtual Topology System (VTS)

Operations / Programmability

& Automation

Automated DCI / WAN

VM

OS

VM

OS

NX-API

Physical Virtual DCI/WAN

Bare Metal Virtualized

BGP-EVPN VXLAN Fabric

VTS

VTS for overlay provisioning and management across Nexus 2000 – Nexus 9000 (2H 2015)


vCenter

REST API

VTS

GUI

Across Nexus PortfolioNexus 2K – 9K

Programmable Fabric

AutomatedSeamless integration with Orchestrators

Overlay provisioning and DCI/WAN integration

Scalable VXLAN ManagementMP-BGP EVPN control plane

High performance virtual forwarding

Open and ProgrammableREST Northbound APIs

Multi-protocol and Multi-hypervisor support

Virtual Topology System (VTS) Overlay Provisioning & Management System

Flexible OverlaysPhysical and virtual overlays

Bare-metal and Virtualized workloads


VXLAN as Data Center Overlay technology

L2 L3 L4VTEP

Local LAN Local LAN Local LAN Local LAN

IP Transport Network

VTEP VTEP VTEP

VXLAN VNI

LAN Segment

Underlay Network:• IP routing – proven, stable, scalable• ECMP – utilize all available network paths

Overlay Network:• Standards-based overlay• Layer-2 extensibility and mobility• Expanded Layer-2 name space • Scalable network domain• Multi-Tenancy

Modes of Operation:• Multicast based flood and learn (No control plane)• BGP EVPN (BGP control plane with MP-BGP Extensions)


Advantages with EVPN Control Plane

Industry standard protocol for multi-vendor support

Built-in Multi tenancy support

Truly scalable with protocol-driven control plane architecture

Fast convergence upon network failures and host movements

Minimize flooding through ARP suppression

Security through VTEP peer-authenticationAdv

anta

ges

of E

VP

N

Con

trol P

lane


VTF

Cisco Network Services Orchestrator VMware vCenter GUI

DVS

Unified Information Model (REST API)

YANG CLI NX-API BGP-EVPN

Virtual Topology System

Service and Infrastructure PolicyInventoryDatabaseResource Management

Policy PlaneC

ontrolPlaneIOS XRvDevice Management

Control Plane FederationMP-BGP

Cisco Nexus 2000, 3000, 5000, and 7000 Series

Cisco Nexus 9000 Series Cisco ASR 9000 Series

Virtual Compute Environment

VTS Architecture


VTS Architecture

Cisco VTS

ToR ToR

Spine Spine

ToR

Hypervisor

VM

x86 Server

Hypervisor

VM

x86 Server

VTF VTF

Hypervisor

VMVM

x86 Server

REST API

DCI

NX-API, CLI, YANG

VTEP

VTEP VTEP

Border Leaf VTEP

VMware vCenter

Virtual Topology System

Service and Infrastructure PolicyInventoryDatabaseResource Management

Policy PlaneC

ontrolPlaneIOS XRvDevice Management


VTS Architecture – Hardware Switches

ToR ToR

Spine Spine

ToRVTEP VTEP

Cisco VTS

Hypervisor

VM

x86 Server

Hypervisor

VM

x86 Server

VTF VTF

Hypervisor

VMVM

x86 Server

REST API

DCI

NX-API, CLI, YANG

VTEP

Border Leaf VTEP

VMware vCenter


VTS Architecture - VTF

Cisco VTS

ToR ToR

Spine Spine

ToR

Hypervisor

VM

x86 Server

Hypervisor

VMVM

x86 Server

REST API

DCI

NX-API, CLI, YANG

VTEP VTEP

Border Leaf VTEP

VMware vCenter Hypervisor

VM

x86 Server

VTF VTF

VTEP

User space, Multi-tenant, line rate packet forwarder

Uses Vector Packet Processing technology

Fully integrated with Intel DPDK

Supports VXLAN, can be extended to support MPLSoGRE, L2TPv3, MPLSoUDP, native MPLS and SR

Programmed by VTS using Restconf/YANG


VTS Functionality

• Discover ToRs, Servers and interconnections• Manage switch and network topology status• Topology information via API or GUI

Discovery

• VXLAN Provisioning (BGP EVPN & Flood/Learn) • VXLAN Overlay management (Add/Modify/Delete)• Multi-tenancy support• Track and Update VNIDs as VM moves• Network facing resource management

Provisioning

• Tenant to VNID mappings and VNID status• VNID to VTEP mappings• VTEP to VLAN and end host mapping• Trace VMs connected to VTEP• VTEP status within a VNID• VXLAN and fabric statistics

Overlay Visibility


Unified Open NX-OS


ExtensibilityAuto DeploymentOptions

Open ApplicationIntegration

ProgrammabilityTool Choice

DevOpsEnabling

POAP NXAPI

Yocto SDK

Standard Open InterfacesOpen Interfaces Automation and

VisibilityAdaptable NXOSAdaptableSDK

Programmable BootStrap and Provisioning

Package and Application Management

Native Application Integration

PXEData

Models

Server Management Tools

OPEN NX-OS - Extensible, Open, Programmable

34


Open NX-OS: Infrastructure Layer Enhancements

35

OPEN BOOTLOADERS & PROVISIONING

OPEN PACKAGE/APPLICATION INTEGRATION

OPEN INTERFACES

OPEN OBJECT BASED API’s (NX-API, Model Driven)

Open NX-OS consistent across both ToR and Modular

Open NXOS


VTEP VTEP VTEP VTEP

• Leverage existing compute deployment infrastructure (PXE/iPXE) for operationalizing NX-OS

• Deploy NX-OS from a web server via HTTPS or TFTP server with support for both IPv4 and IPv6

• NX-OS CLI option added to select boot option either <bootflash(default) > or <pxe>

Boot Server(DHCP & HTTP/TFTP)NX-OS Image Repository

DHCP DISCOVER(v4/v6)

IP Address & File/Image URL

TFTP GET FILE/HTTP URL

http://n9k-dk9….bin..

Validate Image Checksum & Boot

Open NX-OS Bootloaders & ProvisioningiPXE

36


•Ability to third party packages in Secure Guestshell or natively in NX-OS kernel

• Install all third party applications (Puppet/Chef, etc) as RPMs

•Daemon managed via standard Linux interfaces•Built-in support for YUM package manager•Patching and upgrade using standard rpm/yum workflows

• NX-OS processes(BGP) can be upgraded/patched via “yum update”

37

Package as RPM

C app with standard Linux

constructs

Open Embedded 64

bit Build Environment

Cisco/Local

Repository

RPM local repository

RPM uploadYUM Install

Linux Daemon

Linux Kernel

• Raw Socket • Netdevs• Libpcap

init.d

Monitoring

server

ASIC

Build Server Target Switch

Open NX-OS Package Management via YUM/RPMLXC and Native Daemons


•Leverage Linux command toolkit for monitoring configuration and troubleshooting

# tcpdump -w file.pcap -i eth1-1Use ethtool to display detailed interface statistics:

#ethtool –S eth2-1 Use ifconfig to change mtu for an interface to

jumbo MTU:#ifconfig eth2-1 mtu 9000

Use ip route to add a static route for a given interface:#ip route add 203.0.113.0/24 via 198.51.100.2

dev eth2-1Leverage bash for NX-OS scripting automation

vsh –c “show interface brief” | grep up | awk/sed

38

Open NX-OS Linux InterfacesBash Access

Cisco Confidential 39© 2013-2014 Cisco and/or its affiliates. All rights reserved. 39

• Tool provides a convenient way for network engineers to get up to speed with scripting and automation via web browser interface

• Available on all Nexus platforms.

• CLI commands embedded in structured input and output (JSON/XML) via HTTP/HTTPS

• Use “feature nxapi” to enable access on the platform

Open NX-OS ProgrammabilityNX-API Developer Sandbox


Updates on Nexus Portfolio Offerings

Programmable NetworkApplication Centric Infrastructure

NEW! Unified Open NX-OS Release for Nexus 3000 and Nexus 9000 (Q3 2015)• Enhancements to NX-API – object store

and model driven• Native 3rd party RPM applications

integration (tcollector, Nagios, Ganglia, Puppet / Chef etc.)

• Linux utilities support for seamless tool integration across compute and network

• SDK for custom application integration

NEW! ACI Release for Nexus 9000 (Shipping June 2015 )• Microsoft Azure and System Center

Integration• Programmability examples: vCenter plug-

in, ACI toolkit etc. • Simplified operations • Stretched fabric, multiple destinations

from 30KMs to 150KMs• Group-based policy on Openstack• New ACI ecosystem partners (CliQr)

DBDB

Web Web App Web App

NEW! Common NX-API across N2K-N9K (2H 2015)

Programmable Fabric

NEW! Virtual Topology System (VTS) for software overlay provisioning and management across for Nexus 2K-9K (2H 2015)• Standards-based fabric

support on Nexus 5600/7x00 with VXLAN BGP EVPN (shipping with Nexus 9000 today)

VTS


Nexus 9000® SeriesYour Deployment, This Makes it Happen!

Cisco Nexus 9300Platform Fixed Switches

NX-OS and ACI

Choice of Fabric Architectures

Feature Consistency with Silicon Innovations

Cisco Nexus 9500Platform Modular Switches

Nexus 9516 – Best of Interop Data Center 2014 APIC – Best of Interop SDN 2015

In partnership with:

The Data Center Network Evolution

Technology

Transcript of The Data Center Network Evolution