Mastering ACI and - clnv.s3.amazonaws.com · Mastering ACI and OpenStack Domenico Dastoli Technical...

Mastering ACI and OpenStack

Domenico Dastoli

Technical Marketing Engineer INSBU

BRKACI-3456

[email protected]

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#BRKACI-3456


ACI connects Virtual and Physical World

4BRKACI-3456

Agenda

• ACI, Virtualisation and VMM Domains

• ACI and OpenStack

• Options to Install OpenStack and ACI plugin

• Operate OpenStack: ML2 mode and GBP mode

• External Network connectivity

• Demo

• Q&A

Virtualisation and VMM Domains


Remote PoD Multi-Pod / Multi-Site Hybrid Cloud Extension

ACI Anywhere

IP

WAN

IP

WAN

Remote Location Public CloudOn Premise

Security Everywhere Policy EverywhereAnalytics Everywhere

7BRKACI-3456

ACI Anywhere - VisionAny Workload, Any Location, Any Cloud


ACI Fabric

App DBWeb

WWW

QoS

Filter

QoS

Service

QoS

Filter

Cisco ACI – The basicsLogical Network Provisioning of Stateless Hardware

BRKACI-3456 8

Scale-Out Penalty Free Overlay

APIC


Hypervisor Interaction with ACITwo modes of Operation

• ACI Fabric as an IP-Ethernet Transport

• Encapsulations manually allocated

• Separate Policy domains for Physical and Virtual

VLAN 10 VLAN 10 VLAN 100

Non-Integrated Mode

• ACI Fabric as a Policy Authority

• Encapsulations Normalized and dynamically provisioned

• Integrated Policy domains across Physical and Virtual

APP WEB DB

Integrated Mode

DB

BRKACI-3456 9

APIC APIC


vCenter

DVS/AVE

Relationship (VMM Domain) is formed between APIC and Virtual Machine Manager (VMM)

Multiple VMMs likely on a single ACI Fabric

There is 1:1 relationship between a Distributed Virtual Switch and VMM Domain

Hypervisor Integration with ACI

SCVMM

10

OpenStack

BRKACI-3456

Kubernetes

APIC

RHEV


EP

EP

EP

EP

EP

EP EP

EP

EP

EP

EP

EP

EP

EP

VMM Domain 1

4K EPGsVMM Domain 2

4K EPGs

16M Virtual Networks VLAN ID only gives 4K EPGs (12 bits)

Scale by creating pockets of 4K EPGs

Map EPGs to VMM Domain based on scope of live migration

Place VM anywhere

Live migrate within VMM domain

Hypervisor Integration with ACIVMM Domains & VLAN Encapsulation

BRKACI-3456 11

APIC


Hypervisor Integration with ACIVMM Domains & VLAN Encapsulation

EP

EP

EP EP

VMM Domain 1

4K EPGsVMM Domain 2

4K EPGs

VLAN 5

VLAN 16

16M Virtual Networks

VNID 6032

VLAN ID only gives 4K EPGs (12 bits)

Scale by creating pockets of 4K EPGs

Map EPGs to VMM Domain based on scope of live migration

Place VM anywhere

Live migrate within VMM domain

BRKACI-3456 12

APIC

ACI and OpenStack


OpenStack Controller

ACI + OpenStack – With OpFlex SupportFull Policy Based Network Automation Extended to the Hypervisor

• Open Source OpFlex agent extends ACI into the host

• OpFlex Proxy exposes new open API in ACI fabric

OpFlex for OVS

OS nodes OVS OpFlex Agent

OpFlex Proxy

Solutions with Major OpenStack Distributions

14

APIC Unified Plugin

BRKACI-3456

APIC


Why Cisco ACI and OpenStack?

Distributed, Scalable

Virtual Networking

• Full Neutron Node datapath replace

• Fully distributed Layer 2, anycast

gateway, DHCP, and metadata

• Distributed NAT and floating

IP address

Hardware-Accelerated

Performance

• Automatic VXLAN tunnels at top of

rack (ToR)

• No wasted CPU cycles for tunneling

• Optional use of SRIOV

Operations and

Telemetry

• Troubleshooting across physical and

virtual environments

• Health scores and capacity planning

per tenant network

Integrated Overlay

and Underlay

• Fully managed underlay network

through Cisco® APIC

• Capability to connect physical servers

and multiple hypervisors to overlay

networks

BRKACI-3456 15

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16

What is the ACI Unified Plugin for OpenStack?

BRKACI-3456


Neutron ML2

The Modular Layer 2 (ml2) plugin is a framework allowing OpenStack Networking to simultaneously utilize the variety of layer 2 networking technologies.

Drivers within ml2 implement separately extensible sets of network types and of mechanisms for accessing networks of those types.

17

• Type DriversEach available network type is managed by an ml2 TypeDriver.TypeDrivers maintain any needed type-specific network state, andperform provider network validation and tenant network allocation.The ml2 plugin currently includes drivers for the local, flat, vlan, gre,opflex and vxlan network types.

• Mechanism DriversEach networking mechanism is managed by an ml2 MechanismDriver. The MechanismDriver is responsible for taking the information established by the TypeDriver and ensuring that it is properly applied given the specific networking mechanisms that have been enabled.

Neutron Server

ML2 Plug-in

Type Manager Mechanism Manager

API Extensions

GR

E

TypeD

river

Cis

co

AP

IC

VLA

N

TypeD

river

VX

LA

N

TypeD

river

Cis

co

Nexu

s

Mic

rosoft

Hyper-V

Layer 2

Popula

tion

Lin

ux

Brid

ge

Open

vS

witc

h

SR

-IOV

OpF

lex

TypeD

river

BRKACI-3456


ACI ML2 Mechanism Driver

When running the ACI integration, The following Type and Mechanism Drivers will be used:

• Type Driversopflex • Mechanism Driversapic_aim

18

Neutron Server

ML2 Plug-in

Type Manager Mechanism Manager

API Extensions

GR

E

Typ

eD

rive

r

VX

LA

N

Typ

eD

rive

r

Cis

co

Ne

xu

s

Mic

roso

ft

Hyper-V

La

ye

r 2

Po

pu

latio

n

Lin

ux B

ridg

e

Open

vS

witc

h

SR

-IOV

BRKACI-3456

VL

AN

Typ

eD

rive

r

Cis

co

apic

_aim

Op

Fle

x

Typ

eD

rive

r


APIC ML2 options

• Opflex mode allows creation of neutron networks based on

• VLAN

• VXLAN

• APIC AIM Mechanism driver enables the user to deploy OpenStack projects in:

• Neutron standard ML2 mode

• Group Based Policy (GBP) mode

19BRKACI-3456


ML2 vs GBP mode

BRKACI-3456


ML2 – APIC Mapping

Neutron Object APIC Object

Project Tenant

Network EPG + BD

Subnet Subnet

Router Contract

Security Group + Rule N / A

Iptables rules maintained per host

21

• With the ML2 Standard Neutron model, the following mapping happens.

• All the operations are done on OpenStack through Horizon, CLI or Heat

BRKACI-3456


ACI Tenant Creation with ML2 model

22BRKACI-3456



23BRKACI-3456



24BRKACI-3456



25BRKACI-3456



26BRKACI-3456



27BRKACI-3456



28BRKACI-3456


GBP – APIC Mapping

GBP Object APIC Object

Project Tenant

L3 Policy VRF

L2 Policy Bridge Domain + Subnet

Policy Group Endpoint Group

Policy Ruleset Contract

29

• With the GBP Model the following mapping happens.

• GBP offers much more granularity and flexibility compare to standard neutron.

• GBP comes with CLI, Heat and Horizon plugins

BRKACI-3456


ACI Tenant Creation with GBP model

30BRKACI-3456



31BRKACI-3456



32BRKACI-3456



33BRKACI-3456



34BRKACI-3456



35BRKACI-3456



36BRKACI-3456



37BRKACI-3456


GBP Policy Mapping

38BRKACI-3456

Bridge Domain

EPG WEB

EPG APP

EPG DB

EPG DHCP

subnet

dhcpserver

VRF


GBP L3 PolicyGBP Policy Mapping

39BRKACI-3456

Bridge Domain

EPG WEB

EPG APP

EPG DB

EPG DHCP

subnet

dhcpserver

VRF


GBP L2 Policy

GBP Policy Mapping

40BRKACI-3456

Bridge Domain

EPG WEB

EPG APP

EPG DB

EPG DHCP

subnet

dhcpserver

VRF


GBP

Policy Groups

GBP Policy Mapping

41BRKACI-3456

Bridge Domain

EPG WEB

EPG APP

EPG DB

EPG DHCP

subnet

dhcpserver

VRF


GBP Policy Mapping

42BRKACI-3456

Bridge Domain

EPG WEB

EPG APP

EPG DB

EPG DHCP

subnet

dhcpserver

VRF


ML2 vs GBP model – what is best?

• GBP:

• Application Centric

• Security groups are created as ACI contracts AND OVS rules. So they are visible on ACI and will be enforced both in HW (ACI leaf) and SW (OVS).

• Introduces new REST APIs: if any existing templates, you will need to adapt

• ML2:

• Network Centric

• Standard way of creating neutron networks

• REST API will not change: any heat or CLI template will keep working

• Security Groups not visible in ACI: they are implemented by OS as OVS rules

43BRKACI-3456


What are the components and how do they work?

BRKACI-3456


What’s installed on the controller and compute node?

[heat-admin@overcloud-controller-0 ~]$ sudo yum list | grep @aci-repo

aci-integration-module.noarch 0.6.0-162.el7 @aci-repo

agent-ovs.x86_64 1:1.5.0-63.el7.centos @aci-repo

apicapi.noarch 1.1.0-170.el7 @aci-repo

neutron-opflex-agent.noarch 2:6.1.0-26.el7 @aci-repo

openstack-dashboard-gbp.noarch 6.0.0-53.el7 @aci-repo

openstack-heat-gbp.noarch 6.0.0-53.el7 @aci-repo

openstack-neutron-gbp.noarch 6.2.0-53.el7 @aci-repo

[heat-admin@overcloud-controller-0 ~]$

[heat-admin@overcloud-compute-0 ~]$ sudo yum list | grep @aci-repo

agent-ovs.x86_64 1:1.5.0-63.el7.centos @aci-repo

neutron-opflex-agent.noarch 2:6.1.0-26.el7 @aci-repo

openstack-neutron-gbp.noarch 6.2.0-53.el7 @aci-repo

[heat-admin@overcloud-compute-0 ~]$

controller

compute

BRKACI-3456 45


A key component:

AIM Daemon

A new component:

ACI Integration Module

The AIM daemon is running on the Controller nodes and is responsible to configure ACI through REST API call based on the OpenStack policy model defined.

46BRKACI-3456


Architecture: APIC Integration Manager

Description

• APIC Integration Database and APIC Integration

Manager (AIM) introduced as central point of

storing plugin configuration.

• AIM uses the OpenStack database.

• AIM continuously synchronizes with APIC using

APIC Integration Daemon (AID).

• Group-Based Policies are mapped into Neutron API

and then AIM. Neutron APIs are mapped to AIM

directly.

RouterSecurity

Group

Netwo

rkRule

Set

Policy

Group

Policy

Group

Group-Based

Policy

Neutron API

APIC Integration Database (AIM)

APIC Unified Plugin

AID

processes

OpenStack

Controller

BRKACI-3456 47


OpenStack Tenant

Instantiate VMs

Create Application Policy

Web WebWebWeb AppApp4

3

5ACI

Fabric

Automatically Push

Network Profiles to

APIC and keep it

sync

Push Policy

Create Network, Subnet,

Security Groups, PolicyNETWORK ROUTING SECURITY

1

2

DB DB

HYPERVISOR HYPERVISOR HYPERVISOR

NOVANEUTRON

OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH

The AIM Daemon at work: the workflow

APIC

48

C2EPG

APPEPG DB

C1EPG

WEB

Application Network Profile

AIM

BRKACI-3456


Neutron Opflex Agent

Neutron Opflex Agent

The Neutron Opflex Agent runs on both the compute and the controller. It is responsible to communicate with the neutron server.

49BRKACI-3456


Agent OVS

Agent OVS

The Agent OVS runs on the compute and controller nodes. It is responsible to communicate with the OVS and the leaf node to register to ACI fabric.

50BRKACI-3456


OpFlex Architecture

• Neutron-opFlex-agent:

Receives updates from Neutron

about new endpoints and

updates EP and Service files

• Agent-OVS: Runs OpFlex

protocol with ACI leaf proxy.

• Agent-OVS Programs open

vswitch via OpenFlow

OpFlex policy (ACI infra VLAN)

OpenFlow

Endpoint Files

EndpointInformation:RabbitMQ

Neutron-Opflex-Agent

Agent-OVS

Open vSwitch

Neutron Server(s)

OpenStack

Node

51BRKACI-3456


What is the Endpoint file?

52

Routing settings

VM IP address

Network Policy including floating IP if any assigned

VM name

For each VM, the Neutron Opflex Agent creates a .ep file local to the node with all the information of the VM.

BRKACI-3456

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56BRKACI-3456

Distributed, Scalable

Virtual Networking

• Full Neutron Node datapath replace

• Fully distributed Layer 2, anycast

gateway, DHCP, and metadata

• Distributed NAT and floating

IP address

Hardware-Accelerated

Performance

• Automatic VXLAN tunnels at top of

rack (ToR)

• No wasted CPU cycles for tunneling

• Optional use of SRIOV

Operations and

Telemetry

• Troubleshooting across physical and

virtual environments

• Health scores and capacity planning

per tenant network

Integrated Overlay

and Underlay

• Fully managed underlay network

through Cisco® APIC

• Capability to connect physical servers

and multiple hypervisors to overlay

networks

Closer look to the enhancement with the ACI plugin


Routing and Policy Enforcement is done on the host

Tenant Networks

ComputeHost

Neutron Server(s)

OVS

Management/API Network

• Traditionally in OpenStack the

routing is done on the servers

hosting neutron services only.

• With ACI integration the opflex-

agent is taking care of the

routing of the VMs. Since each

compute node has a opflex-

agent, the routing is done in a

distributed manner.

• Also, the opflex-agent performs

local policy enforcement through

OVS rules locally on the same

hypervisor where the instance

lives.

DESCRIPTION

57

Neutron L3

Agent

EndpointFile

Agent-OVS

OpenFlow

BRKACI-3456


DHCP Function

Tenant Networks

ComputeHost

Neutron Server(s)

Agent-OVS


DHCP Allocation and Options

DHCP DORA

Neutron DHCPAgent DNSmasq neutron-

opflex-agent

EndpointFile

• Traditionally VMs are

getting IP from Neutron

DHCP Server

• Agent-OVS learns info of

the VM from Endpoint

Files

• Agent-OVS responds to

the VMs with DHCP

responses

• DHCP allocation and

options passed back to

Neutron server.

DESCRIPTION

58BRKACI-3456


Metadata Function

Tenant Networks

NovaComputeHostOpenStack Controller

Agent-OVS


neutron-opflex-agent

Nova-APIMetadataService

neutron-metadata-agent

VM Metadata

ServiceFile

VMMeta-data

• Traditionally in OS VMs

get the meatadata

information from the

service running on

Neutron Server

• Neutron metadata agent is

reading the Service File

• Metadata agent locally

performs proxy

• Metadata agent updates

the neutron server with

VM Metadata

DESCRIPTION

59BRKACI-3456


L3-Out VRFRID: 7.7.7.7Tenant VRFs

Compute Node

Open vSwitchw/Local NAT

ACI Fabric External RouterWith IP routes to:SNAT: 10.1.2.0/24Floating: 10.1.3.0/24

SNAT Subnet IP:10.1.2.1/24

Floating Subnet IP:10.1.3.1/24

Link Subnet IP:10.1.1.2/30

Link Subnet IP:10.1.1.1/30

NAT/External Traffic

Non-NATTenant Traffic

NAT Function performed in the OVS locally

• Floating IP configured by

OpenStack Neutron using

standard mechanism

• OVS performs NAT

function using OpenFlow

rules from OpFlex agent

for Floating IP

DESCRIPTION

60BRKACI-3456


About the OpenStack Infrastructure network

BRKACI-3456


Required connectivity for hosts

• Typically there will be a number of networks required for OpenStack:

• Internal API Network (VLAN)

• Storage Network (VLAN)

• Storage Management Network (VLAN)

• Provisioning Network (Native VLAN)

• External Network (VLAN)

62

Note:

Controller node requires connectivity to the APIC controller. External Network can be used for this purpose.

BRKACI-3456


Physical connectivity with ACI

OpenStack node will need:

• At least two NICs per server configured as bond interface (for redundancy)

• One NIC for provisioning network

BRKACI-3456 63


ACI Configuration for OpenStack Node connectivity

To provide connectivity between hosts, it is required to pre provision a tenant on ACI with the appropriate configuration.

This tenant could be either dedicated to the OpenStack infrastructure, or it could be shared with other infrastructure hosts.

Note that this infrastructure tenant will provide the underlay connectivity for the host, therefore it will be updated only if necessary to modify the OpenStack node connectivity (i.e. adding a node).

BRKACI-3456 65


ACI OpenStack Infrastructure tenant

66BRKACI-3456

VRF: Main_VRF

BD_MGMT

L3 unicast enabled

Default GW IP defined

Tenant: OpenStack_Infra

EPG-ExternalNet

VLAN104EPG-InternalApi

VLAN105

EPG-StorageNet

VLAN106EPG-StorageMgmt

VLAN107

- Two BDs:

- BD_MGMT provides OOB connectivity (in this design this provides connectivity both

Internet and APIC)

- BD_OSP is only switching but we keep L3 enabled to learn IP from the hosts

- EPGs have static bindings to the interfaces of the host

EPG-Provisioning

Native-VLAN

BD_OpenStack_Infra

L3 Unicast Enabled

Limit IP Learning to Subnet: Disabled


EPG Static binding

EPG-

Provisioning

Native

EPG-

InternalApi

VLAN105

EPG-

StorageNet

VLAN106

EPG-

StorMgmt

VLAN107

EPG-

External

VLAN104

OpenStack nodes will have NIC interfaces statically bound to ACI End Point Group.

On ACI side an individual interface will be configured for Provisioning network. The bond interfaces of

the host will be connected to a VPC pair on ACI leaf switches.

bond0

BRKACI-3456 67


• Define the EPG and the specific static binding for each network required:

• ExternalNet

• InternalAPI

• StorageMgmt

• StorageNet

OpenStack Infrastructure Tenant

BRKACI-3456 69


OpenStack Tenant Network (VM datapath)

BRKACI-3456


OpenStack Tenant Network

• Neutron provides each tenant with their own networks using either VLAN segregation (where each tenant network is a network VLAN), or tunneling (through VXLAN). Network traffic is isolated within each tenant network. Each tenant network has an IP subnet associated with it, and network namespaces means that multiple tenant networks can use the same address range without causing conflicts.

71BRKACI-3456

• ACI Plugin allows to use as encapsulation mode:

• VLAN

• VXLAN


OpenStack Tenant Network with ACI with VLAN

• In VLAN encapsulation mode:

• The user will define a pool of VLAN (VMM Domain Pool)

• Each OpenStack project network will automatically pick one VLAN from the pool

• The ACI Access policy of the Leaf ports will allow all the VLANs defined in the VLAN pool

72

The bond0 could be the same interface used for the OpenStack infra traffic.

However this could be also a dedicated bond for tenant traffic.

BRKACI-3456

VMM Domain Pool: 200-300

bond0

Tenant1

net1

VLAN 200

Tenant1

net2

VLAN 201

Tenant2

net1

VLAN 220

Tenant3

Net1

VLAN 230

VLAN trunk


OpenStack Tenant Network with ACI with VLAN: Scalability

• If you need to scale more than 4k VLANs:

1. You can use VXLAN

2. You can create multiple VMM domain and assign nodes to those:Allows you to use multiple VMM Domains with potentially overlapping VLAN pool ranges in a single OpenStack deployment

73

VMM Domain1 Pool: 200-300

bond0

VLAN trunk VLAN trunk

VMM Domain2 Pool: 200-300

Compute-1

BRKACI-3456

Compute-2

bond0


OpenStack Tenant Network with ACI with VXLAN

• In VXLAN encapsulation mode:

• The ACI Access policy of the Leaf ports will allow the ACI Infra VLAN.

• Each OpenStack project network will automatically pick one VXLAN

• The VXLAN will be encapsulated into a tunnel using the ACI infra VLAN

74

The bond0 could be the same interface used for the OpenStack infra traffic or could be a dedicated bond for tenant traffic.

For better performance, server NICs should be capable of VXLAN offload

Blade systems are not supported with VXLAN encapsulation

BRKACI-3456

VMM Domain: VXLAN

bond0

Tenant1

net1

VXLAN 200

Tenant1

net2

VXLAN 201

Tenant2

net1

VXLAN 220

Tenant3

Net1

VXLAN 230

VXLAN are encapsulated into tunnel

using ACI Infra VLAN


What if you want to provision VLAN to 3rd party?

BRKACI-3456


Hierarchical port binding (HPB)

opflex main segment

(VLAN or VXLAN)

ACI Leaf

Switch

3rd party opflex

Compute Host

non-opflex

using local vlan segment

Hierarchical Port Binding allows to create different network types:

• Opflex networks would be created onto ACI

• vlan or other network types can be created to bind special 3rd party agent or mech driver asks for vlan port binding

BRKACI-3456 76


SR-IOV support with ACI

VNF

OpenStack Controller

Group-Based Policy

VNFVNF VNF VNF

NIC NICSRIOVSRIOV

ML2

VLANs

Another use case for HPB is SR-IOV enabled hosts:

• GBP or ML2 options

• GBP – Reintroduces security policies via groups / rulesets in the fabric

• Can mix opflex and SR-IOV on the same physnet

BRKACI-3456 78


• Many OpenStack customers are interested connecting VMs to both IPv4 and IPv6 networks

• This feature adds support in the ACI OpenStack plugins for dual stack operation

• OpenStack neutron address scopes are automatically mapped to ACI VRFs

• Each IPv4 address scope maps to a unique VRF in ACI. A IPv6 address scope may include multiple IPv4 address scopes will be provisioned on these VRFs

IPv6 Dual Stack

BRKACI-3456 79

Installation of OpenStack


OpenStack Support

• Cisco is committed to provide support to the main OpenStack distributions:

• Other distributions is supported with specific agreements with the 3rd party vendor, i.e.

81BRKACI-3456


Installation of OpenStack and ACI Plugin

• On Cisco.com:

• https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html#OpenStack_Installation_Guides

• Manual installation:

• Prone to errors and discouraged. Moving forward we will limit the support for production environments while documentation will be always provided.

• RHEL OSP Director – full support for automated installation and upgrade

• Canonical Juju Charms – full support for automated installation and upgrade

82BRKACI-3456

https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html#OpenStack_Installation_Guides

Operate OpenStack


ML2 model

BRKACI-3456


ML2 – APIC Mapping

Neutron Object APIC Object

Project Tenant

Network EPG + BD

Subnet Subnet

Router Contract

Security Group + Rule N/A

Iptables rules maintained per host

85

• With the ML2 Standard Neutron model, the following mapping happens.

• All the operations are done on OpenStack through Horizon, CLI or Heat

BRKACI-3456


• A new Tenant is created and a new EPG and unicast disabled BD is created

• Unicast routing will stay disabled until a router is created in OS

• The BD is attached to a generic unroutedVRF created in common tenant

What happens on ACI

BRKACI-3456 90


Security Groups with ML2 model

BRKACI-3456

• In ML2 policy mode the router created corresponds to a permit any contract in ACI.

• Security groups are defined in OpenStack and controlled there.

• They will be reflected in policy defined in OVS rules.

92


GBP model

BRKACI-3456


GBP – APIC Mapping

GBP Object APIC Object

Project Tenant

L3 Policy VRF

L2 Policy Bridge Domain

Policy Group Endpoint Group

Policy Ruleset Contract

95

• With the GBP Model the following mapping happens.

• GBP offers much more granularity and flexibility compare to standard neutron.

• GBP comes with CLI, Heat and Horizon plugins

BRKACI-3456


Create GBP groups (ACI EPGs):

[stack@dom-undercloud ~]$ gbp group-create epg101 --l2-policy l2pnet101

BRKACI-3456

[stack@dom-undercloud ~]$ gbp l2policy-create --l3-policy Main_VRF l2pnet102



I can add more EPG, both in the same Bridge Domain, or in others:

98


In order to allow communication, I need to create policy actions, classifiers, rules and rulesets within GBP.

GBP

Policy Classifier

Policy Rule

Policy Ruleset

ACI

Filter Entry

Filter

Contract

BRKACI-3456 99


Next, define a Policy Rule, referencing the classifier created in the last step:

[stack@dom-undercloud ~]$ gbp policy-rule-create ping-policy-rule --classifier icmp-traffic --actions allow

Created a new policy_rule:

+--------------------------+------------------------------------------------------------------------------------------+

| Field | Value |

+--------------------------+------------------------------------------------------------------------------------------+

| apic:distinguished_names | {"Forward-FilterEntries": ["uni/tn-common/flt-pr_3ecd614d-717b-483c-8e5c-c5f335d40a88/e |

| | -os-entry-0"], "Reverse-FilterEntries": ["uni/tn-common/flt-reverse-pr_3ecd614d-717b- |

| | 483c-8e5c-c5f335d40a88/e-os-entry-1", "uni/tn-common/flt-reverse-pr_3ecd614d-717b-483c- |

| | 8e5c-c5f335d40a88/e-os-entry-2", "uni/tn-common/flt-reverse-pr_3ecd614d-717b-483c-8e5c- |

| | c5f335d40a88/e-os-entry-3", "uni/tn-common/flt-reverse-pr_3ecd614d-717b-483c-8e5c- |

| | c5f335d40a88/e-os-entry-4"]} |

| description | |

| enabled | True |

| id | 3ecd614d-717b-483c-8e5c-c5f335d40a88 |

| name | ping-policy-rule |

| policy_actions | 2070e9ff-4de9-46ea-a81e-772906982adf |

| policy_classifier_id | 88c8e3c0-d9c5-4e6b-9992-a56b539e0b98 |

| project_id | 5b8945dba07a43e0b32efea4f1bc3fdf |

| shared | False |

| status | BUILD |



+--------------------------+------------------------------------------------------------------------------------------+

102


Let’s now apply these rules to our EPGs:

[stack@dom-undercloud ~]$ gbp group-update epg101 --provided-policy-rule-sets "icmp-policy-rule-set=true"

Updated policy_target_group: epg101

[stack@dom-undercloud ~]$ gbp group-update epg102 --consumed-policy-rule-sets "icmp-policy-rule-set=true"

Updated policy_target_group: epg102

BRKACI-3456 104


We can now ping between the two networks!

Compute

Node1

ACI Fabric

epg101 epg102

epg101 EPG Epg102 EPGContract

Compute

Node2epg102

OVS rules do the routing andenforcement on the host

Inter host enforcement isdone on ACI leaf switches.

BRKACI-3456 105

External Network


Shared L3 out

External Connectivity

• Connectivity for a tenant can be either shared or dedicated.

• A shared external network is visible by all OpenStack projects.

• A dedicated connectivity for the OpenStack project.

• It would be possible to have a mixed environment both with shared and dedicated external connectivity.

107BRKACI-3456

net1 net2 net3 net4

Tenant Pasta&Co Tenant Pizza&Co

Dediacated

L3 out

net1 net2 net3 net4

Tenant Pasta&Co Tenant Pizza&Co

Dediacated

L3 out

WWW WWW WWW


Physical Layout

108BRKACI-3456

Compute2Controller

APIC

Compute1

• L3out is defined on ACI.

• The external router is defined with a dynamic or static protocol

L3Out

OSPF/

BGP/

static


How to create the L3out on ACI

• Although the OpenStack plugin could create automatically an L3out on ACI, the best practice is to create it manually

• Defining manually an L3out supports all the L3out features:

• VPC

• Dynamic routing protocols

• Route engineering

• Etc.

• The L3out can be created with XML templates or in any ways you are familiar with.

• Once the L3out is available, ACI AIM plugin on OpenStack can import it and start controlling the L3out.

109BRKACI-3456


Dedicated Tenant External Network

BRKACI-3456


Creation of the L3out Dedicated

111BRKACI-3456

• A dedicated L3out must be created in the OpenStack created tenant.

• In the L3out creation, it must be specified:

• Interfaces and their IP information

• Dynamic routing if any

• External EPG

• You should NOT add any contract as they will be added later automatically by the plugin.

• If you require SNAT or FIP, the L3 out must be defined in a different VRF from the one created by OpenStack!


Query ACI for external networks

• Through the ACI Integration Module (AIM) controller, it is possible to query ACI for the existing and available external networks.

112BRKACI-3456

[heat-admin@overcloud-controller-0 ~]$ aimctl manager external-network-find

+--------------------------------------+-------------------+--------+

| tenant_name | l3out_name | name |

|--------------------------------------+-------------------+--------|

| common | l3out1 | extEpg |

| prj_4ec99ec19a0f4f00808f18d82d7032af | l3out1-DefaultVRF | extEpg |

| prj_5d0431309d5d45a1836dfa0a8beb6ef0 | l3out1-DefaultVRF | extEpg |

| prj_97390b780c7545d393d9314d34e69cfa | externalNet | extEpg |

+--------------------------------------+-------------------+--------+


Import External Networks from ACI to OpenStack

113BRKACI-3456

[heat-admin@overcloud-controller-0 ~]$ aimctl manager external-network-get

prj_97390b780c7545d393d9314d34e69cfa externalNet extEpg

+-------------------------+--------------------------------------------------------------------------+

| Property | Value |

|-------------------------+--------------------------------------------------------------------------|

| tenant_name | prj_97390b780c7545d393d9314d34e69cfa |

| l3out_name | externalNet |

| name | extEpg |

| monitored | True |

| consumed_contract_names | [] |

| provided_contract_names | [] |

| dn | uni/tn-prj_97390b780c7545d393d9314d34e69cfa/out-externalNet/instP-extEpg |

+-------------------------+--------------------------------------------------------------------------+

+---------------+-----------------------------------------+


|---------------+-----------------------------------------|

| resource_type | ExternalNetwork |

| resource_root | tn-prj_97390b780c7545d393d9314d34e69cfa |

| sync_status | synced |

| health_score | 100 |

| id | 3e368bc8-e83d-4c8a-b269-6c7873464def |

+---------------+-----------------------------------------+

• AIM controller manager will import the external network


Create OpenStack External Network

114BRKACI-3456

[stack@dom-undercloud ~]$ neutron net-create external-net-CL --router:external --apic:distinguished_names type=dict

ExternalNetwork=uni/tn-prj_97390b780c7545d393d9314d34e69cfa/out-externalNet/instP-extEpg

+----------------------------+----------------------------------------------------------------------------------------+

| Field | Value |

+----------------------------+----------------------------------------------------------------------------------------+

| admin_state_up | True |

| apic:distinguished_names | {"ExternalNetwork": "uni/tn-prj_97390b780c7545d393d9314d34e69cfa/out-externalNet |

| | /instP-extEpg", "BridgeDomain": "uni/tn-prj_97390b780c7545d393d9314d34e69cfa/BD-EXT- |

| | externalNet", "VRF": "uni/tn-prj_97390b780c7545d393d9314d34e69cfa/ctx-DefaultVRF", |

| | "EndpointGroup": "uni/tn-prj_97390b780c7545d393d9314d34e69cfa/ap-OpenStack/epg-EXT- |

| | externalNet"} |

| apic:nat_type | distributed |

| apic:synchronization_state | synced |

| id | f085fe67-42e1-4b3c-8951-e5d9932222ca |

| is_default | False |

| name | external-net-CL |




| provider:segmentation_id | |


| router:external | True |

| shared | False |

| status | ACTIVE |

| subnets | |

+----------------------------+----------------------------------------------------------------------------------------+

Creating neutron external network bound to the L3out imported with the aimctl manager.


External SNAT or Floating IP Pool Definition

115BRKACI-3456

[stack@dom-undercloud ~]$ neutron subnet-create external-net-CL 10.104.21.0/24 --name ext-subnet --disable-dhcp

--gateway 10.104.21.1 --apic:snat_host_pool True

Created a new subnet:

+----------------------------+--------------------------------------------------+

| Field | Value |

+----------------------------+--------------------------------------------------+

| allocation_pools | {"start": "10.104.21.2", "end": "10.104.21.254"} |

| apic:distinguished_names | {} |

| apic:snat_host_pool | True |

| apic:synchronization_state | N/A |

| cidr | 10.104.21.0/24 |


| enable_dhcp | False |

| gateway_ip | 10.104.21.1 |

| host_routes | |

| id | 5344832d-dd03-40d7-a4d2-3f04c86fbb9d |

| ip_version | 4 |

| ipv6_address_mode | |

| ipv6_ra_mode | |

| name | ext-subnet |

| network_id | f085fe67-42e1-4b3c-8951-e5d9932222ca |


| service_types | |

| subnetpool_id | |

| tenant_id | 97390b780c7545d393d9314d34e69cfa |

+----------------------------+--------------------------------------------------+

[stack@dom-undercloud ~]$ openstack router set --external-gateway external-net-CL CLrouter

Creating neutron external network SNAT pool and attaching the router to the external net.


SNAT Pool

116BRKACI-3456

Each Hypervisor will be assigned with one IP from the pool and the VMs will be NATted with the IP of the hypervisor.


The External network in ACI

117BRKACI-3456

The External Network EPG will be created in the tenant itself.

A contract to allow connectivity between the EPG and the L3out will be created automatically.

1. VM traffic reaches OVS

2. OVS applies NAT rules

3. The NATted IP in ACI is represented by the external EPG

4. Traffic is sent to external router through ACI


Shared Tenant External Network

BRKACI-3456


• The shared external network must be defined in the Common tenant in ACI

Create L3 out on ACI – Shared

BRKACI-3456 119


Same as before, you use the aimctl manager to import the external network

120BRKACI-3456

[heat-admin@overcloud-controller-0 ~]$ aimctl manager external-network-find

+--------------------------------------+--------------+--------+

| tenant_name | l3out_name | name |

|--------------------------------------+--------------+--------|

| common | l3out1 | extEpg |

| prj_11fa0c41388f4d3fbf3f2f6d6184f687 | externalNet | extEpg |

+--------------------------------------+--------------+--------+

[heat-admin@overcloud-controller-0 ~]$ aimctl manager external-network-get common l3out1 extEpg

+-------------------------+---------------------------------------+


|-------------------------+---------------------------------------|

| tenant_name | common |

| l3out_name | l3out1 |

| name | extEpg |

| nat_epg_dn | |

| display_name | |

| monitored | True |

| consumed_contract_names | [] |

| provided_contract_names | [] |

| dn | uni/tn-common/out-l3out1/instP-extEpg |

+-------------------------+---------------------------------------+

[heat-admin@overcloud-controller-0 ~]$


External Network

121BRKACI-3456

[stack@dom-undercloud ~]$ neutron net-create external-net-CL --router:external --shared --apic:distinguished_names

type=dict ExternalNetwork=uni/tn-common/out-l3out1/instP-extEpg

+----------------------------+----------------------------------------------------------------------------------------+

| Field | Value |

+----------------------------+----------------------------------------------------------------------------------------+

| admin_state_up | True |

| apic:distinguished_names | {"ExternalNetwork": "uni/tn-common/out-l3out1/instP-extEpg", "BridgeDomain": "uni/tn- |

| | common/BD-osp11_s2_EXT-l3out1", "VRF": "uni/tn-common/ctx-external_vrf", |

| | "EndpointGroup": "uni/tn-common/ap-osp11_s2_OpenStack/epg-EXT-l3out1"} |

| apic:external_cidrs | 0.0.0.0/0 |

| apic:nat_type | distributed |

| apic:synchronization_state | synced |

| availability_zone_hints | |

| availability_zones | |

| id | b90bfad9-4ed3-477f-996a-4222ae0768dd |

| is_default | False |

| name | external-net-CL |


| project_id | 11fa0c41388f4d3fbf3f2f6d6184f687 |



| provider:segmentation_id | |


| router:external | True |

| shared | True |

| status | ACTIVE |

| subnets | |

+----------------------------+----------------------------------------------------------------------------------------+

Creating neutron external network bound to the L3out imported with the aimctl manager.


SNAT Pool

123BRKACI-3456

Same as before, each Hypervisor will be assigned with one IP from the pool and the VMs will be NATtedwith the IP of the hypervisor. This time the SNAT IP will appear in the Common Tenant in ACI.


Floating IP in ACI

125BRKACI-3456

Floating Subnet will be visible in ACI and when you assign a FIP to a VM this will appear in the operational tab of the external EPG.


Creation of opflex networks.

BRKACI-3456

Demo Time!

Binding of OpenStack VMs to those networks.

Adding connectivity to a bare metal server and a vSphere virtual machine.


controller compute1compute2 ESXi Bare Metal

BRKACI-3456 128


APIC

APIC

APIC

controller compute1compute2 Bare Metal

EPG green-OS

EPG Orange-OS

Bridge Domain Orange 192.168.100.254/24

Bridge Domain Green 192.168.200.254/24

ESXi

BRKACI-3456 129


APIC

APIC

APIC


EPG green-OS

EPG Orange-OS

EPG green-mixed



BRKACI-3456 130


APIC

APIC

APIC


EPG green-OS

EPG Orange-OS

EPG green-mixed



BRKACI-3456 131


APIC

APIC

APIC


EPG green-OS

EPG Orange-OS



Contract allow-ICMP

Allow ICMP

Contract allow-SSH

Allow TCP:22

EPG green-mixed

BRKACI-3456 132


Are we there yet?

134BRKACI-3456


ACI connects Virtual and Physical World

135BRKACI-3456


Documentation

• APIC OpenStack Plugin Installation Guides:

• http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/openstack/b_ACI_with_OpenStack_OpFlex_Architectural_Overview.html

• http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/openstack/b_ACI_with_OpenStack_OpFlex_Deployment_Guide_for_Red_Hat.html

• http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/openstack/b_ACI_with_OpenStack_OpFlex_Deployment_Guide_for_Ubuntu.html

137BRKACI-3456

For YourReference

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/openstack/b_ACI_with_OpenStack_OpFlex_Architectural_Overview.html

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/openstack/b_ACI_with_OpenStack_OpFlex_Deployment_Guide_for_Red_Hat.html

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/openstack/b_ACI_with_OpenStack_OpFlex_Deployment_Guide_for_Ubuntu.html


Documentation (Cont.)

• APIC GBP Plugin Datasheet:

• http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/openstack-at-cisco/datasheet-c78-734181.html

• APIC OpenStack Plugin Datasheet:

• http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/openstack-at-cisco/datasheet-c78-732353.html

• GBP WhitePaper:

• http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-733126.html

• GBP wiki:

• https://wiki.openstack.org/wiki/GroupBasedPolicy

For YourReference

BRKACI-3456 138

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/openstack-at-cisco/datasheet-c78-734181.html

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/openstack-at-cisco/datasheet-c78-732353.html

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-733126.html

https://wiki.openstack.org/wiki/GroupBasedPolicy


Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#BRKACI-3456


• Please complete your Online Session Evaluations after each session

• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.

Complete Your Online Session Evaluation

http://ciscolive.com/Online

http://www.ciscolive.com/global/on-demand-library/


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Tech Circle

• Meet the Engineer 1:1 meetings

• Related sessions

141BRKACI-3456

Thank you

Mastering ACI and - clnv.s3.amazonaws.com · Mastering ACI and OpenStack Domenico Dastoli Technical...

Documents

Transcript of Mastering ACI and - clnv.s3.amazonaws.com · Mastering ACI and OpenStack Domenico Dastoli Technical...