ACI Universe

A tenant is a logical container for application, Concepts Policy Model Application Profile (fvAp)
network and security policies that enable an ACI uses the concepts of Tenants, Private In the policy model, EPGs are tightly coupled with VLANs. For traffic to flow, an EPG must be AP’s define the policies, services and relationships
administrator to exercise domain-based access Networks, Bridge Domains, and Subnets in a deployed on a leaf port with a VLAN in a physical, VMM, L2out, L3out, or Fiber Channel domain. between EPGs. Each AP contains one or more EPG
control. A tenant is a unit of isolation from a policy hierarchy to contain routed traffic. that can communicate with the other EPGs in the
perspective, but it does not represent a private The domain profile associated to the EPG contains the VLAN instance profile. same AP and with EPGs in other AP’s according to the
network. The system provides the following four Some relationships are linked, others are like The domain profile contains both the VLAN instance profile (VLAN pool) and the attachable contract rules.
kinds of tenants: parent-child: Access Entity Profile (AEP), which are associated directly with application EPGs.
- BD is linked to a VRF End Point Groups
1. User Tenants - Subnets are children of BDs The AEP deploys the associated application EPGs to all the ports to which it is attached, and EPG’s are basic unit of policy enforcement, a logical
Defined by the administrator according to the needs - BD’s are children of Tenants automates the task of assigning VLANs. entity that contains a collection of physical or virtual
of users. They contain policies that govern the network endpoints. Endpoints are devices connected
operation of resources such as applications, Policy Universe to the network directly or indirectly. They have an
databases, web servers, network-attached storage, Solid Lines address (identity), a location, attributes and can be
virtual machines, and so on. = object contain the ones below 1 1 physical or virtual. Endpoints within the same EPG can
Dotted Lines communicate freely without restrictions.
n
2. Common Tenant = Indicate a relationship
Provided by the system but can be configured by the Tenant End Points
fabric administrator. It contains policies that govern (user, common or infra) It stands for hosts, in other words MAC address with
the operation of resources accessible to all tenants, IP(s)
such as L4 to L7 services, etc... 1 1 1 1 1 1 - sometimes MAC only
- IP in EP is always /32
3. Infrastructure Tenant Private Network, Context or VRF Instance
Provided by the system but can be configured by the A Virtual Routing and Forwarding (VRF) object (fvCtx)
fabric administrator. It contains policies that govern n n n n n n 1
or context is a tenant network. A VRF is a unique
the operation of infrastructure resources such as the Outside Application Bridge Layer 3 forwarding and application policy domain. A
VRF Contract Filter Access
fabric VXLAN overlay. Network Profile Domain VRF provides IP address space isolation for tenants. A
1 1 1 n 1 n 1 tenant can have multiple VRFs.
4. Management Tenant n
n n
Provided by the system but can be configured by the Domain Profile Bridge Domain (BD)
fabric administrator. It contains policies that govern n
n Subject (Phys, VMM, L2out, L3out A BD is a Layer 2 (broadcast) domain that must be
the operation of fabric management functions. or Fibre Channel) associated with a Tenant‘s Private Network. A BD may
Subnet 1
Contract contain one or more Subnets that can provide routing
Contracts are rules that specify how communications n services for associated EPG’s. It’s a set of logical ports
between EPGs take place. VLAN that share the same flooding or broadcast
Instance Profile characteristics. Like a virtual LAN (VLAN), bridge
Subject 1 domains span multiple devices.
n n n n
A subject is a sub-application running behind an EPG.
Attachable Access Entity Subnet
Endpoint Group
Filter 1 Profile A Subnet is defined by an IP address/mask and
ACI uses a whitelist model: all communication is 1 provides a routing gateway service for EPG’s that are
blocked by default; communication must be given 1 associated to the Subnet‘s parent BD. A BD can
explicit permission. A filter is a TCP/IP header field, Criterion contain multiple Subnets, but a Subnet is contained
such as a L3 protocol type or L4 ports, that are used Encap, IP, MAC within a single BD.
to specify the type of traffic that can be
communicated and how it occurs between EPGs. Common Pervasive Gateway (CPG):
Allows us to have a virtual MAC and virtual IP which is
Consumer/Provider common (the same) across multiple ACI Fabrics.
An EPG that consumes/provides a service. *pervasive = spreading widely throughout an area

Typically you will have only 1 Subnet per BD, so in that sense a Bridge Domain “resembles” a Contracts are not directly linked to Filters. Contracts have a child object called a
VLAN, and a Subnet “resembles” a VLAN interface. Subject and it is the Subjects that are linked to Filters. For a Contract to pass traffic, the
EPGs are children of APs, AP is a child of Tenant, tenant owns AP, AP owns
EPG either has to Consume a Contract or Provide a Contract.
EPGs. But each EPG must be linked to a BD.

Application Centric Infrastructure (ACI) Concepts

 ACI Switches & Interfaces
Access Policies
Access policies refer to the configuration that is applied for physical and virtual devices attached to the fabric. Broken into a few major areas:
Access policies Switch Policy: Interface Policy:
Govern the operation of Global Policy:
Switch Policy Interface Policy
switch access ports that Pool (Vlan / VXLAN)
Define protocol/feature configurations Define protocol / feature configurations
provide connectivity to Pool of encapsulations that can be allocated within the fabric.
Switch Policy Group Interface Policy Group
resources such as storage, Domain (Physical / VMM / L2/L3 External)
Select which policies should be applied Used to specify which interface policies to be applied to a particular interface type.
compute, L2 and L3 Administrative domain which selects a vlan pool for allocation
Switch Profile It also associates an AEP (which defines which domains are allowed on the interface)
connectivity, vm hypervisors, Attachable Access Entity Profile (AEP)
Associate policy groups to switches or Interface Profile
L4 to L7 devices, ... Selects 1 or more domains and is referenced/applied by interface policy groups.
interfaces, through the use of selectors Associate policy groups to switches or interfaces, through the use of selectors
Fabric policies Domains
Domains
Govern the operation of the A domainare
Domains is configured to be associated
a place to deposit a bunch ofwith a VLAN
VLANs via pool.
a VLAN
switch fabric ports, including EPGs
Pool. are
Theyconfigured
also link theto use the VLANs
physical associated
configuration (viawith a
the AEP) to
such functions as NTP synch, domain. Domains link the physical configuration (via
the Logical Tenant configuration. Just how this is done depends AEP) to
IS-IS, BGP route reflectors, the Logical Tenant configuration. Just how this is
on the type of Domain involved. There are 4 type of Domains: done depends
DNS and so on. The fabric on the type of Domain involved. There are 4 type of domains:
MO contains objects such as Physical Domains
power supplies, fans, Interface Policy Types: Physical
Permit a Domains
set of VLANs on a given port (or PC/VPC) where
chassis, and so on. Link-level Physical Domain Profiles
directly attached hosts or(physDomP)
a switch might be attached. This
CDP Permit VLANs
makes it possibleon atoport/PC/VPC
then map one where directly
of the allowedattached hosts
VLANs to a
LLDP Attachable Access Entity Profile (AEP) or a switch might
particular EPG. be attached. This makes it possible to map
Port-channel / LAG Used to group domains with similar one of the allowed VLANs to a particular EPG.
Port-channel member requirements. By grouping domains VMM Domains (Virtual Machine Manager)
Spanning-tree into AEPs and associating them, the VMM
PermitDomains (VirtualonMachine
a set of VLANs a given Manager)
port (or PC/VPC) where a
Switch Policy Types: VMM
Storm Control Interface Policy Groups (IPG) fabric knows where the various devices directlyDomain
attachedProfiles
(e.g.) (vmmDomP)
ESXi host might be attached. A VMM
VPC Domain Permit VLANs on a port/PC/VPC where a directlywith
attached
Data plane policing There are 3 types of Interface Policy Groups: in the domain live and the APIC can Domain also provides a mechanism to integrate the (e.g.)
Spanning-tree (MST) ESXi host might
MCP push the VLANs and policy where it related VMM sobethatattached.
the VLANsA VMM Domain
associated alsothe
with provides a
VMM can
BFD mechanism to integrate
L2 (Vlan local / global) Access Port Policy Groups (APPG) needs to be. Every Interface Policy be dynamically allocatedwith and the related VMM
automatically so thattothe
allocated EPGs. VLAN Pool
Fibre-channel SAN/Node VLANs can be dynamically and automatically allocated to EPGs.
Firewall - Set of policies for a group of ports that are allowed Group needs to be associated with an A VLAN Pool is a
Switch Profiles (SP) to use the same VLAN set. AEP. An AEP as a collection point for a Layer 2 External Domains collection of VLAN
Can define a single (leaf) switch, or a Interface Profiles group of access ports, PC and VPC. Layer
Permit2 aExternal Domains
set of VLANs on a given port (or PC/VPC) where a
Access IDs assigned in
number of switches, and this profile can Interface Profiles define a Port Channel Interface Policy Groups (PCIPG) Bridged outside network
L2Out connects. This makes Domain Profiles
it possible (l2extDomP)
to map one of the blocks. Every
1 be linked to a number of Interface range of interfaces - Define a set of policies for a single PC A container that allows you to define Permit
allowedVLANs
VLANsontoaaport/PC/VPC where an External
Node Profile/Interface Profile. Bridged Domain needs a
Profiles defining a range of interfaces. - This PCIPG becomes the identifier for the PC which VLANs – via its relationship to Network (L2Out) connects. This makes it possible to map one
n single VLAN Pool
Access Port Selectors Domains – are allowed to exist on of the 3allowed
Layer External VLANs
Domainsto a Node Profile/Interface Profile.
Domain Profile associated with it.
n Switch Selectors Each physical port can only Virtual Port Channel Interface Policy Groups (VPCIPG) which Attachable Entities (Ports/PC/ Permit a set of VLANs on a given port (or PC/VPC) where a
(Phys, VMM, L2out, L3out Every AEP can be
A Switch Profile has at least one child belong to one named Access - Define a set of policies for a single VPC VPC) – via its relationship to Interface Layer
L3Out3connects.
External This
Domainsmakes it possible to map one of the
or Fibre Channel) associated with a
1 entity called a Switch Selector. Port Selector. - This VPCIPG becomes the identifier for the PC Profiles Routed
allowedoutside
VLANs to network
a NodeDomain Profiles (l3extDomP)
Profile/Interface Profile. bunch of Domains.
Permit VLANs on a port/PC/VPC where an External Routed
n Network (L3Out) connects. This makes it possible to map one
VLAN of the allowed VLANs to a Node Profile/Interface Profile.
Instance Profile
1 Interface Profile Fibre Channel Domains
n Fibre Channel Domain Profiles (fcDomP)
Access Port Access Port
Attachable Access Entity Used to connect Fibre Channel VLANs and VSANs.
Selector Policy Group
Profile
Switch Profile Access Port Access Port Domain VLAN Pool
Selector Policy Group Attachable
Switch Access Entity Policy
Selector (AEP) Domain VLAN Pool
Interface Profile
Access Port Access Port The vlan containers
To maximise flexibility: Selector Policy Group
- Define 2 kinds of Switch Profiles:
- For VPC (for each pair) switchport trunk allowed vlans 1000-1010
- 2 Switch Selectors for the 2 switches
interface range switch1/eth/1/1-10, switch2/eth/1/1-10 - Create a VLAN pool
- For single-attached devices (each leaf switch)
cdp enable - Create a domain (physical, l2/l3 external or VMM) and associate pool
- 1 Switch Selector for that switch
speed 1000 - Associate domain to AEP
- Define only 1 Interface Profile for each Switch Profile
- Define one Access Port Selector for every single port - Associate interface policy group to AEP
- DO NOT define interface ranges, like 1/1-10
- Define 10 Interface Selectors, each with 1 port We specified what domains and corresponding pools are allowed per interface in the fabric!

Application Centric Infrastructure (ACI) Concepts

 ACI CONCEPTS
Software defined networking
New Approach
Can be defined as a new approach to design, implement and manage networks that is based on the concept of separating the
With the classical networking approach, both of these functions as co-exist on the same network device. Using the SDN approach, as
network control plane and data plane, where the control plane provides an abstracted centralized view of the network.
highlighted earlier these two functions are separated, in which the control plane is Centralized and the forwarding plane is ket distributed. As
a result, SDN provide the ability of administering traffic and deploying services centrally to address changing business need s, without having
Control plane
to touch each individual switch or router in the forwarding plane.
Think of it as the brain of the network where all the intelligence happens such as routing function ( path peering, path sele ction
etc.)
OpFlex
The APIC centrally push policies to the underlying infrastructure using an extensible policy protocol designed to exchange ab stract policy
Data plane
between a network controller and a set of smart devices capable of rendering policy called OpFlex.
Think of it as the muscular part where all the heavy load of the traffic forwarding happens.
Profile Leaf Switches Spine Switches Fabric
Named entity that contains the necessary config All workloads connect to leaf switches. The leaf switches used in an The ACI fabric forwards traffic primarily based on host lookups. A mapping Based on an IP fabric supporting routing to the edge with an integrated
details for implementing 1 or more instances of a ACI fabric are ToR switches. They are divided into 4 main types database stores the info about the ToR switch on which each IP address overlay for host routing. All tenant traffic within the fabric is carried
policy, eg, a switch node profile for a routing policy based on their hardware. resides. This info is stored in the fabric cards of the spine switches. through the overlay decoupling of endpoint identity, location, and
would contain all BGP info. associated policy; all are independent from the underlying topology.
Border Leafs
Alias Provide L2 or L3 external connectivity to outside networks, Site
Spine Spine
A changeable name for a given object. supporting routing protocols to exchange routes with external The APIC cluster domain or single fabric, treated as an ACI region and
routers. availability zone.
Label
Label matching is used to determine which consumer Service Leafs Atomic Counters
and provider EPGs can communicate. A label Connect to L4 to L7 service appliances, such as firewall, load Allow you to gather statistics about traffic between leafs.
matching algorithm is used determine this balancer, and such. The connectivity between the service leaf and
communication. the service appliance can be L2 or L3. Cisco Application Virtual Switch (AVS)
Microsegmentation with ACI Distributed virtual switch that is integrated with the ACI architecture as
Provides the ability to automatically assign endpoints Compute Leafs a virtual leaf and managed by the APIC. It offers different forwarding
to logical security zones called EPGs based on various Connect to compute systems, supports individual port, port and encapsulation options and extends across many virtualized hosts
network-based or VM-based attributes. channel, and virtual port channel (vPC) interfaces. and data centers defined by the VMware vCenter server.

Quota Management IP Storage Leafs Configuration Zones

Enables an admin to limit what managed objects can Connect to IP storage systems, supports individual port, port Leaf Leaf Leaf Leaf Configuration zones divide the ACI fabric into different zones that can be
be added under a given tenant or globally across channel, and virtual port channel (vPC) interfaces. updated with configuration changes at different times.
tenants. vzAny GOLF
The vzAny managed object provides a convenient Also known as Layer 3 EVPN Services for Fabric WAN. It enables much
App Center way of associating all EPGs in a VRF instance to 1 or
Host custom applications running on the controller. more efficient and scalable ACI fabric WAN connectivity. It uses the BGP-
more contracts, instead of creating a separate L2 L3
APIC EVPN protocol over OSPF for WAN routers that are connected to spine
contract relation for each EPG. switches.
Cisco ACI Optimizer Application Policy Infrastructure
L3 Out
APIC tool that enables you to determine how many Policy L2 Out Controller (APIC)
A routed L3 connection.
leaf switches you will need for your network and Named entity that contains specifications for An L2 Out is a bridged Replicated synchronized clustered
ACI routed connections
suggests how to deploy each application and external controlling some aspect of system behavior. For (L2) connection controller. Provides a unified point
perform IP forwarding
EPG on each leaf switch without violating any example, a L3 Outside Network Policy would between an ACI fabric of automation and management,
according to the protocol
constraints. contain the BGP protocol. and an outside L2 policy programming, application
selected, such as BGP,
network (a switch) deployment, and health
OSPF, or EIGRP.
monitoring for the ACI fabric.

REST API Using the REST API Management Information Tree (MIT)
The APIC REST API is a programmatic A hierarchical tree containing all the
http(s):// host:port /api /{mo|class} /{dn|classname} .{xml|json} ?[options]
interface that uses REST managed objects (MOs) of a system.
architecture. The API accepts and Specify Specify filters, The ACI MIT is also called the
http or APIC API
returns HTTP or HTTPS messages Managed Distinguished name Encoding for selectors or modifiers Management Information Model (MIM)
https host and Oper
that contain JSON or XML Object or Class or Object Class response to query, joined using
protocol port ator
documents. The REST API is the Operator ampersnad (&) Tags
interface into the MIT and allows Schema Object tags simplify API operations. In
manipulation of the object model API Inspector an API operation, an objector group of
With Multi-Site, the Schema is a container for single
state. The same REST interface is Provides a real-time display of the REST API commands that the objects is referenced by the tag name
or multiple templates used for defining policies.
used by the Cisco APIC CLI, GUI, and APIC processes to perform GUI interactions. instead of by the distinguished
SDK. The REST API also provides an Managed Object (MO) name(DN). Tags are child objects of the
interface through which other Distinguished Name (DN) item they tag; besides the name, they
An abstract representation of network resources that
information can be retrieved. A unique name that describes a MO and its place in the MIT have no other properties.
are managed.

Application Centric Infrastructure (ACI) Concepts

 Extending a L2 Domain outside the fabric L3 Routes
Extend an EPG to legacy switches Basic Connectivity Type pf Fabric Routes – Internal Routes
L2OUT Create the L3Out
EPG is extended to external devices using static-path bindings to ports (with desired encap/vlan). • Associate VRF and L3 Domain
• Create Logical Node Profile and
The leaf will learn the endpoint information and assign the traffic (by matching the port and VLAN associate fabric nodes to the L3Out.
ID) to the proper EPG, and then enforce the policy. • Create Logical Interface Profile
• Specify Path attributes containing
The endpoint learning, data forwarding, and policy enforcement remain the same whether the physical interface, encapsulation, and IPs
endpoint is directly attached to the leaf port or if it is behind a L2 network (if the VLAN is enabled
in the L2 network).
Spanning Tree
Classical behavior
There are three requirements to advertise Internal Routes out an L3Out:
- STP BPDUs (PVST or MST) are generated by each switch in the STP
1. The BD must be associated with the L3Out*
topology.
- The association adds prefix entry to route map controlling advertised routes
- STP root is elected and interface forwarding is calculated to prevent loops
2. A contract must exists between an EPG within the BD and an external EPG on the L3Out.
by blocking some interfaces.
- The contract creates internal BD route on border leaf (cannot advertise route until it
- All interfaces with best-path (highest bandwidth) towards root bridge will Type pf Fabric Routes
exists locally)
be forwarding.
3. The subnet must have a public scope (Advertise Externally)
- Backup paths will be put in a blocking state by the switch.
- Topology changes (TC) trigger MAC addresses to be flushed in received Type pf Fabric Routes – External Routes
VLAN, allowing traffic reconvergence based on new topology.
ACI floods BPDUs in the fabric encap
- ACI leaves don’t participate in spanning tree, so they don’t generate BPDUs
or block any ports.
- STP BPDUs (PVST or MST) are flooded within the fabric/EPG encap
(allocated per vlan encap in a domain)
- Leaves flush endpoints in the EPG if a TC BPDU is received Spanning Tree
Domain policy determines which EPGs to flush for MST domain TCs
Internal Routes
MST BPDUs are untagged & require an untagged/native EPG to be deployed Subnets defined under BD are internal routes and create static pervasive routes within the
on all interfaces connected to MST domain (includes L3outs using SVIs) fabric.
ACI MST Configuration
External Routes External Routes from ospf, eigrp, or static are redistributed on the border leaf into the
- Configuration is fabric-wide and supports multiple regions for use within different tenants/
Routes learned via a routing protocol or static routes configured under an L3Out. These local bgp process.
domains.
- Any ports connecting to MST switches within the same region MUST have untagged static- path. routes are redistributed into MP-BGP and advertise to all leafs that contain the VRF
The bgp route is exported into MP-BGP with a route-target (RT) of the corresponding VRF.
- Each MST region should have it’s own EPG for BPDU flooding
Transit Routes Each leaf in the fabric with the VRF present will import the RT and install the route.
Routes advertised between L3Outs. External routes on the non-originating border leaf will be seen as bgp learned routes.
Fabric -> Access Policies -> Switch Policies -> Spanning Tree -> default
- Add a Region Policy
External Routes are controlled via Import Route Control flag
- Add a Domain Policy for each MST instance within the region (instance 0 is implicit)
- Add vlan blocks
Type pf Fabric Routes – Transit Routes
Common Mistakes that cause loops
Missing untagged/native EPG in MST region
MST BPDUs are sent untagged by switches and will only be accepted by leaf if an EPG is deployed
with an untagged/native EPG path binding. All interfaces connected to a common MST region
should have the same EPG deployed (this is to ensure BPDU is flooded to all of the MST switches
connected to fabric).

Multiple fabric encaps used for same EPG

BPDUs are flooded within the fabric encap of an EPG (allocated based on domain/vlan pool).
In order for BPDUs to be flooded properly, all interfaces within the EPG that are connected to
external bridges MUST reside in the same physical or L2 external domain and vlan encapsulation.

STP Link Type Must Be Shared In this example, external route ext-S1 is a Transit Route when advertised out L3Out-2.
Since BPDU’s are flooded, ACI acts as a HUB from an STP Perspective. Full Duplex Links default to If OSPF or EIGRP on L3Out-2, ext-S1 is redistributed from BGP into the IGP and advertised.
Spanning-Tree Link-Type PTP. If multiple switches connect to ACI on separate links, Link-Type Transit Routes are controlled via Export Route Control flag
must be set to Shared to allow processing of multiple BPDU’s on the same interface.

Mis-Cabling Protocol
Mis-Cabling Protocol can be used to detect loops. With MCP, a special frame is sent out with a
multicast destination MAC so that the downstream devices will flood it. MCP Can be sent on a per
VLAN basis. If that frame is received back on a leaf in the fabric, it will err-disable the interface if
ONE of the following conditions are met:
1. MD5 Digest is the same
2. Send time is within ~2s of receive time

Application Centric Infrastructure (ACI) Concepts

 Fabric (Access Policies)
Switch Policies Fabric (Access Policies)
Switch Policy Group Switch Profile + Switch Selector Interface Profile + Interface Selector
Switch Profile

Switch Selector Policies Switch Selector Interface Selector Domain + VLAN Pool
Attached
Switch STP (switch) VPC DOMAIN Switch Policy Group Interface Policy Group Access Entity Physical Domain VLAN Pool
Profile

Application Profile
Interface Policies
EPG
Interface Policy Group
Deploy on Bridge Domain
Tenant
static port
Policies IP Subnet
Interface Profile

1/1
Interface Selector CDP LLDP Port-Channel

Interface
STP (int) MCP Link-Level

Global Policies

Access Entity Profiles Fabric (Access Policies)

Switch Profile + Switch Selector
Switch Selector
Physical & External Domains Switch Policy Group

Switch Profile
Physical Domains External L2 Domains External L3 Domains Interface Profile + Interface Selector
Switch Selector
Interface Selector
Interface Profile Interface Policy Group

Pools Access Port

Selector
Attached
VLANs
Access Port Access Entity Profile
Policy Group
Attachable
Domain + VLAN Pool
Access Entity Policy VLAN
Tenant (AEP) Physical Domain
Pool
Application Profile The vlan
containers
Domain Domain
Application Profile
Endpoint Group
VLAN VLAN EPG
Pool Pool
Bridge Domain Tenant
IP Subnet
Networking Security Policy

Deploy on
Contract
Domain

static port
Subject
Subnet
Bridge

Filter
VRF

1/1

Application Centric Infrastructure (ACI) Concepts

 RELATION BETWEEN TENANTS AND ACCESS POLICIES || How both worlds tie together
Switch Profile
Switch Selector - An AEP provisions the VLAN pool on the leaf.
Spine-101 Spine-102 - The VLANs are not actually enabled on the port.
Interface Profile - No traffic flows unless an EPG is deployed on the port.
Access Port Selector - Without VLAN pool deployment using an AEP, a VLAN
is not enabled on the leaf port even if an EPG is
Interface Policy Group
provisioned.
Access Port Policy Group

Physical Domain Profile

VLAN Pool AEP
Cisco Nexus 9396PX

Leaf-1001 Leaf-1002 Leaf-1003 Leaf-1004

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

BCN 1

STS

ACT 2

The domain
Tenant Customer /
Tenant Tenant Customer / profile contains
(use r, common or infra) Group / BU The AEP deploys the
(user, common or infra) (user, common or infra) Group / BU VLAN pool &
AEP which are Private Network associated EPGs to all
Routing Table
Private Network Private Network Routing Table associated VRF Instance the ports to which it
Switch Profile VRF
VRF Instance VRF Instance VRF directly with is attached, and
Switch Selector automates the task of
EPGs L2 Boundary
L2 Boundary Bridge Domain assigning VLANs
Interface Profile Bridge Domain Bridge Domain
Access Port Selector Subnet IP Space(s)
Subnet Subnet IP Space(s)
Interface Policy Group
Application Profile
Access Port Policy Group Application Profile Application Profile Group of
Group of EPG End Points
Physical Domain Profile EPG EPG EPG End Points
VLAN Pool AEP
Cisco Nexus 9396PX

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

BCN 1

STS

ACT 2

Tenants & Access Policies

Fabric – Access Policy Fabric Side Configuration:
Interface Policy Step 1: Create a static VLAN Pool (what VLAN id you will be using on your interface)
AAEP Interface Switch Step 2: Create Physical Domain and bind VLAN Pool to the domain
Group
(Access, PC or VPC) Profile Profile Step 3: Create AAEP and bind domain to the AAEP
Domain Step 4: Create Interface Policies (generic configs that you would want on your interface)
Interface Policy 1 Interface Leaf
Step 5: Create Interface Policy Group (interface policies need to be activated on the interface)
VLAN Pool Selector Selector
Interface Policy 2 → This is the place where you also attach the AAEP.
Step 6: Create Leaf Interface Profiles (select the interfaces and attach interface policy group)
Interface Policy n Physical Port / Leaf Step 7: Create Switch Profile (Everything done till step 6 is just abstract)
→ When you bind leaf interface profile to the switch profile the config makes sense)

Static Port Binding Bridge Domain Tenant Side Configuration:

(VLAN ENCAP) VRF
Step1: Create a VRF
Subnet Step2: Create a Bridge Domain (Associate your BD with the VRF and create the subnet
EPG
Provided
Step 3: Create Application Profile (AP is nothing but a container for the EPGs)
Contract Step 4: Create an Application EPG (You will have to associate BD while creating EPG)
Consumed Filter
Subject Step 5: Attach Physical Domain to the EPG (attach your Physical domain option).
Application Profile Step 6: Static port binding in EPG (static path of the Leaf/Port configured for the server)
Tenant → Also specify the VLAN encap ID from the vlan pool created
Step 7: Attach contract to the EPG

Application Centric Infrastructure (ACI) Concepts

 ACI Policy Enforcement ACI Preferred Group ACI Contracts and Resource Utilization
Allow any any for a subset of EPGs Contract created between E2 and E3
Leaf derives source EPG pcTag based on: EPGs that are part of the preferred group do not require contracts to BD-B1 and BD-B2 each have a subnet defined. Subnet int-S1 on BD-B1
- Match in EP database communicate with each other exists on L1 and L3, while subnet int-S2 for BD-B2 exists on L6
src MAC for L2 traffic or src IP for L3 traffic
- Longest-prefix match against src IP EPGs and External EPGs can be configured to included or excluded When creating the contract between E2 and E3:
(IP-based EPG or L3Out external EPG) from the preferred group. - Program contract rule between 2 E2 and E3 in TCAM. Add Static
- Ingress port + encap - EPGs which are excluded, have hardware rules route for int-S1 created on L6 pointing to spine proxy.
programmed to prevent communication to EPGs - Program contract rule between 2 E2 and E3 in TCAM. Add Static
Leaf derives destination EPG pcTag based on: which are included route for int-S2 created on L3 pointing to spine proxy.
- Match in EP database
dst MAC for L2 traffic or dst IP for L3 traffic Only recommended if the majority of EPGs require unenforced policy Contracts are only programmed on leafs that have provider/consumer
Policy is created based on contract - Longest-prefix match against dst IP EPGs. BD routes are only programmed on leafs that need them!
between EPGs with support for L2/L3/ (L3Out external EPG or shared-services) Deny rules are installed for EPGs outside of the preferred groups
L4 filters similar to traditional ACLs.
Rules are programmed with scope of VRF. Contracts can still be used to enable communication between
Allow traffic between all EPGs Policy lookup is always: excluded and included EPGs
without a contract by setting the VRF (VRF, src- - EPG, dst- - EPG,filter).
to unenforced mode

Contract Scope
The contract scope will limit which providers and consumers
can participate within the same contract.

VRF
Contract can be applied between EPGs within the same VRF.

Application Profile
Contract can be applied between EPGs within the same AP.

Tenant
Contract can be applied between EPGs within the same tenant.
Contracts contribute to both
Global y policy AND routing entries
Contract can be applied between any EPGs within the fabric. on leafs!
Global contracts not in common tenant need to be exported in
order to be consumed by EPG in a different tenant.
Consumers of global contracts will use the ‘Consumer Contract
Interface’ Option.

Application Centric Infrastructure (ACI) Concepts

 Shared Services and Route Leaking
What is a shared service? What happens in the fabric? Shared Service Forwarding
3) Spine performs proxy lookup for H3
IP in VRF-V2. Normal Proxy behavior From Provider E1 to
2) L1 performs layer3 lookup for H3 to forward packet to VTEP of L6 Consumer E4
in VRF-V1 and hits LPM entry for H3
subnet. LPM entry points to proxy S1 S2 Policy Applied
with VNID e rewrite info for VRF-V2.
on egress L6
Packet is sent to Spine Anycast IPv4 3 (consumer VRF)
Proxy VTEP with VRF-V2 VNID EPG- 4) L6 performs layer3 lookup
E1 set in VXLAN header. No policy 2 on H3 destination IP in VRF-
applied in provider VRF V2. Hit in local EP database
4
and derives destination EPG-
L1 L2 L3 L4 L5 L6 E4 L6 applies policy between
EPG-E1 and EPG-E6

1) H1 sends packet
toward gateway in 1 5 5) If permitted, traffic
H1 H3 forwarded to H3 with
EPG-E1 with
BD-B1 BD-B2 appropriate encap
destination IP of H3
EPG-E1 EPG-E4
VRF-V1 VRF-V2
Shared Service ( Route Leaking) enables traffic between endpoints in EPG-E1 is now a shared service provider. Shared Service Forwarding
different VRFs. A shared service EPG provider is an EPG that provides a It is reallocated a fabric unique pcTag (<16384).
contract consumed by an EPG in a different VRF. 3) Spine performs proxy lookup for H1
From Consumer E4 IP in VRF-V1. s If unknown drops the 2) L6 performs layer3
All subnets on consumer BD programmed in provider VRF. to Provider E1 lookup for H1 in VRF-V2 and
Restrictions packet. Else forward to VTEP of L1
- Provider Subnet must be defined under the provider EPG. hits LPM entry for H1
Provider subnet programmed in consumer VRF with pcTag of subnet. LPM entry points to
- Both provider and consumer subnets must have scope set to shared provider EPG. S1 S2 Policy Applied
- Contract needs correct scope proxy with VNID rewrite
on egress L6 info for VRF-V1 and pcTag
- VzAny not supported as provider 3 (consumer VRF)
4) L1 performs layer3 lookup of EPG-E1. L6 applies policy
on H1 destination IP in VRF- between EPG-E4 and EPG-
V1. Hit in local EP database 2 E1 in consumer VRF-V2. If
4 permitted, packet is sent to
and derives destination EPG-
E1 Policy already applied by L1 L2 L3 L4 L5 L6 Spine Anycast IPv4 Proxy
L6 VTEP with VRF-V1 VNID and
EPG-E4 set in VXLAN
5) Traffic is forwarded 5 1 1) H3 sends packet
to H1 with H1 H3 toward gateway in
appropriate encap BD-B1 BD-B2 EPG-E4 with
destination IP of H1
EPG-E1 EPG-E4
VRF-V1 VRF-V2

Application Centric Infrastructure (ACI) Concepts

 ACI ARCHITECTURE
Single APIC Cluster/Single Domain Multiple APIC Clusters/Multiple Domains
Stretched ACI
Stretched Fabric Partially meshed design that Dual-Fabric Connected (L2 and L3 Extension)
ACI Fabric connects Cisco ACI leaf and spine
switches distributed in multiple ACI Fabric 1 ACI Fabric 2
Pod 1 APIC Pod 2 locations. The stretched fabric is a
Cluster single ACI fabric. The sites are one
administration domain and one
availability zone. Administrators
are able to manage the sites as
one entity; configuration changes
made on any Cisco APIC controller
node are applied to devices across
the sites.
L2/L3
Multi-Pod (from 2.0 release)
Multi-Site
Multipod
IP Network Enables provisioning a more fault-
Pod ‘A’ Pod ‘n’ tolerant fabric comprised of Site ‘A’ Site ‘n’
multiple pods with isolated control
plane protocols. Also, multipod Template
provides more flexibility with In an ACI Multi-Site
MP-BGP-EVPN MP-BGP-EVPN configuration, templates are
regard to the full mesh cabling
... between leaf and spine switches. ... framework to hold policies and
Multipod uses MP-BGP EVPN as configuration objects that are
the control-plane communication pushed to the different sites.
protocol between the ACI spine These templates reside within
APIC switches in different pods. schemas that are defined for
each site.
Cluster

Multi-Site
Controller

Pod:
A pod is a leaf-and-spine network sharing a common control plane (Intermediate System–to–
Intermediate System [ISIS], Border Gateway Protocol [BGP], Council of Oracle Protocol [COOP], etc.). A
pod can be considered a single network fault domain.

Fabric:
A fabric is the set of leaf and spines nodes under the control of the same APIC domain. Each fabric
represents a separate tenant change domain, because every configuration and policy change applied in the
APIC is applied across the fabric. A Cisco ACI fabric thus can be considered an availability zone.

Multi-Pod:
A Multi-Pod design consists of a single APIC domain with multiple leaf-and-spine networks
(pods) interconnected. As a consequence, a Multi-Pod design is functionally a fabric (a single availability
zone), but it does not represent a single network failure domain, because each pod runs a separate
instance of control-plane protocols.

Multi-Site:
A Multi-Site design is the architecture interconnecting multiple APIC cluster domains with their associated
pods. A Multi-Site design could also be called a Multi-Fabric design, because it interconnects separate
availability zones (fabrics), each deployed either as a single pod or multiple pods (a Multi-Pod design).

Application Centric Infrastructure (ACI) Concepts

 Summary
Ethernet Frame
Ethernet
MAC Header PAYLOAD FCS
Ethernet Frame
Tagged
Ethernet MAC Header 802.1Q PAYLOAD FCS
Ethernet Frame containing IP packet
IP
MAC Header 802.1Q IPv4 Header PAYLOAD FCS
Ethernet Frame containing TCP packet
TCP
MAC Header 802.1Q IPv4 Header TCP Heade r PAYLOAD FCS
Ethernet Frame containing UDP packet
UDP
MAC Header 802.1Q IPv4 Header UDP Header PAYLOAD FCS
OUTER INNER
VXLAN MAC Header 802.1Q IPv4 Header UDP Header VXLAN Header MAC Header IPv4 Header UDP Header PAYLOAD FCS

MAC Header
Ethernet Frame
MAC Header PAYLOAD FCS

Destination MAC Address (DMAC)

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

... Source MAC Address (SMAC)

32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

...
64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

EtherType
96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111

MAC w/802.1Q Header

Ethernet Frame
MAC Header 802.1Q PAYLOAD FCS

Destination MAC Address (DMAC)

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

... Source MAC Address (SMAC)

32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

...
64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

Tag Protocol Identifier (0x8100) PCP / COS DEI VLAN Identifier

96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127

EtherType
128 129 130 131 132 133 134 135

Application Centric Infrastructure (ACI) Concepts

 IPv4 Header
Ethernet Frame containing IP packet
MAC Header 802.1Q IPv4 Header PAYLOAD FCS

Version Header Length DSCP ECN Total Length

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

D M
Identification R Fragment Offset
F F
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

Time To Live (TTL) Protocol Header Checksum

64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

Source IP Address
96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127

Destination IP Address
128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159

TCP Header
Ethernet Frame containing TCP packet
MAC Header 802.1Q IPv4 Header TCP Header PAYLOAD FCS

Source Port Destination Port

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Sequence Number
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

Acknowledgement Number
64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

Flags/Control Bits
Header Length Reserved Window Size
N C E U A P R S F
96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127

Checksum Urgent Pointer

128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159

UDP Header
Ethernet Frame containing UDP packet
MAC Header 802.1Q IPv4 Header UDP Header PAYLOAD FCS

Source Port Destination Port

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Length Checksum
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

VXLAN Header
OUTER INNER
MAC Header 802.1Q IPv4 Header UDP Header VXLAN Header MAC Header IPv4 Header UDP Header PAYLOAD FCS

Flags
SP DP Source Group
I
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Virtual Network Identifier (VNID) Reserved

32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

Application Centric Infrastructure (ACI) Concepts

 ACI Virtualization
Supported Vendors EPG Policy Resolution and Deployment Immediacy Guidelines for Deleting VMM Domains
ACI supports virtual machine managers (VMMs): Whenever an EPG associates to a VMM domain, the administrator can choose the Follow the sequence below to assure that the APIC request to delete a VMM domain
- Cisco Application Centric Infrastructure Virtual Edge resolution and deployment preferences to specify when a policy should be pushed automatically triggers the associated VM controller (for example VMware vCenter or
- Cisco Application Virtual Switch (AVS) into leaf switches. Microsoft SCVMM) to complete the process normally, and that no orphan EPGs are
- Cloud Foundry stranded in the ACI fabric.
- Kubernetes Resolution Immediacy
- Microsoft System Center Virtual Machine Manager (SCVMM) Pre-provision 1. The VM administrator must detach all the VMs from the port groups (in the case of
- OpenShift Specifies that a policy (for example, VLAN, VXLAN binding, contracts, or filters) is VMware vCenter) or VM networks (in the case of SCVMM), created by the APIC. In
- OpenStack downloaded to a leaf switch even before a VM controller is attached to the virtual the case of Cisco AVS, the VM admin also needs to delete vmk interfaces associated
- Red Hat Virtualization (RHV) switch. This pre-provisions the configuration on the switch. with the Cisco AVS.
- VMware Virtual Distributed Switch (VDS)
VMM Domain VLAN Pool Association When using pre-provision immediacy, policy is downloaded to ACI leaf switch 2. The ACI administrator deletes the VMM domain in the APIC. The APIC triggers
A VMM domain can associate with only one dynamic VLAN pool. By default, the assignment of VLAN identifiers to regardless of CDP/LLDP neighborship. Even without a hypervisor host connected deletion of VMware VDS or Cisco AVS or SCVMM logical switch and associated
EPGs that are associated with VMM domains is done dynamically by the APIC. While dynamic allocation is the to the VMM switch. objects.
default and preferred configuration, an administrator can statically assign a VLAN identifier to an EPG instead. In
that case, the identifiers used must be selected from encapsulation blocks in the VLAN pool associated with the Immediate Note: The VM administrator should not delete the virtual switch or associated objects
VMM domain, and their allocation type must be changed to static. Specifies that EPG policies (including contracts and filters) are downloaded to the (such as port groups or VM networks); allow the APIC to trigger the virtual switch
associated leaf switch software upon ESXi host attachment to a DVS. LLDP or deletion upon completion of step 2 above. EPGs could be orphaned in the APIC if the
The APIC provisions VMM domain VLAN on leaf ports based on EPG events, either statically binding on leaf ports OpFlex permissions are used to resolve the VM controller to leaf node VM administrator deletes the virtual switch from the VM controller before the VMM
or based on VM events from controllers such as VMware vCenter or Microsoft SCVMM. attachments. domain is deleted in the APIC.

VMM Domain EPG Association The policy will be downloaded to leaf when you add host to the VMM switch.
The ACI fabric associates tenant application profile EPGs to VMM domains, either automatically by an CDP/LLDP neighborship from host to leaf is required.
orchestration component such as Microsoft Azure, or by an APIC administrator creating such configurations. An
EPG can span multiple VMM domains and a VMM domain can contain multiple EPGs. On Demand
Specifies that a policy (for example, VLAN, VXLAN bindings, contracts, or filters) is
pushed to the leaf node only when an ESXi host is attached to a DVS and a VM is
placed in the port group (EPG).

The policy will be downloaded to leaf when host is added to VMM switch and
virtual machine needs to be placed into port group (EPG). CDP/LLDP neighborship
from host to leaf is required.

With both immediate and on demand, if host and leaf lose LLDP/CDP neighborship
the policies are removed.

Deployment Immediacy
Once the policies are downloaded to the leaf software, deployment immediacy
can specify when the policy is pushed into the hardware policy content-
addressable memory (CAM).

Immediate
Specifies that the policy is programmed in the hardware policy CAM as soon as the
policy is downloaded in the leaf software.

On demand
Specifies that the policy is programmed in the hardware policy CAM only when
the first packet is received through the data path. This process helps to optimize
End points (EP) of the same color are part of the same EPG. All the green EPs
the hardware space.
are in the same EPG even though they are in two different VMM domains.

Application Centric Infrastructure (ACI) Concepts

 ACI Virtualization
1. Configuring Virtual Machine Networking Policies 3. Creating VDS Uplink Port Groups
Creating a vCenter Domain Profile Using the GUI
The APIC integrates with third-party VM manager (VMM) to extend the benefits of ACI to the virtualized Each VMM domain appears in the vCenter as a vSphere Distributed Switch (VDS). The
1. In the Navigation pane, right-click Switch Policies, and then click Configured Interfaces, PC, and VPC.
infrastructure. The APIC enables the ACI policies inside the VMM system to be used by its administrator. virtualization administrator associates hosts to the VDS created by the APIC and selects
2. In the Configured Interfaces, PC, and VPC dialog box, perform the following actions:
The following modes of Cisco ACI and VMware VMM integration are supported: which vmnics to use for the specific VDS. The configuration of the VDS uplinks are
performed from the APIC controller by changing the vSwitch configuration from the Attach
- Expand Configured Switch Interfaces.
Vmware VDS Entity Profile (AEP) that is associated with the VMM domain. You can find the AEP in the
- Click the + icon.
When integrated with Cisco ACI, the VMware vSphere Distributed Switch (VDS) enables you to configure VM APIC GUI in the Fabric Access Policies configuration area.
- Make sure that the Quick radio button is chosen.
networking in the ACI fabric.
- From the Switches drop-down list, choose the appropriate leaf ID. 4. Creating a Trunk Port Group
- Click the + icon to configure the switch interfaces. Trunk port group must be tenant independent.
Cisco Application Virtual Switch (AVS)
- In the Interface Type area, check the appropriate radio button.
For infoout how to install and configure the Cisco AVS with the Cisco ACI, see details in Cisco ACI with Cisco AVS.
- In the Interfaces field, enter the desired interface range. 1. Log in to the APIC GUI.
Mapping ACI and VMware Constructs - In the Interface Selector Name field, the selector name automatically populates. 2. On the menu bar, choose VM NETWORKING.
Cisco APIC Terms Cisco APIC Terms - In the Interface Policy Group area, choose the Create One radio button. 3. Cchoose VMware > Domain_name > Trunk Port Groups and right-click Create Trunk Port
VM controller vCenter (Datacenter) - From the Link Level Policy drop-down list, choose the desired link level policy. Group.
Virtual Machine Manager (VMM) Domain vSphere Distributed Switch (VDS) - From the CDP Policy drop-down list, choose the desired CDP policy. 4. In the Create Trunk Port Group dialog box, perform the following actions:
Endpoint group (EPG) Port group - In the Attached Device Type area, choose ESX Hosts. - In the Name field, enter the EPG name.
VMware VDS Parameters Managed By APIC - In the Domain area, make sure that the Create One radio button is chosen. - For the Promiscuous Mode buttons, click either Disabled or Enabled. The default is
Vmware VDS Default Value Configurable using APIC Policy - In the Domain Name field, enter the domain name. Disabled.
Name VMM domain Name Yes (derived from Domain) - In the VLAN area, make sure that the Create One radio button is chosen. - For the Trunk Portgroup Immediacy buttons, click either Immediate or On Demand. The
Description “APIC Virtual Switch” No - In the VLAN Range field, enter the VLAN range as appropriate. default is On Demand.
Folder Name VMM domain Name Yes (derived from Domain) - Use a range of at least 200 VLAN numbers. Don’t define a range that includes your manually - For the MAC changes buttons, click either Disabled or Enabled. The default is Enabled.
Version Highest supported by vCenter Yes assigned infra VLAN. - For the Forged transmits buttons, click either Disabled or Enabled. The default is
Discovery Protocol LLDP Yes Enabled.
Uplink Ports and Uplink Names 8 No - In the vCenter Login Name field, enter the login name. - In the VLAN Ranges field, choose the + icon and enter the VLAN range (vlan-100 vlan-
Uplink Name Prefix Uplink No - (Optional) From the Security Domains drop-down list, choose the appropriate security domain. 200).
Maximum MTU 9000 Yes - In the Password field, enter a password. Note : If you do not specify a VLAN Range, the VLAN list will be taken from the domain’s
LACP policy Disabled Yes - In the Confirm Password field, reenter the password. VLAN namespace.
Port mirroring 0 sessions Yes - Expand vCenter. - Click Update.
Alarms 2 alarms added at the folder level No 5. Click Submit.
VDS Port Group Parameters Managed by APIC 3. In the Create vCenter Controller dialog box, enter the appropriate information, and click OK.
VMware VDS Port Group Default Value Configurable using APIC Policy 4. In the Configure Interface, PC, And VPC dialog box, complete the following actions:
Name Tenant Name | AP Name | EPG Name Yes (derived from EPG)
Port Binding Static Binding No If you do not specify policies in the Port Channel Mode and the vSwitch Policy areas, the same policies
VLAN Picked from VLAN Pool Yes that you configured earlier in this procedure will take effect for the vSwitch.
Load Balancing algorithm Derived based on port-channel policy on APIC Yes
Promiscuous Mode Disabled Yes - From the Port Channel Mode drop-down list, choose a mode.
Forged transmit Disabled Yes - In the vSwitch Policy area, click the desired radio button to enable CDP or LLDP.
Mac Change Disabled Yes - From the NetFlow Exporter Policy drop-down list, choose a policy or create one.
Block All Ports False No - Choose values from the Active Flow Time0ut, Idle Flow Timeout, and Sampling Rate drop-down lists.
2. Creating a VMM Domain Profile - Click SAVE twice and then click SUBMIT.
vCenter Domain Operational Workflow
APIC admin configures vCenter domain policies in the APIC and provides the following vCenter connectivity info: 5. Verify the new domain and profiles, by performing the following actions:

- The vCenter IP address and credentials, VMM domain policies, and VMM domain SPAN - On the menu bar, choose VM Networking > Inventory.
- Policies (VLAN pools, domain type such as VMware VDS, ...) - In the Navigation pane, expand VMware > Domain_name > vCenter_name.
- Connectivity to physical leaf inerfaces (using AEPs)
In the Work pane, under Properties, view the VMM domain name to verify that the controller is online.
1. APIC automatically connects to vCenter. In the Work pane, the vCenter properties are displayed including the operational status. The displayed
2. APIC creates the VDS matching the name of the VMM domain. information confirms that connection from the APIC controller to the vCenter server is established, and
3. vCenter admin adds the ESX host to the APIC VDS and assigns the ESX host hypervisor ports as uplinks on the the inventory is available.
APIC VDS. These uplinks must connect to the ACI leaf switches.
4. APIC learns the location of the hypervisor host using LLDP or CDP info of the hypervisors. Configuring Endpoint Retention Using the GUI
5. APIC admin creates and associates EPG policies. 1. Log in to Cisco APIC.
6. APIC admin associates EPG policies to VMM domains. 2. Choose VM Networking > Inventory. A Sequential Illustration of the vCenter Domain Operational Workflow
7. APIC automatically creates port groups in the VMware vCenter under the VDS. This process provisions the network 3. In the left navigation pane, expand the VMware folder and then click the vCenter domain that you
policy in the VMware vCenter. The port group name is a concatenation of the tenant name, the AP name, and the created earlier.
EPG name. The port group is created under the VDS, and it was created earlier by the APIC. 4. In the central Domain work pane, make sure that the Policy and General tabs are selected.
8. The vCenter admin or compute management tool instantiates and assigns VMs to the port groups. 5. In the End Point Retention Time (seconds) counter, choose the number of seconds to retain
9. The APIC learns about the VM placements based on the vCenter events. The APIC automatically pushes the EPG endpoints before they are detached. You can choose between 0 and 600 seconds. The default is 0.
and its associated policy to the ACI fabric. 6. Click Submit.

Application Centric Infrastructure (ACI) Concepts

 Contract Subject Filter
DB-permit-all-ctr DB-permit-all-sbj DB-permit-all-flt

DB-permit-all-ctr
External EPG
L3out-ext-epg
0.0.0.0/0

L3out
DB-legacy-SB-ospf-L3out

Application Network Profile

DB-EXADATA-ap

Bridge Domain EPG

VRF DB-EXADATA-client-bd DB-EXADATA-client-epg
DB-vrf 10.168.8.1/24 VLAN 233

Bridge Domain EPG

DB-EXADATA-backup-bd DB-EXADATA-backup-epg
10.174.2.1/24 VLAN 234

Tenants - DB
Fabric - Access Policies
Domain
BM-Physdom

VLAN Pool AAEP

BM-VL-St-Pool BM-AAEP

Interface Policy Group Interface Policy Group Interface Policy Group

dm03db01-client-vpc-IntPolGrp dm03db02-client-vpc-IntPolGrp BM-AccessPort-IntPolGrp
Interface Policy Interface Policy Interface Policy
Disable-CDP-IntPol Disable-CDP-IntPol Disable-CDP-IntPol
Interface Selector Interface Selector Interface Selector Interface Selector
dm03db01-IntSelect dm03db02-IntSelect dm03db01-dr-IntSelect dm03db02-dr-IntSelect
(1/9) (1/10) (1/11) (1/11)
Interface Policy Interface Policy Interface Policy
enable-MCP enable-MCP enable-MCP
Interface Profile Interface Profile Interface Profile Interface Profile
Leaf1101-1102-IntProf Leaf1101-1102-IntProf Leaf1101-IntProf Leaf1102-IntProf
Interface Policy Interface Policy Interface Policy
Disable-LLDP-IntPol Disable-LLDP-IntPol Disable-LLDP-IntPol
Switch Profile Switch Profile Switch Profile Switch Profile
Leaf1101-1102-LeafProf Leaf1101-1102-LeafProf Leaf1101-LeafProf Leaf1102-LeafProf
Interface Policy Interface Policy
PC-LACP-Active-IntPol PC-LACP-Active-IntPol
Leaf Leaf
Leaf Leaf
1101 1101
1101 1102
1102 1102

Application Centric Infrastructure (ACI) Concepts

 Switch Profile: SP_LEAF_101
Switch Selector: SS_LEAF_101

Interface Profile: INP_LEAF_101

1/1 1/2 1/3 1/4 1/5 1/6 1/7 1/8 1/9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 1/19 1/20 1/21 1/22 1/23 1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31 1/32 1/33 1/34 1/35 1/36 1/37 1/38 1/39 1/40 1/41 1/42

Port Channel Interface Policy Group Virtual Port Channel Interface Policy Group Access Port Policy Group
PhysicalDomain PhysicalDomain
Attachable Access Entity Profile The AEP defines the range of allowed VLANS but it does
VLAN Pool Access Entity Policy VLAN Pool Access Entity Policy not provision them. No traffic flows unless an EPG is
Domain deployed on the port. Without defining a VLAN pool in
an AEP, a VLAN is not enabled on the leaf port even if
VLAN Pool an EPG is provisioned.

Application Profile A particular VLAN is provisioned or enabled on the leaf

port that is based on EPG events either statically
Bridge Domain Subnet EPG 1 binding on a leaf port or based on VM events from
EPG is associated external controllers.
to a Domain
Tenant Bridge Domain Subnet EPG 2 Attached entity profiles can be associated directly with
VRF EPGs, which deploy the associated EPGs to all those
Bridge Domain Subnet EPG 3 ports associated with the AEP. The AEP has a
configurable generic function (infraGeneric), which
contains a relation to an EPG (infraRsFuncToEpg) that is
Bridge Domain Subnet EPG 3
EPG is deployed deployed on all interfaces that are part of the selectors
on a Static port that are associated with the attachable entity profile.

PHYSICAL SWITCH / PHYSICAL INTERFACES

1/1 1/2 1/3 1/4 1/5 1/6 1/7 1/8 1/9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 1/19 1/20 1/21 1/22 1/23 1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31 1/32 1/33 1/34 1/35 1/36 1/37 1/38 1/39 1/40 1/41 1/42

Application Centric Infrastructure (ACI) Concepts

Switch Profile
LeafProf_Leaf-1001

Switch Selector Switch Policy Group

Leaf-1001 default
Interface Selector: 1/01
Interface Selector: 1/02
Interface Selector: 1/03
Interface Selector: 1/04
Interface Selector: 1/05
Interface Selector: 1/06
Interface Selector: 1/07
Interface Selector: 1/08
Interface Selector: 1/09
Interface Selector: 1/10
Interface Selector: 1/11
Interface Selector: 1/12 Tenant
Interface Selector: 1/13 tn-office-bs
Interface Selector: 1/14
Interface Selector: 1/15 VFR
Interface Selector: 1/16 vrf-off-bs
Interface Selector: 1/17
Interface Selector: 1/18 Application Profile
Interface Selector: 1/19 ap-off-bs
Interface Selector: 1/20
Interface Selector: 1/21 Bridge Domain
Interface Selector: 1/22 bd-off-bs-prd
Interface Selector: 1/23
Interface Profile Interface Selector: 1/24 IP Subnet:
IntPro_Leaf-1001 Interface Selector: 1/25 10.168.1.0/24
Interface Selector: 1/26
Interface Selector: 1/27 End Point Group
Interface Selector: 1/28 epg-off-bs-prd
Interface Selector: 1/29
Interface Selector: 1/30
Interface Selector: 1/31 Physical Domain
Interface Selector: 1/32 Bridge Domain
DOM_Physical
Interface Selector: 1/33 bd-off-bs-acc
Interface Selector: 1/34 VLAN Pools:
Interface Selector: 1/35 IP Subnet:
VLANP_Physical_Servers_100_199
Interface Selector: 1/36 10.169.1.0/24
Interface Selector: 1/37
Interface Selector: 1/38 End Point Group
Interface Selector: 1/39 epg-off-bs-acc
Physical Domain
Interface Selector: 1/40 DOM_VMM
Interface Selector: 1/41 Interface Policy Group Access Entity Profile
Interface Selector: 1/42 IntPolGrp_AccessPorts AEP_MIVB VLAN Pools:
Interface Selector: 1/43 Bridge Domain
VLANP_VMM_200_299
Interface Selector: 1/44 bd-off-bs-dev
Interface Selector: 1/45
Interface Selector: 1/46 IP Subnet:
Interface Policy Group
Interface Selector: 1/47 10.170.1.0/24
IntPolGrp_VPC_2 Physical Domain
Interface Selector: 1/48 DOM_External
End Point Group
epg-off-bs-dev
Interface Policy Group VLAN Pools:
Switch Profile IntPolGrp_VPC_1 VLANP_External_300_399
LeafProf_Leaf-1001
Bridge Domain
Switch Selector Switch Policy Group bd-off-bs-tst
Leaf-1001 default
Interface Selector: 1/01 IP Subnet:
Interface Selector: 1/02 10.170.1.128/24
Interface Selector: 1/03
Interface Selector: 1/04 End Point Group
Interface Selector: 1/05 epg-off-bs-tst
Interface Profile
Interface Selector: 1/06
IntPro_Leaf-1001
Interface Selector: 1/07
Interface Selector: 1/08
Interface Selector: 1/09
Interface Selector: 1/10
Interface Selector: 1/11

Application Centric Infrastructure (ACI) Concepts