PART-01 (VXLAN)

VXLAN (Virtual Extensible LAN) was developed by Cisco, VMware, and other networking leaders in 2011
to address the limitations of traditional VLANs in modern data centers and cloud environments.
Traditional VLANs are limited by a 12-bit VLAN identifier, which supports only 4,096 unique VLANs, and
they lack the flexibility needed for large, multi-tenant data centers. VXLAN was introduced as a way to
address these issues by providing scalable Layer 2 segmentation over a Layer 3 network infrastructure.

What is VXLAN?

VXLAN (Virtual Extensible LAN) is a network virtualization technology that enables the extension of Layer
2 networks over a Layer 3 infrastructure. It encapsulates Ethernet frames inside UDP packets, allowing
Layer 2 traffic to traverse IP networks. VXLAN uses a 24-bit identifier called the VXLAN Network Identifier
(VNI), which can support up to 16 million unique segments.

In short VXLAN is a Layer 2 overlay technology that encapsulates Ethernet frames within UDP packets,
allowing Layer 2 traffic to traverse a Layer 3 IP underlay network. VXLAN is capable of performing both
bridging and routing

Why do data centers need VXLAN?

Modern data centers are growing rapidly, continually overcoming limitations faced in the past, such as
issues with STP, limited VLAN scalability, and large MAC tables. VXLAN plays an important role in
addressing these challenges.

Traditional Layer 2 Network Limitations

Traditional Layer 2 networks face several challenges that can restrict scalability and performance in data
centers:

1. Spanning Tree Protocol (STP) Limitations:

o Spanning Tree is used to prevent loops in Layer 2 networks by blocking redundant paths.
However, this wastes network bandwidth since blocked links go unused.

o To improve efficiency, Layer 3 networking is generally preferred in data centers for its
support of Equal-Cost Multi-Path (ECMP) routing, which enables the use of multiple paths
for load balancing. Some applications, however, still require Layer 2 connectivity, making
this a challenge.

2. VLAN Limitations:

o The VLAN ID is only 12 bits, allowing for a maximum of 4,094 usable VLANs, which limits
scalability. Large-scale data centers and multi-tenant environments need more
segmentation to accommodate numerous customers and internal divisions.
 o For instance, a service provider with 500 customers would be limited to just 8 VLANs per
customer if using traditional VLANs, which may not be sufficient.

3. Large MAC Address Tables:

o Server virtualization has led to an exponential increase in MAC addresses, as each virtual
machine (VM) or container has its own MAC address. This means each switchport
connected to a virtualized server may need to store multiple MAC addresses.

o Data center switches like ToR (Top of Rack) switches need large MAC address tables to
keep track of all VMs and containers on each physical server, especially in large,
virtualized environments.

How VXLAN Solves These Layer 2 Issues

VXLAN addresses these Layer 2 limitations by creating a virtual Layer 2 network over a Layer 3
IP infrastructure. Here’s how it solves each problem:

1. Avoiding Spanning Tree Issues:

o VXLAN operates over a Layer 3 network, which eliminates the need for Spanning Tree
Protocol to prevent loops. Instead, VXLAN uses IP routing and ECMP in the underlay
network, allowing multiple paths to be used simultaneously for load balancing and
redundancy.
o This provides a highly scalable network with efficient use of available bandwidth and
avoids the blocked paths caused by Spanning Tree.
2. Increased VLAN Scalability:
o VXLAN uses a 24-bit VXLAN Network Identifier (VNI), allowing for 16 million unique
identifiers (compared to the 4,094 possible VLANs in traditional networks). This scalability
is essential for large data centers and cloud providers supporting multiple tenants.
o Each VNI can represent a separate Layer 2 segment, providing flexibility and enabling fine-
grained network segmentation for each tenant or application.
3. Optimized MAC Address Table Requirements:
o In a VXLAN environment, switches do not need to keep track of all MAC addresses across
the entire data center. Instead, the VXLAN Tunnel Endpoints (VTEPs) maintain MAC-to-IP
mapping tables.
o This mapping is distributed across VTEPs, allowing for more efficient use of switch
resources and reducing the burden on MAC address tables.

Some related terminology to understand before configuring VXLAN.

VNI (VXLAN Network Identifier)

• A 24-bit identifier that uniquely identifies each VXLAN segment or virtual network, allowing up
to 16 million unique VXLAN segments.

• Each VNI represents a distinct Layer 2 domain (similar to a VLAN in traditional networking) but
with higher scalability.
VTEP (VXLAN Tunnel Endpoint)

The VXLAN tunnel endpoint (VTEP) is the device that’s responsible for encapsulating and de-
encapsulating layer 2 traffic. This device is the connection between the overlay and the underlay
network. The VTEP comes in two forms:

• Software (host-based)
• Hardware (gateway)

Software(host-based) based VTEP

• VXLAN functionality can be implemented in software within hypervisors (e.g., VMware NSX,
Open vSwitch).

• This allows virtual switches on servers to act as VTEPs, creating an overlay network directly on
compute infrastructure.

• Software-based VXLAN is commonly used in virtualized data centers and cloud environments.

Hardwar(gateway)e based VTEP

• Many modern network devices (switches, routers) support VXLAN in hardware, which enables
higher throughput and lower latency.
 • VXLAN-capable switches from vendors like Cisco, Arista, and Juniper can act as hardware-based
VTEPs, handling VXLAN encapsulation/decapsulation efficiently.

• Hardware VTEPs are typically used in high-performance data centers requiring scalability.

Each VTEP has two interfaces types:

• VTEP IP interface: Connects the VTEP to the underlay network with a unique IP address. This
interface encapsulates and de-encapsulates Ethernet frames.

• VNI interface: A virtual interface that keeps network traffic separated on the physical interface.
Similar to an SVI interface.

Please note that A VTEP can have multiple VNI interfaces, but they associate with the same VTEP IP
interface.

Overlay Network:

• The overlay is the logical, virtual Layer 2 network created using VXLAN encapsulation that sits on
top of the physical network infrastructure. This virtual network allows for Layer 2 connectivity
across the Layer 3 (IP) network.

• Each virtual network in VXLAN is identified by a unique VXLAN Network Identifier (VNI),
creating isolated tenant networks.

Underlay Network:

• The underlay network is the physical Layer 3 IP network infrastructure over which VXLAN
packets are transmitted.
 • It provides IP connectivity and routing between VXLAN Tunnel Endpoints (VTEPs), ensuring
efficient and scalable data transport.

VXLAN Frame Format

The VXLAN frame encapsulates the original Ethernet frame with additional headers:

1. Outer Ethernet Header: Contains the source and destination MAC addresses of VTEP devices.
2. Outer IP Header: Contains the source and destination IP addresses of VTEPs.
3. Outer UDP Header: VXLAN uses UDP for transport; port 4789 is the default.
4. VXLAN Header:
o The VXLAN header has a 24-bit VNI field used to identify the Layer 2 segment (VLAN).
5. Original Ethernet Frame: The original Layer 2 Ethernet frame is encapsulated within the VXLAN
packet.
VXLAN Packet Walkthrough:

1. Encapsulation at Source VTEP:

o A Layer 2 frame arrives at the source VTEP. The VTEP encapsulates it with VXLAN
headers, using the destination VTEP’s IP address in the underlay.

2. Routing Across Underlay:

o The VXLAN packet is routed across the Layer 3 network, traversing routers and switches
based on IP.

3. Decapsulation at Destination VTEP:

o The destination VTEP receives the VXLAN packet, decapsulates it, and forwards the
original Ethernet frame to the end device within the corresponding VLAN.

VXLAN Control Plane

VXLAN can use multiple control plane methods to establish communication between VTEPs:

Static VXLAN Configuration:

• VTEPs can be configured manually to forward traffic between known VTEP peers. This lacks
dynamic learning and is suitable only for small networks.

Multicast-Based Control Plane:

• Uses multicast groups to forward broadcast and unknown unicast traffic to all VTEPs in a VNI.

• Suitable for small or simple networks, but not recommended for large-scale deployments due to
scalability limits.

MP-BGP EVPN (Multiprotocol BGP with Ethernet VPN):

• BGP EVPN is a scalable, dynamic VXLAN control plane that distributes MAC-to-IP mappings
across VTEPs.

• It offers efficient, dynamic learning and MAC address distribution, reducing the need for
flooding and broadcast traffic.