Connecting Customers Using BGP or OSPF

MPLS Layer 3 VPNs

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-1
Objectives
• Configure an OSPF PE-CE routing session
• Configure a BGP PE-CE routing session
• Describe how to troubleshoot MPLS VPNs

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-2
OSPF as the PE-CE Routing Protocol

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-3
OSPF in an MPLS VPN Routing Model
OSPF Area 0 (Backbone Area) • OSPF divides a network into areas,
all of them linked through the
backbone (Area 0).
• Areas could correspond to individual
Area Border Router Area Border Router
sites from an MPLS VPN
perspective.

Area 1 Area 2 Area 3

• From the customer perspective, an

MPLS VPN-based network has a
BGP backbone with IGP running at BGP Backbone
customer sites.
• Redistribution between IGP and
BGP is performed to propagate
PE Router PE Router
customer routes across the MPLS
VPN backbone.
CE Router
Site IGP Site IGP Site IGP
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-4
OSPF in an MPLS VPN Routing Model:
OSPF-BGP Redistribution Issue
2. An OSPF route is redistributed into BGP.

BGP Backbone 3. The MP-BGP route is propagated to

other PE routers.

4. The MP-BGP route is

redistributed into OSPF.
PE Router PE Router
5. The OSPF route is
propagated as an
external route into
other sites.

Area 1 Area 2 Area 3

1. A local subnetwork is announced to the PE router
as type 1 or type 2 LSA.

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-5
OSPF in an MPLS VPN Routing Model: Caveats
• The OSPF route type is not preserved when the OSPF route is
redistributed into BGP.
• All OSPF routes from a site are inserted as external (type 5 LSA) routes
into other sites.
• The result is that OSPF route summarization and stub areas are hard to
implement.
Conclusion: MPLS VPNs must extend the classic OSPF-BGP routing
model.

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-6
OSPF Superbackbone
• OSPF Area 0 might extend into individual sites.
• The MPLS VPN backbone has to become a superbackbone for OSPF.
BGP Backbone

PE Router PE Router

Area 0 Area 2 Area 0 Area 3

• OSPF between sites will not use normal OSPF-BGP redistribution.
• OSPF continuity must be provided across the MPLS VPN backbone:
- Internal OSPF routes should remain internal OSPF routes.
- External routes should remain external routes.
- OSPF metrics should be preserved.
• CE routers run standard OSPF software.
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-7
OSPF Superbackbone Route Propagation
2. The PE router propagates the route into the
superbackbone. Route summarization can be
OSPF Superbackbone performed on the area boundary.

3. The route from the superbackbone

is inserted into other areas as an
interarea route.

ABR ABR
4. The interarea route
is propagated into
other areas.

Area 0 Area 2 Area 0 Area 3

1. A local subnetwork is announced to the PE

router as type 1 or type 2 LSA.

• Extended BGP communities are used to propagate OSPF route types across the
BGP backbone.
• OSPF cost is copied into the MED attribute.

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-8
OSPF Superbackbone Implementation
BGP
10.0.0.0/8
Backbone OSPF RT = 1:1:0
Internal OSPF routes MED = 768

• The OSPF route type is copied into the PE PE

10.0.0.0/8
extended BGP community on redistribution 10.0.0.0/8
LSA type 3
LSA Type 1
into BGP. OSPF cost 768
OSPF Cost 768
Area 1 Area 2
• The egress PE router performs interarea
transformation. BGP
10.0.0.0/8
Backbone OSPF RT = 1:5:1
External OSPF routes MED = 768

• Routes are propagated in the same PE PE

way as internal OSPF routes across 10.0.0.0/8
10.0.0.0/8
LSA Type 5
the superbackbone. Non-OSPF
LSA Type 5
E2 Metric 20
E2 Metric 20
Route Area 1 Area 2
• The external metric and route type
are preserved. BGP
10.0.0.0/8
Backbone MED = 3
Routes from other routing protocols
• Routes from the MP-BGP backbone that did not PE PE
10.0.0.0/8
originate in OSPF are still subject to standard 10.0.0.0/8 LSA Type 5
redistribution behavior when inserted into OSPF. Hop Count 3 E2 Metric 20

RIP Area 2
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-9
Configuring PE-CE OSPF Routing
Follow these steps to configure OSPF as the PE-CE routing
protocol:
• Configure a per-VRF copy of OSPF.
• Configure redistribution of MP-BGP into OSPF.
• Configure redistribution of OSPF into MP-BGP.

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-10
Configuring PE-CE OSPF Routing: Cisco IOS and
IOS XE Software
router(config)#
router ospf process-id vrf vrf-name
... Standard OSPF parameters ...
• This command starts the per-VRF OSPF routing process.

router(config-router)#
redistribute bgp as-number subnets
• This command redistributes MP-BGP routes into OSPF. The
subnets keyword is mandatory for proper operation.
router(config)#
router bgp as-number
address-family ipv4 vrf vrf-name
redistribute ospf process-id [match [internal]
[external-1] [external-2]]
• OSPF-BGP route redistribution is configured with the
redistribute command under the proper address-family
command.
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-11
Configuring PE-CE OSPF Routing: Cisco IOS XR
Software
RP/0/RP0/CPU0:router(config-ospf)#
vrf vrf-name
... Standard OSPF parameters ...
• This command starts the per-VRF OSPF routing process.

RP/0/RP0/CPU0:router(config-ospf-vrf)#
redistribute bgp as-number
• This command redistributes MP-BGP routes into OSPF.

RP/0/RP0/CPU0:router(config)#
router bgp as-number
vrf vrf-name
address-family ipv4 unicast
redistribute ospf process-id [match {external [1|2] |
internal}]
• OSPF-BGP route redistribution is configured under the proper
address-family command.
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-12
Routing Loops Between MP-BGP and OSPF
2. The OSPF route is received by a PE
router, redistributed into MP-BGP, and
propagated across the MPLS VPN
BGP Backbone backbone.
3. The route from the
superbackbone is inserted as the
interarea route.

PE Router PE Router PE Router

5. The other PE router
would redistribute the
route back into BGP.

4. The OSPF route

is propagated
across the area.

Area 1 Area 2
1. The local subnetwork is announced to the PE
router.
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-13
OSPF Down Bit
• A down bit has been introduced in the options field of the OSPF LSA header.
• PE routers set the down bit when redistributing routes from MP-BGP into OSPF.
• PE routers never redistribute OSPF routes with the down bit set into MP-BGP.
2. An OSPF route is received by a PE router, redistributed into
MP-BGP, and propagated across the MPLS VPN backbone.
BGP Backbone
3. The route from the superbackbone is inserted
as the interarea route.

PE Router PE Router PE Router

Down The route is never redistributed
back into the MP-BGP backbone.

4. The OSPF route is propagated

with the down bit set.

Area 1 Area 2
1. The local subnetwork is announced without the down bit.

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-14
Packet Forwarding Across the MPLS VPN
Backbone
2. The OSPF route is propagated with the down 3. Because of administrative distances,
bit set. an OSPF route is preferred over an
MP-IBGP route. Packet flow across
the network is not optimal.
BGP Backbone

PE Router PE Router PE Router

Down

Another OSPF or
Area 1 Area 2 Non-OSPF Site
1. The OSPF route is received by a PE router
and redistributed into MP-BGP and OSPF.

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-15
Packet Forwarding Across the MPLS VPN
Backbone (Cont.)
1. The OSPF route is propagated with the down 2. The OSPF route is ignored because
bit set. the down bit is set.

BGP Backbone

PE Router PE Router PE Router

Down

Another OSPF or
Area 1 Area 2 Non-OSPF Site
Packet flow across the network is optimal.

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-16
Sham Link
• OSPF prefers intra-area paths to interarea paths.
• The path over a backdoor link will always be selected.
• A sham link is a logical intra-area link.
• It is carried by the superbackbone.
• A sham link is required only
High-Bandwidth
between two VPN sites
BGP Backbone
that belong to the same
area and have a backdoor
link for backup purposes.
• OSPF adjacency is PE Router PE Router
established across the Low-Bandwidth
sham link. Backdoor Link

• LSA flooding occurs

across the sham link.
Site 1 Area 1 Site 2

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-17
Sham Link LSA Propagation
2. The site 1 PE redistributes 3. The site 2 PE receives the OSPF type 1
the OSPF route into MP- LSA for the selected route from two
BGP because the selected directions. The OSPF cost of the sham
OSPF route was not High-Bandwidth link has been configured so that the
sham link is preferred.
received via a sham link.
BGP Backbone
Preferred Path
P

LSA 1
BG

Preferred Sham Link

PE Router 1. The LSA is propagated to the PE Router

Site 1 PE and to Site 2 PE to 4. The site 2 PE is not
redistributing the
LSA 1

LSA 1
allow the best path selection.
selected OSPF
Area 1 route into MP-BGP
LSA 1 because the
preferred route was
received via a sham
Low-Bandwidth link.
Backdoor Link
Site 1 Site 2
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-18
Configuring Sham Links
• A separate /32 address space is BGP Backbone AS 64500
required in each PE router for each
sham link. PE Router PE Router
Sham Link
• This /32 address space:
- Is required so that OSPF packets can gi 0/2/0/0
be sent over the VPN backbone to the
remote end of the sham link Area 1

- Must belong to the VRF Backdoor

- Must not be advertised by OSPF OSPF Process 11 Link
Site 1 Site 2
- Must be advertised by BGP

Router(config-router)# Cisco IOS and IOS XE

area area-id sham-link source-address destination-address cost number

RP/0/RP0/CPU0:router(config-ospf)# Cisco IOS XR

vrf vrf-name
area area-id
sham-link source-address destination-address cost number

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-19
BGP as the PE-CE Routing Protocol

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-20
Configuring per-VRF BGP Routing Contexts

Router(config)#
router bgp as-number
Cisco IOS address-family ipv4 vrf vrf-name
and IOS XE ... Per-VRF BGP definitions ...

RP/0/RP0/CPU0:Router(config)#
router bgp as-number
Cisco IOS vrf vrf-name
XR address-family ipv4 unicast
... Per-VRF BGP definitions ...

• Select a per-VRF BGP context with the address-family command.

• Configure CE EBGP neighbors in the VRF context, not in the global
BGP configuration.
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-21
Configuring per-VRF BGP Routing Contexts
Example
router bgp 64501
neighbor 10.0.1.1 remote-as 64500
network 10.1.1.0 mask 255.255.255.0
MPLS VPN Backbone CE-BGP-A2
AS 64500
Cisco IOS and IOS XE Cisco IOS XR

CE-BGP-A1 CE-BGP-A3

PE-X PE-Y

router bgp 64500

ip vrf Customer_A address-family vpnv4 unicast
rd 64501:1 vrf Customer_A
route-target both 64500:1 rd 64500:1
! address-family ipv4 unicast
router bgp 64500 !
address-family ipv4 vrf Customer_A neighbor 10.2.1.1
neighbor 10.1.1.1 remote-as 64501 remote-as 64502
neighbor 10.1.1.1 activate update-source GigabitEthernet0/0/0/0
address-family ipv4 unicast

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-22
Limiting the Number of Routes in a VRF
• Service providers offering MPLS VPN services are at risk of denial-of-
service attacks similar to those aimed at service providers offering BGP
connectivity:
- Any customer can generate any number of routes, using resources in the PE
routers.
- Therefore, the resources that are used by a single customer have to be
limited.
• Cisco IOS Software offers two solutions:
- You can limit the number of routes received from a BGP neighbor.
- You can limit the total number of routes in a VRF.

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-23
Limiting the Number of Prefixes Received from a
BGP Neighbor
Router(config-router-af)#
Cisco IOS
neighbor ip-address maximum-prefix maximum [threshold]
and IOS
[warning-only]
XE

RP/0/RP0/CPU0:Router(config-bgp-nbr-af)#
Cisco
IOS XR maximum-prefix maximum [threshold] [warning-only]

• Control how many prefixes can be received from a neighbor.

• Optional threshold parameter specifies the percentage where a warning
message is logged (the default is 75 percent).
• Optional warning-only keyword specifies the action on exceeding the
maximum number (the default is to drop peering).

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-24
Limiting the Total Number of VRF Routes
• The VRF maximum routes limit command limits the number of routes that are
imported into a VRF:
- Routes coming from CE routers
- Routes coming from other PE routers (imported routes)
• The route limit is configured for each VRF.
• If the number of routes exceeds the route limit:
- A syslog message (Cisco IOS and IOS XE Software) is generated.
- A SNMP trap (Cisco IOS XR Software) is generated.
- Cisco IOS, IOS XE, and IOS XR Software can be configured to reject routes (optional).

Router(config-vrf)#
Cisco IOS
and IOS XE maximum routes limit {warn-threshold | warn-only}

Cisco IOS RP/0/RSP0/CPU0:Router(config-vrf-af)#

XR maximum prefix limit [threshold]

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-25
Limiting the Total Number of VRF Routes Example

Customer A P-Network
AS 64501 AS 64500
4 3
1 2 VPN-IPv4 Update: VPN-IPv4 Update:
IPv4 Update: IPv4 Update: RD:192.168.60.0/24 RD:192.168.61.0/24
192.168.0.5/32 192.168.50.0/24 RT = 64500:2 RT = 64500:2

CE-BGP-A1 PE-X PE-Y

5
IPv4 Update:
192.168.50.0/24

Cisco IOS and IOS XE Cisco IOS XR

ip vrf Customer_A vrf Customer_A
rd 64500:2 address-family ipv4 unicast
route-target both 64500:2 import route-target 64500:2
maximum routes 4 75 export route-target 64500:2
maximum prefix 4 75

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-26
“Same AS Number” Issue
The customer wants to reuse an AS number on several sites:
• CE-BGP-A1 announces network 10.1.0.0/16 to PE-Site-X.
• The prefix announced by CE-BGP-A1 is propagated to PE-Site-Y as an
internal route through MP-BGP.
• PE-Site-Y prepends AS 64500 to the AS path and propagates the prefix to
CE-BGP-A2.
• CE-BGP-A2 drops the update because AS 64501 is already in the AS path.

Site A P-Network Site B

AS 64501 AS 64500 AS 64501

CE-BGP-A1 PE-X PE-Y CE-BGP-A2

10.1.0.0/16 64501 i 10.1.0.0/16 64501 10.1.0.0/16 64501 64501

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-27
as-override
• New AS path update procedures have been implemented to reuse an AS
number on all VPN sites.
• The procedures allow the use of private and public AS numbers.
• The same AS number may be used for all sites.
• With as-override configured, the AS path update procedure on the PE router is
as follows:
- If the first AS number in the AS path is equal to the neighboring AS, it is replaced with
the provider AS number.
- If the first AS number has multiple occurrences (because of AS path prepend), all
occurrences are replaced with the provider AS number.
- After this operation, the provider AS number is prepended to the AS path.

Router(config-router-af)#
Cisco IOS
and IOS XE neighbor ip-address as-override
RP/0/RP0/CPU0:router(config-bgp-vrf-nbr-af)#
Cisco IOS
XR as-override

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-28
as-override Example
• PE-Site-Y replaces AS 64501 with AS 64500 in the AS path, prepends
another copy of AS 64500 to the AS path, and propagates the prefix.
Cisco IOS and IOS XE Cisco IOS XR
router bgp 64500
router bgp 64500
vrf Customer_2
address-family ipv4 vrf Customer_A
neighbor 10.1.1.1
neighbor 10.1.1.1 remote-as 64501
remote-as 64501
neighbor 10.1.1.1 activate
address-family ipv4 unicast
neighbor 10.1.1.1 activate
as-override

Site A P-Network Site B

AS 64501 AS 64500 AS 64501

CE-BGP-A1 PE-X PE-Y CE-BGP-A2

10.1.0.0/16 64501 i 10.1.0.0/16 64501 10.1.0.0/16 64500 64500

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-29
allowas-in
The BGP route is rejected because the PE3 router sees its own AS number
in the AS path.
Customer A:
Customer A: VPN
VPN Hub Site
Site Spoke 1 EBGP Update EBGP Update
as-path (64501) as-path (64501) AS 64503
AS 64501 CE3
AS1
VRFa
CE1

Customer A: VPN PE1

Site Spoke 2 IBGP Update

AS 64502 PE3

PE2 VRFb

CE2 EBGP Update EBGP Update CE4

as-path as-path
(1 64503 1 64501) (1 64503 1 64501)
router BGP 1
address-family IPv4 VRF Customer 1
neighbor CE4 allowas-in
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-30
Implementing SOO for Loop Prevention
AS path-based BGP loop prevention is bypassed with the as‑override and
allowas-in features. 10.1.0.0/16 64500 64500

Site B AS 64501

10.1.0.0/16 64500 64500

Site A P-Network
AS 64501 AS 64500
CE-BGP-A3

CE-BGP-A1 PE-X PE-Y

CE-BGP-A2

10.1.0.0/16 64501 i 10.1.0.0/16 64501 10.1.0.0/16 64500 64500

• Sets the SOO value for a BGP neighbor

Cisco IOS Router(config-router-af)#
and IOS XE neighbor ip-address soo AS:nn

Cisco IOS RP/0/RP0/CPU0:Router(config-bgp-vrf-nbr-af)#

XR site-of-origin AS:nn
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-31
Troubleshooting MPLS VPNs

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-32
Preliminary Steps in MPLS VPN Troubleshooting
Perform basic MPLS troubleshooting:
• Is Cisco Express Forwarding enabled?
• Are labels for IGP routes generated and propagated?
• Are large labeled packets propagated across the MPLS backbone
(maximum transmission unit issues)?

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-33
Verifying the Routing Information Flow

2. Are routes redistributed into MP-BGP

with the proper extended 5. Are VPNv4 routes inserted into
communities? VRFs on other PE routers?

P-Network 6. Are VPNv4 routes redistributed

from BGP into the PE-CE routing
protocol?
CE-Spoke P CE-Spoke
7. Are IPv4 routes propagated
to other CE routers?
PE-1 PE-2

3. Are VPNv4 routes propagated

CE-Spoke to other PE routers? CE-Spoke
1. Are CE routes received
by a PE router? 4. Is the BGP route selection
process working correctly?

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-34
Routing Information Flow
Troubleshooting Commands

show route vrf

show bgp vpnv4 vrf vrf-name ip-prefix show bgp ip-prefix
debug bgp show vrf detail

P-Network

CE-Spoke P CE-Spoke
show route

PE-1 PE-2

show bgp vpnv4 unicast ip-prefix

CE-Spoke CE-Spoke
show route vrf vrf-name
show bgp vpnv4 unicast vrf vrf-name ip-prefix

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-35
Verifying the Data Flow
Is there an end-to-end LSP
tunnel between the PE
routers? Is the Cisco Express
Forwarding entry correct on
the ingress PE router?
P-Network

CE-Spoke P CE-Spoke
Is Cisco Express Forwarding
enabled on the ingress PE
router interface?
PE-1 PE-2

CE-Spoke CE-Spoke
Is the LFIB entry on the
egress PE router correct?

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-36
Validating Cisco Express Forwarding Status

show cef vrf vrf-name ip-prefix/length detail

P-Network

CE-Spoke P CE-Spoke
show cef interface

PE-1 PE-2

CE-Spoke CE-Spoke

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-37
Validating the End-to-End Label-Switched Path
• Check for summarization issues. The BGP next hop should be
reachable as a host route.
• Quick check—If TTL propagation is disabled, the trace from PE-2 to
PE-1 should contain only one hop.
• If needed, check LFIB values hop by hop.
• Check for MTU issues on the path. MPLS VPN requires a larger label
header than pure MPLS.

P-Network

CE-Spoke P CE-Spoke

PE-1 PE-2

CE-Spoke CE-Spoke

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-38
Validating the LFIB Status

show cef vrf vrf-name ip-prefix/length detail

show mpls forwarding vrf vrf-name value detail

P-Network

CE-Spoke P CE-Spoke

PE-1 PE-2

CE-Spoke CE-Spoke

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-39
Troubleshooting Command Summary
Cisco IOS and IOS XE Cisco IOS XR
show ip ospf database Control Plane show ospf database
show ip bgp show bgp
show ip eigrp topology Routing Protocol show eigrp topology

show ip route IP Routing Table (RIB) show route

show mpls ldp bindings Label Exchange Protocol show mpls ldp bindings
(LFIB)

Data Plane
show ip cef show cef
show ip cef vrf show cef vrf
IP Forwarding Table (FIB)

show mpls forwarding-table Label Forwarding Table

show mpls forwarding
show mpls forwarding-table vrf (LFIB)
show mpls forwarding vrf

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-40
Summary
• OSPF as a PE-CE routing protocol is implemented as a separate
routing process.
• BGP is very scalable and predictable as a PE-CE routing protocol.
• MPLS VPN troubleshooting has two main steps: verifying routing
information flow and verifying proper data flow.

© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-41
© 2011 Cisco and/or its affiliates. All rights reserved. SPEDGE v1.0—2-42