Consider the above diagram.

RTA, RTB and RTC form a single cluster with

RTC being the RR. According to RTC, RTA and RTB are clients and anything
else is a non-client. Remember that clients of an RR are pointed at using
the “neighbor <ip-address> route-reflector-client” command. The same RTD
is the RR for its clients RTE and RTF; RTG is a RR in a third cluster.
Note that RTD, RTC and RTG are fully meshed but routers within a cluster
are not. When a route is received by a RR, it will do the following
depending on the peer type:
RTA
AS 100
RTB
RTC
(RR)
RTD
RTE
RTF
(RR)
1.1.1.1
2.2.2.2
3.3.3.3
4.4.4.4
6.6.6.6 5.5.5.5
(RR)
AS200
RTG
7.7.7.7
8.8.8.8
AS300
12.12.12.12
1/26/96-Rev: A1.2 Page 70 Sam Halabi-cisco Systems
1- Route from a non-client peer: reflect to all the clients within the
cluster.
2- Route from a client peer: reflect to all the non-client peers and also
to the client peers.
3- Route from an EBGP peer: send the update to all client and non-client
peers.
The following is the relative BGP configuration of routers RTC, RTD and
RTB:
RTC#
router bgp 100
neighbor 2.2.2.2 remote-as 100
neighbor 2.2.2.2 route-reflector-client
neighbor 1.1.1.1 remote-as 100
neighbor 1.1.1.1 route-reflector-client
neighbor 7.7.7.7 remote-as 100
neighbor 4.4.4.4 remote-as 100
neighbor 8.8.8.8 remote-as 200
RTB#
router bgp 100
neighbor 3.3.3.3 remote-as 100
neighbor 12.12.12.12 remote-as 300
RTD#
router bgp 100
neighbor 6.6.6.6 remote-as 100
neighbor 6.6.6.6 route-reflector-client
neighbor 5.5.5.5 remote-as 100
neighbor 5.5.5.5 route-reflector-client
neighbor 7.7.7.7 remote-as 100
neighbor 3.3.3.3 remote-as 100
As the IBGP learned routes are reflected, it is possible to have the
routing information loop. The Route-Reflector scheme has a few methods to
avoid this:
1- Originator-id: this is an optional, non transitive BGP attribute that
is four bytes long and is created by a RR. This attribute will carry
the router-id (RID) of the originator of the route in the local AS. Thus,
due to poor configuration, if the routing information comes back to the
originator, it will be ignored.
2- Cluster-list: this will be discussed in the next section.
1/26/96-Rev: A1.2 Page 71 Sam Halabi-cisco Systems
23.1 Multiple RRs within a cluster
Usually, a cluster of clients will have a single RR. In this case, the
cluster will be identified by the router-id of the RR. In order to
increase redundancy and avoid single points of failure, a cluster might
have more than one RR. All RRs in the same cluster need to be configured
with a 4 byte cluster-id so that a RR can recognize updates from RRs in
the same cluster.
A cluster-list is a sequence of cluster-ids that the route has passed.
When a RR reflects a route from its clients to non-clients outside of the
cluster, it will append the local cluster-id to the cluster-list. If this
update has an empty cluster-list the RR will create one. Using this
attribute, a RR can identify if the routing information is looped back to
the same cluster due to poor configuration. If the local cluster-id is
found in the cluster-list, the advertisement will be ignored.
In the above diagram, RTD, RTE, RTF and RTH belong to one cluster with
both RTD and RTH being RRs for the same cluster. Note the redundancy in
that RTH has a fully meshed peering with all the RRs. In case RTD goes
down, RTH will take its place. The following are the configuration of
RTH, RTD, RTF and RTC:
RTA
AS 100
RTB
RTC
(RR)
RTD
RTE
RTF
(RR)
1.1.1.1
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
6.6.6.6
(RR)
AS200
RTG
7.7.7.7
8.8.8.8
(RR)
RTH
9.9.9.9
10.10.10.10
11.11.11.11
AS400
AS300
AS500
13.13.13.13
1/26/96-Rev: A1.2 Page 72 Sam Halabi-cisco Systems
RTH#
router bgp 100
neighbor 4.4.4.4 remote-as 100
neighbor 5.5.5.5 remote-as 100
neighbor 5.5.5.5 route-reflector-client
neighbor 6.6.6.6 remote-as 100
neighbor 6.6.6.6 route-reflector-client
neighbor 7.7.7.7 remote-as 100
neighbor 3.3.3.3 remote-as 100
neighbor 9.9.9.9 remote-as 300
bgp route-reflector 10 (This is the cluster-id)
RTD#
router bgp 100
neighbor 10.10.10.10 remote-as 100
neighbor 5.5.5.5 remote-as 100
neighbor 5.5.5.5 route-reflector-client
neighbor 6.6.6.6 remote-as 100
neighbor 6.6.6.6 route-reflector-client
neighbor 7.7.7.7 remote-as 100
neighbor 3.3.3.3 remote-as 100
neighbor 11.11.11.11 remote-as 400
bgp route-reflector 10 (This is the cluster-id)
RTF#
router bgp 100
neighbor 10.10.10.10 remote-as 100
neighbor 4.4.4.4 remote-as 100
neighbor 13.13.13.13 remote-as 500
RTC#
router bgp 100
neighbor 1.1.1.1 remote-as 100
neighbor 1.1.1.1 route-reflector-client
neighbor 2.2.2.2 remote-as 100
neighbor 2.2.2.2 route-reflector-client
neighbor 4.4.4.4 remote-as 100
neighbor 7.7.7.7 remote-as 100
neighbor 10.10.10.10 remote-as 100
neighbor 8.8.8.8 remote-as 200
Note that we did not need the cluster command for RTC because only one RR
exists in that cluster.
1/26/96-Rev: A1.2 Page 73 Sam Halabi-cisco Systems
An important thing to note, is that peer-groups were not used in the
above configuration. If the clients inside a cluster do not have direct
IBGP peers among one another and they exchange updates through the RR,
peer-goups should not be used. If peer groups were to be configured, then
a potential withdrawal to the source of a route on the RR would be sent
to all clients inside the cluster and could cause problems.
The router sub-command bgp client-to-client reflection is enabled by
default on the RR.If BGP client-to-client reflection were turned off on
the RR and redundant BGP peering was made between the clients, then using
peer groups would be alright.
23.2 RR and conventional BGP speakers
It is normal in an AS to have BGP speakers that do not understand the
concept of route reflectors. We will call these routers conventional BGP
speakers. The route reflector scheme will allow such conventional BGP
speakers to coexist. These routers could be either members of a client
group or a non-client group. This would allow easy and gradual migration
from the current IBGP model to the route reflector model. One could start
creating clusters by configuring a single router as RR and making other
RRs and their clients normal IBGP peers. Then more clusters could be created
gradually.
Example:
RTA
AS 100
RTB
RTD RTC
RTE
RTF
(RR)
1.1.1.1
2.2.2.2
3.3.3.3
4.4.4.4
6.6.6.6 5.5.5.5
AS200
8.8.8.8
AS300
13.13.13.13
AS400 14.14.14.14
1/26/96-Rev: A1.2 Page 74 Sam Halabi-cisco Systems
In the above diagram, RTD, RTE and RTF have the concept of route reflection.
RTC, RTA and RTB are what we call conventional routers and cannot
be configured as RRs. Normal IBGP mesh could be done between these routers
and RTD. Later on, when we are ready to upgrade, RTC could be made a
RR with clients RTA and RTB. Clients do not have to understand the route
reflection scheme; it is only the RRs that would have to be upgraded.
The following is the configuration of RTD and RTC:
RTD#
router bgp 100
neighbor 6.6.6.6 remote-as 100
neighbor 6.6.6.6 route-reflector-client
neighbor 5.5.5.5 remote-as 100
neighbor 5.5.5.5 route-reflector-client
neighbor 3.3.3.3 remote-as 100
neighbor 2.2.2.2 remote-as 100
neighbor 1.1.1.1 remote-as 100
neighbor 13.13.13.13 remote-as 300
RTC#
router bgp 100
neighbor 4.4.4.4 remote-as 100
neighbor 2.2.2.2 remote-as 100
neighbor 1.1.1.1 remote-as 100
neighbor 14.14.14.14 remote-as 400
When we are ready to upgrade RTC and make it a RR, we would remove the
IBGP full mesh and have RTA and RTB become clients of RTC.
23.3 Avoiding looping of routing information
We have mentioned so far two attributes that are used to prevent potential
information looping: the originator-id and the cluster-list.
Another means of controlling loops is to put more restrictions on the set
clause of out-bound route-maps.
The set clause for out-bound route-maps does not affect routes reflected
to IBGP peers.
More restrictions are also put on the nexthop-self which is a per neighbor
configuration option. When used on RRs the nexthop-self will only
affect the nexthop of EBGP learned routes because the nexthop of
reflected routes should not be changed.
1/26/96-Rev: A1.2 Page 75 Sam Halabi-cisco Systems
24.0 Route Flap Dampening
Route dampening (introduced in Cisco IOS version 11.0) is a mechanism to minimize
the
instability caused by route flapping and oscillation over the network. To
accomplish this,
criteria are defined to identify poorly behaved routes. A route which is flapping
gets a penalty
for each flap (1000). As soon as the cumulative penalty reaches a predefined
“suppress-
limit”, the advertisement of the route will be suppressed. The penalty will be

neighbor 192.208.10.6 remote-as 100

RTB is configured for route dampening with default parameters. Assuming
the EBGP link to RTD is stable, RTB’s BGP table would look like this:
RTB
S1 192.208.10.6
S0/0 192.208.10.5
S0 203.250.15.2
AS100
AS300
RTD
L0 192.208.10.174
1/26/96-Rev: A1.2 Page 77 Sam Halabi-cisco Systems
RTB#sh ip bgp
BGP table version is 24, local router ID is 203.250.15.2 Status codes: s
suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 192.208.10.0 192.208.10.5 0 0 300 i
*> 203.250.15.0 0.0.0.0 0 32768 i
In order to simulate a route flap, I will do a “clear ip bgp
192.208.10.6” on RTD. RTB’s BGP table will look like this:
RTB#sh ip bgp
BGP table version is 24, local router ID is 203.250.15.2 Status codes: s
suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
h 192.208.10.0 192.208.10.5 0 0 300 i
*> 203.250.15.0 0.0.0.0 0 32768 i
The BGP entry for 192.208.10.0 has been put in a “history” state. Which
means that we do not have a best path to the route but information about
the route flapping still exists.
RTB#sh ip bgp 192.208.10.0
BGP routing table entry for 192.208.10.0 255.255.255.0, version 25
Paths: (1 available, no best path)
300 (history entry)
192.208.10.5 from 192.208.10.5 (192.208.10.174)
Origin IGP, metric 0, external
Dampinfo: penalty 910, flapped 1 times in 0:02:03
The route has been given a penalty for flapping but the penalty is still
below the “suppress limit” (default is 2000). The route is not yet suppressed.
If the route flaps few more times we will see the following:
1/26/96-Rev: A1.2 Page 78 Sam Halabi-cisco Systems
RTB#sh ip bgp
BGP table version is 32, local router ID is 203.250.15.2 Status codes:
s suppressed, d damped, h history, * valid, > best, i - internal Origin
codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*d 192.208.10.0 192.208.10.5 0 0 300 i
*> 203.250.15.0 0.0.0.0 0 32768 i
RTB#sh ip bgp 192.208.10.0
BGP routing table entry for 192.208.10.0 255.255.255.0, version 32
Paths: (1 available, no best path)
300, (suppressed due to dampening)
192.208.10.5 from 192.208.10.5 (192.208.10.174)
Origin IGP, metric 0, valid, external
Dampinfo: penalty 2615, flapped 3 times in 0:05:18 , reuse in 0:27:00
The route has been dampened (suppressed). The route will be reused when
the penalty reaches the “reuse value”, in our case 750 (default).The
dampening information will be purged when the penalty becomes less than
half of the reuse-limit, in our case (750/2=375). The Following are the
commands used to show and clear flap statistics information:
show ip bgp flap-statistics
(displays flap statistics for all the paths)
show ip bgp-flap-statistics regexp <regexp>
(displays flap statistics for all paths that match the regexp)
show ip bgp flap-statistics filter-list <list>
(displays flap statistics for all paths that pass the filter)
show ip bgp flap-statistics A.B.C.D m.m.m.m
(displays flap statistics for a single entry)
show ip bgp flap-statistics A.B.C.D m.m.m.m longer-prefixes
(displays flap statistics for more specific entries)
show ip bgp neighbor [dampened-routes] | [flap-statistics]
(displays flap statistics for all paths from a neighbor)
clear ip bgp flap-statistics (clears flap statistics for all routes)
clear ip bgp flap-statistics regexp <regexp>
(clears flap statistics for all the paths that match the regexp)
clear ip bgp flap-statistics filter-list <list>
(clears flap statistics for all the paths that pass the filter)
clear ip bgp flap-statistics A.B.C.D m.m.m.m
(clears flap statistics for a single entry)
clear ip bgp A.B.C.D flap-statistics
(clears flap statistics for all paths from a neighbor)
1/26/96-Rev: A1.2 Page 79 Sam Halabi-cisco Systems
25.0 How BGP selects a Path
Now that we are familiar with the BGP attributes and terminology, the
following list indicates how BGP selects the best path for a particular
destination. Remember that we only select one path as the best path. We
put that path in our routing table and we propagate it to our BGP
neighbors.
Path selection is based on the following:
1-If NextHop is inaccessible do not consider it.
2-Prefer the largest Weight.
3-If same weight prefer largest Local Preference.
4-If same Local Preference prefer the route that the
specified router has originated.
5-If no route was originated prefer the shorter AS path.
6-If all paths are external prefer the lowest origin code
(IGP<EGP<INCOMPLETE).
7-If origin codes are the same prefer the path with the lowest
MULTI_EXIT_DISC.
8-If path is the same length prefer External path over Internal.
9-If IGP synchronization is disabled and only internal path remain prefer
the path through the closest IGP neighbor.
10-Prefer the route with the lowest ip address value for BGP router ID.
The ollowing is a design example thatis intended to show the configuration
and routing tables as they actually appear on the C.isco routers.