BGP Fast Reroute
This chapter provides information about BGP Fast Reroute.
Topics in this chapter include:
Applicability
This chapter was initially written for SR OS Release 14.0.R7, but the CLI in the current edition is based on SR OS Release 20.10.R1.
Overview
Border Gateway Protocol (BGP) is a key protocol for ISPs, supporting inter-Autonomous System (inter-AS) and intra-Autonomous System (intra-AS) applications with many address families. Additionally, ISPs need to maintain the service level agreements with their customers, even in case of network failures.
MPLS Fast Reroute (FRR) is often used to provide resiliency to intra-AS services, and relies on alternate label switched paths being established through the network. Traffic is switched to the alternate path in case of a failure of the primary path.
However, the traffic for inter-AS services crosses the boundaries of multiple ASs, so to provide resiliency, BGP FRR can be used. Before a network failure occurs, multiple paths must be received for a prefix to take advantage of this feature. When a prefix has a backup path and its primary paths fail, the affected traffic is rapidly diverted to the backup path without waiting for the control plane to reconverge. When many prefixes share the same primary paths, and in some cases also the same backup path, the time to divert traffic to the backup path is independent of the number of prefixes; this is also known as Prefix Independent Convergence (PIC). The traffic goes back to the primary paths when those paths are restored. Multiple primary paths can be active simultaneously when Equal Cost Multi Path (ECMP) applies.
Within SR OS, two BGP FRR functions are supported: Core PIC and Edge PIC. Core PIC describes a scenario where a link or node on the path to the BGP next-hop fails, but the BGP next-hop remains reachable; see Core PIC. Edge PIC describes a scenario where an edge node or edge link fails, which results in a change of the BGP next-hop; see Edge PIC.
Within SR OS, Core PIC is enabled by default and cannot be disabled. Therefore, this chapter will describe the use of Edge PIC.
BGP FRR is supported for different BGP address families in the base router context or within a specific vprn context. This chapter will focus on the IPv4 address family within the base router context.
The following SR OS supported features can be used to allow BGP to maintain multiple paths through an autonomous system:
BGP best external
BGP add-paths
Convergence goes through several phases, which also apply to BGP:
detect the network failure
distribute updated routing information, and update the network topology
calculate new routes, and optionally change next-hops
update the forwarding plane
Several mechanisms are available to enhance BGP network convergence, such as:
Bidirectional Forwarding Detection (BFD)
Minimum Router Advertisement Interval (MRAI)
BGP peer tracking
This chapter describes the use of BFD and MRAI for faster network convergence.
Configuration
The example topology used in this chapter is shown in BGP FRR topology, and has the following characteristics:
iBGP sessions are established between AS 65537 routers using RR-6 as route reflector with P-2, P-3, P-4, and PE-5 as route reflector clients.
eBGP sessions are established between P-2 and P-3 of AS 65537 and PE-1 of AS 65536.
PE-1 advertises a BGP route for prefix 172.16.1.0/24 to P-2 and P-3.
P-2 changes the local preference to 150 for the route advertised to its route reflector RR-6.
P-2 and P-3 advertise a BGP route for prefix 172.16.5.0/24 to PE-1.
These characteristics enforce traffic for destination 172.16.1.0/24 to leave AS 65537 via P-2. P-2 (and also PE-5) learns the destination and the local preference via route reflector RR-6. But because P-3's own local preference is lower (default LP=100), it stops advertising prefix 172.16.1.0/24 toward RR-6, so that P-4 is aware of the path via P-2 only.
The objective is for P-4 to receive multiple copies of the 172.16.1.0/24 prefix with redundant next-hops, to provide for faster convergence under failure. Considering the characteristics previously listed for the topology, two features contribute for achieving this goal:
Using BGP best external
Using BGP add-paths
The BGP add-paths feature is required in scenarios with route-reflectors, possibly combined with the BGP best external feature. The BGP best external feature can be used without BGP add-paths in scenarios when the BGP peers are in a full mesh.
As a result, multiple exit paths for prefix 172.16.1.0/24 leaving AS 65537 are available, improving convergence time on the iBGP peers because they only need to update their FIBs if they lose the primary route.
BGP best external
P-3 is configured with the BGP best external feature, as follows:
# on P-3:
configure
router
bgp
loop-detect discard-route
advertise-inactive
split-horizon
advertise-external ipv4
group "eBGP_AS65536"
peer-as 65536
neighbor 10.0.13.1
exit
exit
group "iBGP_AS65537"
next-hop-self
peer-as 65537
neighbor 192.0.2.6
exit
exit
no shutdown
exit
In this output, advertise-external is activated for the IPv4 address family only. It can also be activated for the IPv6, label-IPv4, and label-IPv6 address families.
Although it is not necessary to also enable BGP best external on P-2, it is not uncommon to also configure this feature on other autonomous system border routers.
P-3 advertises prefix 172.16.1.0/24 toward the route reflector RR-6, as follows:
*A:P-3# show router bgp neighbor 192.0.2.6 advertised-routes
===============================================================================
BGP Router ID:192.0.2.3 AS:65537 Local AS:65537
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
i 172.16.1.0/24 100 None
192.0.2.3 None 0
65536 -
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The BGP best external feature is sufficient for providing alternate paths in a fully meshed autonomous system, and could be used in conjunction with the BGP add-paths feature. The BGP add-paths feature is a requirement in scenarios with route reflectors.
BGP add-paths
P-2, P-3, P-4 and RR-6 are configured with the BGP add-paths feature. PE-5 does not require the add-paths feature, because the alternate path to 172.16.1.0/24 starts in P-4.
# on P-2, P-3, P-4, RR-6:
configure
router
bgp
group "iBGP_AS65537"
add-paths
ipv4 send 2 receive
The BGP configuration on P-2 is as follows:
# on P-2:
configure
router
bgp
loop-detect discard-route
advertise-inactive
split-horizon
group "eBGP_AS65536"
import "Import_LP150"
peer-as 65536
neighbor 10.0.12.1
exit
exit
group "iBGP_AS65537"
next-hop-self
peer-as 65537
add-paths
ipv4 send 2 receive
exit
neighbor 192.0.2.6
exit
exit
no shutdown
exit
The BGP configuration for P-3 and P-4 is very similar and is not shown here.
The BGP configuration on RR-6 then is as follows:
# on RR-6:
configure
router
bgp
loop-detect discard-route
split-horizon
group "iBGP_AS65537"
cluster 6.6.6.6
advertise-inactive
peer-as 65537
add-paths
ipv4 send 2 receive
exit
neighbor 192.0.2.2
exit
neighbor 192.0.2.3
exit
neighbor 192.0.2.4
exit
neighbor 192.0.2.5
exit
exit
no shutdown
exit
The default behavior of a route reflector is to only consider the best path. By enabling the add-paths feature on RR-6, multiple paths are considered.
Both P-2 and P-3 advertise route 172.16.1.0/24 to RR-6, as follows:
*A:P-2# show router bgp neighbor 192.0.2.6 advertised-routes
===============================================================================
BGP Router ID:192.0.2.2 AS:65537 Local AS:65537
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
i 172.16.1.0/24 150 None
192.0.2.2 1 0
65536 -
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
*A:P-3# show router bgp neighbor 192.0.2.6 advertised-routes
===============================================================================
BGP Router ID:192.0.2.3 AS:65537 Local AS:65537
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
i 172.16.1.0/24 100 None
192.0.2.3 1 0
65536 -
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
For more examples of the BGP add-paths feature, see the BGP Add-Paths chapter and the BGP Multipath chapter.
Backup path
P-4 is the place in the topology where an alternate path is created. The data plane part of the Edge PIC configuration is performed by enabling the backup-path command within the bgp context. In the following, backup-paths are considered for the IPv4 address family only, but the IPv6, label-IPv4, and label-IPv6 address families are allowed too.
# on PE-1, P-4:
configure
router
bgp
backup-path ipv4
In this way, BGP considers all alternate paths which are present through the BGP best external and BGP add-paths feature. The BGP configuration on P-4 is as follows:
# on P-4:
configure
router
bgp
loop-detect discard-route
split-horizon
backup-path ipv4
group "iBGP_AS65537"
peer-as 65537
add-paths
ipv4 send 2 receive
exit
neighbor 192.0.2.6
exit
exit
no shutdown
exit
In the default BGP behavior, without the backup-path command, two BGP routes exist. Both routes are valid, but only the first one is the best path (indicated by ">"), as follows:
*A:P-4# show router bgp routes 172.16.1.0/24
===============================================================================
BGP Router ID:192.0.2.4 AS:65537 Local AS:65537
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 172.16.1.0/24 150 None
192.0.2.2 3 10
65536 -
*i 172.16.1.0/24 100 None
192.0.2.3 1 10
65536 -
-------------------------------------------------------------------------------
Routes : 2
===============================================================================
The routing table then is as follows:
*A:P-4# show router route-table protocol bgp
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
172.16.1.0/24 Remote BGP 00h17m19s 170
192.168.24.1 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
With the backup-path command, again both BGP routes are valid; the first route is the best path, and now the second route is explicitly marked to be a backup path (indicated by "b"), as follows:
*A:P-4# show router bgp routes 172.16.1.0/24
===============================================================================
BGP Router ID:192.0.2.4 AS:65537 Local AS:65537
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 172.16.1.0/24 150 None
192.0.2.2 3 10
65536 -
ub*i 172.16.1.0/24 100 None
192.0.2.3 1 10
65536 -
-------------------------------------------------------------------------------
Routes : 2
===============================================================================
Now the routing table is as follows. The "B" flag indicates that a BGP backup path is available.
*A:P-4# show router route-table protocol bgp
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
172.16.1.0/24 [B] Remote BGP 00h00m16s 170
192.168.24.1 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
To show both routes, use the following command:
*A:P-4# show router route-table protocol bgp alternative
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
Alt-NextHop Alt-
Metric
-------------------------------------------------------------------------------
172.16.1.0/24 Remote BGP 00h01m36s 170
192.168.24.1 0
172.16.1.0/24 (Backup) Remote BGP 00h01m36s 170
192.168.34.1 0
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
Backup = BGP backup route
LFA = Loop-Free Alternate nexthop
S = Sticky ECMP requested
===============================================================================
The currently active next-hop in the forwarding path is 192.168.24.1, as follows:
*A:P-4# show router fib 1 172.16.1.0/24 all
===============================================================================
FIB Display
===============================================================================
Prefix [Flags] Protocol Installed
NextHop
-------------------------------------------------------------------------------
172.16.1.0/24 BGP Y
192.168.24.1 (int-P-4-P-2)
-------------------------------------------------------------------------------
Total Entries : 1
-------------------------------------------------------------------------------
===============================================================================
The active and standby next-hops are also programmed into the forwarding path, as follows:
*A:P-4# show router fib 1 172.16.1.0/24 extensive
===============================================================================
FIB Display (Router: Base)
===============================================================================
Dest Prefix : 172.16.1.0/24
Protocol : BGP
Installed : Y
Indirect Next-Hop : 192.0.2.2
QoS : Priority=n/c, FC=n/c
Source-Class : 0
Dest-Class : 0
ECMP-Weight : 1
Resolving Next-Hop : 192.168.24.1
Interface : int-P-4-P-2
ECMP-Weight : 1
Indirect Next-Hop : 192.0.2.3
QoS : Priority=n/c, FC=n/c
Source-Class : 0
Dest-Class : 0
ECMP-Weight : 1
Backup-Path : Yes
Resolving Next-Hop : 192.168.34.1
Interface : int-P-4-P-3
ECMP-Weight : 1
===============================================================================
Total Entries : 1
===============================================================================
In summary, two paths are available out of P-4 and leading to 172.16.1.0/24 in the remote AS, but only one is installed in the forwarding plane. The active route is P-4-P-2-PE-1; the backup route is P-4-P-3-PE-1. A traceroute command confirms the active path, as follows:
*A:PE-5# traceroute no-dns 172.16.1.1 source 172.16.5.1
traceroute to 172.16.1.1 from 172.16.5.1, 30 hops max, 40 byte packets
1 192.168.45.1 0.722 ms 0.662 ms 0.646 ms
2 192.168.24.1 1.22 ms 1.21 ms 1.21 ms
3 172.16.1.1 3.09 ms 1.78 ms 1.74 ms
Faster convergence through BFD
As already described, BFD can help speed up BGP convergence, mainly when detecting network failure. In the following, BFD is enabled on the eBGP sessions, and on the IS-IS protocol.
The BFD parameters are defined at interface level, enabling BFD for an application is done in the application context. Because PE-1 only has eBFD sessions toward P-2 and P-3, it is enabled at the global BGP level, but it can also be enabled at the group or neighbor level.
# on PE-1:
configure
router
interface "int-PE-1-P-2"
address 10.0.12.1/30
port 1/1/1
bfd 100 receive 100 multiplier 3
no shutdown
exit
interface "int-PE-1-P-3"
address 10.0.13.1/30
port 1/1/2
bfd 100 receive 100 multiplier 3
no shutdown
exit
bgp
loop-detect discard-route
bfd-enable
split-horizon
backup-path ipv4
group "eBGP_AS65537"
export "AS65537_172.16.1.0/24"
peer-as 65537
neighbor 10.0.12.2
exit
neighbor 10.0.13.2
exit
exit
no shutdown
exit
Because the BFD configuration for P-2 and P-3 is very similar, it is only shown for P-2, as follows:
# for P-2:
configure
router
interface "int-P-2-P-4"
address 192.168.24.1/30
port 1/1/1
bfd 100 receive 100 multiplier 3
no shutdown
exit
interface "int-P-2-PE-1"
address 10.0.12.2/30
port 1/1/2
bfd 100 receive 100 multiplier 3
no shutdown
exit
interface "system"
address 192.0.2.2/32
no shutdown
exit
BFD is enabled for group eBGP_AS65536 only, at group level, as follows:
# on P-2:
configure
router
bgp
loop-detect discard-route
advertise-inactive
split-horizon
group "eBGP_AS65536"
import "Import_LP150"
peer-as 65536
bfd-enable
neighbor 10.0.12.1
exit
exit
group "iBGP_AS65537"
next-hop-self
peer-as 65537
add-paths
ipv4 send 2 receive
exit
neighbor 192.0.2.6
exit
exit
exit
BFD for IS-IS is enabled at the IS-IS interface level, and is enabled for IPv4 only, as follows.
# on P-2:
configure
router
isis 0
area-id 49.0001
interface "system"
no shutdown
exit
interface "int-P-2-P-4"
interface-type point-to-point
bfd-enable ipv4
no shutdown
exit
no shutdown
exit
Faster convergence through MRAI
Adjusting the BGP MRAI also can help speed up network convergence, using the following command:
configure router bgp min-route-advertisement
- min-route-advertisement <seconds>
- no min-route-advertisement
<seconds> : [1..255]
Lowering the MRAI puts a higher load on the CPM, so a trade-off must be made between convergence time and processing load.
Switchover
To demonstrate a switchover scenario, a failure is introduced by disabling port 1/1/1 on PE-1, as follows:
# on PE-1:
configure
port 1/1/1
shutdown
The path through the network is PE-5-P-4-P-3-PE-1, as follows:
*A:PE-5# traceroute no-dns 172.16.1.1 source 172.16.5.1
traceroute to 172.16.1.1 from 172.16.5.1, 30 hops max, 40 byte packets
1 192.168.45.1 0.698 ms 0.695 ms 0.698 ms
2 192.168.34.1 1.21 ms 1.21 ms 1.15 ms
3 172.16.1.1 1.73 ms 1.71 ms 1.70 ms
On P-4, traffic is now diverted to P-3, and the BGP routes are as follows:
*A:P-4# show router bgp routes 172.16.1.0/24
===============================================================================
BGP Router ID:192.0.2.4 AS:65537 Local AS:65537
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 172.16.1.0/24 100 None
192.0.2.3 1 10
65536 -
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The route table is as follows:
*A:P-4# show router route-table protocol bgp
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
172.16.1.0/24 Remote BGP 00h01m41s 170
192.168.34.1 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The forwarding plane is reprogrammed to send traffic for the 172.16.1.0/24 subnet to P-3, as follows:
*A:P-4# show router fib 1 172.16.1.0/24 extensive
===============================================================================
FIB Display (Router: Base)
===============================================================================
Dest Prefix : 172.16.1.0/24
Protocol : BGP
Installed : Y
Indirect Next-Hop : 192.0.2.3
QoS : Priority=n/c, FC=n/c
Source-Class : 0
Dest-Class : 0
ECMP-Weight : 1
Resolving Next-Hop : 192.168.34.1
Interface : int-P-4-P-3
ECMP-Weight : 1
===============================================================================
Total Entries : 1
===============================================================================
Bringing port 1/1/1 on PE-1 up again will result in the path PE-5-P-4-P-2-PE-1 being reactivated. Switchback takes longer, because the external BGP session needs to be re-established, and routes have to be relearned.
Conclusion
BGP FRR provides ISPs the means to offer backup paths with fast switchover times when used in combination with short failure detection times and short advertisement intervals. By guaranteeing service in case of network failures, ISPs can provide enhanced service offerings to their customers.