BGP Multipath
This chapter provides information about BGP Multipath.
Topics in this chapter include:
Applicability
This chapter was initially written for SR OS Release 14.0.R4, but the CLI in the current edition is based on SR OS Release 21.10.R1. Configurable BGP multipath parameters per address family and selective BGP multipath is supported in SR OS Release 19.5.R1, and later.
Overview
When BGP multipath is enabled, traffic can be forwarded to an IP prefix destination over multiple BGP paths that are considered equal by the BGP decision process. BGP multipath is supported in base router and VPRNs, both for iBGP and eBGP. The multi-path command specifies the maximum number of BGP paths that each BGP RIB can submit to the route table for an IP prefix. The equal cost multipath (ECMP) limit defines how many paths are selected for installation in the forwarding information base (FIB). Traffic in the data path that matches the IP prefix is load-balanced across the ECMP next hops on a per-packet hash calculation.
As described in chapter Separate BGP RIBs for Labeled Routes, labeled routes and unlabeled routes do not mix.
BGP multipath can be configured as follows:
The multi-path commands present in the base router and VPRN bgp contexts are configurable on a global level or more specific, within an address family context (ipv4, ipv6, label-ipv4, and label-ipv6). Following parameters are possible:
*A:PE-5# configure router bgp multi-path ? [no] ipv4 - Configure ipv4 multi-path maximum-paths [no] ipv6 - Configure ipv6 multi-path maximum-paths [no] label-ipv4 - Configure label-ipv4 multi-path maximum-paths [no] label-ipv6 - Configure label-ipv6 multi-path maximum-paths [no] maximum-paths - Configure multi-path maximum-pathsmaximum-paths | ipv4 | ipv6 | label-ipv4 | label-ipv6 max-paths [ebgp ebgp-max-paths] [ibgp ibgp-max-paths] [restrict {same-neighbor-as | exact-as-path}] [unequal-cost]
*A:PE-5# configure router bgp multi-path maximum-paths ? - maximum-paths <max-paths> [ebgp <ebgp-max-paths>] [ibgp <ibgp-max-paths>] [restrict {same-neighbor-as|exact-as-path}] [unequal-cost] - no maximum-paths <max-paths> : [1..64] <ebgp-max-paths> : [1..64] <ibgp-max-paths> : [1..64]multi-path configuration per address family (ipv4 | ipv6 | label-ipv4 | label-ipv6) overrules the generic maximum-paths configuration.
max-paths is the default maximum number of paths. It is overruled by ebgp-max-paths and ibgp-max-paths. However, if there is no maximum set for the number of eBGP paths or iBGP paths, then the maximum number of paths is set by max-paths.
ebgp-max-paths specifies the maximum number of paths that can be used when the best path is eBGP. If configured, ebgp-max-paths overrides the configured max-paths for eBGP paths.
ibgp-max-paths specifies the maximum number of paths that can be used when the best path is iBGP. If configured, ibgp-max-paths overrides the configured max-paths for iBGP paths.
restrict same-neighbor-as forces multipaths to have the same (shortest) AS path length (unless as-path-ignore is configured) and, for the paths with that length, the same neighbor AS.
restrict exact-as-path forces multipaths to have the exact same AS paths.
unequal-cost allows to use routes with different next-hop costs in multipath ECMP sets.
The ebgp-ibgp-equal command is added to the best-path-selection contexts in base router and VPRN bgp contexts. When this command is configured, as follows, the BGP decision process skips the step that prefers eBGP over iBGP. This enables load-balancing between eBGP and iBGP paths.
*A:PE-5# configure router bgp best-path-selection ? - best-path-selection [no] always-compare* - Determine how the Multi-Exit Discriminator (MED) path attribute is used in the BGP route selection process [no] as-path-ignore - Determine whether the AS Path is used in determining the best BGP route [no] compare-origin* - Enable/Disable compare validation state [no] d-path-length-* - Enable/disable d-path-length-ignore [no] deterministic-* - Enable/Disable deterministic Multi-Exit Discriminator [no] ebgp-ibgp-equal - Determine whether EBGP and IBGP learned paths are considered equal [no] ignore-nh-metr* - Enable/Disable ignore next-hop metric [no] ignore-router-* - Enable/Disable ignore router-id [no] origin-invalid* - Enable/Disable origin invalid unusable routes.
The ebgp-ibgp-equal command is not to be confused with the eibgp-loadbalance command in a VPRN, that is used to provide ECMP over BGP-VPN (imported routes) and BGP routes. It is called eibgp-loadbalance because, in such scenarios, BGP-VPN is typically used between iBGP peers and BGP is used between eBGP peers. However, this is not always the case, so the name can be misleading.
Entire BGP groups or a selection of BGP neighbors can be configured as multipath-eligible. If a route is learned for an IPv4, IPv6, label-IPv4, or label-IPv6 prefix, and the associated maximum number of paths is N (which can depend on the address family and whether the best path was received from an eBGP or iBGP peer), then one of the following three rules applies:
If the best path came from a neighbor marked as multipath-eligible, then only paths marked as multipath-eligible are candidates for the BGP multipath and the best N are chosen for installation as ECMP next-hops.
If none of the paths from the set of all possible multipaths came from a neighbor marked as multipath-eligible, the best N are chosen.
If the best path did not come from a neighbor marked as multipath-eligible and at least one path from the set of all possible multipaths came from a multipath-eligible peer, then only the best path is chosen and all other paths are eliminated.
Configuration
The examples in this section show the multipath BGP configuration in the base router. For BGP multipath in a VPRN, the configuration is similar.
Example topology shows the example configuration with the used IP addresses. PE-5 has eBGP sessions with CEs in different autonomous systems (ASs) and iBGP sessions with PE-6, PE-7, PE-8, and PE-9.
The initial configuration includes:
Cards, MDAs, ports
Router interfaces
IS-IS in AS 64500
LDP in AS 64500
BGP on all nodes (eBGP between CEs and PE-5; iBGP between PEs)
Export policy "export-bgp" accepting routes from protocol direct on all nodes
The BGP configuration on CE-1 is as follows:
# on CE-1:
configure
router
autonomous-system 64501
bgp
split-horizon
group "eBGP"
export "export-bgp"
peer-as 64500
neighbor 172.16.15.2
exit
exit
The BGP configuration on the other nodes that advertise routes to PE-5 is similar.
The BGP configuration on PE-5 is as follows:
# on PE-5:
configure
router
autonomous-system 64500
bgp
split-horizon
group "eBGP"
neighbor 172.16.15.1
peer-as 64501
exit
neighbor 172.16.25.1
peer-as 64502
exit
neighbor 172.16.35.1
peer-as 64503
exit
neighbor 172.16.45.1
peer-as 64504
exit
exit
group "iBGP"
peer-as 64500
neighbor 192.0.2.6
exit
neighbor 192.0.2.7
exit
neighbor 192.0.2.8
exit
neighbor 192.0.2.9
exit
exit
exit
The following will be configured and verified:
BGP multipath with different eBGP and iBGP path limits
BGP multipath with equal eBGP and iBGP path treatment
BGP multipath restricted to the same neighbor AS
BGP multipath restricted to the exact AS path
BGP multipath per address family
Selective BGP multipath
BGP multipath with different eBGP and iBGP path limits
On PE-5, BGP multipath is configured as follows:
# on PE-5:
configure
router
bgp
multi-path
maximum-paths 8 ebgp 2 ibgp 3
exit
It is mandatory to specify a maximum for BGP multipaths, as follows, but that is overruled by the individual limits for eBGP and iBGP. It is optional to configure limits for eBGP and iBGP.
*A:PE-5# configure router bgp multi-path maximum-paths ebgp 2 ibgp 3
^
Error: Missing parameter
It is allowed to specify a lower value for maximum-paths than for either eBGP or iBGP because the configured number of paths for eBGP and iBGP overrule the maximum-paths limitation, as follows:
# on PE-5:
configure
router
bgp
multi-path
maximum-paths 1 ebgp 2 ibgp 3
exit
With this configuration, regardless of the value of maximum-paths, there can be two eBGP routes for the same prefix and three iBGP routes for the same prefix. If the best route is eBGP, the ebgp-max-paths value is 2; if the best route is iBGP, the ibgp-max-paths value is 3. The value for maximum-paths (1) is never used when limits for both eBGP and iBGP are configured.
# on PE-5:
configure
router
bgp
multi-path
maximum-paths 3 ebgp 2
exit
With this configuration, there can be two eBGP routes for the same prefix and three iBGP routes for the same prefix. If the best route is eBGP, the ebgp-max-paths value is 2, and if the best route is iBGP, the max-paths value is 3.
In the following example, all four eBGP neighbors advertise prefix 3.1.0.0/32 to PE-5 and all four iBGP neighbors advertise prefix 3.2.0.0/32 to PE-5. PE-5 receives four eBGP routes for prefix 3.1.0.0/32, but only two are added to the common IP route table, as shown in BGP multipath with eBGP limit 2.
These routes can only be added to the FIB if ECMP is configured to a value at least equal to the number of routes allowed in BGP multipath. By default, ECMP is disabled and only one route is added to the FIB, as shown in eBGP multipath with limit 2 and ECMP disabled.
With ECMP disabled, only one of the four paths is used for prefix 3.1.0.0/32, as follows:
*A:PE-5# show router bgp routes 3.1.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.1.0.0/32 None None
172.16.15.1 None 0
64501 -
*i 3.1.0.0/32 None None
172.16.25.1 None 0
64502 -
*i 3.1.0.0/32 None None
172.16.35.1 None 0
64503 -
*i 3.1.0.0/32 None None
172.16.45.1 None 0
64504 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
In the remainder of the chapter, ECMP is configured with a value of eight, implying that the routes added to the common IP route table will be added to the FIB as well. ECMP is configured on PE-5 as follows:
# on PE-5:
configure
router
ecmp 8
exit
With ECMP configured with a limit of eight, two eBGP paths are used for prefix 3.1.0.0/32.
The first two of the following BGP routes for prefix 3.1.0.0/32 are used on PE-5:
*A:PE-5# show router bgp routes 3.1.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.1.0.0/32 None None
172.16.15.1 None 0
64501 -
u*>i 3.1.0.0/32 None None
172.16.25.1 None 0
64502 -
*>i 3.1.0.0/32 None None
172.16.35.1 None 0
64503 -
*>i 3.1.0.0/32 None None
172.16.45.1 None 0
64504 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
The four iBGP neighbors of PE-5 advertise prefix 3.2.0.0/32 to PE-5. BGP multipath has a limit of three for iBGP routes. Consequently, three BGP routes are added to the common IP route table and to the FIB, as shown in BGP multipath with iBGP limit 3 and ECMP limit 8.
Three iBGP paths are used for prefix 3.2.0.0/32, as follows:
*A:PE-5# show router bgp routes 3.2.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.2.0.0/32 100 None
192.0.2.6 None 10
6 -
u*>i 3.2.0.0/32 100 None
192.0.2.7 None 10
7 -
u*>i 3.2.0.0/32 100 None
192.0.2.8 None 10
8 -
*>i 3.2.0.0/32 100 None
192.0.2.9 None 10
9 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
BGP multipath with equal eBGP and iBGP path treatment
It is optional to specify limits for eBGP and iBGP; an overall multipath limitation is sufficient, such as:
# on PE-5:
configure
router
bgp
multi-path
maximum-paths 6
exit
With this configuration, there can be six routes for the same prefix. These routes can be eBGP or iBGP routes. By default, eBGP routes are preferred and, therefore, only the four eBGP routes are imported in the common IP route table, as shown in BGP multipath with limit 6 and eBGP preferred.
Only the four eBGP paths are used, as follows:
*A:PE-5# show router bgp routes 3.3.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.3.0.0/32 None None
172.16.15.1 None 0
64501 -
u*>i 3.3.0.0/32 None None
172.16.25.1 None 0
64502 -
u*>i 3.3.0.0/32 None None
172.16.35.1 None 0
64503 -
u*>i 3.3.0.0/32 None None
172.16.45.1 None 0
64504 -
*i 3.3.0.0/32 100 None
192.0.2.6 None 10
6 -
*i 3.3.0.0/32 100 None
192.0.2.7 None 10
7 -
*i 3.3.0.0/32 100 None
192.0.2.8 None 10
8 -
*i 3.3.0.0/32 100 None
192.0.2.9 None 10
9 -
-------------------------------------------------------------------------------
Routes : 8
===============================================================================
The BGP decision process prefers eBGP over iBGP, but this step can be skipped by configuring the following:
# on PE-5:
configure
router
bgp
best-path-selection
ebgp-ibgp-equal ipv4
exit
This configuration only skips one step in the BGP decision process. If the best route is still eBGP, the eBGP multipath limit applies; if the best route is iBGP, the iBGP multipath limit applies.
Optionally, other best path selection criteria can also be configured, such as ignore-nh-metric. However, the multi-path configuration can also be configured with the unequal-cost option. This allows to ignore the next-hop cost in BGP multipath ECMP sets, while preserving the next-hop option in the BGP decision process.
# on PE-5:
configure
router
bgp
multi-path
maximum-paths 6 unequal-cost
exit
When all other path options are identical (such as local preference, MED, IGP cost, and other criteria from the BGP decision process), or when the best-path-selection is configured to ignore specific path options, and the only differentiator is an originator ID, the remaining steps in the BGP decision process do not exclude any routes. In that case, six of the eight eligible BGP paths are included in the BGP multipath, as shown in BGP multipath with limit 6, eBGP equal to iBGP, and other path options identical.
From the eight advertised BGP routes for prefix 3.3.0.0/32, six paths are used, as follows:
*A:PE-5# show router bgp routes 3.3.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.3.0.0/32 None None
172.16.15.1 None 0
64501 -
u*>i 3.3.0.0/32 None None
172.16.25.1 None 0
64502 -
u*>i 3.3.0.0/32 None None
172.16.35.1 None 0
64503 -
u*>i 3.3.0.0/32 None None
172.16.45.1 None 0
64504 -
u*>i 3.3.0.0/32 100 None
192.0.2.6 None 10
6 -
u*>i 3.3.0.0/32 100 None
192.0.2.7 None 10
7 -
*>i 3.3.0.0/32 100 None
192.0.2.8 None 10
8 -
*>i 3.3.0.0/32 100 None
192.0.2.9 None 10
9 -
-------------------------------------------------------------------------------
Routes : 8
===============================================================================
BGP multipath restricted to the same neighbor AS
BGP multipath can be configured with the restriction that the neighbor AS must be the same for all the used paths. When all routes have a different neighbor AS, only one path is used. This can be shown for prefix 3.2.0.0/32 that is advertised by the iBGP neighbors. The BGP multipath configuration on PE-5 is as follows:
# on PE-5:
configure
router
bgp
multi-path
maximum-paths 8 ebgp 2 ibgp 3 restrict same-neighbor-as
exit
BGP multipath configured with restriction to the same neighbor AS shows that with the restriction to the same neighbor AS, only one path is used because all BGP routes have a different neighbor AS.
Only one BGP path is used, because all the other routes have a different neighbor AS, as follows:
*A:PE-5# show router bgp routes 3.2.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.2.0.0/32 100 None
192.0.2.6 None 10
6 -
*>i 3.2.0.0/32 100 None
192.0.2.7 None 10
7 -
*>i 3.2.0.0/32 100 None
192.0.2.8 None 10
8 -
*>i 3.2.0.0/32 100 None
192.0.2.9 None 10
9 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
BGP multipath restricted to the same neighbor AS: AS paths with same length shows that the iBGP neighbors also advertise prefix 3.4.0.0/32 with a different AS path, but the AS path is equally long and the neighbor AS is the same. Three of these BGP paths are used.
All iBGP neighbors have the same neighbor AS and an AS path of equal length. Three of the iBGP paths for prefix 3.4.0.0/32 are used, as follows:
*A:PE-5# show router bgp routes 3.4.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.4.0.0/32 100 None
192.0.2.6 None 10
4 6 -
u*>i 3.4.0.0/32 100 None
192.0.2.7 None 10
4 7 -
u*>i 3.4.0.0/32 100 None
192.0.2.8 None 10
4 8 -
*>i 3.4.0.0/32 100 None
192.0.2.9 None 10
4 9 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
The restriction that the neighbor AS must be the same does not overrule the BGP selection criterion that the shorter AS path is preferred. When the AS path is longer for the routes advertised by neighbors 192.0.2.8 and 192.0.2.9, only the BGP paths with the shorter AS path are used, as shown in BGP multipath restricted to the same neighbor AS: AS paths of different lengths.
All BGP routes advertised by the iBGP neighbors have the same neighbor AS, but the AS path is longer for neighbors 192.0.2.8 and 192.0.2.9. The routes advertised by these neighbors will not be selected as best path and will not be added to the route table. Only the two BGP routes with the shorter AS path are used, as follows:
*A:PE-5# show router bgp routes 3.4.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.4.0.0/32 100 None
192.0.2.6 None 10
4 6 -
u*>i 3.4.0.0/32 100 None
192.0.2.7 None 10
4 7 -
*i 3.4.0.0/32 100 None
192.0.2.8 None 10
4 1 8 -
*i 3.4.0.0/32 100 None
192.0.2.9 None 10
4 1 9 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
When the best path selection is configured to ignore the AS path, three paths are used again, as shown in BGP multipath restricted to the same neighbor AS: AS paths of different lengths, AS path ignored.
The best path selection is reconfigured as follows:
# on PE-5:
configure
router
bgp
best-path-selection
as-path-ignore ipv4
exit
Three of the four eligible BGP routes are used, as follows:
*A:PE-5# show router bgp routes 3.4.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.4.0.0/32 100 None
192.0.2.6 None 10
4 6 -
u*>i 3.4.0.0/32 100 None
192.0.2.7 None 10
4 7 -
u*>i 3.4.0.0/32 100 None
192.0.2.8 None 10
4 1 8 -
*>i 3.4.0.0/32 100 None
192.0.2.9 None 10
4 1 9 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
The best selection path settings are restored as follows:
# on PE-5:
configure
router
bgp
best-path-selection
no as-path-ignore
exit
BGP multipath restricted to the exact AS path
The BGP multipath configuration on PE-5 restricts BGP to only use identical AS paths, as follows:
# on PE-5:
configure
router
bgp
multi-path
maximum-paths 8 ebgp 2 ibgp 3 restrict exact-as-path
exit
The four iBGP neighbors advertise prefixes 3.5.0.0/32 and 3.6.0.0/32 to PE-5, see BGP multipath restricted to exact same AS. All AS paths are different. and BGP multipath restricted to exact same AS. All AS paths are identical. The AS paths for prefix 3.5.0.0/32 are not identical, but the neighbor AS is the same, and the AS path is of equal length. The AS paths for prefix 3.6.0.0/32 are identical.
The BGP multipath configuration specifies that the AS paths must be identical, which is not the case for the received BGP routes for prefix 3.5.0.0/32. Only one BGP route is imported in the route table, as shown in BGP multipath restricted to exact same AS. All AS paths are different..
All the BGP routes for prefix 3.5.0.0/32 have a different AS path. Only the BGP route advertised by neighbor 192.0.2.6 is used, as follows:
*A:PE-5# show router bgp routes 3.5.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.5.0.0/32 100 None
192.0.2.6 None 10
5 6 -
*>i 3.5.0.0/32 100 None
192.0.2.7 None 10
5 7 -
*>i 3.5.0.0/32 100 None
192.0.2.8 None 10
5 8 -
*>i 3.5.0.0/32 100 None
192.0.2.9 None 10
5 9 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
However, all the received BGP routes for prefix 3.6.0.0/32 have the same AS path. Three of these BGP paths are used, as shown in BGP multipath restricted to exact same AS. All AS paths are identical.
Three of the four received BGP routes for prefix 3.6.0.0/32 are used, as follows:
*A:PE-5# show router bgp routes 3.6.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.6.0.0/32 100 None
192.0.2.6 None 10
1 3 -
u*>i 3.6.0.0/32 100 None
192.0.2.7 None 10
1 3 -
u*>i 3.6.0.0/32 100 None
192.0.2.8 None 10
1 3 -
*>i 3.6.0.0/32 100 None
192.0.2.9 None 10
1 3 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
BGP multipath per address family
On PE-6, PE-7, PE-8, and PE-9, the address families IPv4, label-IPv4, and label-IPv6 are configured in the context of iBGP neighbor 192.0.2.5. Prefix 3.7.0.0/32 is exported as IPv4 route, whereas prefix 3.8.0.0/32 is exported as label-IPv4 route, and prefix 2001:db8::3:8:0:0/32 as label-IPv6 route.
On PE-5, the address families IPv4, label-IPv4, and label-IPv6 are configured in the context of the ‟iBGP” group, each with a different max-paths setting: maximum two IPv4 paths, maximum three label-IPv4 paths, and maximum four label-IPv6 paths:
# on PE-5:
configure
router Base
bgp
multi-path
ipv4 2 ibgp 2
label-ipv4 3 ibgp 3
label-ipv6 4 ibgp 4
no maximum-paths
exit
group "iBGP"
family ipv4 label-ipv4 label-ipv6
exit
In this example, only iBGP routes are received. Two of the four received IPv4 routes for prefix 3.7.0.0/32 are used:
*A:PE-5# show router bgp routes 3.7.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.7.0.0/32 100 None
192.0.2.6 None 10
7 -
u*>i 3.7.0.0/32 100 None
192.0.2.7 None 10
7 -
*>i 3.7.0.0/32 100 None
192.0.2.8 None 10
7 -
*>i 3.7.0.0/32 100 None
192.0.2.9 None 10
7 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
The last two IPv4 routes from PE-8 and PE-9 are not used because the maximum number of IPv4 iBGP paths (2) is exceeded:
*A:PE-5# show router bgp routes 3.7.0.0/32 hunt | match "MP Exc. Reason"
TieBreakReason : PeerRouterID MP Exc. Reason : MaxPathsExceeded
TieBreakReason : PeerRouterID MP Exc. Reason : MaxPathsExceeded
BGP multipath for the IPv4 address family shows that two of the four received IPv4 routes are used.
Three of the four received label-IPv4 routes for prefix 3.8.0.0/32 are used:
*A:PE-5# show router bgp routes 3.8.0.0/32 label-ipv4
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP LABEL-IPV4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.8.0.0/32 100 None
192.0.2.6 None 10
8 524282
u*>i 3.8.0.0/32 100 None
192.0.2.7 None 10
8 524282
u*>i 3.8.0.0/32 100 None
192.0.2.8 None 10
8 524282
*>i 3.8.0.0/32 100 None
192.0.2.9 None 10
8 524282
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
The last label-IPv4 route from PE-9 is not used because the maximum number of label-IPv4 paths (3) is exceeded:
*A:PE-5# show router bgp routes 3.8.0.0/32 label-ipv4 hunt | match "MP Exc. Reason"
TieBreakReason : PeerRouterID MP Exc. Reason : MaxPathsExceeded
BGP multipath for the label-IPv4 address family shows that three of the received label-IPv4 routes are used.
All four received label-IPv6 routes for prefix 2001:db8::3:8:0:0/128 are used:
*A:PE-5# show router bgp routes 2001:db8::3:8:0:0/128 label-ipv6
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP LABEL-IPV6 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 2001:db8::3:8:0:0/128 100 None
::ffff:192.0.2.6 None 10
8 2
u*>i 2001:db8::3:8:0:0/128 100 None
::ffff:192.0.2.7 None 10
8 2
u*>i 2001:db8::3:8:0:0/128 100 None
::ffff:192.0.2.8 None 10
8 2
u*>i 2001:db8::3:8:0:0/128 100 None
::ffff:192.0.2.9 None 10
8 2
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
BGP multipath for the label-IPv6 address family shows that all four received label-IPv6 routes are used.
Selective BGP multipath
Entire BGP groups or a selection of BGP neighbors can be configured as multipath-eligible. In all preceding examples, all BGP groups and BGP neighbors are—by default—marked as ‛not multipath-eligible’ (no multipath-eligible). In a scenario where all paths originate from neighbors that are not marked as multipath-eligible, the N best routes are chosen.
For prefixes 3.7.0.0/32, 3.8.0.0/32, and 2001:db8::3:8:0:0/128, the best path is the path originating from neighbor 192.0.2.6, based on the (lowest) router ID.
In the following example, only neighbors 192.0.2.7 and 192.0.2.8 are configured as multipath-eligible:
# on PE-5:
configure
router
bgp
multi-path
ipv4 2 ibgp 2
label-ipv4 3 ibgp 3
label-ipv6 4 ibgp 4
no maximum-paths
exit
group "iBGP"
neighbor 192.0.2.6
no multipath-eligible # default
exit
neighbor 192.0.2.7
multipath-eligible
exit
neighbor 192.0.2.8
multipath-eligible
exit
neighbor 192.0.2.9
no multipath-eligible # default
exit
exit
When the best path originates from a neighbor that is configured as no multipath-eligible (default), while at least one path originates from a neighbor that is marked as multipath-eligible, only the best path is used (no multipath in this scenario):
*A:PE-5# show router bgp routes 3.7.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.7.0.0/32 100 None
192.0.2.6 None 10
7 -
*>i 3.7.0.0/32 100 None
192.0.2.7 None 10
7 -
*>i 3.7.0.0/32 100 None
192.0.2.8 None 10
7 -
*>i 3.7.0.0/32 100 None
192.0.2.9 None 10
7 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
The routes originating from PE-7, PE-8, and PE-9 are not used because the best BGP path toward 3.7.0.0/32 originates from PE-6, which is not multipath-eligible:
*A:PE-5# show router bgp routes 3.7.0.0/32 hunt | match "MP Exc. Reason"
TieBreakReason : PeerRouterID MP Exc. Reason : NotMultipathEligible
TieBreakReason : PeerRouterID MP Exc. Reason : NotMultipathEligible
TieBreakReason : PeerRouterID MP Exc. Reason : NotMultipathEligible
Best IPv4 path originates from a non-multipath-eligible BGP neighbor shows that only the best IPv4 route is used when the best path originates from a non-multipath-eligible BGP neighbor.
Also, for the label-IPv4 and label-IPv6 routes, only the best path is used and the other routes are not included in the multipath because the best path is not multipath-eligible.
In the following example, BGP neighbors 192.0.2.6, 192.0.2.8, and 192.0.2.9 are configured as multipath-eligible:
# on PE-5:
configure
router
bgp
multi-path
ipv4 2 ibgp 2
label-ipv4 3 ibgp 3
label-ipv6 4 ibgp 4
no maximum-paths
exit
group "iBGP"
neighbor 192.0.2.6
multipath-eligible
exit
neighbor 192.0.2.7
no multipath-eligible # default
exit
neighbor 192.0.2.8
multipath-eligible
exit
neighbor 192.0.2.9
multipath-eligible
exit
exit
The best path originates from neighbor 192.0.2.6 that is marked as multipath-eligible. In this case, only paths marked as multipath-eligible are candidates for the BGP multipath algorithm and the best N multipath-eligible routes will be chosen (if available): two IPv4 paths, three label-IPv4 paths, and four label-IPv6 paths.
On PE-5, two IPv4 routes are used for prefix 3.7.0.0/32: the best path from neighbor 192.0.2.6 and a path from neighbor 192.0.2.8:
*A:PE-5# show router bgp routes 3.7.0.0/32
===============================================================================
BGP Router ID:192.0.2.5 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 3.7.0.0/32 100 None
192.0.2.6 None 10
7 -
*>i 3.7.0.0/32 100 None
192.0.2.7 None 10
7 -
u*>i 3.7.0.0/32 100 None
192.0.2.8 None 10
7 -
*>i 3.7.0.0/32 100 None
192.0.2.9 None 10
7 -
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
IPv4 route from neighbor 192.0.2.7 is not used because it is not multipath-eligible; IPv4 route from neighbor 192.0.2.9 is not used because the maximum number of IPv4 paths is exceeded, as follows:
*A:PE-5# show router bgp routes 3.7.0.0/32 hunt | match "MP Exc. Reason"
TieBreakReason : PeerRouterID MP Exc. Reason : NotMultipathEligible
TieBreakReason : PeerRouterID MP Exc. Reason : MaxPathsExceeded
Two IPv4 paths from multipath-eligible BGP peers are used shows that two IPv4 routes from multipath-eligible peers are used: the best path originating from PE-6 and the second best path originating from PE-8.
Conclusion
BGP multipath allows the IP routing table to have multiple BGP paths to the same destination. Different path limits can be applied for eBGP and iBGP paths and per address family. It is possible to treat eBGP and iBGP routes as equal. Restrictions can be imposed related to AS path. Specific BGP neighbors or entire BGP groups can be marked as multipath-eligible, resulting in selective BGP multipath behavior.