IS-IS Link Bundling
This chapter provides information about IS-IS link bundling.
Topics in this chapter include:
Applicability
The chapter was initially written for SR OS Release 11.0.R6. However, the CLI in the current edition is based on SR OS Release 20.7.R2.
Overview
Intermediate System to Intermediate System (IS-IS) Link Bundling allows for the grouping of a number of IS-IS interfaces into a single virtual link, called an IS-IS link group. It is used in conjunction with Equal Cost Multipath (ECMP) to dynamically change the metric of parallel IS-IS links if one or more links fail or suffer some sort of performance degradation.
Consider the network in Link bundle schematic, where a Provider Edge router PE-1 connects to a core network comprised of two Provider (P) routers and a single Autonomous System Border Router (ASBR). The links between PE-1 and P-3, and PE-1 and P-4 are 10 Gigabit Ethernet links. The links between ASBR-2 and P-3 and P-4 are both 100 Gigabit links. The link metrics are as shown in Link bundle schematic.
In order to maximize the use of link bandwidth, ECMP is enabled on all routers and set to a value of 2, so that IP traffic flowing between PE1 and P-3, and PE-1 and P-4, is load balanced across the two links.
A default route is injected into the ASBR-2 router and re-distributed via a policy statement into IS-IS, so that traffic flowing from PE-1 to the ASBR is resolved by this route. Traffic flows between PE-1 and ASBR-2, using the path with the lowest IS-IS metric, via P-3 with a metric of 11. The second path PE-1 to ASBR-2 via P-4 has the same bandwidth, but a higher IS-IS metric of 31.
Traffic in the reverse direction flows toward a user subnet described by a static route configured on PE-1, which is redistributed into IS-IS using a policy statement. Once again, the shortest path between ASBR-2 and PE-1 is via P-3, so the bi-directional traffic flow is symmetric.
If one of the links between PE-1 and P-3 fails, traffic still flows via P-3, because the IS-IS metric is unchanged, but this now has less bandwidth than the second path via P-4. It is desirable to make use of the additional bandwidth of the second path, but this requires a change in metric. This can be achieved using IS-IS link bundling.
IS-IS link bundling allows for the creation of a group of IS-IS links, where the failure of a member link allows the metric of the remaining members of the link group to be increased by an offset value.
Using Effect of single link failure on bundle group as an example, the links between PE-1 and P-3 are included in a bundle group. To illustrate the change in metrics, a default static route is configured on ASBR-2 and re-distributed into IS-IS, and the path to this route is monitored at PE-1. Similarly, a static route to subnet 172.16.0.0/16 is configured on PE-1 and redistributed into IS-IS and viewed on ASBR-2.
Should one of the links between PE-1 and P-3 fail, the metric of the remaining members can be increased by an offset, for example 90, so that the metric of the remaining link becomes 10+90 = 100. The IS-IS metric between PE-1 and ASBR-2 via P-3 is now 101. The metric offset is applied to each remaining IS-IS interface individually and is advertised within the IS-IS database as the default cost in the TE-IS neighbors Type Length Variable (TLV).
The path between PE-1 and ASBR-2 via P-4 now has the lowest IS-IS metric, and any affected routers within the IS-IS area will try and re-route the traffic based on the new metric.
The fundamentals of this feature are:
The treatment of all member links in a link group bundle as a single virtual interface.
The increase in metric by a specific offset value of each remaining individual link within the group when a failure of one or more links occurs.
The application of the offset occurs when the number of active links drops below a configured threshold.
The offset is removed when the number of active links within the link group bundle reaches the configured reversion threshold.
A link bundle is required on a router for the thresholds and offsets to apply.
Consider a second and subsequent failure where a link between PE-1 and P-4 also fails, so that there is only one active IS-IS interface between PE-1 and each of its neighboring P routers. This is shown in Double link failure .
In this case, the metric for the remaining link between PE-1 and P-4 can be increased by an offset value of +70 so that the IS-IS metric PE-1 to P-4 becomes 100, the same as that between PE-1 and P-3 when a link has failed.
PE-1 now sees two equal cost paths to the default route – one via P-3 and one via P-4, so there are still two 10Gigabit Ethernet links across which the traffic can be load shared.
This can be summarized using the following table, where ABCD are the 4 links as per Link bundle schematic and link status is Up (U) or Down (D).
ABCD Status |
A (metric,status) |
B (metric,status) |
C (metric,status) |
D (metric,status) |
|---|---|---|---|---|
UUUU |
10 Transmit |
10 Transmit |
30 Idle |
30 Idle |
UDUU |
100 Idle |
Down |
30 Transmit |
30 Transmit |
UDUD |
100 Transmit |
Down |
100 Transmit |
Down |
UUUD |
10 Transmit |
10 Transmit |
100 Idle |
Down |
Configuration
The example topology is shown in Example topology.
The PE-1 router configuration commands are as follows.
# on PE-1:
configure
router
interface "int-PE-1-P-3-1"
address 192.168.13.1/30
port 1/1/1
exit
interface "int-PE-1-P-3-2"
address 192.168.113.1/30
port 1/1/3
exit
interface "int-PE-1-P-4-1"
address 192.168.14.1/30
port 1/1/2
exit
interface "int-PE-1-P-4-2"
address 192.168.114.1/30
port 1/1/4
exit
interface "system"
address 192.0.2.1/32
exit
ecmp 2
The IP router configuration for the remaining routers can be derived from Example topology.
The IS-IS network is a level 1 network.
The IS-IS configuration for PE-1, including the interface metrics is as follows:
# on PE-1:
configure
router
isis
level-capability level-1
area-id 49.0001
advertise-passive-only
level 1
wide-metrics-only
exit
interface "system"
passive
exit
interface "int-PE-1-P-3-1"
interface-type point-to-point
level 1
metric 10
exit
exit
interface "int-PE-1-P-3-2"
interface-type point-to-point
level 1
metric 10
exit
exit
interface "int-PE-1-P-4-1"
interface-type point-to-point
level 1
metric 30
exit
exit
interface "int-PE-1-P-4-2"
interface-type point-to-point
level 1
metric 30
exit
exit
no shutdown
The IS-IS configuration for the remaining routers can be derived from Example topology.
The following configuration is for the static route and export policy on ASBR-2. The configuration of the static route on PE-1 is similar.
# on ASBR-2:
configure
router
static-route-entry 0.0.0.0/0
black-hole
no shutdown
exit
# on PE-1, ASBR-2:
configure
router
policy-options
begin
policy-statement "STATIC-ISIS"
entry 10
from
protocol static
exit
to
level 1
exit
action accept
metric set "@igp@"
exit
exit
exit
commit
exit
isis 0
export "STATIC-ISIS"
exit
Link group configuration
PE-1 contains 2 link groups. The first link group contains the IS-IS interfaces toward P-3. The second contains the interfaces toward P-4.
Each link-group is configured using a unique name, which is unique per router, and the IS-IS interface names are configured within the group as group members.
The metric offset value is the amount by which the IS-IS metric of active member links is increased when the number of links drops below a configured threshold.
The IS-IS link group configuration for PE-1 for the interfaces toward P-3 is as follows:
# on PE-1:
configure
router
isis 0
link-group "Link-Group-PE-1-P-3"
level 1
ipv4-unicast-metric-offset 90
member "int-PE-1-P-3-1"
member "int-PE-1-P-3-2"
revert-members 2
oper-members 2
exit
exit
exit
Similarly, the IS-IS link group for PE-1 for the interfaces toward P-4 is:
# on PE-1:
configure
router
isis 0
link-group "Link-Group-PE-1-P-4"
level 1
ipv4-unicast-metric-offset 70
member "int-PE-1-P-4-1"
member "int-PE-1-P-4-2"
revert-members 2
oper-members 2
exit
exit
exit
Within the link-group, two thresholds are configured:
oper-members threshold
revert-members threshold
If the number of operational links in the link-group drops below the oper-members value, then all interfaces associated with that IS-IS link group have their interface metric increased by the configured offset value. As a result, IS-IS then tries to reroute traffic over lower cost paths.
If the number of operational links in the link-group equals the revert-members threshold value, then all interfaces associated with that IS-IS link group have their interface metric decreased by the configured offset value.
In this configuration, there is a requirement to increase the metric of each interface within a link-group when a single interface fails. This means that the oper-members value is set to 2. In normal working circumstances, when both interfaces are active, the metric used is the configured interface metric. This means that the revert-members value must also be set to 2.
It is not possible to set the oper-members threshold to a value higher than that of the revert-members.
For completeness, the IS-IS configuration of the P-routers is as follows.
# on P-3:
configure
router
isis 0
level-capability level-1
area-id 49.0001
advertise-passive-only
level 1
wide-metrics-only
exit
interface "system"
passive
exit
interface "int-P-3-PE-1-1"
interface-type point-to-point
level 1
metric 10
exit
exit
interface "int-P-3-PE-1-2"
interface-type point-to-point
level 1
metric 10
exit
exit
interface "int-P-3-ASBR-2"
interface-type point-to-point
level 1
metric 1
exit
exit
link-group "Link-Group-P-3-PE-1"
level 1
ipv4-unicast-metric-offset 90
member "int-P-3-PE-1-1"
member "int-P-3-PE-1-2"
revert-members 2
oper-members 2
exit
exit
no shutdown
exit
# on P-4:
configure
router
isis
level-capability level-1
area-id 49.0001
advertise-passive-only
level 1
wide-metrics-only
exit
interface "system"
passive
exit
interface "int-P-4-PE-1-1"
interface-type point-to-point
level 1
metric 30
exit
exit
interface "int-P-4-PE-1-2"
interface-type point-to-point
level 1
metric 30
exit
exit
interface "int-P-4-ASBR-2"
interface-type point-to-point
level 1
metric 1
exit
exit
link-group "Link-Group-P-4-PE-1"
level 1
ipv4-unicast-metric-offset 70
member "int-P-4-PE-1-1"
member "int-P-4-PE-1-2"
revert-members 2
oper-members 2
exit
exit
no shutdown
exit
An overview of all link groups can be shown using the following commands, in this case on node PE-1.
The link group status is as follows:
*A:PE-1# show router isis link-group-status
===============================================================================
Rtr Base ISIS Instance 0 Link-Group Status
===============================================================================
Link-group Mbrs Oper Revert Active Level State
Mbr Mbr Mbr
-------------------------------------------------------------------------------
Link-Group-PE-1-P-3 2 2 2 2 L1 normal
Link-Group-PE-1-P-4 2 2 2 2 L1 normal
===============================================================================
The output for the individual link group members is as follows:
For "Link-Group-PE-1-P-3" at PE-1:
*A:PE-1# show router isis link-group-member-status level 1 "Link-Group-PE-1-P-3"
===============================================================================
Rtr Base ISIS Instance 0 Link-Group Member
===============================================================================
Link-group I/F name Level State
-------------------------------------------------------------------------------
Link-Group-PE-1-P-3 int-PE-1-P-3-1 L1 Up
Link-Group-PE-1-P-3 int-PE-1-P-3-2 L1 Up
-------------------------------------------------------------------------------
Legend: BER = bitErrorRate
===============================================================================
For "Link-Group-PE-1-P-4" at PE-1:
*A:PE-1# show router isis link-group-member-status level 1 "Link-Group-PE-1-P-4"
===============================================================================
Rtr Base ISIS Instance 0 Link-Group Member
===============================================================================
Link-group I/F name Level State
-------------------------------------------------------------------------------
Link-Group-PE-1-P-4 int-PE-1-P-4-1 L1 Up
Link-Group-PE-1-P-4 int-PE-1-P-4-2 L1 Up
-------------------------------------------------------------------------------
Legend: BER = bitErrorRate
===============================================================================
For P-3, the link group status is as follows:
*A:P-3# show router isis link-group-status
===============================================================================
Rtr Base ISIS Instance 0 Link-Group Status
===============================================================================
Link-group Mbrs Oper Revert Active Level State
Mbr Mbr Mbr
-------------------------------------------------------------------------------
Link-Group-P-3-PE-1 2 2 2 2 L1 normal
===============================================================================
For P-3, the link group member status is as follows:
*A:P-3# show router isis link-group-member-status level 1 "Link-Group-P-3-PE-1"
===============================================================================
Rtr Base ISIS Instance 0 Link-Group Member
===============================================================================
Link-group I/F name Level State
-------------------------------------------------------------------------------
Link-Group-P-3-PE-1 int-P-3-PE-1-1 L1 Up
Link-Group-P-3-PE-1 int-P-3-PE-1-2 L1 Up
-------------------------------------------------------------------------------
Legend: BER = bitErrorRate
===============================================================================
Routing table PE-1
In a normal working state, the routing table for PE-1 contains the default route for forwarding traffic toward ASBR-2. Because ECMP is set to a value of 2, two entries are available with next-hops pointing toward P-3, as follows. The metric for each path is 11.
*A:PE-1# show router route-table 0.0.0.0/0
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
0.0.0.0/0 Remote ISIS 00h02m27s 15
192.168.13.2 11
0.0.0.0/0 Remote ISIS 00h02m27s 15
192.168.113.2 11
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
Failure of link member PE-1 to P-3
One of the links between PE-1 and P-3 is put into a failed state by disabling port 1/1/2 on P-3, as per Link failure.
# on P-3:
configure
port 1/1/2
shutdown
The route-table on PE-1 shows that the metric for the default route prefix, 0.0.0.0/0, has increased from 11 to 31, and the next-hops are now interface addresses on P-4, as follows:
*A:PE-1# show router route-table 0.0.0.0/0
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
0.0.0.0/0 Remote ISIS 00h01m14s 15
192.168.14.2 31
0.0.0.0/0 Remote ISIS 00h01m14s 15
192.168.114.2 31
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The link-group status shows that the number of active members has fallen below the oper-members threshold and as a result, the metric offset has been applied.
*A:PE-1# show router isis link-group-status
===============================================================================
Rtr Base ISIS Instance 0 Link-Group Status
===============================================================================
Link-group Mbrs Oper Revert Active Level State
Mbr Mbr Mbr
-------------------------------------------------------------------------------
Link-Group-PE-1-P-3 2 2 2 1 L1 Offset-Applied
Link-Group-PE-1-P-4 2 2 2 2 L1 normal
===============================================================================
Finally, the status of an individual link group member is as follows:
*A:PE-1# show router isis link-group-member-status "Link-Group-PE-1-P-3"
===============================================================================
Rtr Base ISIS Instance 0 Link-Group Member
===============================================================================
Link-group I/F name Level State
-------------------------------------------------------------------------------
Link-Group-PE-1-P-3 int-PE-1-P-3-1 L1 If-Down
Link-Group-PE-1-P-3 int-PE-1-P-3-2 L1 Up
-------------------------------------------------------------------------------
Legend: BER = bitErrorRate
===============================================================================
By examining the IS-IS database on PE-1, it can be seen that the link metric (TE-IS neighbor) toward P-3 has a metric of 100, comprised of the original metric of 10 plus the offset of 90.
*A:PE-1# show router isis database PE-1 detail
===============================================================================
Rtr Base ISIS Instance 0 Database (detail)
===============================================================================
Displaying Level 1 database
-------------------------------------------------------------------------------
LSP ID : PE-1.00-00 Level : L1
Sequence : 0x7 Checksum : 0x3c96 Lifetime : 905
Version : 1 Pkt Type : 18 Pkt Ver : 1
Attributes: L1 Max Area : 3 Alloc Len : 1492
SYS ID : 1920.0000.2001 SysID Len : 6 Used Len : 163
TLVs :
Area Addresses:
Area Address : (3) 49.0001
Supp Protocols:
Protocols : IPv4
IS-Hostname : PE-1
Router ID :
Router ID : 192.0.2.1
I/F Addresses :
I/F Address : 192.0.2.1
I/F Address : 192.168.13.1
I/F Address : 192.168.14.1
I/F Address : 192.168.113.1
I/F Address : 192.168.114.1
TE IS Nbrs :
Nbr : P-3.00
Default Metric : 100
Sub TLV Len : 12
IF Addr : 192.168.113.1
Nbr IP : 192.168.113.2
TE IS Nbrs :
Nbr : P-4.00
Default Metric : 30
Sub TLV Len : 12
IF Addr : 192.168.14.1
Nbr IP : 192.168.14.2
TE IS Nbrs :
Nbr : P-4.00
Default Metric : 30
Sub TLV Len : 12
IF Addr : 192.168.114.1
Nbr IP : 192.168.114.2
TE IP Reach :
Default Metric : 1
Control Info: , prefLen 16
Prefix : 172.16.0.0
Default Metric : 0
Control Info: , prefLen 32
Prefix : 192.0.2.1
Level (1) LSP Count : 1
---snip---
Failure of link member PE-1 to P-4:
If a link between PE-1 and P-4 now fails, simulated by disabling port 1/1/1 on P-4, then the metric offset is applied to the link groups on PE-1 and P-4 as the number of active links has dropped below the oper-members threshold for the link groups Link-Group-PE-1-P-4 on PE-1 and Link-Group-P-4-PE-1on P-4.
# on P-4:
configure
port 1/1/1
shutdown
The routing table for PE-1 now shows that there are still two equal cost paths for the default route prefix advertised by ASBR-2, as follows:
*A:PE-1# show router route-table 0.0.0.0/0
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
0.0.0.0/0 Remote ISIS 00h01m16s 15
192.168.113.2 101
0.0.0.0/0 Remote ISIS 00h01m16s 15
192.168.114.2 101
-------------------------------------------------------------------------------
No. of Routes: 2
The metric for each routing table entry is 101, comprising of a cost of 100 for the PE-1 to P router link, where the link-group offset has been applied, and the cost of 1 for the P router to ASBR-2 router link.
By examining the IS-IS database on the PE-1 router, the updated metric for the link to neighbors P-3 and P-4 can be seen with the offset applied. These are seen in the ‟TE-IS Nbrs” TLV in the following output.
*A:PE-1# show router isis database PE-1 detail
===============================================================================
Rtr Base ISIS Instance 0 Database
===============================================================================
Displaying Level 1 database
-------------------------------------------------------------------------------
LSP ID : PE-1.00-00 Level : L1
Sequence : 0xb Checksum : 0xd372 Lifetime : 1129
Version : 1 Pkt Type : 18 Pkt Ver : 1
Attributes: L1 Max Area : 3 Alloc Len : 1492
SYS ID : 1920.0000.2001 SysID Len : 6 Used Len : 138
TLVs :
Area Addresses:
Area Address : (3) 49.0001
Supp Protocols:
Protocols : IPv4
IS-Hostname : PE-1
Router ID :
Router ID : 192.0.2.1
I/F Addresses :
I/F Address : 192.0.2.1
I/F Address : 192.168.13.1
I/F Address : 192.168.14.1
I/F Address : 192.168.113.1
I/F Address : 192.168.114.1
TE IS Nbrs :
Nbr : P-3.00
Default Metric : 100
Sub TLV Len : 12
IF Addr : 192.168.113.1
Nbr IP : 192.168.113.2
TE IS Nbrs :
Nbr : P-4.00
Default Metric : 100
Sub TLV Len : 12
IF Addr : 192.168.114.1
Nbr IP : 192.168.114.2
TE IP Reach :
Default Metric : 1
Control Info: , prefLen 16
Prefix : 172.16.0.0
Default Metric : 0
Control Info: , prefLen 32
Prefix : 192.0.2.1
Level (1) LSP Count : 1
---snip---
Conclusion
IS-IS link bundling allows service providers to configure multiple IS-IS interfaces as a single link group for ECMP purposes and allow link metric increases if an interface within the bundle group fails. This example provides the configuration for IS-IS link bundling, together with the associated commands and outputs which can be used for verifying and troubleshooting.