IS-IS Link Bundling

Applicability

The chapter was initially written for SR OS Release 11.0.R6. The configuration in the current edition is based on MD-CLI in SR OS Release 20.7.R2.

Overview

Intermediate System to Intermediate System (IS-IS) Link Bundling allows for the grouping of a number of IS-IS interfaces into a single virtual link, called an IS-IS link group. It is used in conjunction with Equal Cost Multipath (ECMP) to dynamically change the metric of parallel IS-IS links if one or more links fail or suffer some sort of performance degradation.

Consider the network in Link bundle schematic, where a Provider Edge router PE-1 connects to a core network comprised of two Provider (P) routers and a single Autonomous System Border Router (ASBR).

The links between PE-1 and P-3, and PE-1 and P-4 are 10 Gigabit Ethernet links. The links between ASBR-2 and P-3 and P-4 are both 100 Gigabit links. The link metrics are as shown in Link bundle schematic.

In order to maximize the use of link bandwidth, ECMP is enabled on all routers and set to a value of 2, so that IP traffic flowing between PE1 and P-3, and PE-1 and P-4, is load balanced across the two links.

A default route is injected into the ASBR-2 router and redistributed via a policy statement into IS-IS, so that traffic flowing from PE-1 to the ASBR is resolved by this route. Traffic flows between PE-1 and ASBR-2, using the path with the lowest IS-IS metric, via P-3 with a metric of 11. The second path PE-1 to ASBR-2 via P-4 has the same bandwidth, but a higher IS-IS metric of 31.

Traffic in the reverse direction flows toward a user subnet described by a static route configured on PE-1, which is redistributed into IS-IS using a policy statement. Once again, the shortest path between ASBR-2 and PE-1 is via P-3, so the bi-directional traffic flow is symmetric.

If one of the links between PE-1 and P-3 fails, traffic still flows via P-3, because the IS-IS metric is unchanged, but this now has less bandwidth than the second path via P-4. It is desirable to make use of the additional bandwidth of the second path, but this requires a change in metric. This can be achieved using IS-IS link bundling.

IS-IS link bundling allows for the creation of a group of IS-IS links, where the failure of a member link allows the metric of the remaining members of the link group to be increased by an offset value.

Using Effect of single link failure on bundle group as an example, the links between PE-1 and P-3 are included in a bundle group.

Figure 2. Effect of single link failure on bundle group

To illustrate the change in metrics, a default static route is configured on ASBR-2 and redistributed into IS-IS, and the path to this route is monitored at PE-1. Similarly, a static route to subnet 172.16.0.0/16 is configured on PE-1 and redistributed into IS-IS and viewed on ASBR-2.

Should one of the links between PE-1 and P-3 fail, the metric of the remaining members can be increased by an offset, for example 90, so that the metric of the remaining link becomes 10 + 90 = 100. The IS-IS metric between PE-1 and ASBR-2 via P-3 is now 101. The metric offset is applied to each remaining IS-IS interface individually and is advertised within the IS-IS database as the default cost in the TE-IS neighbors Type Length Variable (TLV).

The path between PE-1 and ASBR-2 via P-4 now has the lowest IS-IS metric, and any affected routers within the IS-IS area will try and reroute the traffic based on the new metric.

The fundamentals of this feature are:

The treatment of all member links in a link group bundle as a single virtual interface.
The increase in metric by a specific offset value of each remaining individual link within the group when a failure of one or more links occurs.
- The application of the offset occurs when the number of active links drops below a configured threshold.
- The offset is removed when the number of active links within the link group bundle reaches the configured reversion threshold.
- A link bundle is required on a router for the thresholds and offsets to apply.

Consider a second and subsequent failure where a link between PE-1 and P-4 also fails, so that there is only one active IS-IS interface between PE-1 and each of its neighboring P routers. This is shown in Double link failure .

In this case, the metric for the remaining link between PE-1 and P-4 can be increased by an offset value of +70 so that the IS-IS metric PE-1 to P-4 becomes 100, the same as that between PE-1 and P-3 when a link has failed.

PE-1 now sees two equal cost paths to the default route—one via P-3 and one via P-4, so there are still two 10Gigabit Ethernet links across which the traffic can be load shared.

This can be summarized using the following table, where ABCD are the 4 links as per Link bundle schematic and link status is Up (U) or Down (D).

Table 1. Status of the links A, B, C, and D
ABCD Status	A (metric,status)	B (metric,status)	C (metric,status)	D (metric,status)
UUUU	10 Transmit	10 Transmit	30 Idle	30 Idle
UDUU	100 Idle	Down	30 Transmit	30 Transmit
UDUD	100 Transmit	Down	100 Transmit	Down
UUUD	10 Transmit	10 Transmit	100 Idle	Down

Configuration

The example topology is shown in Example topology.

On PE-1, ECMP is set to a value of 2 and the following router interfaces are configured:

# on PE-1:
configure {
    router "Base" {
        ecmp 2
        interface "int-PE-1-P-3-1" {
            port 1/1/1
            ipv4 {                                                            
                primary {
                    address 192.168.13.1
                    prefix-length 30
                }
            }
        }
        interface "int-PE-1-P-3-2" {
            port 1/1/3
            ipv4 {
                primary {
                    address 192.168.113.1
                    prefix-length 30
                }
            }
        }
        interface "int-PE-1-P-4-1" {
            port 1/1/2
            ipv4 {
                primary {
                    address 192.168.14.1
                    prefix-length 30
                }
            }
        }
        interface "int-PE-1-P-4-2" {
            port 1/1/4
            ipv4 {
                primary {
                    address 192.168.114.1
                    prefix-length 30
                }
            }
        }
        interface "system" {
            ipv4 {
                primary {
                    address 192.0.2.1
                    prefix-length 32
                }
            }
        }

The IP router configuration for the remaining routers can be derived from Example topology.

The IS-IS network is a level 1 network.

The IS-IS configuration for PE-1, including the interface metrics is as follows:

# on PE-1:
configure {
    router "Base" {
        isis 0 {
            admin-state enable
            advertise-passive-only true
            level-capability 1
            area-address [49.0001]
            interface "int-PE-1-P-3-1" {
                interface-type point-to-point
                level 1 {
                    metric 10
                }
            }
            interface "int-PE-1-P-3-2" {
                interface-type point-to-point
                level 1 {
                    metric 10
                }
            }
            interface "int-PE-1-P-4-1" {
                interface-type point-to-point
                level 1 {
                    metric 30
                }                                                             
            }
            interface "int-PE-1-P-4-2" {
                interface-type point-to-point
                level 1 {
                    metric 30
                }
            }
            interface "system" {
                passive true
            }
            level 1 {
                wide-metrics-only true
            }
        }

The IS-IS configuration for the remaining routers can be derived from Example topology.

The following configuration is for the static route and export policy on ASBR-2. The configuration of the static route on PE-1 is similar.

# on ASBR-2:
configure {
    router "Base" {
        static-routes {
            route 0.0.0.0/0 route-type unicast {
                blackhole {
                    admin-state enable
                }
            }

# on PE-1, ASBR-2:
configure {
    policy-options {
        policy-statement "STATIC-ISIS" {
            entry 10 {
                from {
                    protocol {
                        name [static]
                    }
                }
                to {
                    level 1
                }
                action {
                    action-type accept
                    metric {
                        set "@igp@"
                    }
                }
            }
        }
    }
    router "Base" {
        isis 0 {
            export-policy ["STATIC-ISIS"]
        }

Link group configuration

PE-1 contains 2 link groups. The first link group contains the IS-IS interfaces toward P-3. The second contains the interfaces toward P-4.

Each link-group is configured using a unique name, which is unique per router, and the IS-IS interface names are configured within the group as group members.

The metric offset value is the amount by which the IS-IS metric of active member links is increased when the number of links drops below a configured threshold.

The IS-IS link group configuration for PE-1 for the interfaces toward P-3 is as follows:

# on PE-1:
configure {
    router "Base" {
        isis 0 {
            link-group "Link-Group-PE-1-P-3" {
                level 1 {
                    ipv4-unicast-metric-offset 90
                    oper-members 2
                    revert-members 2
                    member "int-PE-1-P-3-1" { }
                    member "int-PE-1-P-3-2" { }
                }
            }

Similarly, the IS-IS link group for PE-1 for the interfaces toward P-4 is:

            link-group "Link-Group-PE-1-P-4" {
                level 1 {
                    ipv4-unicast-metric-offset 70
                    oper-members 2
                    revert-members 2
                    member "int-PE-1-P-4-1" { }
                    member "int-PE-1-P-4-2" { }
                }
            }

Within the link-group, two thresholds are configured:

oper-members threshold
revert-members threshold

If the number of operational links in the link-group drops below the oper-members value, then all interfaces associated with that IS-IS link group have their interface metric increased by the configured offset value. As a result, IS-IS then tries to reroute traffic over lower cost paths.

If the number of operational links in the link-group equals the revert-members threshold value, then all interfaces associated with that IS-IS link group have their interface metric decreased by the configured offset value.

In this configuration, there is a requirement to increase the metric of each interface within a link group when a single interface fails. This means that the oper-members value is set to 2. In normal working circumstances, when both interfaces are active, the metric used is the configured interface metric. This means that the revert-members value must also be set to 2.

It is not possible to set the oper-members threshold to a value higher than that of the revert-members.

For completeness, the IS-IS configuration the P-routers is as follows.

# on P-3:
configure {
    router "Base" {
        isis 0 {
            admin-state enable
            advertise-passive-only true
            level-capability 1
            area-address [49.0001]
            interface "int-P-3-ASBR-2" {
                interface-type point-to-point
                level 1 {
                    metric 1
                }
            }
            interface "int-P-3-PE-1-1" {
                interface-type point-to-point
                level 1 {
                    metric 10
                }
            }
            interface "int-P-3-PE-1-2" {
                interface-type point-to-point
                level 1 {
                    metric 10
                }
            }
            interface "system" {
                passive true
            }
            level 1 {
                wide-metrics-only true
            }
            link-group "Link-Group-P-3-PE-1" {
                level 1 {
                    ipv4-unicast-metric-offset 90
                    oper-members 2
                    revert-members 2
                    member "int-P-3-PE-1-1" { }
                    member "int-P-3-PE-1-2" { }
                }
            }

# on PE-4:
configure {
    router "Base" {
        isis 0 {
            admin-state enable
            advertise-passive-only true
            level-capability 1
            area-address [49.0001]
            interface "int-P-4-ASBR-2" {
                interface-type point-to-point
                level 1 {
                    metric 1
                }
            }
            interface "int-P-4-PE-1-1" {
                interface-type point-to-point
                level 1 {
                    metric 30
                }
            }
            interface "int-P-4-PE-1-2" {
                interface-type point-to-point
                level 1 {
                    metric 30
                }
            }
            interface "system" {
                passive true
            }
            level 1 {
                wide-metrics-only true
            }
            link-group "Link-Group-P-4-PE-1" {
                level 1 {
                    ipv4-unicast-metric-offset 70
                    oper-members 2
                    revert-members 2
                    member "int-P-4-PE-1-1" { }
                    member "int-P-4-PE-1-2" { }
                }
            }
        }

An overview of all link groups can be shown using the following commands, in this case on node PE-1.

The link group status on PE-1 is as follows:

[]
A:admin@PE-1# show router isis link-group-status

===============================================================================
Rtr Base ISIS Instance 0 Link-Group Status
===============================================================================
Link-group              Mbrs   Oper   Revert Active Level   State
                               Mbr     Mbr    Mbr           
-------------------------------------------------------------------------------
Link-Group-PE-1-P-3     2      2      2      2      L1      normal
Link-Group-PE-1-P-4     2      2      2      2      L1      normal
===============================================================================

The output for the individual link group members on PE-1 is as follows:

For "Link-Group-PE-1-P-3" at PE-1:

[]
A:admin@PE-1# show router isis link-group-member-status "Link-Group-PE-1-P-3"

===============================================================================
Rtr Base ISIS Instance 0 Link-Group Member
===============================================================================
Link-group            I/F name                Level      State
-------------------------------------------------------------------------------
Link-Group-PE-1-P-3   int-PE-1-P-3-1          L1         Up
Link-Group-PE-1-P-3   int-PE-1-P-3-2          L1         Up
-------------------------------------------------------------------------------
Legend: BER = bitErrorRate
===============================================================================

For "Link-Group-PE-1-P-4" at PE-1:

[]
A:admin@PE-1# show router isis link-group-member-status "Link-Group-PE-1-P-4"
===============================================================================
Rtr Base ISIS Instance 0 Link-Group Member
===============================================================================
Link-group            I/F name                Level      State
-------------------------------------------------------------------------------
Link-Group-PE-1-P-4   int-PE-1-P-4-1          L1         Up
Link-Group-PE-1-P-4   int-PE-1-P-4-2          L1         Up
-------------------------------------------------------------------------------
Legend: BER = bitErrorRate
===============================================================================

For P-3, the link group status is as follows:

[]
A:admin@P-3# show router isis link-group-status 

===============================================================================
Rtr Base ISIS Instance 0 Link-Group Status
===============================================================================
Link-group              Mbrs   Oper   Revert Active Level   State
                               Mbr     Mbr    Mbr           
-------------------------------------------------------------------------------
Link-Group-P-3-PE-1     2      2      2      2      L1      normal
===============================================================================

For P-3, the link group member status is as follows:

[]
A:admin@P-3# show router isis link-group-member-status "Link-Group-P-3-PE-1"

===============================================================================
Rtr Base ISIS Instance 0 Link-Group Member
===============================================================================
Link-group            I/F name                Level      State
-------------------------------------------------------------------------------
Link-Group-P-3-PE-1   int-P-3-PE-1-1          L1         Up
Link-Group-P-3-PE-1   int-P-3-PE-1-2          L1         Up
-------------------------------------------------------------------------------
Legend: BER = bitErrorRate
===============================================================================

Routing table PE-1

In a normal working state, the routing table for PE-1 contains the default route for forwarding traffic toward ASBR-2. Because ECMP is set to a value of 2, two entries are available with next-hops pointing toward P-3, as follows. The metric for each path is 11.

[]
A:admin@PE-1# show router route-table 0.0.0.0/0 
 
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric
-------------------------------------------------------------------------------
0.0.0.0/0                                     Remote  ISIS      00h02m27s  15
       192.168.13.2                                                 11
0.0.0.0/0                                     Remote  ISIS      00h02m27s  15
       192.168.113.2                                                11
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
       B = BGP backup route available
       L = LFA nexthop available
       S = Sticky ECMP requested
===============================================================================

Failure of link member PE-1 to P-3

Link failure shows that one of the links in the active link group between PE-1 and P-3 fails.

One of the links between PE-1 and P-3 is put into a failed state by disabling port 1/1/2 on P-3, as follows:

# on P-3:
configure {
    port 1/1/2 {
        admin-state disable
    }

The route-table on PE-1 shows that the metric for the default route prefix, 0.0.0.0/0, has increased from 11 to 31, and the next-hops are now interface addresses on P-4, as follows:

[]
A:admin@PE-1# show router route-table 0.0.0.0/0 

===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric
-------------------------------------------------------------------------------
0.0.0.0/0                                     Remote  ISIS      00h01m14s  15
       192.168.14.2                                                 31
0.0.0.0/0                                     Remote  ISIS      00h01m14s  15
       192.168.114.2                                                31
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
       B = BGP backup route available
       L = LFA nexthop available
       S = Sticky ECMP requested
===============================================================================

The link-group status shows that the number of active members has fallen below the oper-members threshold and as a result, the metric offset has been applied.

[]
A:admin@PE-1# show router isis link-group-status

===============================================================================
Rtr Base ISIS Instance 0 Link-Group Status
===============================================================================
Link-group              Mbrs   Oper   Revert Active Level   State
                               Mbr     Mbr    Mbr           
-------------------------------------------------------------------------------
Link-Group-PE-1-P-3     2      2      2      1      L1      Offset-Applied
Link-Group-PE-1-P-4     2      2      2      2      L1      normal
===============================================================================

Finally, the status of an individual link group member is as follows:

[]
A:admin@PE-1# show router isis link-group-member-status "Link-Group-PE-1-P-3"

===============================================================================
Rtr Base ISIS Instance 0 Link-Group Member
===============================================================================
Link-group            I/F name                Level      State
-------------------------------------------------------------------------------
Link-Group-PE-1-P-3   int-PE-1-P-3-1          L1         If-Down
Link-Group-PE-1-P-3   int-PE-1-P-3-2          L1         Up
-------------------------------------------------------------------------------
Legend: BER = bitErrorRate
===============================================================================

The following IS-IS database on PE-1 shows that the link metric (TE-IS neighbor) toward P-3 has a metric of 100, comprised of the original metric of 10 plus the offset of 90.

[]
A:admin@PE-1# show router isis database PE-1 detail 

===============================================================================
Rtr Base ISIS Instance 0 Database (detail)
===============================================================================


Displaying Level 1 database
-------------------------------------------------------------------------------
LSP ID    : PE-1.00-00                                  Level     : L1
Sequence  : 0x7                    Checksum  : 0x3c96   Lifetime  : 1099
Version   : 1                      Pkt Type  : 18       Pkt Ver   : 1
Attributes: L1                     Max Area  : 3        Alloc Len : 1492
SYS ID    : 1920.0000.2001         SysID Len : 6        Used Len  : 163

TLVs :
  Area Addresses:
    Area Address : (3) 49.0001
  Supp Protocols:
    Protocols     : IPv4
  IS-Hostname   : PE-1
  Router ID   :
    Router ID   : 192.0.2.1
  I/F Addresses :
    I/F Address   : 192.0.2.1
    I/F Address   : 192.168.13.1
    I/F Address   : 192.168.14.1
    I/F Address   : 192.168.113.1
    I/F Address   : 192.168.114.1
  TE IS Nbrs   :
    Nbr   : P-3.00
    Default Metric  : 100
    Sub TLV Len     : 12
    IF Addr   : 192.168.113.1
    Nbr IP    : 192.168.113.2
  TE IS Nbrs   :
    Nbr   : P-4.00
    Default Metric  : 30
    Sub TLV Len     : 12
    IF Addr   : 192.168.14.1
    Nbr IP    : 192.168.14.2
  TE IS Nbrs   :
    Nbr   : P-4.00
    Default Metric  : 30
    Sub TLV Len     : 12
    IF Addr   : 192.168.114.1
    Nbr IP    : 192.168.114.2
  TE IP Reach   :
    Default Metric  : 1
    Control Info:    , prefLen 16
    Prefix   : 172.16.0.0
    Default Metric  : 0
    Control Info:    , prefLen 32
    Prefix   : 192.0.2.1

Level (1) LSP Count : 1

---snip---

Failure of link member PE-1 to P-4:

Second link failure shows that all link groups only have one active link instead of two.

If a link between PE-1 and P-4 now fails, simulated by disabling port 1/1/1 on P-4, then the metric offset is applied to the link groups on PE-1 and P-4 as the number of active links has dropped below the oper-members threshold for the link groups Link-Group-PE-1-P-4 on PE-1 and Link-Group-P-4-PE-1 on P-4.

# on P-4:
configure {
    port 1/1/1 {
        admin-state disable
    }

The routing table for PE-1 now shows that there are still two equal cost paths for the default route prefix advertised by ASBR-2, as follows:

[]
A:admin@PE-1# show router route-table 0.0.0.0/0 
 
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric
-------------------------------------------------------------------------------
0.0.0.0/0                                     Remote  ISIS      00h01m16s  15
       192.168.113.2                                                101
0.0.0.0/0                                     Remote  ISIS      00h01m16s  15
       192.168.114.2                                                101
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
       B = BGP backup route available
       L = LFA nexthop available
       S = Sticky ECMP requested
===============================================================================

The metric for each routing table entry is 101, comprising of a cost of 100 for the PE-1 to P router link, where the link-group offset has been applied, and the cost of 1 for the P router to ASBR-2 router link.

By examining the IS-IS database on the PE-1 router, the updated metric for the link to neighbors P-3 and P-4 can be seen with the offset applied. These are seen in the ‟TE-IS Nbrs” TLV in the following output.

[]
A:admin@PE-1# show router isis database PE-1 detail

===============================================================================
Rtr Base ISIS Instance 0 Database
===============================================================================

Displaying Level 1 database
-------------------------------------------------------------------------------
LSP ID    : PE-1.00-00                                  Level     : L1
Sequence  : 0x8                    Checksum  : 0x3e0a   Lifetime  : 1129
Version   : 1                      Pkt Type  : 18       Pkt Ver   : 1
Attributes: L1                     Max Area  : 3        Alloc Len : 1492
SYS ID    : 1920.0000.2001         SysID Len : 6        Used Len  : 138

TLVs :
  Area Addresses:
    Area Address : (3) 49.0001
  Supp Protocols:
    Protocols     : IPv4
  IS-Hostname   : PE-1
  Router ID   :
    Router ID   : 192.0.2.1
  I/F Addresses :
    I/F Address   : 192.0.2.1
    I/F Address   : 192.168.13.1
    I/F Address   : 192.168.14.1
    I/F Address   : 192.168.113.1
    I/F Address   : 192.168.114.1
  TE IS Nbrs   :
    Nbr   : P-3.00
    Default Metric  : 100
    Sub TLV Len     : 12
    IF Addr   : 192.168.113.1
    Nbr IP    : 192.168.113.2
  TE IS Nbrs   :
    Nbr   : P-4.00
    Default Metric  : 100
    Sub TLV Len     : 12
    IF Addr   : 192.168.114.1
    Nbr IP    : 192.168.114.2
  TE IP Reach   :
    Default Metric  : 1
    Control Info:    , prefLen 16
    Prefix   : 172.16.0.0
    Default Metric  : 0
    Control Info:    , prefLen 32
    Prefix   : 192.0.2.1

Level (1) LSP Count : 1

---snip---

Conclusion

IS-IS link bundling allows service providers to configure multiple IS-IS interfaces as a single link group for ECMP purposes and allow link metric increases if an interface within the bundle group fails. This example provides the configuration for IS-IS link bundling, together with the associated commands and outputs which can be used for verifying and troubleshooting.