Seamless BFD for SR-TE LSPs

This chapter describes seamless BFD for SR-TE LSPs.

Topics in this chapter include:

Applicability

This chapter was initially written based on SR OS Release 19.10.R1, but the configuration in the current edition is based on SR OS Release 23.3.R3. BFD for RSVP-TE LSPs is supported in SR OS Release 13.0, and later. Seamless BFD for SR-TE LSPs is supported in SR OS Release 19.10.R1, and later.

Overview

Bidirectional Forwarding Detection (BFD) is widely deployed in IP/MPLS networks to rapidly detect failures in the forwarding path between network elements. In this chapter, a comparison is made between classical BFD and seamless BFD (S-BFD).

Classical BFD

Classical BFD, described in RFC 5880, requires little overhead. However, the handshake mechanism to negotiate and set up two-way BFD sessions between network elements can take several seconds. RFC 5880 specifies two modes of operation: asynchronous mode and on-demand mode. Additionally, the BFD echo function loops back BFD echo packets to the sender.

Classical BFD is applied to the interface. In asynchronous mode, sessions are established. Network elements periodically send BFD control packets to one another. Discriminators are used as a session demultiplexer to distinguish between BFD sessions. The transmitting network element generates a unique non-zero discriminator value, which is exchanged as part of the session handshake establishment. Classical BFD handshake shows the classical BFD handshake for a single hop across an IP link.

Figure 1. Classical BFD handshake

BFD for MPLS LSPs

BFD is supported for RSVP-TE LSPs and for LDP LSPs, as described in the "BFD for RSVP-TE and LDP LSPs" chapter in 7450 ESS, 7750 SR, and 7950 XRS MPLS Advanced Configuration Guide for Classic CLI.

BFD for MPLS LSPs is described in RFC 5884. For continuity checks in MPLS LSPs, BFD packets are transmitted using the MPLS encapsulation, so they share fate with the LSP data path.

BFD is bootstrapped using an LSP ping. An MPLS echo request packet is transmitted along the LSP path, including a BFD discriminator TLV containing the head-end BFD discriminator value. The tail end responds with an echo reply packet, using the IP forwarding path, including the tail-end BFD discriminator value.

Afterward, BFD control packets establish a BFD session between the head end and tail end using the discriminator values from the bootstrap session. The egress LER will send a BFD control packet upon receipt.

Each session has its own pair of discriminators, so multiple discriminators are allocated by the system.

S-BFD for SR-TE LSPs

S-BFD is described in RFC 7880. Unlike classical BFD, S-BFD does not rely on the BFD bootstrapping process (handshake) or session state at the tail end of a session. Instead, when S-BFD is initialized, a pair of discriminators are selected by the system for specific purposes (reflector or initiator). S-BFD minimizes the time required to establish BFD sessions, which contributes to its seamless operation. S-BFD relies on the fact that the discriminators are already known by the endpoints for each session, either through configuration or advertisement using unicast protocols.

There are two discriminators, one for each end of the BFD/S-BFD session. From the perspective of the S-BFD initiator (or BFD head end) there is a local 'my discriminator' and a remote 'your discriminator'. The 'your discriminator' matches the remote node's local discriminator, which for BFD is allocated to the session endpoint, and for S-BFD is the reflector discriminator.

Terminology

RFC 7880 S-BFD terms describes the S-BFD terms, as defined by RFC 7880.

Table 1. RFC 7880 S-BFD terms

S-BFD term

Description

Entity

A function on a network node to which the S-BFD mechanism allows remote network nodes to perform continuity tests. An entity can be abstract (for example, reachability) or specific (for example, IP addresses, router IDs, functions).

S-BFD initiator

An S-BFD session on a network node that performs a continuity test to a remote entity by sending S-BFD packets.

BFD discriminator

An identifier for a BFD session at an endpoint of a BFD session.

Initiator

A network node hosting an S-BFD initiator.

S-BFD reflector

An S-BFD session on a network node that listens for incoming S-BFD control packets to local entities and generates response S-BFD control packets.

Responder

A network node hosting an S-BFD reflector.

S-BFD discriminator

A BFD discriminator allocated for an endpoint of an S-BFD session.

Relationship between S-BFD terms shows the relationship between the S-BFD terms described in RFC 7880 S-BFD terms.

Figure 2. Relationship between S-BFD terms

S-BFD implementation in SR OS

Before an application can request the establishment of an S-BFD session, a mapping table of remote discriminators to peer far-end IP addresses must exist. These correspond to the discriminators of the reflector nodes. The mapping can be accomplished in two ways:
  • automatically learned (using opaque OSPF or IS-IS routing extensions) or
  • statically configured

A single S-BFD discriminator is allocated to a reflector in a router instance. The local reflector S-BFD discriminator is statically configured in the CLI and must be in the range from 524288 to 526335. The S-BFD discriminator must not be the same as any discriminator used for classical BFD.

As per RFC 5884, the destination IP address of explicitly label-switched S-BFD control packets must be chosen from the 127/8 range for IPv4 and the TTL of the IP header must be set to 1. The source IP address is a routable address of the sender.

The initiator node uses the following UDP ports for S-BFD control packets:

  • UDP destination port 7784

  • UDP source port, which can be any valid port except 7784, as follows:

    • the same UDP source port for all S-BFD control packets to the same reflector

    • different UDP source ports for S-BFD control packets to different reflectors

    • packets with UDP source port 7784 will be discarded by the reflector

The responder node swaps the UDP source and destination port when sending S-BFD control packets back to the initiator node:

  • received UDP source port = transmitted UDP destination port

  • received UDP destination port = transmitted UDP source port

It also exchanges the 'my discriminator' and 'your discrminitator' values in the reflected S-BFD packet.

S-BFD can be applied to SR-TE LSPs and the SR-TE LSP state can depend on the S-BFD session state.

S-BFD session establishment - continuity check

S-BFD session establishment - continuity check shows the continuity check S-BFD control packets between PE-1 and PE-4. On PE-1, the BFD (initiator) discriminator equals 123; on PE-4, the S-BFD (reflector) discriminator equals 524288. Head-end router PE-1 has a mapping table of remote discriminators to far-end IP addresses; for PE-4, the system ID is 192.0.2.4 and the S-BFD discriminator 524288. There is no INIT state in S-BFD. The mapping between the remote discriminators and the far-end IP addresses is required when the BFD return path is routed; when the BFD return path is controlled, no remote discriminators are used.

Figure 3. S-BFD session establishment - continuity check

The session initiator node PE-1 generates an S-BFD control packet with destination PE-4 (but with an IP DA from the 127/8 range), YourDiscriminator 524288 (= S-BFD (reflector) discriminator value), MyDiscriminator 123 (= BFD (initiator) discriminator value), and admin state up.

The responder node PE-4 responds to PE-1 with an IP DA from the 127/8 range, YourDiscriminator 123, MyDiscriminator 524288, and admin state up. The admin state of the reflector reflects the configured S-BFD local state.

Configuration

Example topology shows the example topology with eight nodes.

Figure 4. Example topology

The initial configuration includes:

  • Cards, MDAs, ports

  • Router interfaces

  • IS-IS as IGP (alternatively, OSPF can be used) with traffic engineering (TE) enabled

  • Segment routing enabled on all nodes

  • MPLS and RSVP enabled on all router interfaces

The following will be configured:

Note:

Even though BFD can use intervals smaller than 1000 ms, the used example setup has its limitations. The nodes in the used example setup are sims and the simulation for CPM-NP or central BFD sessions has the limitation that intervals that are configured with a value smaller than 1000 ms are always negotiated to intervals of 1000 ms. To avoid confusion when the configured intervals differ from the negotiated intervals on sims, a BFD template with intervals of 1000 ms is configured and used in this chapter.

S-BFD for SR-TE LSPs with routed return path

For S-BFD, the S-BFD (reflector) discriminator on the responder (tail-end) node must be known by both end nodes. The mapping between the remote discriminators and the far-end IP addresses can be configured statically or it can be learned dynamically from IGP. On each node, the reflector S-BFD discriminator must be in the range from 524288 to 526335 and the local state must be set to up.

Automated S-BFD distribution

In this example, one SR-TE LSP is established between head end PE-4 and tail end PE-5. On tail end PE-5, the global S-BFD configuration is as follows:

# on PE-5:
configure
    bfd
        seamless-bfd
            reflector "PE-5"
                discriminator 524292
                local-state up
                no shutdown
            exit
        exit

The S-BFD configuration on the other PEs is similar; in this example, it is sufficient to have the global S-BFD configuration on tail end PE-5 only. When the IGP is configured with advertise-router-capability area and traffic-engineering enabled, IGP routing protocol extensions provide the encodings to advertise the S-BFD discriminators as opaque information within the IGP link state information. This way, the remote IP addresses and the S-BFD discrimators are automatically mapped.

When PE-4 sets up an SR-TE LSP to PE-5, it will use a BFD discriminator—for example, 3—and S-BFD (reflector) discriminator 524292 for PE-5. For different LSPs toward PE-5, PE-4 will use different BFD discriminators combined with the same S-BFD (reflector) discriminator 524292.

Static S-BFD configuration

If advertise-router-capability or traffic-engineering are not configured, the S-BFD far-end IP address and its discriminator are statically mapped, as follows. When all SR-TE LSPs have far end PE-5, the mapping for PE-5 is sufficient.

# on PE-4:
configure
    router Base
        bfd
            seamless-bfd
                peer 192.0.2.1
                    discriminator 524288
                exit
                peer 192.0.2.2
                    discriminator 524289
                exit
                peer 192.0.2.3
                    discriminator 524290
                exit
                peer 192.0.2.5
                    discriminator 524292
                exit
                peer 192.0.2.6
                    discriminator 524293
                exit
                peer 192.0.2.7
                    discriminator 524294
                exit
                peer 192.0.2.8
                    discriminator 524295
                exit
            exit

If the initiator receives a valid response from the reflector with an Up state, the initiator declares the S-BFD session as Up.

Note: Traffic engineering in not supported in VPRN or in OSPF3, so S-BFD discriminators cannot be automatically distributed in such cases.

Examples

S-BFD is only supported in CPM-NP on SR OS nodes, so the BFD type must be set to cpm-np. SR-TE LSPs can use CPM-NP BFD templates with a transmit and receive interval of minimum 10 ms. However, due to the simulation limitations on the sims in the example topology, the intervals are configured with a value of 1000 ms, as follows:

# on PE-4:
configure
    router Base
        bfd
            begin
            bfd-template "bfd-cpm-np-1s"
                type "cpm-np"
                transmit-interval 1000
                receive-interval 1000
            exit
            commit

On PE-4, the following paths and SR-TE LSPs are configured:

  • "LSP-PE-4-PE-5_empty_localCSPF" with primary path "empty", which does not contain any explicit hops

  • "LSP-PE-4-PE-5_viaPE-2_localCSPF" with primary path "via-PE-2", which contains 192.0.2.2 as a loose hop

  • "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd" with primary path "via-PE-2" and secondary path "via-PE-3", which contains 192.0.2.3 as a loose hop

Any path computation method can be used. In the following example, the path computation method is local CSPF, as described in the SR-TE LSP Path Computation Using Local CSPF chapter. BFD can be configured per LSP or per path (primary or secondary) in the LSP.

# on PE-4:
configure
    router Base
        mpls
            path "empty"
                no shutdown
            exit
            path "via-PE-2"
                hop 10 192.0.2.2 loose
                no shutdown
            exit
            path "via-PE-3"
                hop 10 192.0.2.3 loose
                no shutdown
            exit
            lsp "LSP-PE-4-PE-5_empty_localCSPF" sr-te
                to 192.0.2.5
                path-computation-method local-cspf
                max-sr-labels 6 additional-frr-labels 2 
                bfd
                    bfd-template "bfd-cpm-np-1s"
                    bfd-enable
                exit
                primary "empty"
                exit
                no shutdown
            exit
            lsp "LSP-PE-4-PE-5_viaPE-2_localCSPF" sr-te
                to 192.0.2.5
                path-computation-method local-cspf
                max-sr-labels 6 additional-frr-labels 2 
                primary "via-PE-2"
                    bfd
                        bfd-template "bfd-cpm-np-1s"
                        bfd-enable
                    exit
                exit
                no shutdown
            exit
            lsp "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd" sr-te
                to 192.0.2.5
                path-computation-method local-cspf
                max-sr-labels 6 additional-frr-labels 2 
                bfd
                    bfd-template "bfd-cpm-np-1s"
                    bfd-enable
                exit
                primary "via-PE-2"
                exit
                secondary "via-PE-3"
                    standby
                exit
                no shutdown
            exit

The head-end or initiator node PE-4 learned the S-BFD reflector discriminator for PE-5 (524292), so the BFD control packets can be sent with both a BFD and S-BFD discriminator value. The BFD control packets follow the data path from head end to tail end. The return path is native IP.

The first S-BFD session on initiator node PE-4 gets BFD discriminator 1, the second BFD discriminator 2, and so on. The S-BFD discriminator for PE-5 remains the same: 524292. For "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd", with primary and secondary path, two S-BFD sessions are established: one with BFD discriminator 3 and another with BFD discriminator 4, as follows:

*A:PE-4# show router bfd seamless-bfd session 
                          lsp-name "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd" detail

===============================================================================
BFD Session
===============================================================================
Prefix         : 192.0.2.5/32
Local Address  : 192.0.2.4
LSP Name       : LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
LSP Index      : 65538                    Path LSP ID      : 18432
Fec Type       : srTe
Oper State     : Up                       Protocols        : mplsLsp
Up Time        : 0d 00:01:29              Up Transitions   : 1
Last Down Time : 0d 00:00:01              Down Transitions : 0
                                          Version Mismatch : 0

Forwarding Information

Local Discr    : 3                        Local State      : Up
Local Diag     : 0 (None)
Local Mode     : Demand
Local Min Tx   : 1000                     Local Mult       : 3
Last Sent (ms) : 0                        Local Min Rx     : 0
Type           : cpm-np
Remote Discr   : 524292                   Remote State     : Up
Remote Diag    : 0 (None)                 Remote Mode      : Async
Remote Min Tx  : 1000                     Remote Mult      : 3
Remote C-flag  : 1
Last Recv (ms) : 0                        Remote Min Rx    : 3
===============================================================================
Prefix         : 192.0.2.5/32
Local Address  : 192.0.2.4
LSP Name       : LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
LSP Index      : 65538                    Path LSP ID      : 18434
Fec Type       : srTe
Oper State     : Up                       Protocols        : mplsLsp
Up Time        : 0d 00:01:29              Up Transitions   : 1
Last Down Time : 0d 00:00:01              Down Transitions : 0
                                          Version Mismatch : 0

Forwarding Information

Local Discr    : 4                        Local State      : Up
Local Diag     : 0 (None)
Local Mode     : Demand
Local Min Tx   : 1000                     Local Mult       : 3
Last Sent (ms) : 0                        Local Min Rx     : 0
Type           : cpm-np
Remote Discr   : 524292                   Remote State     : Up
Remote Diag    : 0 (None)                 Remote Mode      : Async
Remote Min Tx  : 1000                     Remote Mult      : 3
Remote C-flag  : 1
Last Recv (ms) : 0                        Remote Min Rx    : 3
===============================================================================
===============================================================================

In the preceding show command, "Local Discr: 3" and "Local Discr: 4" refer to the BFD discriminator values on the initiator node PE-4, while "Remote Discr: 524292" refers to the S-BFD reflector discriminator value on the responder node PE-5.

The following command shows that the primary path "via-PE-2" goes from PE-4 via PE-2 and PE-1 to PE-5; the secondary path "via-PE-3" goes from PE-4 via PE-3 and PE-7 to PE-5:

*A:PE-4# show router mpls sr-te-lsp "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd" path detail

===============================================================================
MPLS SR-TE LSP LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path  (Detail)
===============================================================================
Legend :
    S      - Strict                      L      - Loose
    A-SID  - Adjacency SID               N-SID  - Node SID
    +      - Inherited
===============================================================================
-------------------------------------------------------------------------------
LSP SR-TE LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path  via-PE-2
-------------------------------------------------------------------------------
LSP Name    : LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path LSP ID      : 18432
From             : 192.0.2.4
To               : 192.0.2.5
Admin State      : Up                      Oper State        : Up
Path Name   : via-PE-2
Path Type        : Primary
Path Admin       : Up                      Path Oper         : Up
---snip---

Explicit Hops    :
                  192.0.2.2(L)
Actual Hops      :
    192.168.24.1(192.0.2.2)(A-SID)               Record Label        : 524285
 -> 192.168.12.1(192.0.2.1)(A-SID)               Record Label        : 524287
 -> 192.168.15.2(192.0.2.5)(A-SID)               Record Label        : 524284

BFD Configuration and State
Template         : None                    Ping Interval     : N/A
Enable           : False                   State             : up
ReturnPathLabel  : None
WaitForUpTimer   : 4 sec                   OperWaitForUpTimer: 4 sec
WaitForUpTmLeft  : 0
StartFail Rsn    : N/A

-------------------------------------------------------------------------------
LSP SR-TE LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path  via-PE-3
-------------------------------------------------------------------------------
LSP Name    : LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path LSP ID      : 18434
From             : 192.0.2.4
To               : 192.0.2.5
Admin State      : Up                      Oper State        : Up
Path Name   : via-PE-3
Path Type        : Standby
Path Admin       : Up                      Path Oper         : Up
---snip---

Explicit Hops    :
                  192.0.2.3(L)
Actual Hops      :
    192.168.34.1(192.0.2.3)(A-SID)               Record Label        : 524287
 -> 192.168.37.2(192.0.2.7)(A-SID)               Record Label        : 524284
 -> 192.168.57.1(192.0.2.5)(A-SID)               Record Label        : 524287
Srlg             : Disabled                Srlg Disjoint     : False

BFD Configuration and State
Template         : None                    Ping Interval     : N/A
Enable           : False                   State             : up
ReturnPathLabel  : None
WaitForUpTimer   : 4 sec                   OperWaitForUpTimer: 4 sec
WaitForUpTmLeft  : 0
StartFail Rsn    : N/A

===============================================================================

The following OAM LSP trace from PE-4 shows that the path goes via PE-2 and PE-1 to PE-5:

*A:PE-4# oam lsp-trace sr-te "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd"
lsp-trace to LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd: 0 hops min, 0 hops max, 176 byte packets
1  192.0.2.2  rtt=2.15ms rc=3(EgressRtr) rsc=3
1  192.0.2.2  rtt=2.76ms rc=8(DSRtrMatchLabel) rsc=2
2  192.0.2.1  rtt=2.95ms rc=3(EgressRtr) rsc=2
2  192.0.2.1  rtt=3.68ms rc=8(DSRtrMatchLabel) rsc=1
3  192.0.2.5  rtt=3.42ms rc=3(EgressRtr) rsc=1
S-BFD session down without failure action

Failure on remote link in primary path shows the two paths of "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd" with a failure on the link between PE-2 and PE-1, which is part of the primary path "via-PE-2". The broken link is remote to the head-end node PE-4. The failure is emulated on PE-2 by disabling the port toward PE-1.

Figure 5. Failure on remote link in primary path

As a result, the BFD session associated with the primary path "via-PE-2" goes down, as follows:

*A:PE-4# show router bfd seamless-bfd session 
                                lsp-name "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd"

===============================================================================
Legend:
  Session Id = Interface Name | LSP Name | Prefix | RSVP Sess Name | Service Id
  wp = Working path   pp = Protecting path
===============================================================================
BFD Session
===============================================================================
Session Id                                        State      Tx Pkts    Rx Pkts
  Rem Addr/Info/SdpId:VcId                      Multipl     Tx Intvl   Rx Intvl
  Protocols                                        Type     LAG Port     LAG ID
  Loc Addr                                                             LAG name
-------------------------------------------------------------------------------
192.0.2.5/32                                       Down          N/A        N/A
  192.0.2.5                                           3         1000          0
  mplsLsp                                        cpm-np          N/A        N/A
  192.0.2.4
192.0.2.5/32                                         Up          N/A        N/A
  192.0.2.5                                           3         1000       1000
  mplsLsp                                        cpm-np          N/A        N/A
  192.0.2.4
-------------------------------------------------------------------------------
No. of BFD sessions: 2
===============================================================================

By default, there is no failure action on the BFD session, so the primary path remains up even when the BFD session on that path is down, as follows:

*A:PE-4# show router mpls sr-te-lsp "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd" path detail

===============================================================================
MPLS SR-TE LSP LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path  (Detail)
===============================================================================
Legend :
    S      - Strict                      L      - Loose
    A-SID  - Adjacency SID               N-SID  - Node SID
    +      - Inherited
===============================================================================
-------------------------------------------------------------------------------
LSP SR-TE LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path  via-PE-2
-------------------------------------------------------------------------------
LSP Name    : LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path LSP ID      : 18432
From             : 192.0.2.4
To               : 192.0.2.5
Admin State      : Up                      Oper State        : Up
Path Name   : via-PE-2
Path Type        : Primary
Path Admin       : Up                      Path Oper         : Up
---snip---

Explicit Hops    :
                  192.0.2.2(L)
Actual Hops      :
    192.168.24.1(192.0.2.2)(A-SID)               Record Label        : 524285
 -> 192.168.12.1(192.0.2.1)(A-SID)               Record Label        : 524287
 -> 192.168.15.2(192.0.2.5)(A-SID)               Record Label        : 524284

BFD Configuration and State
Template         : None                    Ping Interval     : N/A
Enable           : False                   State             : down
ReturnPathLabel  : None
WaitForUpTimer   : 4 sec                   OperWaitForUpTimer: 4 sec
WaitForUpTmLeft  : 0
StartFail Rsn    : N/A
---snip---

The LSP and its paths remain up and the corresponding SR-TE tunnel in the tunnel table remains unchanged, so the traffic using the LSP will be blackholed. The following tunnel table lists three SR-TE tunnels, corresponding to:

  • "LSP-PE-4-PE-5_empty_localCSPF", with next-hop 192.168.48.2 (PE-8)

  • "LSP-PE-4-PE-5_viaPE-2_localCSPF", using path "via-PE-2", with next-hop 192.168.24.1

  • "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd", using path "via-PE-2" (the primary path is used, not the secondary), with next-hop 192.168.24.1.

*A:PE-4# show router tunnel-table protocol sr-te

===============================================================================
IPv4 Tunnel Table (Router: Base)
===============================================================================
Destination           Owner     Encap TunnelId  Pref   Nexthop        Metric
   Color
-------------------------------------------------------------------------------
192.0.2.5/32          sr-te     MPLS  655362    8      192.168.48.2   20
192.0.2.5/32          sr-te     MPLS  655363    8      192.168.24.1   30
192.0.2.5/32          sr-te     MPLS  655364    8      192.168.24.1   30
-------------------------------------------------------------------------------
Flags: B = BGP or MPLS backup hop available
       L = Loop-Free Alternate (LFA) hop available
       E = Inactive best-external BGP route
       k = RIB-API or Forwarding Policy backup hop
===============================================================================

The OAM LSP ping command using the SR-TE LSP "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd" fails, as follows:

*A:PE-4# oam lsp-ping sr-te "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd"
LSP-PING LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd: 96 bytes MPLS payload
Request timed out.

---- LSP LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd PING Statistics ----
1 packets sent, 0 packets received, 100.00% packet loss

The OAM LSP trace command shows that the LSP trace stops at PE-2 (192.0.2.2):

*A:PE-4# oam lsp-trace sr-te "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd"
lsp-trace to LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd: 0 hops min, 0 hops max, 176 byte packets
1  192.0.2.2  rtt=2.41ms rc=3(EgressRtr) rsc=3 
1  192.0.2.2  rtt=2.56ms rc=8(DSRtrMatchLabel) rsc=2 
2  192.168.12.1  * 
3  192.168.12.1  * 
4  192.168.12.1  * 
5  192.168.12.1  * 
6  192.168.12.1  * 
S-BFD session down with failure action

To force a failover to the secondary path or to bring the LSP down when the BFD session goes down, a failure action needs to be configured in the BFD context of the LSP, as follows:

# on PE-4:
configure
    router Base
        mpls
            lsp "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd"
                bfd
                    bfd-template "bfd-cpm-np-1s"
                    bfd-enable
                    failure-action failover-or-down
                    wait-for-up-timer 4    # default; applicable for failure action
                exit

The failure action failover-or-down is the only failure action that is allowed for SR-TE LSPs. An error is raised when attempting to configure failure action down or failure action failover, as follows:

*A:PE-4>config>router>mpls>lsp>bfd# failure-action down 
MINOR: MPLS #1013 LSP parameter conflict - Cannot configure 'failure-action down' for
 SR-TE LSPs

*A:PE-4>config>router>mpls>lsp>bfd# failure-action failover 
MINOR: MPLS #1013 LSP parameter conflict - Cannot configure 'failure-action failover'
 for SR-TE LSPs

When the failure action is configured, the primary path "via-PE-2" goes down and a failover takes place to the secondary path "via-PE-3" (if available). When no secondary paths are available, the LSP is operational down.

When a link or node fails on the primary path, the BFD state goes down for the primary path. The head-end node switches to the best preference standby that is up. When the LSP retry timer expires, the MPLS module initiates a local CSPF request to calculate a new SR-TE path. When it is possible to calculate a new path meeting the path constraints for the primary path, the new path is added to the SR-TE tunnel, and S-BFD for the primary path is started. S-BFD comes up and the LSP metric is set.

By default, the revert timer is zero, so no reversion to the primary path takes place. However, if the revert timer is configured to a non-zero value, the revert timer starts when the S-BFD session comes up. When the revert timer expires, the active path is reprogrammed from secondary to primary. If pce-report-enable is configured, a PCEP status report is sent for each path, so two reports are sent.

The following command shows that the primary path "via-PE-2" is down and the list of actual hops is empty. Therefore, the S-BFD session state is not applicable. The secondary path remains up and the LSP is up.

*A:PE-4# show router mpls sr-te-lsp "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd" path detail

===============================================================================
MPLS SR-TE LSP LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path  (Detail)
===============================================================================
Legend :
    S      - Strict                      L      - Loose
    A-SID  - Adjacency SID               N-SID  - Node SID
    +      - Inherited
===============================================================================
-------------------------------------------------------------------------------
LSP SR-TE LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path  via-PE-2
-------------------------------------------------------------------------------
LSP Name    : LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path LSP ID      : 12800
From             : 192.0.2.4
To               : 192.0.2.5
Admin State      : Up                      Oper State        : Up
Path Name   : via-PE-2
Path Type        : Primary
Path Admin       : Up                      Path Oper         : Down
---snip---

Failure Code     : bfdDown
Failure Node     : 192.0.2.4
Explicit Hops    :
                  192.0.2.2(L)
Actual Hops      :
    No Hops Specified

BFD Configuration and State
Template         : None                    Ping Interval     : N/A
Enable           : False                   State             : notApplicable
ReturnPathLabel  : None
WaitForUpTimer   : 4 sec                   OperWaitForUpTimer: 4 sec
WaitForUpTmLeft  : 0
StartFail Rsn    : N/A

-------------------------------------------------------------------------------
LSP SR-TE LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path  via-PE-3
-------------------------------------------------------------------------------
LSP Name    : LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd
Path LSP ID      : 12802
From             : 192.0.2.4
To               : 192.0.2.5
Admin State      : Up                      Oper State        : Up
Path Name   : via-PE-3
Path Type        : Standby
Path Admin       : Up                      Path Oper         : Up
---snip---

Explicit Hops    :
                  192.0.2.3(L)
Actual Hops      :
    192.168.34.1(192.0.2.3)(A-SID)               Record Label        : 524287
 -> 192.168.37.2(192.0.2.7)(A-SID)               Record Label        : 524284
 -> 192.168.57.1(192.0.2.5)(A-SID)               Record Label        : 524285
Srlg             : Disabled                Srlg Disjoint     : False

BFD Configuration and State
Template         : None                    Ping Interval     : N/A
Enable           : False                   State             : up
ReturnPathLabel  : None
WaitForUpTimer   : 4 sec                   OperWaitForUpTimer: 4 sec
WaitForUpTmLeft  : 0
StartFail Rsn    : N/A

===============================================================================

The tunnel table shows an entry with tunnel ID 655364, which corresponds to "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd", with next-hop 192.168.34.1 (PE-3):

*A:PE-4# show router tunnel-table protocol sr-te

===============================================================================
IPv4 Tunnel Table (Router: Base)
===============================================================================
Destination           Owner     Encap TunnelId  Pref   Nexthop        Metric
   Color
-------------------------------------------------------------------------------
192.0.2.5/32          sr-te     MPLS  655362    8      192.168.14.1   20
192.0.2.5/32          sr-te     MPLS  655363    8      192.168.24.1   30
192.0.2.5/32          sr-te     MPLS  655364    8      192.168.34.1   30
-------------------------------------------------------------------------------
Flags: B = BGP or MPLS backup hop available
       L = Loop-Free Alternate (LFA) hop available
       E = Inactive best-external BGP route
       k = RIB-API or Forwarding Policy backup hop
===============================================================================

The OAM LSP trace using "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd" shows that the active path goes via PE-3 and PE-7 to PE-5, as follows:

*A:PE-4# oam lsp-trace sr-te "LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd"
lsp-trace to LSP-PE-4-PE-5_viaPE-2_localCSPF_2nd: 0 hops min, 0 hops max,
 176 byte packets
1  192.0.2.3  rtt=1.90ms rc=3(EgressRtr) rsc=3
1  192.0.2.3  rtt=2.48ms rc=8(DSRtrMatchLabel) rsc=2
2  192.0.2.7  rtt=3.00ms rc=3(EgressRtr) rsc=2
2  192.0.2.7  rtt=2.88ms rc=8(DSRtrMatchLabel) rsc=1
3  192.0.2.5  rtt=3.41ms rc=3(EgressRtr) rsc=1

S-BFD for SR-TE LSPs with controlled return path

In this mode, a controlled return path for BFD reply packets is configured at the initiating node. The reflector function at the far end of the SR-TE LSP is bypassed, so there is no need to configure reflector discriminators for these sessions.

The initiating node pushes an additional MPLS label on S-BFD packets at the bottom of the stack and the BFD session operates in echo mode. The return path label refers to an MPLS binding SID of an SR policy programmed at the far end of the SR-TE LSP. The SR policy can be used to forward BFD reply packets along an explicit TE path back to the initiator, avoiding the IGP shortest path.

It is possible to configure a specific TE return path for each S-BFD session on an SR-TE LSP at the initiating node. The SR policies can have segments lists with different paths, ensuring the BFD reply packets from different LSP paths do not share the same outcome.

In the following example, initiating node PE-1 has three SR-TE LSPs to far end PE-8:
  • SR-TE LSP "LSP-PE-1-PE-8_empty_localCSPF" with an empty primary path and return path label 20041
  • SR-TE LSP "LSP-PE-1-PE-8_viaPE-2_localCSPF" with primary path "via-PE-2" and return path label 20621
  • SR-TE LSP "LSP-PE-1-PE-8_viaPE-2_localCSPF_2nd" with primary path "via-PE-2" and return path label 20621 and secondary path "via-PE-3" and return path label 20051

The configuration of the paths and the SR-TE LSPs on PE-1 is as follows:

# on PE-1:
configure
    router Base
        bfd
            begin
            bfd-template "bfd-cpm-np-1s"
                type "cpm-np"
                transmit-interval 1000
                receive-interval 1000
            exit
            commit
        exit
        mpls
            path "empty"
                no shutdown
            exit
            path "via-PE-2"
                hop 10 192.0.2.2 loose
                no shutdown
            exit
            path "via-PE-3"
                hop 10 192.0.2.3 loose
                no shutdown
            exit
            lsp "LSP-PE-1-PE-8_empty_localCSPF" sr-te
                to 192.0.2.8
                path-computation-method local-cspf
                max-sr-labels 6 additional-frr-labels 2 
                pce-report enable
                bfd
                    bfd-template "bfd-cpm-np-1s"
                    return-path-label 20041
                    bfd-enable
                    failure-action failover-or-down
                exit
                primary "empty"
                exit
                no shutdown
            exit
            lsp "LSP-PE-1-PE-8_viaPE-2_localCSPF" sr-te
                to 192.0.2.8
                path-computation-method local-cspf
                max-sr-labels 6 additional-frr-labels 2 
                pce-report enable
                bfd
                    failure-action failover-or-down
                exit
                primary "via-PE-2"
                    bfd
                        bfd-template "bfd-cpm-np-1s"
                        return-path-label 20621
                        bfd-enable
                    exit
                exit
                no shutdown
            exit
            lsp "LSP-PE-1-PE-8_viaPE-2_localCSPF_2nd" sr-te
                to 192.0.2.8
                path-computation-method local-cspf
                max-sr-labels 6 additional-frr-labels 2 
                pce-report enable
                bfd
                    failure-action failover-or-down
                exit
                primary "via-PE-2"
                    bfd
                        bfd-template "bfd-cpm-np-1s"
                        return-path-label 20621
                        bfd-enable
                    exit
                exit
                secondary "via-PE-3"
                    standby
                    bfd
                        bfd-template "bfd-cpm-np-1s"
                        return-path-label 20051
                        bfd-enable
                    exit
                exit
                no shutdown
            exit
            no shutdown

The return path labels correspond to binding SIDs in SR policies on PE-8, as follows:

# on PE-8:
configure
    router Base
        mpls-labels
            sr-labels start 32000 end 32999
            reserved-label-block "SRLB1"
                start-label 20000 end-label 21999
            exit
        exit
        segment-routing
            sr-policies
                reserved-label-block "SRLB1"
                static-policy "SR-static-policy-PE-4-PE-1" create
                    binding-sid 20041
                    color 810
                    distinguisher 10020041
                    endpoint 192.0.2.1
                    head-end local
                    segment-list 1 create
                        segment 1 create
                            mpls-label 32004    # node SID of PE-4
                        exit
                        segment 2 create
                            mpls-label 32001    # node SID of PE-1
                        exit
                        no shutdown
                    exit
                    no shutdown
                exit
                static-policy "SR-static-policy-PE-5-PE-1" create
                    binding-sid 20051
                    color 820
                    distinguisher 10020051
                    endpoint 192.0.2.1
                    head-end local
                    segment-list 1 create
                        segment 1 create
                            mpls-label 32005    # node SID of PE-5
                        exit
                        segment 2 create
                            mpls-label 32001    # node SID of PE-1
                        exit
                        no shutdown
                    exit
                    no shutdown
                exit
                static-policy "SR-static-policy-PE-6-PE-2-PE-1" create
                    binding-sid 20621
                    color 830
                    distinguisher 10020621
                    endpoint 192.0.2.1
                    head-end local
                    segment-list 1 create
                        segment 1 create
                            mpls-label 32006    # node SID of PE-6
                        exit
                        segment 2 create
                            mpls-label 32002    # node SID of PE-2
                        exit
                        segment 3 create
                            mpls-label 32001    # node SID of PE-1
                        exit
                        no shutdown
                    exit
                    no shutdown
                exit
                no shutdown
            exit

The tunnel table on PE-8 contains three SR-policy tunnels to PE-1:

*A:PE-8# show router tunnel-table protocol sr-policy

===============================================================================
IPv4 Tunnel Table (Router: Base)
===============================================================================
Destination           Owner     Encap TunnelId  Pref   Nexthop        Metric
   Color
-------------------------------------------------------------------------------
192.0.2.1/32          sr-policy MPLS  917506    14     192.0.2.4      0
   810
192.0.2.1/32          sr-policy MPLS  917507    14     192.0.2.5      0
   820
192.0.2.1/32          sr-policy MPLS  917508    14     192.0.2.6      0
   830
-------------------------------------------------------------------------------
Flags: B = BGP or MPLS backup hop available
       L = Loop-Free Alternate (LFA) hop available
       E = Inactive best-external BGP route
       k = RIB-API or Forwarding Policy backup hop
===============================================================================

Four S-BFD sessions are up between PE-1 and PE-8:

*A:PE-1# show router bfd session

===============================================================================
Legend:
  Session Id = Interface Name | LSP Name | Prefix | RSVP Sess Name | Service Id
  wp = Working path   pp = Protecting path
===============================================================================
BFD Session
===============================================================================
Session Id                                        State      Tx Pkts    Rx Pkts
  Rem Addr/Info/SdpId:VcId                      Multipl     Tx Intvl   Rx Intvl
  Protocols                                        Type     LAG Port     LAG ID
  Loc Addr                                                             LAG name
-------------------------------------------------------------------------------
192.0.2.8/32                                         Up          N/A        N/A
  192.0.2.8                                           3         1000       1000
  mplsLsp                                        cpm-np          N/A        N/A
  192.0.2.1
192.0.2.8/32                                         Up          N/A        N/A
  192.0.2.8                                           3         1000       1000
  mplsLsp                                        cpm-np          N/A        N/A
  192.0.2.1
192.0.2.8/32                                         Up          N/A        N/A
  192.0.2.8                                           3         1000       1000
  mplsLsp                                        cpm-np          N/A        N/A
  192.0.2.1
192.0.2.8/32                                         Up          N/A        N/A
  192.0.2.8                                           3         1000       1000
  mplsLsp                                        cpm-np          N/A        N/A
  192.0.2.1
-------------------------------------------------------------------------------
No. of BFD sessions: 4
===============================================================================

When the SR policies on PE-8 are down, the corresponding BFD sessions on PE-1 go down.

On PE-1, SR-TE LSP "LSP-PE-1-PE-8_viaPE-2_localCSPF_2nd" has a primary path and a standby secondary path. The local discriminator for the primary path is 3; for the secondary path 4. No remote discriminators are used when the return path corresponds to an SR policy, so the remote discrimators equal zero. The return path label is the binding SID of the SR policy in the far end node.

*A:PE-1# show router bfd seamless-bfd session 
                          lsp-name "LSP-PE-1-PE-8_viaPE-2_localCSPF_2nd" detail

===============================================================================
BFD Session
===============================================================================
Prefix         : 192.0.2.8/32
Local Address  : 192.0.2.1
LSP Name       : LSP-PE-1-PE-8_viaPE-2_localCSPF_2nd
LSP Index      : 65538                    Path LSP ID      : 2
Fec Type       : srTe
Return Path    : 20621
Oper State     : Up                       Protocols        : mplsLsp
Up Time        : 0d 00:04:23              Up Transitions   : 1
Last Down Time : 0d 00:00:12              Down Transitions : 0
                                          Version Mismatch : 0

Forwarding Information

Local Discr    : 3                        Local State      : Up
Local Diag     : 0 (None)
Local Mode     : Demand
Local Min Tx   : 1000                     Local Mult       : 3
Last Sent (ms) : 0                        Local Min Rx     : 1000
Type           : cpm-np
Remote Discr   : 0                        Remote State     : Up
Remote Diag    : 0 (None)                 Remote Mode      : Demand
Remote Min Tx  : 1000                     Remote Mult      : 3
Remote C-flag  : 1
Last Recv (ms) : 0                        Remote Min Rx    : 1000
===============================================================================
Prefix         : 192.0.2.8/32
Local Address  : 192.0.2.1
LSP Name       : LSP-PE-1-PE-8_viaPE-2_localCSPF_2nd
LSP Index      : 65538                    Path LSP ID      : 4
Fec Type       : srTe
Return Path    : 20051
Oper State     : Up                       Protocols        : mplsLsp
Up Time        : 0d 00:04:24              Up Transitions   : 1
Last Down Time : 0d 00:00:11              Down Transitions : 0
                                          Version Mismatch : 0

Forwarding Information

Local Discr    : 4                        Local State      : Up
Local Diag     : 0 (None)
Local Mode     : Demand
Local Min Tx   : 1000                     Local Mult       : 3
Last Sent (ms) : 0                        Local Min Rx     : 1000
Type           : cpm-np
Remote Discr   : 0                        Remote State     : Up
Remote Diag    : 0 (None)                 Remote Mode      : Demand
Remote Min Tx  : 1000                     Remote Mult      : 3
Remote C-flag  : 1
Last Recv (ms) : 0                        Remote Min Rx    : 1000
===============================================================================
===============================================================================

Conclusion

Seamless BFD for SR-TE LSPs allows fast connectivity checking of the data plane of the LSP. This can be used to trigger fast failover from the currently active to a standby path.