Multi-Chassis APS and Pseudowire Redundancy Interworking

This chapter describes multi-chassis APS and pseudowire redundancy interworking.

Topics in this chapter include:

Applicability

This chapter was initially written for SR OS Release 6.0.R2, but the CLI in the current edition is based on SR OS Release 19.10.R2.The configuration in this chapter includes the use of the ATM ports. See the Release Notes for information about support of ATM (and other) MDAs on various platforms as well as MC-APS restrictions.

Overview

MC-APS

MC-APS is an extension to the APS feature to provide not only link redundancy but also node level redundancy. It can protect against nodal failure by configuring the working circuit of an APS group on one node while configuring the protect circuit of the same APS group on a different node.

The two nodes connect to each other with an IP link that is used to establish a signaling path between them. The relevant APS groups in both the working and protection routers must have the same group ID and working circuit, and the protect circuit must have compatible configurations (such as the same speed, framing, and port type). Signaling is provided using the direct connection between the two service routers. A heartbeat protocol can be used to add robustness to the interaction between the two routers.

Signaling functionality includes support for:

  • APS group matching between service routers.

  • Verification that one side is configured as a working circuit and the other side is configured as the protect circuit. In case of a mismatch, a trap (incompatible-neighbor) is generated.

  • Change in working circuit status is sent from the working router to keep the protection router in sync.

  • Protection router, based on K1/K2 byte data, member circuit status, and external request, selects the active circuit and informs the working router to activate or de-activate the working circuit.

Pseudowire redundancy

Pseudowire (PW) redundancy provides the ability to protect a pseudowire with a pre-provisioned pseudowire and to switch traffic over to the secondary standby pseudowire in case of a SAP and/or network failure condition. Normally, pseudowires are redundant by the virtue of the SDP redundancy mechanism. For instance, if the SDP is an RSVP LSP and is protected by a secondary standby path and/or by Fast-Reroute paths, the pseudowire is also protected.

However, there are a few applications in which SDP redundancy does not protect the end-to-end pseudowire path when there are two different destination 7x50 PE nodes for the same VLL service. The main use case is the provisioning of dual-homing of a CPE or access node to two 7x50 PE nodes located in different POPs. The other use case is the provisioning of a pair of active and standby BRAS nodes, or active and standby links to the same BRAS node, to provide service resiliency to broadband service subscribers.

Example topology

The setup in this section contains two access nodes and 4 PE nodes. The access nodes can be any ATM switches that support 1+1 bi-directional APS. MC-APS network topology shows the physical topology of the setup. Access node and network resilience (part 2) shows the use of both MC-APS in the access network and pseudowire redundancy in the core network to provide a resilient end-to-end VLL service.

Figure 1. MC-APS network topology
Figure 2. Access node and network resilience (part 1)
Figure 3. Access node and network resilience (part 2)

Configuration

The following configuration should be completed on the PEs before configuring MC-APS:

  • Cards, MDAs and ports

  • Router interfaces

  • IGP configured and converged

  • MPLS

  • SDPs configured between all PE routers

For the IGP, OSPF or IS-IS can be used. MPLS or GRE can be used for the transport tunnels. Also, several protocols can be used for signaling MPLS labels. In this example, OSPF and LDP are used. The following commands can be used to check if OSPF has converged and to make sure the SDPs are up (for example, on PE-1):

*A:PE-1# show router route-table
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric
-------------------------------------------------------------------------------
192.0.2.1/32                                  Local   Local     01h29m06s  0
       system                                                       0
192.0.2.2/32                                  Remote  OSPF      01h23m18s  10
       192.168.12.2                                                 100
192.0.2.3/32                                  Remote  OSPF      01h17m45s  10
       192.168.13.2                                                 100
192.0.2.4/32                                  Remote  OSPF      01h17m30s  10
       192.168.12.2                                                 200
192.168.12.0/30                               Local   Local     01h29m06s  0
       int-PE-1-PE-2                                                0
192.168.13.0/30                               Local   Local     01h29m06s  0
       int-PE-1-PE-3                                                0
192.168.24.0/30                               Remote  OSPF      01h23m18s  10
       192.168.12.2                                                 200
192.168.34.0/30                               Remote  OSPF      01h17m45s  10
       192.168.13.2                                                 200
-------------------------------------------------------------------------------
No. of Routes: 8
Flags: n = Number of times nexthop is repeated
       B = BGP backup route available
       L = LFA nexthop available
       S = Sticky ECMP requested
===============================================================================
*A:PE-1# show service sdp
============================================================================
Services: Service Destination Points
============================================================================
SdpId  AdmMTU  OprMTU  Far End          Adm  Opr         Del     LSP   Sig
----------------------------------------------------------------------------
12     0       1492    192.0.2.2        Up   Up          MPLS    L     TLDP
13     0       1492    192.0.2.3        Up   Up          MPLS    L     TLDP
14     0       1492    192.0.2.4        Up   Up          MPLS    L     TLDP
----------------------------------------------------------------------------
Number of SDPs : 3
----------------------------------------------------------------------------
Legend: R = RSVP, L = LDP, B = BGP, M = MPLS-TP, n/a = Not Applicable
        I = SR-ISIS, O = SR-OSPF, T = SR-TE, F = FPE
============================================================================

Step 1. APS configuration on MSANs

The access nodes can be any ATM switches that support 1+1 bi-directional APS. Here is an example on 7670RSP (Routing Switching Platform).

RSP> configure
RSP> port 1-6-1-1
RSP> options protection type 1+1
RSP> options protection switching bidirect
RSP> options protection
                                                                    (Standby)
#                  Type        Status           Name
1-6-1-1            STM1_IR8    OK
Protection Group Contains:
  Protection Port     : 1-6-1-1     (Standby)
  Working Port        : 1-5-1-1
Protection Type       : 1+1
Switching Type        : Non-Revertive
Switching Mode        : Bi-directional
Wait-To-Restore Timer :  5 minute(s)

Step 2. MC-APS configuration on PE-1 and PE-2

Assuming the link between MSAN and PE-1 is working circuit and the link between MSAN and PE-2 is protection circuit.

Configure APS on the PE-1 port. Specify the system IP address of neighbor node (PE-2) and working-circuit.

# on PE-1
configure
    port 1/2/1
        sonet-sdh
        exit
        no shutdown
    exit
    port aps-1
        aps
            neighbor 192.0.2.2
            working-circuit 1/2/1
        exit
        sonet-sdh
            path sts3
                encap-type atm
                crc 32
                atm
                exit
                no shutdown
            exit
        exit
        no shutdown
    exit

Configure APS on the PE-2 port. Specify the system IP address of neighbor node (PE-1) and protect-circuit instead of working-circuit.

# on PE-2
configure
    port 1/2/1 
        sonet-sdh
        exit
        no shutdown
    exit
    port aps-1
        aps
            neighbor 192.0.2.1
            protect-circuit 1/2/1
        exit
        sonet-sdh
            path sts3
                encap-type atm
                crc 32
                atm
                exit
                no shutdown
            exit
        exit
        no shutdown

The following parameters can optionally be configured under APS.

  • advertise-interval — This command specifies the time interval, in 100s of milliseconds, between 'I am operational' messages sent by both protect and working circuits to their neighbor for multi-chassis APS.

  • hold-time — This command specifies how much time can pass, in 100s of milliseconds, without receiving an advertise packet from the neighbor before the multi-chassis signaling link is considered not operational.

  • revert-time — This command configures the revert-time timer to determine how long to wait before switching back to the working circuit after that circuit has been restored into service.

  • switching-mode — This command configures the switching mode for the APS port which can be bi-directional or uni-directional.

Step 3. Verify the APS status on PE-1.

*A:PE-1# show port aps-1

===============================================================================
SONET/SDH Interface
===============================================================================
Description        : APS Group
Interface          : aps-1                  Speed                : oc3
Admin Status       : up                     Oper Status          : up
Physical Link      : Yes                    Loopback Mode        : none
Single Fiber Mode  : No                     
Clock Source       : loop                   Framing              : sonet
Last State Change  : 01/10/2020 09:38:21    Port IfIndex         : 1358987264
Configured Address : 04:0f:ff:00:02:49      
Hardware Address   : 04:0f:ff:00:02:49      
Last Cleared Time  : N/A                    
J0 String          : 0x01                   Section Trace Mode   : byte
Rx S1 Byte         : 0x00 (stu)             Rx K1/K2 Byte        : 0x00/0x00
Tx S1 Byte         : 0x0f (dus)             Tx DUS/DNU           : Disabled
Rx J0 String (Hex) : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
Cfg Alarm          : loc lrdi lb2er-sf slof slos 
Alarm Status       : 
BER SD Threshold   : 6                      BER SF Threshold     : 3
Hold time up       : 500 milliseconds       Reset On Path Down   : Disabled
Hold time down     : 200 milliseconds

===============================================================================
Port Statistics
===============================================================================
                                                   Input                 Output
-------------------------------------------------------------------------------
Packets                                                0                      0
Discards                                               0                      0
Unknown Proto Discards                                 0 
===============================================================================

Step 4. Verify the MC-APS status and parameters on PE-1 and PE-2

Detailed parameters of the APS configuration on PE-1 can be verified, as follows. The admin/oper status of APS group 1 shows up/up. K1/K2 byte shows N/A as APS 1+1 exchanges that information through the protection circuit. The admin/oper status of the working circuit (the link between MSAN and PE-1) is up/up.

*A:PE-1# show aps detail

===============================================================================
APS Group: aps-1
===============================================================================
Description        : APS Group
Group Id           : 1                    Active Circuit       : 1/2/1
Admin Status       : Up                   Oper Status          : Up
Working Circuit    : 1/2/1                Protection Circuit   : N/A
Switching-mode     : Bi-directional       Switching-arch       : 1+1(sig-only)
Annex B            : No
Revertive-mode     : Non-revertive        Revert-time (min)    :
Rx K1/K2 byte      : N/A
Tx K1/K2 byte      : N/A
Current APS Status : OK
Multi-Chassis APS  : Yes
Neighbor           : 192.0.2.2
Control link state : Up
Advertise Interval : 1000 msec            Hold Time            : 3000 msec
Mode mismatch Cnt  : 0                    Channel mismatch Cnt : 0
PSB failure Cnt    : 0                    FEPL failure Cnt     : 0
-------------------------------------------------------------------------------
 APS Working Circuit - 1/2/1
-------------------------------------------------------------------------------
Admin Status       : Up                   Oper Status          : Up
Current APS Status : OK                   No. of Switchovers   : 0
Last Switchover    : None                 Switchover seconds   : 0
Signal Degrade Cnt : 0                    Signal Failure Cnt   : 0
Last Switch Cmd    : N/A                  Last Exercise Result : N/A
Tx L-AIS           : None
===============================================================================

Detailed parameters of the APS configuration on PE-2 can be verified, as follows. The admin/oper status of APS group 1 shows up/up. Both Rx and Tx of the K1/K2 byte are in the status of 0x00/0x05 (No-Req on Protect) as there is no failure or force-switchover request. The admin/oper status of the protection circuit (the link between MSAN and PE-2) is up/up.

*A:PE-2# show aps detail
 
===============================================================================
APS Group: aps-1
===============================================================================
Description        : APS Group
Group Id           : 1                    Active Circuit       : N/A
Admin Status       : Up                   Oper Status          : Up
Working Circuit    : N/A                  Protection Circuit   : 1/2/1
Switching-mode     : Bi-directional       Switching-arch       : 1+1(sig-only)
Annex B            : No                   
Revertive-mode     : Non-revertive        Revert-time (min)    :  
Rx K1/K2 byte      : 0x00/0x05 (No-Req on Protect)
Tx K1/K2 byte      : 0x00/0x05 (No-Req on Protect)
Current APS Status : OK                   
Multi-Chassis APS  : Yes
Neighbor           : 192.0.2.1
Control link state : Up
Advertise Interval : 1000 msec            Hold Time            : 3000 msec
Mode mismatch Cnt  : 0                    Channel mismatch Cnt : 0
PSB failure Cnt    : 0                    FEPL failure Cnt     : 1
-------------------------------------------------------------------------------
 APS Working Circuit - Neighbor
-------------------------------------------------------------------------------
Admin Status       : N/A                  Oper Status          : N/A
Current APS Status : OK                   No. of Switchovers   : 0
Last Switchover    : None                 Switchover seconds   : 0
Signal Degrade Cnt : 0                    Signal Failure Cnt   : 1
Last Switch Cmd    : No Cmd               Last Exercise Result : Unknown
Tx L-AIS           : None                 
-------------------------------------------------------------------------------
 APS Protection Circuit - 1/2/1
-------------------------------------------------------------------------------
Admin Status       : Up                   Oper Status          : Up
Current APS Status : OK                   No. of Switchovers   : 0
Last Switchover    : None                 Switchover seconds   : 0
Signal Degrade Cnt : 0                    Signal Failure Cnt   : 0
Last Switch Cmd    : No Cmd               Last Exercise Result : Unknown
Tx L-AIS           : None                 
===============================================================================

Step 5. MC-APS configuration on PE-3 and PE-4

The MC-APS configuration on PE-3 and PE-4 is similar to the configuration on PE-1 and PE-2. Configure the working circuit on PE-4 and the protection circuit on PE-3.

Step 6. Pseudowire configuration

Configure an Apipe service on every PE and create endpoints X and Y. Associate the SAPs and spoke SDPs with the endpoints, as shown in Association of SAPs/SDPs and endpoints.

Figure 4. Association of SAPs/SDPs and endpoints

# on PE-1
configure
    service
        apipe 1 name name "Apipe 1" customer 1 create
            endpoint "X" create
            exit
            endpoint "Y" create
            exit
            service-mtu 1400
            sap aps-1:0/32 endpoint "X" create
            exit
            spoke-sdp 13:1 endpoint "Y" create
            exit
            spoke-sdp 14:1 endpoint "Y" create
            exit
            no shutdown
        exit

Syntax aps-1:0/32 specifies the APS group and VPI/VCI of the ATM circuit using the aps-id:vpi/vci format.

Likewise, an Apipe service, with endpoints, SAPs and spoke SDPs must be configured on the other PE routers.

Step 7. Pseudowire verification

The Apipe service is up in PE-1 (MC-APS working circuit), as follows:

*A:PE-1# show service service-using
===============================================================================
Services
===============================================================================
ServiceId    Type      Adm  Opr  CustomerId Service Name
-------------------------------------------------------------------------------
1            Apipe     Up   Up   1          1
---snip---
-------------------------------------------------------------------------------
Matching Services : 3
-------------------------------------------------------------------------------
===============================================================================

The Apipe service is down in PE-2 (MC-APS protect circuit), as follows:

*A:PE-2# show service service-using
===============================================================================
Services
===============================================================================
ServiceId    Type      Adm  Opr  CustomerId Service Name
-------------------------------------------------------------------------------
1            Apipe     Up   Down 1          1
---snip---
-------------------------------------------------------------------------------
Matching Services : 3
-------------------------------------------------------------------------------
===============================================================================

The Apipe service is down in PE-3 (MC-APS protect circuit), as follows:

*A:PE-3# show service service-using
===============================================================================
Services
===============================================================================
ServiceId    Type      Adm  Opr  CustomerId Service Name
-------------------------------------------------------------------------------
1            Apipe     Up   Down 1          1
---snip---
-------------------------------------------------------------------------------
Matching Services : 3
-------------------------------------------------------------------------------
===============================================================================
*A:PE-3#

The Apipe service is up in PE-4 (MC-APS working circuit), as follows:

*A:PE-4# show service service-using
===============================================================================
Services
===============================================================================
ServiceId    Type      Adm  Opr  CustomerId Service Name
-------------------------------------------------------------------------------
1            Apipe     Up   Up   1          1
---snip---
-------------------------------------------------------------------------------
Matching Services : 3
-------------------------------------------------------------------------------
===============================================================================

Note: After configuring ICB spoke-SDPs, the Apipe will be up on all PEs.

Step 8. Verify SDP status

The status of SDP 23:1 on PE-2 can be verified as follows.

Peer Pw Bits shows the status of the pseudowire on the peer node. In this example, both the local node (PE-2) as the remote node (PE-3) are sending the lacIngressFault, lacEgressFault and pwFwdingStandby flags. This is because the Apipe service on these nodes is down because the MC-APS is in protection status.

*A:PE-2# show service id 1 sdp 23:1 detail

===============================================================================
Service Destination Point (Sdp Id : 23:1) Details
===============================================================================
-------------------------------------------------------------------------------
 Sdp Id 23:1  -(192.0.2.3)
-------------------------------------------------------------------------------
Description     : (Not Specified)
SDP Id             : 23:1                     Type              : Spoke
Spoke Descr     : (Not Specified)
Split Horiz Grp    : (Not Specified)
VC Type            : AAL5SDU                  VC Tag            : 0
Admin Path MTU     : 0                        Oper Path MTU     : 1492
Delivery           : MPLS
Far End            : 192.0.2.3                Tunnel Far End    :
Oper Tunnel Far End: 192.0.2.3
LSP Types          : LDP

Admin State        : Up                       Oper State        : Up
MinReqd SdpOperMTU : 1404
Acct. Pol          : None                     Collect Stats     : Disabled
Ingress Label      : 524283                   Egress Label      : 524282
Ingr Mac Fltr-Id   : n/a                      Egr Mac Fltr-Id   : n/a
Ingr IP Fltr-Id    : n/a                      Egr IP Fltr-Id    : n/a
Admin ControlWord  : Preferred                Oper ControlWord  : True
Admin BW(Kbps)     : 0                        Oper BW(Kbps)     : 0
BFD Template       : None
BFD-Enabled        : no                       BFD-Encap         : ipv4
Last Status Change : 01/10/2020 09:46:58      Signaling         : TLDP
Last Mgmt Change   : 01/10/2020 09:46:52 
Endpoint           : Y                        Precedence        : 4
PW Status Sig      : Enabled
Class Fwding State : Down
Flags              : None
Local Pw Bits      : lacIngressFault lacEgressFault pwFwdingStandby
Peer Pw Bits       : lacIngressFault lacEgressFault pwFwdingStandby
Peer Fault Ip      : None
Peer Vccv CV Bits  : lspPing bfdFaultDet
Peer Vccv CC Bits  : pwe3ControlWord mplsRouterAlertLabel

---snip---

===============================================================================

In case of failure, the access link can be protected by MC-APS. An MPLS network failure can be protected by pseudowire redundancy. Node failure can be protected by the combination of MC-APS and pseudowire redundancy.

Step 9. Inter-Chassis Backup (ICB) pseudowire configuration.

Configuring Inter-Chassis Backup (ICB) is optional. It can reduce traffic impact by forwarding traffic on ICB spoke SDPs during MC-APS switchover. The ICB spoke SDP cannot be added to the endpoint if the SAP is not part of an MC-APS (or MC-LAG) instance. Conversely, a SAP which is not part of a MC-APS (or MC-LAG) instance cannot be added to an endpoint which already has an ICB spoke SDP. Forwarding between ICBs is blocked on the same node. The user has to explicitly indicate the spoke SDP is actually an ICB at creation time. Figure 5 shows some setup examples where ICBs are required.

After configuring ICB spoke SDPs, the Apipe will be in admin/oper up/up status on all PE routers.

ICB SDPs are configured and associated to endpoints, as shown in ICB spoke SDPs and association with the endpoints.

Figure 5. ICB spoke SDPs and association with the endpoints

Two ICB spoke SDPs must be configured in the Apipe service on each PE router, one in each endpoint. The same SDP IDs can be used for the ICBs since the far-end will be the same. However, the VC-id must be different. The ICB spoke SDPs must cross, meaning one end should be associated with endpoint X and the other end (on the other PE) should be associated with endpoint Y.

An ICB is always the last forwarding resort. Only one spoke SDP will be forwarding. If there is an ICB and an MC-APS SAP in an endpoint, the ICB will only forward if the SAP goes down. If an ICB resides in an endpoint together with other spoke SDPs the ICB will only forward if there is no other active spoke SDP.

The following shows the additional configuration for ICB on each PE:

# on PE-1
configure
    service
        apipe 1
            spoke-sdp 12:1 endpoint "X" icb create
            exit
            spoke-sdp 12:2 endpoint "Y" icb create
            exit
        exit
    exit
exit
# on PE-2
configure
    service
        apipe 1
            spoke-sdp 21:1 endpoint "Y" icb create
            exit
            spoke-sdp 21:2 endpoint "X" icb create
            exit
        exit
    exit
exit
# on PE-3
configure
    service
        apipe 1
            spoke-sdp 34:1 endpoint "X" icb create
            exit
            spoke-sdp 34:2 endpoint "Y" icb create
            exit
        exit
    exit
exit
# on PE-4
configure
    service
        apipe 1
            spoke-sdp 43:1 endpoint "Y" icb create
            exit
            spoke-sdp 43:2 endpoint "X" icb create
            exit
        exit
    exit
exit

Step 10. Verification of active objects for each endpoint

The following command shows which objects are configured for each endpoint and which is the active object at this moment:

*A:PE-1# show service id 1 endpoint

===============================================================================
Service 1 endpoints
===============================================================================
Endpoint name                : X
Description                  : (Not Specified)
Creation Origin              : manual
Revert time                  : 0
Act Hold Delay               : 0
Tx Active                    : aps-1:0/32
Tx Active Up Time            : 0d 00:38:31
Revert Time Count Down       : never
Tx Active Change Count       : 1
Last Tx Active Change        : 01/10/2020 09:46:45
-------------------------------------------------------------------------------
Members
-------------------------------------------------------------------------------
SAP      : aps-1:0/32                               Oper Status: Up
Spoke-sdp: 12:1 Prec:4 (icb)                        Oper Status: Up
===============================================================================
Endpoint name                : Y
Description                  : (Not Specified)
Creation Origin              : manual
Revert time                  : 0
Act Hold Delay               : 0
Tx Active (SDP)              : 14:1
Tx Active Up Time            : 0d 00:35:40
Revert Time Count Down       : never
Tx Active Change Count       : 2
Last Tx Active Change        : 01/10/2020 10:03:31
-------------------------------------------------------------------------------
Members
-------------------------------------------------------------------------------
Spoke-sdp: 12:2 Prec:4 (icb)                        Oper Status: Up
Spoke-sdp: 13:1 Prec:4                              Oper Status: Up
Spoke-sdp: 14:1 Prec:4                              Oper Status: Up
===============================================================================
===============================================================================

On PE-1, both the SAP and the spoke SDP 14:1 are active. The other objects do not forward traffic.

Step 11. Other types of setups

The following figures show other setups that combine MC-APS and pseudowire redundancy.

Figure 6. Additional setup example 1 (part 1)
Figure 7. Additional setup example 1 (part 2)
Figure 8. Additional setup example 2 (part 1a)
Figure 9. Additional setup example 2 (part 1b)
Figure 10. Additional setup example 2 (part 2)

Forced switchover

MC-APS convergence can be forced with the tools perform aps command:

*A:PE-1# tools perform aps force 
  - force <aps-id> {protect|working} [number <number>]

 <aps-id>             : aps-<group-id>
                          aps            - keyword
                          group-id       - [1..128]
 <protect|working>    : keyword
 <number>             : [1-2]

After the forced switchover, it is important to clear the forced switchover:

*A:PE-1# tools perform aps clear 
  - clear <aps-id> {protect|working} [number <number>]

 <aps-id>             : aps-<group-id>
                          aps            - keyword
                          group-id       - [1..128]
 <protect|working>    : protect|working
 <number>             : [1-2]

Conclusion

In addition to Multi-Chassis LAG, Multi-Chassis APS provides a solution for both network redundancy and access node redundancy. It supports ATM VLL and Ethernet VLL with ATM SAP. Access links and PE nodes are protected by APS and the MPLS network is protected by pseudowire redundancy/FRR. With this feature, Nokia can provide resilient end-to-end solutions.