Operational Groups for EVPN-VXLAN VPWS Services

This chapter describes the Operational Groups for EVPN-VXLAN VPWS Services.

Topics in this chapter include:

Applicability

This chapter was initially written based on SR OS Release 16.0.R5, but the CLI in the current edition corresponds to SR OS Release 21.5.R2. EVPN-VXLAN VPWS and service-level operational groups for VPWS services are supported in SR OS Release 16.0.R1, or later.

Overview

Operational groups on Epipe services are used for fault propagation to other services, such as I-VPLS or R-VPLS services. Epipes with VXLAN destinations are used in some edge PE applications along with port cross-connect (PXC) so that VXLAN networks can be terminated in other VPLS or VPRN services. In such cases, the operational status of the Epipe services terminating VXLAN must override the operational status of the SAPs of the VPLS or VPRN where the Epipe is stitched to.

Operational group on egress VTEP in Epipes with static VXLAN bindings

The Static VXLAN Termination in Epipe Services chapter describes how Epipes with static VXLAN termination are stitched to I-VPLS services. In Epipes with static VXLAN bindings, operational groups can be configured in the egress VTEP context. Epipe with static VXLAN termination shows the example topology with a static VXLAN tunnel between PE-1 and an anycast address on PE-2 and PE-3. The All-Active Multi-Homing Ethernet Segments (AA MH ESs) "vES23_1.101" and "vES23_1.102" are used by the I-VPLSs 101 and 102, which are both stitched to Epipe 1 in PE-2 and PE-3. The SAPs in these I-VPLSs monitor the operational group configured in the egress VTEP context of the Epipe service, so the SAPs will go operationally down when the operational group of the VTEP goes operationally down.

Figure 1. Epipe with static VXLAN termination

On PE-2 and PE-3, Epipe 1 is configured with static VXLAN bindings, as follows. The egress VTEP is 192.0.2.1, which is the system IP address of PE-1. Operational group "op-grp-1" in configured for this egress VTEP. LAG 2 combines PXC ports and is used to stitch Epipe 1 to the I-VPLS services 101 and 102. For a detailed description of the configuration, see the Static VXLAN Termination in Epipe Services chapter.

# on PE-2, PE-3:
configure
    service
        oper-group "op-grp-1" create
        exit
        epipe 1 name "Epipe 1" customer 1 create
            description "Epipe 1 with static VXLAN bindings"
            vxlan-src-vtep 23.23.23.1
            vxlan instance 1 vni 1 create
                egr-vtep 192.0.2.1
                    oper-group "op-grp-1"
                exit
            exit
            sap lag-2:1.* create
                no shutdown
            exit
            no shutdown
        exit

For failure propagation to the stitched I-VPLSs, the SAPs in the I-VPLSs can monitor the operational group "op-grp-1", for I-VPLS 101 on PE-2 and PE-3, as follows:

# on PE-2, PE-3:
configure
    service
        vpls 101 name "I-VPLS 101" customer 1 i-vpls create
            pbb
                backbone-vpls 100
                exit
            exit
            sap lag-1:1.101 create
                monitor-oper-group "op-grp-1"
                no shutdown
            exit
            no shutdown
        exit

When the egress VTEP prefix 192.0.2.1 disappears from the global route-table on PE-2, the VXLAN binding goes down, as follows:

*A:PE-2# show service id 1 vxlan destinations
 
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address                            Egress VNI          Oper      Vxlan
                                                            State     Type
-------------------------------------------------------------------------------
192.0.2.1                               1                   Down      static
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---

When the egress VTEP 192.0.2.1 goes down, the operational group "op-grp-1" goes down too, as follows:

*A:PE-2# show service oper-group "op-grp-1"
 
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group       : op-grp-1
Creation Origin  : manual                          Oper Status: down
Hold DownTime    : 0 secs                          Hold UpTime: 4 secs
Members          : 1                               Monitoring : 2
===============================================================================

When the operational group "op-grp-1" goes down, the monitoring SAP in I-VPLS 101 goes operationally down with flag OperGroupDown, as follows:

*A:PE-2# show service id 101 sap lag-1:1.101 detail 
                                             | match expression "Flags | Oper State"
Admin State        : Up                       Oper State        : Down
Flags              : OperGroupDown
Stp Admin State    : Up                       Stp Oper State    : Down

When this SAP goes down, the entire I-VPLS 101 service goes down on PE-2, as follows:

*A:PE-2# show service id 101 base | match "State"
Admin State       : Up                  Oper State        : Down

Epipes with static VXLAN bindings impose the following restrictions, which cannot be overcome unless a control plane protocol such as BGP-EVPN is used for the VXLAN bindings.

  • When anycast VTEPs on the PEs are used, a change in the vES preference on the DF PE triggers a DF switchover for the I-VPLS service. However, the access PE (PE-1 in Epipe with static VXLAN termination) is unaware and keeps sending the VXLAN traffic to the same PE, unless a change in DF comes with an automatic change in the underlay IGP metrics, which cannot be easily accomplished.

  • Without a control plane, Eth-CFM must be used between PEs and access PEs to detect end-to-end service-level failures.

  • Traffic from the access PE is forwarded to the anycast VTEP, based on underlay IGP metrics. There is no control on a per-service basis.

  • The architecture does not support AA MH for the Epipe service, so the access PEs always send the traffic to one single PE.

The preceding challenges can be addressed by using different VTEPs on the PEs and adding a BGP-EVPN control plane on the Epipe, as described in the next section.

Operational groups in EVPN-VXLAN Epipes

Epipe 2 with EVPN-VXLAN and all-active multi-homing shows EVPN-VXLAN Epipe 2 stitched to I-VPLSs 201 and 202. AA MH ESs "vES23_2.201" and "vES23_2.202" are used by the I-VPLSs 201 and 202 respectively.

Figure 2. Epipe 2 with EVPN-VXLAN and all-active multi-homing

The EVPN-VXLAN VPWS chapter describes the configuration of Epipes with EVPN-VXLAN bindings instead of static VXLAN bindings. The egress VTEP is not configured manually, but dynamically learned through BGP-EVPN. Therefore, the operational group cannot be configured in the egress VTEP context. However, it is possible to configure an operational group in an Epipe at the service level, as follows:

# on PE-2: 
configure
    service
        oper-group "op-grp-2" create
        exit
        epipe 2 name "Epipe 2" customer 1 create
            description "Epipe 2 with EVPN-VXLAN"
            oper-group "op-grp-2"
            vxlan instance 1 vni 2 create
            exit
            bgp
            exit
            bgp-evpn
                local-attachment-circuit AC-23 create
                    eth-tag 123
                exit
                remote-attachment-circuit AC-1 create
                    eth-tag 101
                exit
                evi 2
                vxlan bgp 1 vxlan-instance 1
                    no shutdown
                exit
            exit
            sap lag-2:2.* create
                no shutdown
            exit
            no shutdown

The following shows the error raised when attempting to configure the egress VTEP manually in an Epipe service with BGP-EVPN enabled:

*A:PE-2>configure>service>epipe>vxlan# egr-vtep 192.0.2.1
MINOR: SVCMGR #7894 Cannot configure egr-vtep - service has bgp-evpn

An operational group can be associated with the entire Epipe or with specific objects, such as SAPs or spoke-SDPs, but not simultaneously. The following error is raised when attempting to associate the operational group "op-grp-2"—that is already associated with the Epipe with the SAP on PE-2:

*A:PE-2>config>service>epipe>sap# oper-group "op-grp-2"
MINOR: SVCMGR #1003 Inconsistent value - oper-group already in use as service oper group

The service-level operational group status is derived from the service operational status: when Epipe 2 is operationally down, the operational group "op-grp-2" will be down.

For fault propagation to the stitched I-VPLSs 201 and 202, the SAPs in the I-VPLSs monitor the operational group "op-grp-2", for I-VPLS 201 on PE-2 and PE-3, as follows:

# on PE-2, PE-3:
configure
    service
        vpls 201 name "I-VPLS 201" customer 1 i-vpls create
            pbb
                backbone-vpls 100
                exit
            exit
            sap lag-1:2.201 create
                monitor-oper-group "op-grp-2"
                no shutdown
            exit
            no shutdown
        exit

Configuration

In this section, the following use cases are described:

  • operational group on egress VTEP in Epipes with static VXLAN bindings stitched to I-VPLSs using AA MH ESs

  • service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLSs using AA MH ESs

  • service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLSs using Single-Active (SA) MH ESs

Example topology shows the example topology with four PEs in an autonomous system.

Figure 3. Example topology

The initial configuration includes:

  • Cards, MDAs, ports

  • Router interfaces

  • IS-IS as IGP (level capability 2 in the core between PE-2, PE-3, and PE-4; level capability 1 in the access toward PE-1)

  • LDP between the core routers PE-2, PE-3, and PE-4

Oper group on egress VTEP in Epipes with static VXLAN bindings stitched to I-VPLSs using AA MH ESs

When static VXLAN bindings are used, no BGP-EVPN is required in the access network to and from PE-1; BGP is only configured in the core network. When PE-2 acts as the route reflector, its BGP configuration is as follows:

# on RR PE-2:
configure
    router Base
        autonomous-system 64500
        bgp
            vpn-apply-import
            vpn-apply-export
            rapid-update evpn
            group "CORE"
                family evpn
                type internal
                cluster 192.0.2.2
                split-horizon
                neighbor 192.0.2.3
                exit
                neighbor 192.0.2.4
                exit
            exit

Epipe with static VXLAN termination shows that Epipe 1 is configured in the access network: on PE-1, the VTEP is the system address 192.0.2.1 (default), and on PE-2 and PE-3, the VTEP is a unicast address 23.23.23.1.

On PE-1, Epipe 1 is configured with egress VTEP 23.23.23.1, as follows:

# on PE-1:
configure
    service
        epipe 1 name "Epipe 1" customer 1 create
            description "Epipe 1 with static VXLAN bindings"
            vxlan instance 1 vni 1 create
                egr-vtep 23.23.23.1
                exit
            exit
            sap 1/2/1:1.* create
                no shutdown
            exit
            no shutdown
        exit

On PE-2 and PE-3, the following unicast address is configured:

# on PE-2, PE-3:
configure
    router
        interface "lo23"
            address 23.23.23.0/31
            loopback
        exit

On PE-2 and PE-3, three ports are configured as PXC. PXC 1 is used as Forwarding Path Extension (FPE) and the VXLAN tunnel termination 23.23.23.1 is configured with this FPE, as follows:

# on PE-2, PE-3:
configure
    service
        system
            vxlan
                tunnel-termination 23.23.23.1 fpe 1 create
            exit

PXCs 2 and 3 are used in the internal LAGs that are used to stitch Epipe 1 to I-VPLSs 101 and 102, as follows. LAG 1 will be used in the I-VPLS services; LAG 2 in the Epipe services.

# on PE-2, PE-3:
configure
    lag 1 
        mode hybrid
        encap-type qinq
        port pxc-2.a
        port pxc-3.a
        no shutdown
    exit
    lag 2 
        mode hybrid
        encap-type qinq
        port pxc-2.b
        port pxc-3.b
        no shutdown
    exit

On PE-2 and PE-3, Epipe 1 is configured with source VTEP 23.23.23.1 and egress VTEP 192.0.2.1, as follows. The operational group "op-grp-1" is associated with the egress VTEP. The SAP stitches Epipe 1 to the I-VPLSs 101 and 102.

# on PE-2, PE-3:
configure
    service
        oper-group "op-grp-1" create
        exit
        epipe 1 name "Epipe 1" customer 1 create
            description "Epipe 1 with static VXLAN bindings"
            vxlan-src-vtep 23.23.23.1
            vxlan instance 1 vni 1 create
                egr-vtep 192.0.2.1
                    oper-group "op-grp-1"
                exit
            exit
            sap lag-2:1.* create
                no shutdown
            exit
            no shutdown
        exit

On PE-2, B-VPLS 100 is configured as follows. The configuration is similar on PE-3 and PE-4.

# on PE-2:
configure
    service
        vpls 100 name "B-VPLS-100" customer 1 b-vpls create
            service-mtu 1532
            pbb
                source-bmac 00:00:00:00:00:02
                use-es-bmac
            exit
            bgp
            exit
            bgp-evpn
                evi 100
                mpls bgp 1
                    ingress-replication-bum-label
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            no shutdown
        exit

On PE-2, I-VPLS 101 is configured as follows. The SAP monitors the operational group "op-grp-1" that is configured in Epipe 1. The AA MH ES "vES23_1.101" is used. The configuration of I-VPLS 102 is similar, but it uses AA MH ES "vES23_1.102" with preference value 50 instead. On PE-3, the preference values are reversed: preference value 50 for "vES23_1.101" and 100 for "vES23_1.102".

# on PE-2:
configure
    service
        system
            bgp-evpn
                ethernet-segment "vES23_1.101" virtual create
                    esi 01:00:00:00:00:23:00:00:01:11
                    source-bmac-lsb 23-11 es-bmac-table-size 8
                    es-activation-timer 3
                    service-carving
                        mode manual
                        manual
                            preference create
                                value 100
                            exit
                        exit
                    exit
                    multi-homing all-active
                    lag 1
                    qinq
                        s-tag 1 c-tag-range 101
                    exit
                    no shutdown
                exit
            exit
        exit
        vpls 101 name "I-VPLS 101" customer 1 i-vpls create
            pbb
                backbone-vpls 100
                exit
            exit
            sap lag-1:1.101 create
                monitor-oper-group "op-grp-1"
                no shutdown
            exit
            no shutdown
        exit

On PE-4, I-VPLS 101 is configured as follows:

# on PE-4:
configure
    service
        vpls 101 name "I-VPLS 101" customer 1 i-vpls create
            pbb
                backbone-vpls 100
                exit
            exit
            sap 1/2/1:1.101 create
                no shutdown
            exit
            no shutdown
        exit

To emulate a failure that affects the operational state of the egress VTEP (and also of the Epipe service), the SAP of Epipe 1 on PE-2 is disabled, as follows:

# on PE-2:
configure 
    service 
        epipe "Epipe 1"
            sap lag-2:1.* 
                shutdown

When the SAP is operationally down, Epipe 1 goes down, as follows:

*A:PE-2# show service id 1 base | match "State"
Admin State       : Up                  Oper State        : Down

The egress VTEP 192.0.2.1 is operationally down, as follows:

*A:PE-2# show service id 1 vxlan destinations 

===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address                            Egress VNI          Oper      Vxlan
                                                            State     Type
-------------------------------------------------------------------------------
192.0.2.1                               1                   Down      static
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---

The operational group "op-grp-1" is associated with the egress VTEP, so it goes operationally down, as follows:

*A:PE-2# show service oper-group "op-grp-1" 
 
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group       : op-grp-1
Creation Origin  : manual                          Oper Status: down
Hold DownTime    : 0 secs                          Hold UpTime: 4 secs
Members          : 1                               Monitoring : 2
===============================================================================

This operational group is monitored by the SAPs in I-VPLSs 101 and 102, so these SAPs go down with flag OperGroupDown; for example, for I-VPLS 101 on PE-2:

*A:PE-2# show service id 101 sap lag-1:1.101 | match "Flags" 
Flags              : OperGroupDown

When the SAP goes down, the I-VPLS service goes down, as follows:

*A:PE-2# show service id 101 base | match "State"            
Admin State       : Up                  Oper State        : Down

Even though Epipe 1 on PE-2 is operationally down while Epipe 1 on PE-3 is up, PE-1 is unaware because the VXLAN destination in Epipe 1 remains up, as follows:

*A:PE-1# show service id 1 vxlan destinations 
 
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address                            Egress VNI          Oper      Vxlan
                                                            State     Type
-------------------------------------------------------------------------------
23.23.23.1                              1                   Up        static
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---

With ECMP=1, all traffic from PE-1 is directed to PE-2, regardless of the state of the Epipe on PE-2. The following route table on PE-1 shows that destination prefix 23.23.23.0/31 has next-hop 192.168.12.2, which is an interface address on PE-2.

*A:PE-1# show router route-table 23.23.23.0/31
 
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric
-------------------------------------------------------------------------------
23.23.23.0/31                                 Remote  ISIS      00h06m41s  15
       192.168.12.2                                                 10
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
       B = BGP backup route available
       L = LFA nexthop available
       S = Sticky ECMP requested
===============================================================================

Traffic from the CEs attached to PE-1 is forwarded by PE-1 to PE-2, where it is dropped.

Service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLS using AA MH ESs

BGP must be enabled on all nodes for the EVPN address family, also in the access to and from PE-1. The BGP configuration on PE-1 is as follows:

# on PE-1:
configure
    router Base
        autonomous-system 64500
        bgp
            vpn-apply-import
            vpn-apply-export
            rapid-update evpn
            group "ACCESS"
                family evpn
                type internal
                split-horizon
                neighbor 192.0.2.2
                exit
                neighbor 192.0.2.3
                exit
            exit

On PE-1, the following EVPN-VXLAN Epipe 2 is configured with local Ethernet tag 101 and remote Ethernet tag 123.

# on PE-1:
configure
    service
        epipe 2 name "Epipe 2" customer 1 create
            description "Epipe 2 with EVPN-VXLAN"
            vxlan instance 1 vni 2 create
            exit
            bgp
            exit
            bgp-evpn
                local-attachment-circuit AC-1 create
                    eth-tag 101
                exit
                remote-attachment-circuit AC-23 create
                    eth-tag 123
                exit
                evi 2
                vxlan bgp 1 vxlan-instance 1
                    no shutdown
                exit
            exit
            sap 1/2/1:2.* create
                no shutdown
            exit
            no shutdown
        exit

On PE-2, the following EVPN-VXLAN Epipe 2 is configured with local Ethernet tag 123 and remote Ethernet tag 101. The operational group "op-grp-2" is associated with Epipe 2. The configuration on PE-3 is identical.

# on PE-2:
configure
    service
        oper-group "op-grp-2" create
        exit
        epipe 2 name "Epipe 2" customer 1 create
            description "Epipe 2 with EVPN-VXLAN"
            oper-group "op-grp-2"
            vxlan instance 1 vni 2 create
            exit
            bgp
            exit
            bgp-evpn
                local-attachment-circuit AC-23 create
                    eth-tag 123
                exit
                remote-attachment-circuit AC-1 create
                    eth-tag 101
                exit
                evi 2
                vxlan bgp 1 vxlan-instance 1
                    no shutdown
                exit
            exit
            sap lag-2:2.* create
                no shutdown
            exit
            no shutdown
        exit

The configuration of B-VPLS 100 remains unchanged and the configuration of the I-VPLSs 201 and 202 resembles the configuration of VPLSs 101.

When there is no failure, the egress VTEP for Epipe 2 on PE-1 is 192.0.2.2, which is the system IP address of PE-2, as follows:

*A:PE-1# show service id 2 vxlan destinations 
 
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address                            Egress VNI          Oper      Vxlan
                                                            State     Type
-------------------------------------------------------------------------------
192.0.2.2                               2                   Up        evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---

To emulate a failure that affects the operational state of the Epipe service, the SAP in Epipe 2 is disabled, as follows:

# on PE-2:
configure 
    service 
        epipe "Epipe 2"
            sap lag-2:2.* 
                shutdown

When the SAP goes down, the Epipe goes down, as follows:

*A:PE-2# show service id 2 base | match "State"
Admin State       : Up                  Oper State        : Down

On PE-1, the egress VTEP for Epipe 2 is 192.0.2.3, which is the system IP address of PE-3, as follows:

*A:PE-1# show service id 2 vxlan destinations 

===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address                            Egress VNI          Oper      Vxlan
                                                            State     Type
-------------------------------------------------------------------------------
192.0.2.3                               2                   Up        evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---

The operational group "op-grp-2" follows the state of Epipe 2, so it goes down, as follows. As a consequence, the monitoring SAPs for this operational group also go down.

*A:PE-2# show service oper-group "op-grp-2" detail
 
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group       : op-grp-2
Creation Origin  : manual                          Oper Status: down
Hold DownTime    : 0 secs                          Hold UpTime: 4 secs
Members          : 1                               Monitoring : 2
===============================================================================
 
===============================================================================
Member Services for OperGroup: op-grp-2
===============================================================================
Svc Id
-------------------------------------------------------------------------------
2
-------------------------------------------------------------------------------
Service Entries found: 1
===============================================================================
 
===============================================================================
Monitoring SAPs for OperGroup: op-grp-2
===============================================================================
PortId                          SvcId      Ing.  Ing.    Egr.  Egr.   Adm  Opr
                                           QoS   Fltr    QoS   Fltr
-------------------------------------------------------------------------------
lag-1:2.201                     201        1     none    1     none   Up   Down
lag-1:2.202                     202        1     none    1     none   Up   Down
-------------------------------------------------------------------------------
SAP Entries found: 2
===============================================================================

The SAPs in I-VPLSs 201 and 202 go down with the OperGroupDown flag, as follows:

*A:PE-2# show service id 201 sap lag-1:2.201 detail | match "Flags" context all
Flags              : OperGroupDown
*A:PE-2# show service id 202 sap lag-1:2.202 detail | match "Flags" context all
Flags              : OperGroupDown

When the SAPs go down, the I-VPLSs 201 and 202 also go down, as follows:

*A:PE-2# show service id 201 base | match "State"
Admin State       : Up                  Oper State        : Down
*A:PE-2# show service id 202 base | match "State"
Admin State       : Up                  Oper State        : Down

Even with this failure on PE-2, traffic can still flow between the CEs, as follows:

*A:PE-4# ping router 21 172.16.21.11 rapid 
PING 172.16.21.11 56 data bytes
!!!!!
---- 172.16.21.11 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.51ms, avg = 4.84ms, max = 9.30ms, stddev = 2.24ms. 

The following FDB for I-VPLS 201 on PE-4 shows that MAC address 00:ca:fe:00:21:11 of CE-1-21 is reachable via AA MH ES with ES-BMAC 00:00:00:00:23:21:

*A:PE-4# show service id 201 fdb detail
 
===============================================================================
Forwarding Database, Service 201
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
            Transport:Tnl-Id                         Age
-------------------------------------------------------------------------------
201        00:ca:fe:00:21:11 eES-BMAC:               L/90     07/15/21 13:50:28
                             00:00:00:00:23:21
201        00:ca:fe:00:21:41 sap:1/2/1:2.201         L/90     07/15/21 13:56:09
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================

Service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLS using SA MH ESs

Epipe 3 with EVPN-VXLAN and SA MH ES shows the example topology with an SA MH ES used by the I-VPLSs.

Figure 4. Epipe 3 with EVPN-VXLAN and SA MH ES

The configuration of Epipe 3 resembles the configuration of Epipe 2: the same Ethernet tags are used, only the VNI, EVI, and SAPs are different.

On PE-2 and PE-3, PXC 4 is configured to stitch Epipe 3 to I-VPLSs 301 and 302. The PXC port will be used in the SA MH ES.

On PE-2 and PE-3, Epipe 3 is configured as follows:

# on PE-2, PE-3:
configure
    service
        oper-group "op-grp-3" create
        exit
        epipe 3 name "Epipe 3" customer 1 create
            description "EVPN-VXLAN Epipe 3"
            oper-group "op-grp-3"
            vxlan instance 1 vni 3 create
            exit
            bgp-evpn
                local-attachment-circuit AC-23 create
                    eth-tag 123
                exit
                remote-attachment-circuit AC-1 create
                    eth-tag 101
                exit
                evi 3
                vxlan bgp 1 vxlan-instance 1
                    no shutdown
                exit
            exit
            sap pxc-4.b:3.* create
                no shutdown
            exit
            no shutdown
        exit

On PE-2, I-VPLS 301 uses SA MH ES "vES23_3.*", and is configured as follows.

# on PE-2:
configure
    service
        system
            bgp-evpn
                ethernet-segment "vES23_3.*" virtual create
                    esi 01:00:00:00:00:23:00:00:03:01
                    source-bmac-lsb 23-34 es-bmac-table-size 8
                    es-activation-timer 3
                    service-carving
                        mode manual
                        manual
                            preference create
                                value 100
                            exit
                        exit
                    exit
                    multi-homing single-active
                    port pxc-4.a
                    qinq
                        s-tag-range 3
                    exit
                    no shutdown
                exit
            exit
        exit
        vpls 301 name "I-VPLS 301" customer 1 i-vpls create
            pbb
                backbone-vpls 100
                exit
            exit
            stp
                shutdown
            exit
            sap pxc-4.a:3.301 create
                monitor-oper-group "op-grp-3"
                no shutdown
            exit
            no shutdown
        exit

The configuration is similar on PE-3, but with source-bmac-lsb 23-35 and preference 50.

When Epipe 3 on PE-2 is operationally up, the egress VTEP for Epipe 3 on PE-1 is 192.0.2.2, as follows:

*A:PE-1# show service id 3 vxlan destinations 

===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address                            Egress VNI          Oper      Vxlan
                                                            State     Type
-------------------------------------------------------------------------------
192.0.2.2                               3                   Up        evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---

To emulate a failure on PE-2 that affects the operational state of the Epipe service, the SAP in Epipe 3 is disabled, as follows:

# on PE-2:
configure
    service
        epipe "Epipe 3"
            sap pxc-4.b:3.*
                shutdown

When the SAP goes down, Epipe 3 goes down on PE-2, as follows:

*A:PE-2# show service id 3 base | match "State"
Admin State       : Up                  Oper State        : Down

When Epipe 3 on PE-2 goes operationally down, the egress VTEP for Epipe 3 on PE-1 is 192.0.2.3, as follows:

*A:PE-1# show service id 3 vxlan destinations 

===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address                            Egress VNI          Oper      Vxlan
                                                            State     Type
-------------------------------------------------------------------------------
192.0.2.3                               3                   Up        evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---

The operational group "op-grp-3" follows the state of Epipe 3 on PE-2, so it goes down. Also, the monitoring SAPs for this operational group go down.

*A:PE-2# show service oper-group "op-grp-3" detail
 
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group       : op-grp-3
Creation Origin  : manual                          Oper Status: down
Hold DownTime    : 0 secs                          Hold UpTime: 4 secs
Members          : 1                               Monitoring : 2
===============================================================================
 
===============================================================================
Member Services for OperGroup: op-grp-3
===============================================================================
Svc Id
-------------------------------------------------------------------------------
3
-------------------------------------------------------------------------------
Service Entries found: 1
===============================================================================
 
===============================================================================
Monitoring SAPs for OperGroup: op-grp-3
===============================================================================
PortId                          SvcId      Ing.  Ing.    Egr.  Egr.   Adm  Opr
                                           QoS   Fltr    QoS   Fltr
-------------------------------------------------------------------------------
pxc-4.a:3.301                   301        1     none    1     none   Up   Down
pxc-4.a:3.302                   302        1     none    1     none   Up   Down
-------------------------------------------------------------------------------
SAP Entries found: 2
===============================================================================

The SAPs in I-VPLSs 301 and 302 on PE-2 go down with the OperGroupDown flag, as follows:

*A:PE-2# show service id 301 sap pxc-4.a:3.301 | match "Flags" context all 
Flags              : StandByForMHProtocol
                     OperGroupDown
*A:PE-2# show service id 302 sap pxc-4.a:3.302 | match "Flags" context all 
Flags              : StandByForMHProtocol
                     OperGroupDown

When the SAPs go down, the I-VPLSs go down on PE-2, as follows:

*A:PE-2# show service id 301 base | match "State"
Admin State       : Up                  Oper State        : Down
*A:PE-2# show service id 302 base | match "State"
Admin State       : Up                  Oper State        : Down

When the initial DF PE-2 goes down for the I-VPLSs 301 and 302, PE-3 becomes the new DF. The connectivity between the CEs is preserved, as follows:

*A:PE-4# ping router 31 172.16.31.11 rapid 
PING 172.16.31.11 56 data bytes
!!!!!
---- 172.16.31.11 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.68ms, avg = 4.15ms, max = 4.40ms, stddev = 0.264ms

The following FDB for I-VPLS 301 on PE-4 shows that the frames toward MAC address 00:ca:fe:00:31:11 of CE-1-31 are sent via PE-3 (192.0.2.3):

*A:PE-4# show service id 301 fdb detail
 
===============================================================================
Forwarding Database, Service 301
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
            Transport:Tnl-Id                         Age
-------------------------------------------------------------------------------
301        00:ca:fe:00:31:11 b-mpls:                 L/120    07/15/21 14:16:25
                             192.0.2.3:524279
           ldp:65537
301        00:ca:fe:00:31:41 sap:1/2/1:3.301         L/120    07/15/21 14:12:33
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================

PE-3 is now the DF for I-VPLS 301, as follows:

*A:PE-3# show service id 301 ethernet-segment 

===============================================================================
SAP Ethernet-Segment Information
===============================================================================
SAP                   Eth-Seg                          Status
-------------------------------------------------------------------------------
pxc-4.a:3.301         vES23_3.*                        DF
===============================================================================
No sdp entries
No vxlan instance entries

Conclusion

Some service providers use VXLAN as a next-generation access technology used between the MSANs (or access PEs) and core PE routers. EVPN-VXLAN Epipes can be stitched using PXC to other services, such as I-VPLS. Operational groups can be defined in the Epipe for fault propagation to the SAPs of the services where the Epipe is stitched to.