Operational Groups for EVPN-VXLAN VPWS Services
This chapter describes the Operational Groups for EVPN-VXLAN VPWS Services.
Topics in this chapter include:
Applicability
This chapter was initially written based on SR OS Release 16.0.R5, but the CLI in the current edition corresponds to SR OS Release 21.5.R2. EVPN-VXLAN VPWS and service-level operational groups for VPWS services are supported in SR OS Release 16.0.R1, or later.
Overview
Operational groups on Epipe services are used for fault propagation to other services, such as I-VPLS or R-VPLS services. Epipes with VXLAN destinations are used in some edge PE applications along with port cross-connect (PXC) so that VXLAN networks can be terminated in other VPLS or VPRN services. In such cases, the operational status of the Epipe services terminating VXLAN must override the operational status of the SAPs of the VPLS or VPRN where the Epipe is stitched to.
Operational group on egress VTEP in Epipes with static VXLAN bindings
The Static VXLAN Termination in Epipe Services chapter describes how Epipes with static VXLAN termination are stitched to I-VPLS services. In Epipes with static VXLAN bindings, operational groups can be configured in the egress VTEP context. Epipe with static VXLAN termination shows the example topology with a static VXLAN tunnel between PE-1 and an anycast address on PE-2 and PE-3. The All-Active Multi-Homing Ethernet Segments (AA MH ESs) "vES23_1.101" and "vES23_1.102" are used by the I-VPLSs 101 and 102, which are both stitched to Epipe 1 in PE-2 and PE-3. The SAPs in these I-VPLSs monitor the operational group configured in the egress VTEP context of the Epipe service, so the SAPs will go operationally down when the operational group of the VTEP goes operationally down.
On PE-2 and PE-3, Epipe 1 is configured with static VXLAN bindings, as follows. The egress VTEP is 192.0.2.1, which is the system IP address of PE-1. Operational group "op-grp-1" in configured for this egress VTEP. LAG 2 combines PXC ports and is used to stitch Epipe 1 to the I-VPLS services 101 and 102. For a detailed description of the configuration, see the Static VXLAN Termination in Epipe Services chapter.
# on PE-2, PE-3:
configure
service
oper-group "op-grp-1" create
exit
epipe 1 name "Epipe 1" customer 1 create
description "Epipe 1 with static VXLAN bindings"
vxlan-src-vtep 23.23.23.1
vxlan instance 1 vni 1 create
egr-vtep 192.0.2.1
oper-group "op-grp-1"
exit
exit
sap lag-2:1.* create
no shutdown
exit
no shutdown
exit
For failure propagation to the stitched I-VPLSs, the SAPs in the I-VPLSs can monitor the operational group "op-grp-1", for I-VPLS 101 on PE-2 and PE-3, as follows:
# on PE-2, PE-3:
configure
service
vpls 101 name "I-VPLS 101" customer 1 i-vpls create
pbb
backbone-vpls 100
exit
exit
sap lag-1:1.101 create
monitor-oper-group "op-grp-1"
no shutdown
exit
no shutdown
exit
When the egress VTEP prefix 192.0.2.1 disappears from the global route-table on PE-2, the VXLAN binding goes down, as follows:
*A:PE-2# show service id 1 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.1 1 Down static
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---
When the egress VTEP 192.0.2.1 goes down, the operational group "op-grp-1" goes down too, as follows:
*A:PE-2# show service oper-group "op-grp-1"
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group : op-grp-1
Creation Origin : manual Oper Status: down
Hold DownTime : 0 secs Hold UpTime: 4 secs
Members : 1 Monitoring : 2
===============================================================================
When the operational group "op-grp-1" goes down, the monitoring SAP in I-VPLS 101 goes operationally down with flag OperGroupDown, as follows:
*A:PE-2# show service id 101 sap lag-1:1.101 detail
| match expression "Flags | Oper State"
Admin State : Up Oper State : Down
Flags : OperGroupDown
Stp Admin State : Up Stp Oper State : Down
When this SAP goes down, the entire I-VPLS 101 service goes down on PE-2, as follows:
*A:PE-2# show service id 101 base | match "State"
Admin State : Up Oper State : Down
Epipes with static VXLAN bindings impose the following restrictions, which cannot be overcome unless a control plane protocol such as BGP-EVPN is used for the VXLAN bindings.
When anycast VTEPs on the PEs are used, a change in the vES preference on the DF PE triggers a DF switchover for the I-VPLS service. However, the access PE (PE-1 in Figure 1) is unaware and keeps sending the VXLAN traffic to the same PE, unless a change in DF comes with an automatic change in the underlay IGP metrics, which cannot be easily accomplished.
Without a control plane, Eth-CFM must be used between PEs and access PEs to detect end-to-end service-level failures.
Traffic from the access PE is forwarded to the anycast VTEP, based on underlay IGP metrics. There is no control on a per-service basis.
The architecture does not support AA MH for the Epipe service, so the access PEs always send the traffic to one single PE.
The preceding challenges can be addressed by using different VTEPs on the PEs and adding a BGP-EVPN control plane on the Epipe, as described in the next section.
Operational groups in EVPN-VXLAN Epipes
Epipe 2 with EVPN-VXLAN and all-active multi-homing shows EVPN-VXLAN Epipe 2 stitched to I-VPLSs 201 and 202. AA MH ESs "vES23_2.201" and "vES23_2.202" are used by the I-VPLSs 201 and 202 respectively.
The EVPN-VXLAN VPWS chapter describes the configuration of Epipes with EVPN-VXLAN bindings instead of static VXLAN bindings. The egress VTEP is not configured manually, but dynamically learned through BGP-EVPN. Therefore, the operational group cannot be configured in the egress VTEP context. However, it is possible to configure an operational group in an Epipe at the service level, as follows:
# on PE-2:
configure
service
oper-group "op-grp-2" create
exit
epipe 2 name "Epipe 2" customer 1 create
description "Epipe 2 with EVPN-VXLAN"
oper-group "op-grp-2"
vxlan instance 1 vni 2 create
exit
bgp
exit
bgp-evpn
local-attachment-circuit AC-23 create
eth-tag 123
exit
remote-attachment-circuit AC-1 create
eth-tag 101
exit
evi 2
vxlan bgp 1 vxlan-instance 1
no shutdown
exit
exit
sap lag-2:2.* create
no shutdown
exit
no shutdown
The following shows the error raised when attempting to configure the egress VTEP manually in an Epipe service with BGP-EVPN enabled:
*A:PE-2>configure>service>epipe>vxlan# egr-vtep 192.0.2.1
MINOR: SVCMGR #7894 Cannot configure egr-vtep - service has bgp-evpn
An operational group can be associated with the entire Epipe or with specific objects, such as SAPs or spoke-SDPs, but not simultaneously. The following error is raised when attempting to associate the operational group "op-grp-2"—that is already associated with the Epipe with the SAP on PE-2:
*A:PE-2>config>service>epipe>sap# oper-group "op-grp-2"
MINOR: SVCMGR #1003 Inconsistent value - oper-group already in use as service oper group
The service-level operational group status is derived from the service operational status: when Epipe 2 is operationally down, the operational group "op-grp-2" will be down.
For fault propagation to the stitched I-VPLSs 201 and 202, the SAPs in the I-VPLSs monitor the operational group "op-grp-2", for I-VPLS 201 on PE-2 and PE-3, as follows:
# on PE-2, PE-3:
configure
service
vpls 201 name "I-VPLS 201" customer 1 i-vpls create
pbb
backbone-vpls 100
exit
exit
sap lag-1:2.201 create
monitor-oper-group "op-grp-2"
no shutdown
exit
no shutdown
exit
Configuration
In this section, the following use cases are described:
operational group on egress VTEP in Epipes with static VXLAN bindings stitched to I-VPLSs using AA MH ESs
service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLSs using AA MH ESs
service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLSs using Single-Active (SA) MH ESs
Example topology shows the example topology with four PEs in an autonomous system.
The initial configuration includes:
Cards, MDAs, ports
Router interfaces
IS-IS as IGP (level capability 2 in the core between PE-2, PE-3, and PE-4; level capability 1 in the access toward PE-1)
LDP between the core routers PE-2, PE-3, and PE-4
Oper group on egress VTEP in Epipes with static VXLAN bindings stitched to I-VPLSs using AA MH ESs
When static VXLAN bindings are used, no BGP-EVPN is required in the access network to and from PE-1; BGP is only configured in the core network. When PE-2 acts as the route reflector, its BGP configuration is as follows:
# on RR PE-2:
configure
router Base
autonomous-system 64500
bgp
vpn-apply-import
vpn-apply-export
rapid-update evpn
group "CORE"
family evpn
type internal
cluster 192.0.2.2
split-horizon
neighbor 192.0.2.3
exit
neighbor 192.0.2.4
exit
exit
Epipe with static VXLAN termination shows that Epipe 1 is configured in the access network: on PE-1, the VTEP is the system address 192.0.2.1 (default), and on PE-2 and PE-3, the VTEP is a unicast address 23.23.23.1.
On PE-1, Epipe 1 is configured with egress VTEP 23.23.23.1, as follows:
# on PE-1:
configure
service
epipe 1 name "Epipe 1" customer 1 create
description "Epipe 1 with static VXLAN bindings"
vxlan instance 1 vni 1 create
egr-vtep 23.23.23.1
exit
exit
sap 1/2/1:1.* create
no shutdown
exit
no shutdown
exit
On PE-2 and PE-3, the following unicast address is configured:
# on PE-2, PE-3:
configure
router
interface "lo23"
address 23.23.23.0/31
loopback
exit
On PE-2 and PE-3, three ports are configured as PXC. PXC 1 is used as Forwarding Path Extension (FPE) and the VXLAN tunnel termination 23.23.23.1 is configured with this FPE, as follows:
# on PE-2, PE-3:
configure
service
system
vxlan
tunnel-termination 23.23.23.1 fpe 1 create
exit
PXCs 2 and 3 are used in the internal LAGs that are used to stitch Epipe 1 to I-VPLSs 101 and 102, as follows. LAG 1 will be used in the I-VPLS services; LAG 2 in the Epipe services.
# on PE-2, PE-3:
configure
lag 1
mode hybrid
encap-type qinq
port pxc-2.a
port pxc-3.a
no shutdown
exit
lag 2
mode hybrid
encap-type qinq
port pxc-2.b
port pxc-3.b
no shutdown
exit
On PE-2 and PE-3, Epipe 1 is configured with source VTEP 23.23.23.1 and egress VTEP 192.0.2.1, as follows. The operational group "op-grp-1" is associated with the egress VTEP. The SAP stitches Epipe 1 to the I-VPLSs 101 and 102.
# on PE-2, PE-3:
configure
service
oper-group "op-grp-1" create
exit
epipe 1 name "Epipe 1" customer 1 create
description "Epipe 1 with static VXLAN bindings"
vxlan-src-vtep 23.23.23.1
vxlan instance 1 vni 1 create
egr-vtep 192.0.2.1
oper-group "op-grp-1"
exit
exit
sap lag-2:1.* create
no shutdown
exit
no shutdown
exit
On PE-2, B-VPLS 100 is configured as follows. The configuration is similar on PE-3 and PE-4.
# on PE-2:
configure
service
vpls 100 name "B-VPLS-100" customer 1 b-vpls create
service-mtu 1532
pbb
source-bmac 00:00:00:00:00:02
use-es-bmac
exit
bgp
exit
bgp-evpn
evi 100
mpls bgp 1
ingress-replication-bum-label
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
no shutdown
exit
On PE-2, I-VPLS 101 is configured as follows. The SAP monitors the operational group "op-grp-1" that is configured in Epipe 1. The AA MH ES "vES23_1.101" is used. The configuration of I-VPLS 102 is similar, but it uses AA MH ES "vES23_1.102" with preference value 50 instead. On PE-3, the preference values are reversed: preference value 50 for "vES23_1.101" and 100 for "vES23_1.102".
# on PE-2:
configure
service
system
bgp-evpn
ethernet-segment "vES23_1.101" virtual create
esi 01:00:00:00:00:23:00:00:01:11
source-bmac-lsb 23-11 es-bmac-table-size 8
es-activation-timer 3
service-carving
mode manual
manual
preference create
value 100
exit
exit
exit
multi-homing all-active
lag 1
qinq
s-tag 1 c-tag-range 101
exit
no shutdown
exit
exit
exit
vpls 101 name "I-VPLS 101" customer 1 i-vpls create
pbb
backbone-vpls 100
exit
exit
sap lag-1:1.101 create
monitor-oper-group "op-grp-1"
no shutdown
exit
no shutdown
exit
On PE-4, I-VPLS 101 is configured as follows:
# on PE-4:
configure
service
vpls 101 name "I-VPLS 101" customer 1 i-vpls create
pbb
backbone-vpls 100
exit
exit
sap 1/2/1:1.101 create
no shutdown
exit
no shutdown
exit
To emulate a failure that affects the operational state of the egress VTEP (and also of the Epipe service), the SAP of Epipe 1 on PE-2 is disabled, as follows:
# on PE-2:
configure
service
epipe "Epipe 1"
sap lag-2:1.*
shutdown
When the SAP is operationally down, Epipe 1 goes down, as follows:
*A:PE-2# show service id 1 base | match "State"
Admin State : Up Oper State : Down
The egress VTEP 192.0.2.1 is operationally down, as follows:
*A:PE-2# show service id 1 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.1 1 Down static
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---
The operational group "op-grp-1" is associated with the egress VTEP, so it goes operationally down, as follows:
*A:PE-2# show service oper-group "op-grp-1"
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group : op-grp-1
Creation Origin : manual Oper Status: down
Hold DownTime : 0 secs Hold UpTime: 4 secs
Members : 1 Monitoring : 2
===============================================================================
This operational group is monitored by the SAPs in I-VPLSs 101 and 102, so these SAPs go down with flag OperGroupDown; for example, for I-VPLS 101 on PE-2:
*A:PE-2# show service id 101 sap lag-1:1.101 | match "Flags"
Flags : OperGroupDown
When the SAP goes down, the I-VPLS service goes down, as follows:
*A:PE-2# show service id 101 base | match "State"
Admin State : Up Oper State : Down
Even though Epipe 1 on PE-2 is operationally down while Epipe 1 on PE-3 is up, PE-1 is unaware because the VXLAN destination in Epipe 1 remains up, as follows:
*A:PE-1# show service id 1 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
23.23.23.1 1 Up static
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---
With ECMP=1, all traffic from PE-1 is directed to PE-2, regardless of the state of the Epipe on PE-2. The following route table on PE-1 shows that destination prefix 23.23.23.0/31 has next-hop 192.168.12.2, which is an interface address on PE-2.
*A:PE-1# show router route-table 23.23.23.0/31
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
23.23.23.0/31 Remote ISIS 00h06m41s 15
192.168.12.2 10
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
Traffic from the CEs attached to PE-1 is forwarded by PE-1 to PE-2, where it is dropped.
Service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLS using AA MH ESs
BGP must be enabled on all nodes for the EVPN address family, also in the access to and from PE-1. The BGP configuration on PE-1 is as follows:
# on PE-1:
configure
router Base
autonomous-system 64500
bgp
vpn-apply-import
vpn-apply-export
rapid-update evpn
group "ACCESS"
family evpn
type internal
split-horizon
neighbor 192.0.2.2
exit
neighbor 192.0.2.3
exit
exit
On PE-1, the following EVPN-VXLAN Epipe 2 is configured with local Ethernet tag 101 and remote Ethernet tag 123.
# on PE-1:
configure
service
epipe 2 name "Epipe 2" customer 1 create
description "Epipe 2 with EVPN-VXLAN"
vxlan instance 1 vni 2 create
exit
bgp
exit
bgp-evpn
local-attachment-circuit AC-1 create
eth-tag 101
exit
remote-attachment-circuit AC-23 create
eth-tag 123
exit
evi 2
vxlan bgp 1 vxlan-instance 1
no shutdown
exit
exit
sap 1/2/1:2.* create
no shutdown
exit
no shutdown
exit
On PE-2, the following EVPN-VXLAN Epipe 2 is configured with local Ethernet tag 123 and remote Ethernet tag 101. The operational group "op-grp-2" is associated with Epipe 2. The configuration on PE-3 is identical.
# on PE-2:
configure
service
oper-group "op-grp-2" create
exit
epipe 2 name "Epipe 2" customer 1 create
description "Epipe 2 with EVPN-VXLAN"
oper-group "op-grp-2"
vxlan instance 1 vni 2 create
exit
bgp
exit
bgp-evpn
local-attachment-circuit AC-23 create
eth-tag 123
exit
remote-attachment-circuit AC-1 create
eth-tag 101
exit
evi 2
vxlan bgp 1 vxlan-instance 1
no shutdown
exit
exit
sap lag-2:2.* create
no shutdown
exit
no shutdown
exit
The configuration of B-VPLS 100 remains unchanged and the configuration of the I-VPLSs 201 and 202 resembles the configuration of VPLSs 101.
When there is no failure, the egress VTEP for Epipe 2 on PE-1 is 192.0.2.2, which is the system IP address of PE-2, as follows:
*A:PE-1# show service id 2 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.2 2 Up evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---
To emulate a failure that affects the operational state of the Epipe service, the SAP in Epipe 2 is disabled, as follows:
# on PE-2:
configure
service
epipe "Epipe 2"
sap lag-2:2.*
shutdown
When the SAP goes down, the Epipe goes down, as follows:
*A:PE-2# show service id 2 base | match "State"
Admin State : Up Oper State : Down
On PE-1, the egress VTEP for Epipe 2 is 192.0.2.3, which is the system IP address of PE-3, as follows:
*A:PE-1# show service id 2 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.3 2 Up evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---
The operational group "op-grp-2" follows the state of Epipe 2, so it goes down, as follows. As a consequence, the monitoring SAPs for this operational group also go down.
*A:PE-2# show service oper-group "op-grp-2" detail
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group : op-grp-2
Creation Origin : manual Oper Status: down
Hold DownTime : 0 secs Hold UpTime: 4 secs
Members : 1 Monitoring : 2
===============================================================================
===============================================================================
Member Services for OperGroup: op-grp-2
===============================================================================
Svc Id
-------------------------------------------------------------------------------
2
-------------------------------------------------------------------------------
Service Entries found: 1
===============================================================================
===============================================================================
Monitoring SAPs for OperGroup: op-grp-2
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
lag-1:2.201 201 1 none 1 none Up Down
lag-1:2.202 202 1 none 1 none Up Down
-------------------------------------------------------------------------------
SAP Entries found: 2
===============================================================================
The SAPs in I-VPLSs 201 and 202 go down with the OperGroupDown flag, as follows:
*A:PE-2# show service id 201 sap lag-1:2.201 detail | match "Flags" context all
Flags : OperGroupDown
*A:PE-2# show service id 202 sap lag-1:2.202 detail | match "Flags" context all
Flags : OperGroupDown
When the SAPs go down, the I-VPLSs 201 and 202 also go down, as follows:
*A:PE-2# show service id 201 base | match "State"
Admin State : Up Oper State : Down
*A:PE-2# show service id 202 base | match "State"
Admin State : Up Oper State : Down
Even with this failure on PE-2, traffic can still flow between the CEs, as follows:
*A:PE-4# ping router 21 172.16.21.11 rapid
PING 172.16.21.11 56 data bytes
!!!!!
---- 172.16.21.11 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.51ms, avg = 4.84ms, max = 9.30ms, stddev = 2.24ms.
The following FDB for I-VPLS 201 on PE-4 shows that MAC address 00:ca:fe:00:21:11 of CE-1-21 is reachable via AA MH ES with ES-BMAC 00:00:00:00:23:21:
*A:PE-4# show service id 201 fdb detail
===============================================================================
Forwarding Database, Service 201
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
201 00:ca:fe:00:21:11 eES-BMAC: L/90 07/15/21 13:50:28
00:00:00:00:23:21
201 00:ca:fe:00:21:41 sap:1/2/1:2.201 L/90 07/15/21 13:56:09
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
Service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLS using SA MH ESs
Epipe 3 with EVPN-VXLAN and SA MH ES shows the example topology with an SA MH ES used by the I-VPLSs.
The configuration of Epipe 3 resembles the configuration of Epipe 2: the same Ethernet tags are used, only the VNI, EVI, and SAPs are different.
On PE-2 and PE-3, PXC 4 is configured to stitch Epipe 3 to I-VPLSs 301 and 302. The PXC port will be used in the SA MH ES.
On PE-2 and PE-3, Epipe 3 is configured as follows:
# on PE-2, PE-3:
configure
service
oper-group "op-grp-3" create
exit
epipe 3 name "Epipe 3" customer 1 create
description "EVPN-VXLAN Epipe 3"
oper-group "op-grp-3"
vxlan instance 1 vni 3 create
exit
bgp-evpn
local-attachment-circuit AC-23 create
eth-tag 123
exit
remote-attachment-circuit AC-1 create
eth-tag 101
exit
evi 3
vxlan bgp 1 vxlan-instance 1
no shutdown
exit
exit
sap pxc-4.b:3.* create
no shutdown
exit
no shutdown
exit
On PE-2, I-VPLS 301 uses SA MH ES "vES23_3.*", and is configured as follows.
# on PE-2:
configure
service
system
bgp-evpn
ethernet-segment "vES23_3.*" virtual create
esi 01:00:00:00:00:23:00:00:03:01
source-bmac-lsb 23-34 es-bmac-table-size 8
es-activation-timer 3
service-carving
mode manual
manual
preference create
value 100
exit
exit
exit
multi-homing single-active
port pxc-4.a
qinq
s-tag-range 3
exit
no shutdown
exit
exit
exit
vpls 301 name "I-VPLS 301" customer 1 i-vpls create
pbb
backbone-vpls 100
exit
exit
stp
shutdown
exit
sap pxc-4.a:3.301 create
monitor-oper-group "op-grp-3"
no shutdown
exit
no shutdown
exit
The configuration is similar on PE-3, but with source-bmac-lsb 23-35 and preference 50.
When Epipe 3 on PE-2 is operationally up, the egress VTEP for Epipe 3 on PE-1 is 192.0.2.2, as follows:
*A:PE-1# show service id 3 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.2 3 Up evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---
To emulate a failure on PE-2 that affects the operational state of the Epipe service, the SAP in Epipe 3 is disabled, as follows:
# on PE-2:
configure
service
epipe "Epipe 3"
sap pxc-4.b:3.*
shutdown
When the SAP goes down, Epipe 3 goes down on PE-2, as follows:
*A:PE-2# show service id 3 base | match "State"
Admin State : Up Oper State : Down
When Epipe 3 on PE-2 goes operationally down, the egress VTEP for Epipe 3 on PE-1 is 192.0.2.3, as follows:
*A:PE-1# show service id 3 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.3 3 Up evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
---snip---
The operational group "op-grp-3" follows the state of Epipe 3 on PE-2, so it goes down. Also, the monitoring SAPs for this operational group go down.
*A:PE-2# show service oper-group "op-grp-3" detail
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group : op-grp-3
Creation Origin : manual Oper Status: down
Hold DownTime : 0 secs Hold UpTime: 4 secs
Members : 1 Monitoring : 2
===============================================================================
===============================================================================
Member Services for OperGroup: op-grp-3
===============================================================================
Svc Id
-------------------------------------------------------------------------------
3
-------------------------------------------------------------------------------
Service Entries found: 1
===============================================================================
===============================================================================
Monitoring SAPs for OperGroup: op-grp-3
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
pxc-4.a:3.301 301 1 none 1 none Up Down
pxc-4.a:3.302 302 1 none 1 none Up Down
-------------------------------------------------------------------------------
SAP Entries found: 2
===============================================================================
The SAPs in I-VPLSs 301 and 302 on PE-2 go down with the OperGroupDown flag, as follows:
*A:PE-2# show service id 301 sap pxc-4.a:3.301 | match "Flags" context all
Flags : StandByForMHProtocol
OperGroupDown
*A:PE-2# show service id 302 sap pxc-4.a:3.302 | match "Flags" context all
Flags : StandByForMHProtocol
OperGroupDown
When the SAPs go down, the I-VPLSs go down on PE-2, as follows:
*A:PE-2# show service id 301 base | match "State"
Admin State : Up Oper State : Down
*A:PE-2# show service id 302 base | match "State"
Admin State : Up Oper State : Down
When the initial DF PE-2 goes down for the I-VPLSs 301 and 302, PE-3 becomes the new DF. The connectivity between the CEs is preserved, as follows:
*A:PE-4# ping router 31 172.16.31.11 rapid
PING 172.16.31.11 56 data bytes
!!!!!
---- 172.16.31.11 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.68ms, avg = 4.15ms, max = 4.40ms, stddev = 0.264ms
The following FDB for I-VPLS 301 on PE-4 shows that the frames toward MAC address 00:ca:fe:00:31:11 of CE-1-31 are sent via PE-3 (192.0.2.3):
*A:PE-4# show service id 301 fdb detail
===============================================================================
Forwarding Database, Service 301
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
301 00:ca:fe:00:31:11 b-mpls: L/120 07/15/21 14:16:25
192.0.2.3:524279
ldp:65537
301 00:ca:fe:00:31:41 sap:1/2/1:3.301 L/120 07/15/21 14:12:33
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
PE-3 is now the DF for I-VPLS 301, as follows:
*A:PE-3# show service id 301 ethernet-segment
===============================================================================
SAP Ethernet-Segment Information
===============================================================================
SAP Eth-Seg Status
-------------------------------------------------------------------------------
pxc-4.a:3.301 vES23_3.* DF
===============================================================================
No sdp entries
No vxlan instance entries
Conclusion
Some service providers use VXLAN as a next-generation access technology used between the MSANs (or access PEs) and core PE routers. EVPN-VXLAN Epipes can be stitched using PXC to other services, such as I-VPLS. Operational groups can be defined in the Epipe for fault propagation to the SAPs of the services where the Epipe is stitched to.