Operational Groups for EVPN-VXLAN VPWS Services
This chapter describes the Operational Groups for EVPN-VXLAN VPWS Services.
Topics in this chapter include:
Applicability
This chapter was initially written based on SR OS Release 16.0.R5, but the MD-CLI in the current edition corresponds to SR OS Release 21.7.R1. EVPN-VXLAN VPWS and service-level operational groups for VPWS services are supported in SR OS Release 16.0.R1, or later.
Overview
Operational groups on Epipe services are used for fault propagation to other services, such as I-VPLS or R-VPLS services. Epipes with VXLAN destinations are used in some edge PE applications along with port cross-connect (PXC) so that VXLAN networks can be terminated in other VPLS or VPRN services. In such cases, the operational status of the Epipe services terminating VXLAN must override the operational status of the SAPs of the VPLS or VPRN where the Epipe is stitched to.
Operational group on egress VTEP in Epipes with static VXLAN bindings
The Static VXLAN Termination in Epipe Services chapter describes how Epipes with static VXLAN termination are stitched to I-VPLS services. In Epipes with static VXLAN bindings, operational groups can be configured in the egress VTEP context. Epipe with static VXLAN termination shows the example topology with a static VXLAN tunnel between PE-1 and an anycast address on PE-2 and PE-3. The All-Active Multi-Homing Ethernet Segments (AA MH ESs) "vES23_1.101" and "vES23_1.102" are used by the I-VPLSs 101 and 102, which are both stitched to Epipe 1 in PE-2 and PE-3. The SAPs in these I-VPLSs monitor the operational group configured in the egress VTEP context of the Epipe service, so the SAPs will go operationally down when the operational group of the VTEP goes operationally down.
On PE-2 and PE-3, Epipe 1 is configured with static VXLAN bindings, as follows. The egress VTEP is 192.0.2.1, which is the system IP address of PE-1. Operational group "op-grp-1" in configured for this egress VTEP. LAG 2 combines PXC ports and is used to stitch Epipe 1 to the I-VPLS services 101 and 102. For a detailed description of the configuration, see the Static VXLAN Termination in Epipe Services chapter.
# on PE-2, PE-3:
configure {
service {
oper-group "op-grp-1" {
}
epipe "Epipe 1" {
admin-state enable
description "Epipe 1 with static VXLAN bindings"
service-id 1
customer "1"
sap lag-2:1.* {
}
vxlan {
source-vtep 23.23.23.1
instance 1 {
vni 1
egress-vtep {
ip-address 192.0.2.1
oper-group "op-grp-1"
}
}
}
}
For failure propagation to the stitched I-VPLSs, the SAPs in the I-VPLSs can monitor the operational group "op-grp-1", for I-VPLS 101 on PE-2 and PE-3, as follows:
# on PE-2, PE-3:
configure {
service {
vpls "I-VPLS 101" {
admin-state enable
service-id 101
customer "1"
pbb-type i-vpls
pbb {
backbone-vpls "B-VPLS-100" {
isid 101
}
}
sap lag-1:1.101 {
monitor-oper-group "op-grp-1"
}
}
When the egress VTEP prefix 192.0.2.1 disappears from the global route-table on PE-2, the VXLAN binding goes down, as follows:
[/]
A:admin@PE-2# show service id 1 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.1 1 Down static
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN VXLAN ES Dest
===============================================================================
I Eth Seg Id TEP Address VNI Last Changed
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
When the egress VTEP 192.0.2.1 goes down, the operational group "op-grp-1" goes down too, as follows:
[/]
A:admin@PE-2# show service oper-group "op-grp-1"
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group : op-grp-1
Creation Origin : manual Oper Status: down
Hold DownTime : 0 secs Hold UpTime: 4 secs
Members : 1 Monitoring : 2
===============================================================================
When the operational group "op-grp-1" goes down, the monitoring SAP in I-VPLS 101 goes operationally down with flag OperGroupDown, as follows:
[/]
A:admin@PE-2# show service id 101 sap lag-1:1.101 detail | match 'Flags | Oper State'
Admin State : Up Oper State : Down
Flags : OperGroupDown
Stp Admin State : Up Stp Oper State : Down
When this SAP goes down, the entire I-VPLS 101 service goes down on PE-2, as follows:
[/]
A:admin@PE-2# show service id 101 base | match 'State'
Admin State : Up Oper State : Down
Epipes with static VXLAN bindings impose the following restrictions, which cannot be overcome unless a control plane protocol such as BGP-EVPN is used for the VXLAN bindings.
-
When anycast VTEPs on the PEs are used, a change in the vES preference on the DF PE triggers a DF switchover for the I-VPLS service. However, the access PE (PE-1 in Figure 1) is unaware and keeps sending the VXLAN traffic to the same PE, unless a change in DF comes with an automatic change in the underlay IGP metrics, which cannot be easily accomplished.
-
Without a control plane, Eth-CFM must be used between PEs and access PEs to detect end-to-end service-level failures.
-
Traffic from the access PE is forwarded to the anycast VTEP, based on underlay IGP metrics. There is no control on a per-service basis.
-
The architecture does not support AA MH for the Epipe service, so the access PEs always send the traffic to one single PE.
The preceding challenges can be addressed by using different VTEPs on the PEs and adding a BGP-EVPN control plane on the Epipe, as described in the next section.
Operational groups in EVPN-VXLAN Epipes
Epipe 2 with EVPN-VXLAN and all-active multi-homing shows EVPN-VXLAN Epipe 2 stitched to I-VPLSs 201 and 202. AA MH ESs "vES23_2.201" and "vES23_2.202" are used by the I-VPLSs 201 and 202 respectively.
The EVPN-VXLAN VPWS chapter describes the configuration of Epipes with EVPN-VXLAN bindings instead of static VXLAN bindings. The egress VTEP is not configured manually, but dynamically learned through BGP-EVPN. Therefore, the operational group cannot be configured in the egress VTEP context. However, it is possible to configure an operational group in an Epipe at the service level, as follows:
# on PE-2:
configure {
service {
oper-group "op-grp-2" {
}
epipe "Epipe 2" {
admin-state enable
description "Epipe 2 with EVPN-VXLAN"
service-id 2
customer "1"
oper-group "op-grp-2"
bgp 1 {
}
sap lag-2:2.* {
}
vxlan {
instance 1 {
vni 2
}
}
bgp-evpn {
evi 2
local-attachment-circuit "AC-23" {
eth-tag 123
}
remote-attachment-circuit "AC-1" {
eth-tag 101
}
vxlan 1 {
admin-state enable
vxlan-instance 1
}
}
}
The following shows the error messages raised when attempting to configure the egress VTEP manually in an Epipe service with BGP-EVPN enabled:
[ex:/configure service epipe "Epipe 2" vxlan instance 1 egress-vtep]
A:admin@PE-2# ip-address 192.0.2.1
*[ex:/configure service epipe "Epipe 2" vxlan instance 1 egress-vtep]
A:admin@PE-2# commit
MINOR: SVCMGR #12: configure service epipe "Epipe 2" vxlan instance 1 egress-vtep ip-address - Inconsistent Value error - configuration incompatible with bgp-evpn - configure service epipe "Epipe 2" bgp-evpn
MINOR: MGMT_CORE #4001: configure service epipe "Epipe 2" vxlan instance 1 egress-vtep ip-address - egress vtep not supported with bgp-evpn - configure service epipe "Epipe 2" bgp-evpn
An operational group can be associated with the entire Epipe or with specific objects, such as SAPs or spoke-SDPs, but not simultaneously. The following error is raised when attempting to associate the operational group "op-grp-2"—that is already associated with the Epipe with the SAP on PE-2:
[ex:/configure service epipe "Epipe 2" sap lag-2:2.*]
A:admin@PE-2# oper-group "op-grp-2"
*[ex:/configure service epipe "Epipe 2" sap lag-2:2.*]
A:admin@PE-2# commit
MINOR: SVCMGR #12: configure service epipe "Epipe 2" sap lag-2:2.* oper-group - Inconsistent Value error - cannot monitor or belong to a service oper-group within the same service - configure service epipe "Epipe 2" oper-group
The service-level operational group status is derived from the service operational status: when Epipe 2 is operationally down, the operational group "op-grp-2" will be down.
For fault propagation to the stitched I-VPLSs 201 and 202, the SAPs in the I-VPLSs monitor the operational group "op-grp-2", for I-VPLS 201 on PE-2 and PE-3, as follows:
# on PE-2, PE-3:
configure {
service {
vpls "I-VPLS 201" {
admin-state enable
service-id 201
customer "1"
pbb-type i-vpls
pbb {
backbone-vpls "B-VPLS-100" {
isid 201
}
}
sap lag-1:2.201 {
monitor-oper-group "op-grp-2"
}
}
Configuration
In this section, the following use cases are described:
-
operational group on egress VTEP in Epipes with static VXLAN bindings stitched to I-VPLSs using AA MH ESs
-
service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLSs using AA MH ESs
-
service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLSs using Single-Active (SA) MH ESs
Example topology shows the example topology with four PEs in an autonomous system.
The initial configuration includes:
-
Cards, MDAs, ports
-
Router interfaces
-
IS-IS as IGP (level capability 2 in the core between PE-2, PE-3, and PE-4; level capability 1 in the access toward PE-1)
-
LDP between the core routers PE-2, PE-3, and PE-4
Oper group on egress VTEP in Epipes with static VXLAN bindings stitched to I-VPLSs using AA MH ESs
When static VXLAN bindings are used, no BGP-EVPN is required in the access network to and from PE-1; BGP is only configured in the core network. When PE-2 acts as the route reflector, its BGP configuration is as follows:
# on RR PE-2:
configure {
router "Base" {
autonomous-system 64500
bgp {
vpn-apply-export true
vpn-apply-import true
rapid-update {
evpn true
}
group "CORE" {
type internal
split-horizon true
family {
evpn true
}
cluster {
cluster-id 192.0.2.2
}
}
neighbor "192.0.2.3" {
group "CORE"
}
neighbor "192.0.2.4" {
group "CORE"
}
Epipe with static VXLAN termination shows that Epipe 1 is configured in the access network: on PE-1, the VTEP is the system address 192.0.2.1 (default), and on PE-2 and PE-3, the VTEP is a unicast address 23.23.23.1.
On PE-1, Epipe 1 is configured with egress VTEP 23.23.23.1, as follows:
# on PE-1:
configure {
service {
epipe "Epipe 1" {
admin-state enable
description "Epipe 1 with static VXLAN bindings"
service-id 1
customer "1"
sap 1/2/1:1.* {
}
vxlan {
instance 1 {
vni 1
egress-vtep {
ip-address 23.23.23.1
}
}
}
}
On PE-2 and PE-3, the following unicast address is configured:
# on PE-2, PE-3:
configure {
router "Base" {
interface "lo23" {
loopback
ipv4 {
primary {
address 23.23.23.0
prefix-length 31
}
}
On PE-2 and PE-3, three ports are configured as PXC. PXC 1 is used as Forwarding Path Extension (FPE) and the VXLAN tunnel termination 23.23.23.1 is configured with this FPE, as follows:
# on PE-2, PE-3:
configure {
service {
system {
vxlan {
tunnel-termination 23.23.23.1 {
fpe-id 1
}
}
PXCs 2 and 3 are used in the internal LAGs that are used to stitch Epipe 1 to I-VPLSs 101 and 102, as follows. LAG "lag-1" will be used in the I-VPLS services; LAG "lag-2" in the Epipe services.
# on PE-2, PE-3:
configure {
lag "lag-1" {
admin-state enable
encap-type qinq
mode hybrid
max-ports 64
port pxc-2.a {
}
port pxc-3.a {
}
}
lag "lag-2" {
admin-state enable
encap-type qinq
mode hybrid
max-ports 64
port pxc-2.b {
}
port pxc-3.b {
}
}
On PE-2 and PE-3, Epipe 1 is configured with source VTEP 23.23.23.1 and egress VTEP 192.0.2.1, as follows. The operational group "op-grp-1" is associated with the egress VTEP. The SAP stitches Epipe 1 to the I-VPLSs 101 and 102.
# on PE-2, PE-3:
configure {
service {
oper-group "op-grp-1" {
}
epipe "Epipe 1" {
admin-state enable
description "Epipe 1 with static VXLAN bindings"
service-id 1
customer "1"
sap lag-2:1.* {
}
vxlan {
source-vtep 23.23.23.1
instance 1 {
vni 1
egress-vtep {
ip-address 192.0.2.1
oper-group "op-grp-1"
}
}
}
}
On PE-2, B-VPLS 100 is configured as follows. The configuration is similar on PE-3 and PE-4.
# on PE-2:
configure {
service {
vpls "B-VPLS-100" {
admin-state enable
service-id 100
customer "1"
service-mtu 1532
pbb-type b-vpls
pbb {
source-bmac {
address 00:00:00:00:00:02
use-es-bmac-lsb true
}
}
bgp 1 {
}
bgp-evpn {
evi 100
mpls 1 {
admin-state enable
ingress-replication-bum-label true
auto-bind-tunnel {
resolution any
}
}
}
}
On PE-2, I-VPLS 101 is configured as follows. The SAP monitors the operational group "op-grp-1" that is configured in Epipe 1. The AA MH ES "vES23_1.101" is used. The configuration of I-VPLS 102 is similar, but it uses AA MH ES "vES23_1.102" with preference value 50 instead. On PE-3, the preference values are reversed: preference value 50 for "vES23_1.101" and 100 for "vES23_1.102".
# on PE-2:
configure {
service {
system {
bgp {
evpn {
ethernet-segment "vES23_1.101" {
admin-state enable
type virtual
esi 0x01000000002300000111
multi-homing-mode all-active
df-election {
es-activation-timer 3
service-carving-mode manual
manual {
preference {
value 100
}
}
}
association {
lag "lag-1" {
virtual-ranges {
qinq {
s-tag-c-tag 1 c-tag-start 101 {
c-tag-end 101
}
}
}
}
}
pbb {
source-bmac-lsb 0x2311
}
}
}
}
}
vpls "I-VPLS 101" {
admin-state enable
service-id 101
customer "1"
pbb-type i-vpls
pbb {
backbone-vpls "B-VPLS-100" {
isid 101
}
}
sap lag-1:1.101 {
monitor-oper-group "op-grp-1"
}
}
On PE-4, I-VPLS 101 is configured as follows:
# on PE-4:
configure {
service {
vpls "I-VPLS 101" {
admin-state enable
service-id 101
customer "1"
pbb-type i-vpls
pbb {
backbone-vpls "B-VPLS-100" {
isid 101
}
}
sap 1/2/1:1.101 {
}
}
To emulate a failure that affects the operational state of the egress VTEP (and also of the Epipe service), the SAP of Epipe 1 on PE-2 is disabled, as follows:
# on PE-2:
configure {
service {
epipe "Epipe 1" {
sap lag-2:1.* {
admin-state disable
When the SAP is operationally down, Epipe 1 goes down, as follows:
[/]
A:admin@PE-2# show service id 1 base | match 'State'
Admin State : Up Oper State : Down
The egress VTEP 192.0.2.1 is operationally down, as follows:
[/]
A:admin@PE-2# show service id 1 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.1 1 Down static
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN VXLAN ES Dest
===============================================================================
I Eth Seg Id TEP Address VNI Last Changed
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
The operational group "op-grp-1" is associated with the egress VTEP, so it goes operationally down, as follows:
[/]
A:admin@PE-2# show service oper-group "op-grp-1"
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group : op-grp-1
Creation Origin : manual Oper Status: down
Hold DownTime : 0 secs Hold UpTime: 4 secs
Members : 1 Monitoring : 2
===============================================================================
This operational group is monitored by the SAPs in I-VPLSs 101 and 102, so these SAPs go down with flag OperGroupDown; for example, for I-VPLS 101 on PE-2:
[/]
A:admin@PE-2# show service id 101 sap lag-1:1.101 detail | match 'Flags'
Flags : OperGroupDown
When the SAP goes down, the I-VPLS service goes down, as follows:
[/]
A:admin@PE-2# show service id 101 base | match 'State'
Admin State : Up Oper State : Down
Even though Epipe 1 on PE-2 is operationally down while Epipe 1 on PE-3 is up, PE-1 is unaware because the VXLAN destination in Epipe 1 remains up, as follows:
[/]
A:admin@PE-1# show service id 1 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
23.23.23.1 1 Up static
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN VXLAN ES Dest
===============================================================================
I Eth Seg Id TEP Address VNI Last Changed
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
With ECMP=1, all traffic from PE-1 is directed to PE-2, regardless of the state of the Epipe on PE-2. The following route table on PE-1 shows that destination prefix 23.23.23.0/31 has next-hop 192.168.12.2, which is an interface address on PE-2.
[/]
A:admin@PE-1# show router route-table 23.23.23.0/31
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
23.23.23.0/31 Remote ISIS 00h03m57s 15
192.168.12.2 10
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
Traffic from the CEs attached to PE-1 is forwarded by PE-1 to PE-2, where it is dropped.
Service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLS using AA MH ESs
BGP must be enabled on all nodes for the EVPN address family, also in the access to and from PE-1. The BGP configuration on PE-1 is as follows:
# on PE-1:
configure {
router "Base" {
autonomous-system 64500
bgp {
vpn-apply-import true
vpn-apply-export true
rapid-update {
evpn true
}
group "ACCESS" {
type internal
split-horizon true
family {
evpn true
}
}
neighbor 192.0.2.2 {
group "ACCESS"
}
neighbor 192.0.2.3 {
group "ACCESS"
}
On PE-1, the following EVPN-VXLAN Epipe 2 is configured with local Ethernet tag 101 and remote Ethernet tag 123.
# on PE-1:
configure {
service {
epipe "Epipe 2" {
admin-state enable
description "Epipe 2 with EVPN-VXLAN"
service-id 2
customer "1"
bgp 1 {
}
sap 1/2/1:2.* {
}
vxlan {
instance 1 {
vni 2
}
}
bgp-evpn {
evi 2
local-attachment-circuit "AC-1" {
eth-tag 101
}
remote-attachment-circuit "AC-23" {
eth-tag 123
}
vxlan 1 {
admin-state enable
vxlan-instance 1
}
}
}
On PE-2, the following EVPN-VXLAN Epipe 2 is configured with local Ethernet tag 123 and remote Ethernet tag 101. The operational group "op-grp-2" is associated with Epipe 2. The configuration on PE-3 is identical.
# on PE-2:
configure {
service {
oper-group "op-grp-2" {
}
epipe "Epipe 2" {
admin-state enable
description "Epipe 2 with EVPN-VXLAN"
service-id 2
customer "1"
oper-group "op-grp-2"
bgp 1 {
}
sap lag-2:2.* {
}
vxlan {
instance 1 {
vni 2
}
}
bgp-evpn {
evi 2
local-attachment-circuit "AC-23" {
eth-tag 123
}
remote-attachment-circuit "AC-1" {
eth-tag 101
}
vxlan 1 {
admin-state enable
vxlan-instance 1
}
}
}
The configuration of B-VPLS 100 remains unchanged and the configuration of the I-VPLSs 201 and 202 resembles the configuration of VPLSs 101.
When there is no failure, the egress VTEP for Epipe 2 on PE-1 is 192.0.2.2, which is the system IP address of PE-2, as follows:
[/]
A:admin@PE-1# show service id 2 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.2 2 Up evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN VXLAN ES Dest
===============================================================================
I Eth Seg Id TEP Address VNI Last Changed
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
To emulate a failure that affects the operational state of the Epipe service, the SAP in Epipe 2 is disabled, as follows:
# on PE-2:
configure {
service {
epipe "Epipe 2" {
sap lag-2:2.* {
admin-state disable
When the SAP goes down, the Epipe goes down, as follows:
[/]
A:admin@PE-2# show service id 2 base | match 'State'
Admin State : Up Oper State : Down
On PE-1, the egress VTEP for Epipe 2 is 192.0.2.3, which is the system IP address of PE-3, as follows:
[/]
A:admin@PE-1# show service id 2 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.3 2 Up evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN VXLAN ES Dest
===============================================================================
I Eth Seg Id TEP Address VNI Last Changed
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
The operational group "op-grp-2" follows the state of Epipe 2, so it goes down, as follows. As a consequence, the monitoring SAPs for this operational group also go down.
[/]
A:admin@PE-2# show service oper-group "op-grp-2" detail
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group : op-grp-2
Creation Origin : manual Oper Status: down
Hold DownTime : 0 secs Hold UpTime: 4 secs
Members : 1 Monitoring : 2
===============================================================================
===============================================================================
Member Services for OperGroup: op-grp-2
===============================================================================
Svc Id
-------------------------------------------------------------------------------
2
-------------------------------------------------------------------------------
Service Entries found: 1
===============================================================================
===============================================================================
Monitoring SAPs for OperGroup: op-grp-2
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
lag-1:2.201 201 1 none 1 none Up Down
lag-1:2.202 202 1 none 1 none Up Down
-------------------------------------------------------------------------------
SAP Entries found: 2
===============================================================================
The SAPs in I-VPLSs 201 and 202 go down with the OperGroupDown flag, as follows:
[/]
A:admin@PE-2# show service id 201 sap lag-1:2.201 detail | match 'Flags'
Flags : OperGroupDown
[/]
A:admin@PE-2# show service id 202 sap lag-1:2.202 detail | match 'Flags'
Flags : OperGroupDown
When the SAPs go down, the I-VPLSs 201 and 202 also go down, as follows:
[/]
A:admin@PE-2# show service id 201 base | match 'State'
Admin State : Up Oper State : Down
[/]
A:admin@PE-2# show service id 202 base | match 'State'
Admin State : Up Oper State : Down
Even with this failure on PE-2, traffic can still flow between the CEs, as follows:
[/]
A:admin@PE-4# ping 172.16.21.11 router-instance "VPRN 21" interval 0.1 output-format summary
PING 172.16.21.11 56 data bytes
!!!!!
---- 172.16.21.11 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.40ms, avg = 4.02ms, max = 4.57ms, stddev = 0.388ms
The following FDB for I-VPLS 201 on PE-4 shows that MAC address 00:ca:fe:00:21:11 of CE-1-21 is reachable via AA MH ES with ES-BMAC 00:00:00:00:23:21:
[/]
A:admin@PE-4# show service id 201 fdb detail
===============================================================================
Forwarding Database, Service 201
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
201 00:ca:fe:00:21:11 eES-BMAC: L/90 08/10/21 16:56:31
00:00:00:00:23:21
201 00:ca:fe:00:21:41 sap:1/2/1:2.201 L/90 08/10/21 16:56:24
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
Service-level operational group in EVPN-VXLAN Epipes stitched to I-VPLS using SA MH ESs
Epipe 3 with EVPN-VXLAN and SA MH ES shows the example topology with an SA MH ES used by the I-VPLSs.
The configuration of Epipe 3 resembles the configuration of Epipe 2: the same Ethernet tags are used, only the VNI, EVI, and SAPs are different.
On PE-2 and PE-3, PXC 4 is configured to stitch Epipe 3 to I-VPLSs 301 and 302. The PXC port will be used in the SA MH ES.
On PE-2 and PE-3, Epipe 3 is configured as follows:
# on PE-2, PE-3:
configure {
service {
oper-group "op-grp-3" {
}
epipe "Epipe 3" {
admin-state enable
description "EVPN-VXLAN Epipe 3"
service-id 3
customer "1"
oper-group "op-grp-3"
sap pxc-4.b:3.* {
}
vxlan {
instance 1 {
vni 3
}
}
bgp-evpn {
evi 3
local-attachment-circuit "AC-23" {
eth-tag 123
}
remote-attachment-circuit "AC-1" {
eth-tag 101
}
vxlan 1 {
admin-state enable
vxlan-instance 1
}
}
}
On PE-2, I-VPLS 301 uses SA MH ES "vES23_3.*", and is configured as follows.
# on PE-2:
configure {
service {
oper-group "op-grp-3" {
}
system {
bgp {
evpn {
ethernet-segment "vES23_3.*" {
admin-state enable
type virtual
esi 0x01000000002300000301
multi-homing-mode single-active
df-election {
es-activation-timer 3
service-carving-mode manual
manual {
preference {
value 100
}
}
}
association {
port pxc-4.a {
virtual-ranges {
qinq {
s-tag 3 {
end 3
}
}
}
}
}
pbb {
source-bmac-lsb 0x2334
}
}
}
}
}
vpls "I-VPLS 301" {
admin-state enable
service-id 301
customer "1"
pbb-type i-vpls
pbb {
backbone-vpls "B-VPLS-100" {
isid 301
}
}
sap pxc-4.a:3.301 {
monitor-oper-group "op-grp-3"
}
}
The configuration is similar on PE-3, but with source-bmac-lsb 23-35 and preference 50.
When Epipe 3 on PE-2 is operationally up, the egress VTEP for Epipe 3 on PE-1 is 192.0.2.2, as follows:
[/]
A:admin@PE-1# show service id 3 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.2 3 Up evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN VXLAN ES Dest
===============================================================================
I Eth Seg Id TEP Address VNI Last Changed
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
To emulate a failure on PE-2 that affects the operational state of the Epipe service, the SAP in Epipe 3 is disabled, as follows:
# on PE-2:
configure {
service {
epipe "Epipe 3" {
sap pxc-4.b:3.* {
admin-state disable
When the SAP goes down, Epipe 3 goes down on PE-2, as follows:
[/]
A:admin@PE-2# show service id 3 base | match 'State'
Admin State : Up Oper State : Down
When Epipe 3 on PE-2 goes operationally down, the egress VTEP for Epipe 3 on PE-1 is 192.0.2.3, as follows:
[/]
A:admin@PE-1# show service id 3 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
VTEP Address Egress VNI Oper Vxlan
State Type
-------------------------------------------------------------------------------
192.0.2.3 3 Up evpn
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN VXLAN ES Dest
===============================================================================
I Eth Seg Id TEP Address VNI Last Changed
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
The operational group "op-grp-3" follows the state of Epipe 3 on PE-2, so it goes down. Also, the monitoring SAPs for this operational group go down.
[/]
A:admin@PE-2# show service oper-group "op-grp-3" detail
===============================================================================
Service Oper Group Information
===============================================================================
Oper Group : op-grp-3
Creation Origin : manual Oper Status: down
Hold DownTime : 0 secs Hold UpTime: 4 secs
Members : 1 Monitoring : 2
===============================================================================
===============================================================================
Member Services for OperGroup: op-grp-3
===============================================================================
Svc Id
-------------------------------------------------------------------------------
3
-------------------------------------------------------------------------------
Service Entries found: 1
===============================================================================
===============================================================================
Monitoring SAPs for OperGroup: op-grp-3
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
pxc-4.a:3.301 301 1 none 1 none Up Down
pxc-4.a:3.302 302 1 none 1 none Up Down
-------------------------------------------------------------------------------
SAP Entries found: 2
===============================================================================
The SAPs in I-VPLSs 301 and 302 on PE-2 go down with the OperGroupDown flag, as follows:
[/]
A:admin@PE-2# show service id 301 sap pxc-4.a:3.301 detail | match 'Flags' post-lines 1
Flags : StandByForMHProtocol
OperGroupDown
[/]
A:admin@PE-2# show service id 302 sap pxc-4.a:3.302 detail | match 'Flags' post-lines 1
Flags : StandByForMHProtocol
OperGroupDown
When the SAPs go down, the I-VPLSs go down on PE-2, as follows:
[/]
A:admin@PE-2# show service id 301 base | match 'State'
Admin State : Up Oper State : Down
[/]
A:admin@PE-2# show service id 302 base | match 'State'
Admin State : Up Oper State : Down
When the initial DF PE-2 goes down for the I-VPLSs 301 and 302, PE-3 becomes the new DF. The connectivity between the CEs is preserved, as follows:
[/]
A:admin@PE-4# ping 172.16.31.11 router-instance "VPRN 31" interval 0.1 output-format summary
PING 172.16.31.11 56 data bytes
!!!!!
---- 172.16.31.11 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.56ms, avg = 3.89ms, max = 4.94ms, stddev = 0.526ms
The following FDB for I-VPLS 301 on PE-4 shows that the frames toward MAC address 00:ca:fe:00:31:11 of CE-1-31 are sent via PE-3 (192.0.2.3):
[/]
A:admin@PE-4# show service id 301 fdb detail
===============================================================================
Forwarding Database, Service 301
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
301 00:ca:fe:00:31:11 b-mpls: L/270 08/10/21 17:08:17
192.0.2.3:524281
ldp:65539
301 00:ca:fe:00:31:41 sap:1/2/1:3.301 L/270 08/10/21 17:04:55
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
PE-3 is now the DF for I-VPLS 301, as follows:
[/]
A:admin@PE-3# show service id 301 ethernet-segment
===============================================================================
SAP Ethernet-Segment Information
===============================================================================
SAP Eth-Seg Status
-------------------------------------------------------------------------------
pxc-4.a:3.301 vES23_3.* DF
===============================================================================
No sdp entries
No vxlan instance entries
Conclusion
Some service providers use VXLAN as a next-generation access technology used between the MSANs (or access PEs) and core PE routers. EVPN-VXLAN Epipes can be stitched using PXC to other services, such as I-VPLS. Operational groups can be defined in the Epipe for fault propagation to the SAPs of the services where the Epipe is stitched to.