EVPN Interconnect Ethernet Segments in Dual EVPN-VXLAN Instance VPLS Services
This chapter provides information about EVPN Interconnect Ethernet Segments in Dual EVPN-VXLAN Instance VPLS Services.
Topics in this chapter include:
Applicability
The information and configuration in this chapter are based on SR OS Release 21.7.R1. EVPN multi-homing on dual VXLAN instance VPLS services is supported on SR OS Release 19.10.R1, and later.
Overview
Some service providers are deploying large Data Centers (DCs) where SR OS routers are used as leaf switches in a VXLAN fabric. In those cases, all-active multi-homing can provide redundancy and maximize the bandwidth utilization.
SR OS supports Interconnect Ethernet Segments (I-ESs) for VXLAN as per RFC 9014. Chapter EVPN Interconnect Ethernet Segments (I-ESs) describes how I-ESs allow Data Center Gateways (DCGWs) with two BGP instances (one for EVPN-MPLS and one for EVPN-VXLAN) to handle redundancy in VXLAN access networks, as supported in SR OS 15.0.R4, and later.
This chapter describes similar scenarios with EVPN-VXLAN in the core network instead of EVPN-MPLS. The following scenarios are supported with I-ES in VXLAN instance 1:
dual instance VPLS with two EVPN-VXLAN instances
dual instance VPLS with one EVPN-VXLAN instance and one static VXLAN instance
dual instance VPLS with one EVPN-VXLAN instance and one EVPN-MPLS instance
The first two of these scenarios are described in this chapter.
CLI
Sample topology shows VPLS 1 with different EVPN-VXLAN instances: VXLAN instance 1 in DC 1 (and DC2) and VXLAN instance 2 in the WAN.
On DCGW PE-2, the following all-active I-ES is configured for VXLAN instance 1 and service id 1:
# on DCGW PE-2:
configure
service
system
bgp-auto-rd-range 192.0.2.2 comm-val 1 to 1000
bgp-evpn
ethernet-segment "I-ES-23_1" virtual create
esi 00:23:23:23:23:23:23:00:00:01
service-carving
mode manual
manual
preference create
value 100
exit
evi 1
exit
exit
multi-homing all-active
network-interconnect-vxlan 1
service-id
service-range 1
exit
no shutdown
exit
exit
The following command configures VPLS 1 with dual EVPN-VXLAN instance. VXLAN instance 1 is a member of the I-ES and VXLAN instance 2 is configured with mh-mode network and auto-disc-route-advertisement:
# on DCGW PE-2:
configure
service
vpls 1 name "VPLS 1" customer 1 create
vxlan instance 1 vni 11 create
rx-discard-on-ndf bum
exit
vxlan instance 2 vni 12 create
exit
bgp
route-distinguisher auto-rd
route-target export target:64500:11 import target:64500:11
exit
bgp 2
route-distinguisher auto-rd
route-target export target:64500:12 import target:64500:12
exit
bgp-evpn
evi 1
vxlan bgp 1 vxlan-instance 1
ecmp 2
default-route-tag 11
auto-disc-route-advertisement
no shutdown
exit
vxlan bgp 2 vxlan-instance 2
ecmp 2
default-route-tag 12
auto-disc-route-advertisement
mh-mode network
no shutdown
exit
exit
stp
shutdown
exit
sap 1/2/1:1 create
no shutdown
exit
no shutdown
exit
By default, the multi-homing mode for EVPN-VXLAN is access, but for VXLAN instance 2, it is modified to mh-mode network. The following error is raised when attempting to configure VXLAN instance 1—as a member of an I-ES—with mh-mode network:
*A:PE-2>config>service>vpls>bgp-evpn>vxlan# mh-mode network
MINOR: SVCMGR #7886 cannot modify evpn - not supported when vxlan instance is a member of an ethernet-segment
With mh-mode network configured, it is mandatory to configure auto-disc-route-advertisement; for mh-mode access, it is optional. When auto-disc-route-advertisement is enabled in an access instance associated to an I-ES, AD per-ES/EVI routes and MAC/IP routes are advertised for the I-ES.
The following AD per-EVI route is sent by DCGW PE-2:
13 2021/09/07 14:35:47.456 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.1
"Peer 1: 192.0.2.1: UPDATE
Peer 1: 192.0.2.1 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:1 ESI: 00:23:23:23:23:23:23:00:00:01,
tag: 0 Label: 11 (Raw Label: 0xb) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
origin:64500:23
target:64500:11
bgp-tunnel-encap:VXLAN
"
For MAC routes and their ESI value for an access VXLAN instance, the following redistribution considerations apply.
With mh-mode access and auto-disc-route-advertisement configured, MAC routes are redistributed from the instance network to the instance access with the I-ESI if present, regardless of the original ESI.
With mh-mode access and no auto-disc-route-advertisement, MAC routes are redistributed with zero ESI, regardless of the original ESI.
The following EVPN-MAC route is sent by DCGW PE-2 with I-ESI 00:23:23:23:23:23:23:00:00:01 of "I-ES-23_1":
79 2021/09/07 14:36:18.380 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.1
"Peer 1: 192.0.2.1: UPDATE
Peer 1: 192.0.2.1 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 89
Flag: 0x90 Type: 14 Len: 44 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-MAC Len: 33 RD: 192.0.2.2:1 ESI: 00:23:23:23:23:23:23:00:00:01,
tag: 0, mac len: 48 mac: 00:ca:fe:ca:fe:05, IP len: 0, IP: NULL, label1: 11
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
origin:64500:23
target:64500:11
bgp-tunnel-encap:VXLAN
"
The following ES route is sent by DCGW PE-2:
17 2021/09/07 14:35:47.456 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.1
"Peer 1: 192.0.2.1: UPDATE
Peer 1: 192.0.2.1 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 71
Flag: 0x90 Type: 14 Len: 34 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-ETH-SEG Len: 23 RD: 192.0.2.2:0
ESI: 00:23:23:23:23:23:23:00:00:01, IP-Len: 4 Orig-IP-Addr: 192.0.2.2
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
df-election::DF-Type:Preference/DP:0/DF-Preference:100/AC:1
target:23:23:23:23:23:23
"
The following commands are not supported when mh-mode network is configured:
proxy-arp/nd
assisted replication
source-vtep-security
Attempting to enable these unsupported commands while a BGP-EVPN VXLAN instance has mh-mode network triggers error messages, as follows:
*A:PE-2>config>service>vpls# proxy-arp
MINOR: SVCMGR #8005 Cannot create proxy arp - not supported when a bgp-evpn vxlan instance has mh-mode network
*A:PE-2>config>service>vpls# proxy-nd
MINOR: SVCMGR #8008 Cannot create proxy nd - not supported when a bgp-evpn vxlan instance has mh-mode network
*A:PE-2>config>service>vpls>vxlan# assisted-replication replicator
MINOR: SVCMGR #8111 Cannot change assisted-replicated role - not supported when vxlan instance is in use by bgp-evpn with mh-mode network
*A:PE-2>config>service>vpls>vxlan# source-vtep-security
MINOR: SVCMGR #7897 Cannot modify vxlan instance - not supported when vxlan instance is in use by bgp-evpn with mh-mode network
Local bias
When EVPN-VXLAN is used in the instance network of a dual-instance VPLS service, local bias—as described in RFC 8365—is used for split horizon in all-active I-ESs. In VXLAN, there is no multicast label or multicast BMAC, so BUM traffic is identified by the MAC destination address. The modified forwarding rules for the I-ES-sourced BUM traffic for ingress PE and egress PE are as follows:
ingress PE
The Non-Designated Forwarder (NDF) must discard BUM traffic, so one of the following two commands must be configured in VXLAN instance 1.
no send-imet-ir-on-ndf
rx-discard-on-ndf bum
BUM frames received on any SAP or I-ES VXLAN binding are flooded to:
local non-ES and single-active DF ES SAPs
local all-active ES SAPs (DF and NDF)
EVPN-VXLAN destinations (BUM frames received on an I-ES VXLAN binding follow SHG rules, so they can only be forwarded to EVPN-VXLAN destinations belonging to the other VXLAN instance.)
egress PE
Look up source VTEP for BUM frames received on EVPN-VXLAN.
If the source VTEP matches a PE with which the local PE shares an ES and a VXLAN service, then the local PE does not forward to the shared local ESs (this includes port, lag, and network-interconnect-VXLAN ESs).
The local PE forwards to local ESs that are not shared but only when in DF state.
EVPN-VXLAN network interconnect VXLAN multi-homing and local bias shows the BUM forwarding with local bias procedures in multi-instance VPLS services.
In the example, GW1 and GW2 are configured with no send-imet-ir-on-ndf. TOR1 generates BUM traffic that will only reach DF GW1 and is forwarded as follows.
Ingress PE GW1 forwards to CE-1 and EVPN-VXLAN destinations GW2, GW3, and GW4.
Egress PE GW2 identifies the source VTEP as a PE with which I-ES-1 is shared, so it does not forward the BUM frames to the local I-ES. PE GW2 forwards only to the non-shared ES and local SAPs, in this case, to CE-2.
Egress PE GW3 receives the BUM traffic with a source VTEP that does not match any PE with which GW3 shares an ES, so it forwards to all ESs that are DF, in this case, to CE-3.
Egress PE GW4 receives the BUM traffic with a source VTEP that does not match any PE with which GW4 shares an ES, so it forwards to all ESs that are DF, in this case, to TOR2 through I-ES-2.
Local bias with static VXLAN on I-ES
When a static VXLAN instance coexists with an EVPN-VXLAN instance in the same VPLS service, traffic blackholes may occur when the static VXLAN instance is associated to an all-active I-ES. This is because, when multi-homing is used with an EVPN-VXLAN network instance, the NDF PE always discards unknown unicast traffic to the static VXLAN instance (this is not the case with EVPN-MPLS if the unknown traffic has a BUM label).
All-active I-ES NDF PE-5 drops unknown unicast traffic shows the packet blackhole for unknown unicast traffic at all-active I-ES NDF PE-5.
In the event that the remote PE-1 has learned the destination MAC address MAC6 via I-ES-45 EVPN destination, but the DCGWs PE-4 and PE-5 do not know MAC6, regular aliasing procedures allow that PE-1 sends unicast traffic with destination MAC6 to the NDF PE-5, which does not know MAC6 and drops all unknown unicast traffic, creating a blackhole for the flow.
When a static VXLAN instance coexists with an EVPN-VXLAN instance in the same VPLS service, Nokia recommends using a single-active I-ES or an anycast solution without I-ES instead of an all-active I-ES.
Configuration
Sample topology shows the sample topology with six SR OS nodes:
The initial configuration includes:
Cards, MDAs, and ports
Router interfaces
IS-IS on all interfaces (level 1 in the DCs; level 2 in the WAN)
BGP is configured for the EVPN address family. PE-1 acts as Route Reflector (RR) in DC 1 and PE-6 as RR in DC 2; no RR is used in the WAN.
The BGP configuration on RR PE-1 in DC 1 is as follows. The BGP configuration on RR PE-6 in DC2 is similar.
# on PE-1:
configure
router Base
autonomous-system 64500
bgp
family evpn
vpn-apply-import
vpn-apply-export
cluster 192.0.2.1
rapid-withdrawal
rapid-update evpn
group "DC"
type internal
neighbor 192.0.2.2
exit
neighbor 192.0.2.3
exit
exit
On DCGWs PE-2 and PE-3, BGP is configured as follows. The policies are explained in the next section.
# on PE-2, PE-3:
configure
router Base
autonomous-system 64500
bgp
family evpn
vpn-apply-import
vpn-apply-export
rapid-withdrawal
rapid-update evpn
group "DC"
type internal
import "drop SOO-DCGW-23"
export "export DC routes and add SOO"
neighbor 192.0.2.1
exit
exit
group "WAN"
type internal
export "export WAN routes only"
neighbor 192.0.2.4
exit
neighbor 192.0.2.5
exit
exit
On DCGWs PE-4 and PE-5, BGP is configured as follows. The policies are explained in the next section.
# on PE-4, PE-5:
configure
router
autonomous-system 64500
bgp
family evpn
vpn-apply-import
vpn-apply-export
rapid-withdrawal
rapid-update evpn
group "DC"
type internal
import "drop SOO-DCGW-45"
export "export DC routes and add SOO"
neighbor 192.0.2.6
exit
exit
group "WAN"
type internal
export "export WAN routes only"
neighbor 192.0.2.2
exit
neighbor 192.0.2.3
exit
exit
The following examples are configured:
All-active multi-homing I-ESs in dual EVPN-VXLAN instance VPLS
All-active multi-homing for I-ESs shows the example topology with the service VPLS 1 on all nodes and two all-active I-ESs:
On PE-1, VPLS 1 is configured as follows. The configuration on PE-6 is similar.
# on PE-1:
configure
service
system
bgp-auto-rd-range 192.0.2.1 comm-val 1 to 1000 # on PE-6: 192.0.2.6
exit
vpls 1 name "VPLS 1" customer 1 create
vxlan instance 1 vni 11 create
exit
bgp
route-distinguisher auto-rd
route-target export target:64500:11 import target:64500:11
exit
bgp-evpn
evi 1
vxlan bgp 1 vxlan-instance 1
ecmp 2
no shutdown
exit
exit
sap 1/2/1:1 create
exit
no shutdown
exit
On DCGW PE-2, the following all-active multi-homing I-ES is configured for VXLAN instance 1 and service id 1. The configuration on DCGW PE-3 is similar, but the preference value is 150 instead of 100.
# on PE-2:
configure
service
system
bgp-evpn
ethernet-segment "I-ES-23_1" virtual create
esi 00:23:23:23:23:23:23:00:00:01
service-carving
mode manual
manual
preference create
value 100 # on PE-3: preference value 150
exit
evi 1
exit
exit
multi-homing all-active
network-interconnect-vxlan 1
service-id
service-range 1
exit
no shutdown
exit
exit
On DCGWs PE-4 and PE-5, the following I-ES is configured:
# on PE-4, PE-5:
configure
service
system
bgp-evpn
ethernet-segment "I-ES-45_1" virtual create
esi 00:45:45:45:45:45:45:00:00:01
service-carving
mode auto
exit
multi-homing all-active
network-interconnect-vxlan 1
service-id
service-range 1
exit
no shutdown
exit
exit
exit
On DCGWs PE-2, PE-3, PE-4, and PE-5, VPLS 1 is configured as follows. The rx-discard-on-ndf bum command makes the NDF drop any BUM traffic in VXLAN instance 1. VXLAN instance 2 is configured with mh-mode network and auto-disc-route-advertisement.
# on PE-2, PE-3, PE-4, PE-5:
configure
service
vpls 1 name "VPLS 1" customer 1 create
vxlan instance 1 vni 11 create
rx-discard-on-ndf bum
exit
vxlan instance 2 vni 12 create
exit
bgp
route-distinguisher auto-rd
route-target export target:64500:11 import target:64500:11
exit
bgp 2
route-distinguisher auto-rd
route-target export target:64500:12 import target:64500:12
exit
bgp-evpn
evi 1
vxlan bgp 1 vxlan-instance 1
ecmp 2
default-route-tag 11
auto-disc-route-advertisement
no shutdown
exit
vxlan bgp 2 vxlan-instance 2
ecmp 2
default-route-tag 12
auto-disc-route-advertisement
mh-mode network
no shutdown
exit
exit
stp
shutdown
exit
sap 1/2/1:1 create
no shutdown
exit
no shutdown
exit
On PE-2 and PE-3, the following policies are configured.
The import policy "drop SOO-DCGW-23" in group "DC" is used to drop all VXLAN instance 1 routes between PE-2 and PE-3.
The export policy "export WAN routes only" in group "WAN" is applied to avoid sending VXLAN instance 1 routes to the WAN PEs.
The export policy "export DC routes and add SOO" in group "DC" is used to tag VXLAN instance 1 routes with community "SOO-23".
# on PE-2, PE-3:
configure
router Base
policy-options
begin
community "SOO-23"
members "origin:64500:23"
exit
policy-statement "drop SOO-DCGW-23" # import in group "DC"
entry 10
from
community "SOO-23"
family evpn
exit
action drop
exit
exit
default-action accept
exit
exit
policy-statement "export WAN routes only" # export in group "WAN"
entry 10
from
tag 11
family evpn
exit
action drop
exit
exit
default-action accept
exit
exit
policy-statement "export DC routes and add SOO" # export in group "DC"
entry 10
from
tag 11
family evpn
exit
action accept
community add "SOO-23"
exit
exit
default-action accept
exit
exit
commit
On PE-4 and PE-5, the following policies are configured:
# on PE-4, PE-5:
configure
router Base
policy-options
begin
community "SOO-45"
members "origin:64500:45"
exit
policy-statement "drop SOO-DCGW-45" # import in group "DC"
entry 10
from
community "SOO-45"
family evpn
exit
action drop
exit
exit
default-action accept
exit
exit
policy-statement "export WAN routes only" # export in group "WAN"
entry 10
from
tag 11
family evpn
exit
action drop
exit
exit
default-action accept
exit
exit
policy-statement "export DC routes and add SOO" # export in group "DC"
entry 10
from
tag 11
family evpn
exit
action accept
community add "SOO-45"
exit
exit
default-action accept
exit
exit
commit
For VPLS 1, PE-2 is DF and PE-3 is NDF in the I-ES "I-ES-23_1":
*A:PE-2# show service id 1 ethernet-segment
No sap entries
No sdp entries
===============================================================================
VXLAN Ethernet-Segment Information
===============================================================================
VXLAN Instance Eth-Seg Status
-------------------------------------------------------------------------------
1 I-ES-23_1 DF
===============================================================================
*A:PE-3# show service id 1 ethernet-segment
No sap entries
No sdp entries
===============================================================================
VXLAN Ethernet-Segment Information
===============================================================================
VXLAN Instance Eth-Seg Status
-------------------------------------------------------------------------------
1 I-ES-23_1 NDF
===============================================================================
PE-4 is NDF and PE-5 is DF in the I-ES "I-ES-45_1":
*A:PE-4# show service vxlan-instance-using ethernet-segment
===============================================================================
VXLAN Ethernet-Segment Information
===============================================================================
SvcId VXLAN Instance ES Name Status
-------------------------------------------------------------------------------
1 1 I-ES-45_1 NDF
===============================================================================
*A:PE-5# show service vxlan-instance-using ethernet-segment
===============================================================================
VXLAN Ethernet-Segment Information
===============================================================================
SvcId VXLAN Instance ES Name Status
-------------------------------------------------------------------------------
1 1 I-ES-45_1 DF
===============================================================================
On leaf PE-1, the VXLAN destinations in VXLAN instance 1 are the following:
*A:PE-1# show service id 1 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
Instance VTEP Address Egress VNI EvpnStatic Num
Mcast Oper State L2 PBR SupBcasDom MACs
-------------------------------------------------------------------------------
1 192.0.2.2 11 evpn 0
BUM Up No No
1 192.0.2.3 11 evpn 0
BUM Up No No
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 2
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-VXLAN Ethernet Segment Dest
===============================================================================
Instance Eth SegId Num. Macs Last Change
-------------------------------------------------------------------------------
1 00:23:23:23:23:23:23:00:00:01 5 09/07/2021 14:35:57
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
On DCGW PE-2, the VXLAN destinations in VXLAN instances 1 and 2 are the following:
*A:PE-2# show service id 1 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
Instance VTEP Address Egress VNI EvpnStatic Num
Mcast Oper State L2 PBR SupBcasDom MACs
-------------------------------------------------------------------------------
1 192.0.2.1 11 evpn 1
BUM Up No No
2 192.0.2.3 12 evpn 1
BUM Up No No
2 192.0.2.4 12 evpn 1
BUM Up No No
2 192.0.2.5 12 evpn 1
BUM Up No No
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 4
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-VXLAN Ethernet Segment Dest
===============================================================================
Instance Eth SegId Num. Macs Last Change
-------------------------------------------------------------------------------
2 00:45:45:45:45:45:45:00:00:01 1 09/07/2021 14:36:31
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
ECMP 2 is configured, so aliasing is used. PE-1 can reach the I-ES "I-ES-23_1" in VXLAN instance 1 via PE-2 and PE-3:
*A:PE-1# show service id 1 vxlan esi 00:23:23:23:23:23:23:00:00:01
===============================================================================
BGP EVPN-VXLAN Ethernet Segment Dest
===============================================================================
Instance Eth SegId Num. Macs Last Change
-------------------------------------------------------------------------------
1 00:23:23:23:23:23:23:00:00:01 5 09/07/2021 14:35:57
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-VXLAN Dest TEP Info
===============================================================================
Instance TEP Address Egr VNI Last Change
-------------------------------------------------------------------------------
1 192.0.2.2 11 09/07/2021 14:35:47
1 192.0.2.3 11 09/07/2021 14:35:57
-------------------------------------------------------------------------------
Number of entries : 2
-------------------------------------------------------------------------------
===============================================================================
In a similar way, PE-4 can reach the I-ES "I-ES-23_1" via PE-2 and PE-3 in VXLAN instance 2:
*A:PE-4# show service id 1 vxlan esi 00:23:23:23:23:23:23:00:00:01
===============================================================================
BGP EVPN-VXLAN Ethernet Segment Dest
===============================================================================
Instance Eth SegId Num. Macs Last Change
-------------------------------------------------------------------------------
2 00:23:23:23:23:23:23:00:00:01 1 09/07/2021 14:36:10
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-VXLAN Dest TEP Info
===============================================================================
Instance TEP Address Egr VNI Last Change
-------------------------------------------------------------------------------
2 192.0.2.2 12 09/07/2021 14:36:10
2 192.0.2.3 12 09/07/2021 14:36:10
-------------------------------------------------------------------------------
Number of entries : 2
-------------------------------------------------------------------------------
===============================================================================
The following command on PE-2 shows the ES information for "I-ES-23_1": DF status, DF candidate list, VXLAN instance service range, and so on:
*A:PE-2# show service system bgp-evpn ethernet-segment name "I-ES-23_1" all
===============================================================================
Service Ethernet Segment
===============================================================================
Name : I-ES-23_1
Eth Seg Type : Virtual
Admin State : Enabled Oper State : Up
ESI : 00:23:23:23:23:23:23:00:00:01
Oper ESI : 00:23:23:23:23:23:23:00:00:01
Auto-ESI Type : None
AC DF Capability : Include
Multi-homing : allActive Oper Multi-homing : allActive
ES SHG Label : 524287
Source BMAC LSB : None
VXLAN Instance Id : 1
ES Activation Timer : 3 secs (default)
Oper Group : (Not Specified)
Svc Carving : manual Oper Svc Carving : manual
Cfg Range Type : lowest-pref
-------------------------------------------------------------------------------
DF Pref Election Information
-------------------------------------------------------------------------------
Preference Preference Last Admin Change Oper Pref Do No
Mode Value Value Preempt
-------------------------------------------------------------------------------
revertive 100 09/07/2021 14:35:47 100 Disabled
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
EVI Ranges
-------------------------------------------------------------------------------
From To
-------------------------------------------------------------------------------
1 1
-------------------------------------------------------------------------------
ISID Ranges: <none>
===============================================================================
===============================================================================
EVI Information
===============================================================================
EVI SvcId Actv Timer Rem DF
-------------------------------------------------------------------------------
1 1 0 yes
-------------------------------------------------------------------------------
Number of entries: 1
===============================================================================
-------------------------------------------------------------------------------
DF Candidate list
-------------------------------------------------------------------------------
EVI DF Address
-------------------------------------------------------------------------------
1 192.0.2.2
1 192.0.2.3
-------------------------------------------------------------------------------
Number of entries: 2
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
===============================================================================
---snip---
===============================================================================
Vxlan Instance Service Ranges
===============================================================================
Svc Range Start Svc Range End Last Changed
-------------------------------------------------------------------------------
1 1 09/07/2021 14:35:47
-------------------------------------------------------------------------------
Number of Entries: 1
===============================================================================
When traffic is sent between CE-1 and CE-6, the EVPN-MAC routes are sent with I-ESI. The FDB for VPLS 1 on PE-1 shows I-ESI 00:23:23:23:23:23:23:00:00:01 of "I-ES-23_1" as source identifier for MAC address 00:ca:fe:ca:fe:06 of CE-6:
*A:PE-1# show service id 1 fdb detail
===============================================================================
Forwarding Database, Service 1
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
1 00:ca:fe:ca:fe:01 sap:1/2/1:1 L/127 09/07/21 14:48:43
1 00:ca:fe:ca:fe:06 eES: Evpn 09/07/21 14:48:43
00:23:23:23:23:23:23:00:00:01
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
On PE-2, the FDB for VPLS 1 shows I-ESI 00:45:45:45:45:45:45:00:00:01 of "I-ES-45_1" as source identifier for MAC address 00:ca:fe:ca:fe:06 of CE-6:
*A:PE-2# show service id 1 fdb detail
===============================================================================
Forwarding Database, Service 1
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
1 00:ca:fe:ca:fe:01 vxlan-1: Evpn 09/07/21 14:48:43
192.0.2.1:11
1 00:ca:fe:ca:fe:06 eES: Evpn 09/07/21 14:48:43
00:45:45:45:45:45:45:00:00:01
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
Single-active multi-homing I-ES when static VXLAN coexists with EVPN-VXLAN in the same VPLS
I-ES with EVPN-VXLAN in DC 1 and static VXLAN in DC2 shows the sample topology for VPLS 2 with static VXLAN in DC 2 and the single-active "I-ES-45_2" on PE-4 and PE-5.
The configuration for VPLS 2 on PE-1, PE-2, and PE-3 is similar to the configuration for VPLS 1, so only the configuration on PE-4, PE-5, and PE-6 is shown.
On PE-6, VPLS 2 is configured with static VXLAN using non-anycast VTEP addresses:
# on PE-6:
configure
service
system
bgp-auto-rd-range 192.0.2.6 comm-val 1 to 1000
exit
vpls 2 name "VPLS 2" customer 1 create
vxlan instance 1 vni 21 create
egr-vtep 192.0.2.4 create
exit
egr-vtep 192.0.2.5 create
exit
exit
sap 1/2/1:2 create
no shutdown
exit
no shutdown
exit
To avoid blackholes, the I-ES between DCGWs PE-4 and PE-5 must not be all-active.
On PE-4 and PE-5, the single-active I-ES "I-ES-45_2" is configured as follows:
# on PE-4, PE-5:
configure
service
system
bgp-evpn
ethernet-segment "I-ES-45_2" virtual create
esi 00:45:45:45:45:45:45:00:00:02
service-carving
mode auto
exit
multi-homing single-active
network-interconnect-vxlan 1
service-id
service-range 2
exit
no shutdown
exit
exit
On PE-4 and PE-5, VPLS 2 is configured as follows:
# on PE-4, PE-5:
configure
service
vpls 2 name "VPLS 2" customer 1 create
vxlan instance 1 vni 21 create
egr-vtep 192.0.2.6 create
exit
exit
vxlan instance 2 vni 22 create
exit
bgp 2
route-distinguisher auto-rd
route-target export target:64500:22 import target:64500:22
exit
bgp-evpn
evi 2
vxlan bgp 2 vxlan-instance 2
ecmp 2
default-route-tag 22
auto-disc-route-advertisement
mh-mode network
no shutdown
exit
exit
stp
shutdown
exit
sap 1/2/1:2 create # optional SAP toward local CE
no shutdown
exit
no shutdown
exit
The policies on all DCGWs must be modified with tag 21 for VXLAN instance 1 in VPLS 2, as follows:
# on PE-2, PE-3:
configure
router Base
policy-options
begin
policy-statement "export WAN routes only"
entry 20
from
tag 21
family evpn
exit
action drop
exit
exit
default-action accept
exit
exit
policy-statement "export DC routes and add SOO"
entry 20
from
tag 21
family evpn
exit
action accept
community add "SOO-23"
exit
exit
default-action accept
exit
exit
commit
DCGW PE-5 is NDF for "I-ES-45_2":
*A:PE-5# show service id 2 ethernet-segment
No sap entries
No sdp entries
===============================================================================
VXLAN Ethernet-Segment Information
===============================================================================
VXLAN Instance Eth-Seg Status
-------------------------------------------------------------------------------
1 I-ES-45_2 NDF
===============================================================================
On PE-5, the status of VXLAN instance 1 in VPLS 2 is mhStandby, as follows:
*A:PE-5# show service id 2 vxlan
===============================================================================
VPLS VXLAN
===============================================================================
Vxlan Src Vtep IP: N/A
===============================================================================
Vxlan Instance
===============================================================================
VXLAN Instance VNI AR Oper-flags VTEP
security
-------------------------------------------------------------------------------
1 21 none mhStandby disabled
2 22 none none disabled
-------------------------------------------------------------------------------
Number of Entries : 2
-------------------------------------------------------------------------------
===============================================================================
The VXLAN destinations in VPLS 2 on PE-5 are the following:
*A:PE-5# show service id 2 vxlan destinations
===============================================================================
Egress VTEP, VNI
===============================================================================
Instance VTEP Address Egress VNI EvpnStatic Num
Mcast Oper State L2 PBR SupBcasDom MACs
-------------------------------------------------------------------------------
1 192.0.2.6 21 static 0
- Up No No
2 192.0.2.2 22 evpn 1
BUM Up No No
2 192.0.2.3 22 evpn 1
BUM Up No No
2 192.0.2.4 22 evpn 1
BUM Up No No
-------------------------------------------------------------------------------
Number of Egress VTEP, VNI : 4
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-VXLAN Ethernet Segment Dest
===============================================================================
Instance Eth SegId Num. Macs Last Change
-------------------------------------------------------------------------------
2 00:23:23:23:23:23:23:00:00:02 1 09/07/2021 14:53:05
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
An anycast solution without I-ES can also be configured when an EVPN-VXLAN coexists with a static VXLAN.
Conclusion
Service providers can use I-ESs for better bandwidth utilization and redundancy in large DCs. EVPN all-active multi-homing I-ESs can be used in dual EVPN-VXLAN instance VPLS services. However, when a static VXLAN instance coexists with EVPN-VXLAN in the same VPLS, a single-active multi-homing I-ES (or an anycast solution without I-ES) is required to avoid blackholes.