EVPN for MPLS Tunnels in Epipe Services (EVPN-VPWS)
This chapter provides information about EVPN for MPLS tunnels in Epipe services (EVPN-VPWS).
Topics in this chapter include:
Applicability
This chapter was initially written for SR OS Release 14.0.R4, but the MD-CLI in the current edition is based on SR OS Release 22.10.R1. Ethernet Virtual Private Network - Virtual Private Wire Service (EVPN-VPWS) is supported in SR OS Release 14.0.R1 and later. EVPN-VPWS in multi-homing scenarios is supported in SR OS Release 14.0.R4 and later.
Chapter EVPN for MPLS Tunnels is prerequisite reading.
Overview
Service providers prefer an optimized, standardized, and unified control plane for VPNs. EVPN-VPWS is supported in MPLS networks that also run EVPN-MPLS in VPLS services. From a control plane perspective, EVPN-VPWS is a simplified point-to-point version of RFC 7432 – BGP MPLS-Based Ethernet VPN, because there is no need to advertise MAC routes in VPWS. EVPN-VPWS is described in RFC 8214 – Virtual Private Wire Service Support in Ethernet VPN.
EVPN-VPWS supports all-active multi-homing (per-flow load-balancing multi-homing) as well as single-active multi-homing (per-service load-balancing multi-homing), using the same Ethernet segments (ESs) used for EVPN-MPLS VPLS services. EVPN-VPWS uses route-type 1 and route-type 4; it does not use route-types 2, 3, or 5, because MAC/IP routes, inclusive multicast, or IP-prefix routes are not required.
The figure Route types and NLRIs for EVPN-VPWS shows the encoding of the required extensions for the route-types 1 and 4 for EVPN-VPWS.
Two sub-types are defined for route-type 1. Route-type 4 has no sub-types. The route types used for EVPN-VPWS have the following purposes:
-
Route-type 1 - Auto-discovery per EVPN instance (AD per-EVI). This route type is used in all EVPN-VPWS scenarios, with or without multi-homing. For EVPN-VPWS, the Ethernet tag field is encoded with the local Attachment Circuit (AC) of the advertising PE. This value is configured using the service>epipe>bgp-evpn>local-attachment-circuit>eth-tag <value> command. The route distinguisher (RD), MPLS label, and the Ethernet segment ID (ESI) are encoded as for EVPN-MPLS. The MPLS label field is used as service label. In case of multi-homing, AD per-EVI routes containing the same ESI are used to provide aliasing and a backup path to the PEs part of the ES. The L2 MTU is encoded with the service MTU configured in the Epipe. The following flags are used for EVPN-VPWS:
-
Flag C is set if a control word is configured in the service.
-
Flag P is set if the advertising PE is primary PE.
-
If no multi-homing is used, there is no primary PE (P=0).
-
In all-active multi-homing, all PEs in the ES are primary (P=1).
-
In single-active multi-homing, only one PE per-EVI in the ES is primary (P=1).
-
-
Flag B is set if the advertising PE is backup PE.
-
The B-flag is only set in case of single-active multi-homing and only for one PE, even if more than two PEs are present in the same single-active ES. The backup PE is the winner of the second Designated Forwarder (DF) election (excluding the DF). The remaining non-DF PEs send B=0.
-
If there is no multi-homing, the ESI, flag P, and flag B will be zero.
-
-
Route-type 1 - AD per Ethernet segment (AD per-ES). Same encoding as for EVPN-MPLS. AD per-ES is only used in multi-homing scenarios where it is advertised per ES from the PE. It carries the ESI label (used for split-horizon, but only for VPLS services and not for Epipe services) and can affect procedures such as the DF election, as well as the aliasing on remote PEs.
-
Route-type 4 - ES route. Same encoding as for EVPN-MPLS. Route-type 4 is only used in multi-homing scenarios. This route advertises a local configured ES. The exchange of this route can discover remote PEs that are part of the same ES and the DF election algorithm among them.
Configuration
The figure EVPN-VPWS example topology shows the example topology that will be used throughout this chapter.
The example topology consists of six SR OS nodes with the following initial configuration:
-
Network (or hybrid) ports interconnect the core PEs with configured router interfaces.
-
MTU-1 is a pure Ethernet aggregator. The ports toward the core PEs are access ports. Likewise, the ports on PE-2 and PE-3 toward MTU-1 are access ports.
-
Core PEs and MTU-6 run IS-IS on all router interfaces. Point-to-point adjacencies are established for the exchange of system IP addresses.
-
Link LDP is configured between all PEs, and toward/from MTU-6.
-
EVPN uses BGP for exchanging reachability at service level. Therefore, BGP peering sessions must be established among the core PEs for the EVPN family. Although typically a separate router is used, in this chapter, PE-2 is used as route reflector (RR) with the following BGP configuration:
# on RR PE-2: configure { router "Base" { autonomous-system 64500 bgp { vpn-apply-export true vpn-apply-import true rapid-withdrawal true peer-ip-tracking true split-horizon true rapid-update { evpn true } group "internal" { peer-as 64500 family { evpn true } cluster { cluster-id 192.0.2.2 } } neighbor "192.0.2.3" { group "internal" } neighbor "192.0.2.4" { group "internal" } neighbor "192.0.2.5" { group "internal" } }The BGP configuration on the other PEs is as follows:
# on PE-3, PE-4, PE-5: configure { router "Base" { autonomous-system 64500 bgp { vpn-apply-export true vpn-apply-import true rapid-withdrawal true peer-ip-tracking true split-horizon true rapid-update { evpn true } group "internal" { peer-as 64500 family { evpn true } } neighbor "192.0.2.2" { group "internal" } }
The following EVPN-VPWS scenarios are described in the following sections:
EVPN for MPLS tunnels in Epipe services without multi-homing
BGP-EVPN can be enabled in Epipe services with either SAPs or spoke-SDPs at the access, as shown in the figure Example topology for EVPN-VPWS without multi-homing.
On PE-2, Epipe 1 is configured as follows:
# on PE-2:
configure {
service {
epipe "Epipe-1" {
admin-state enable
service-id 1
customer "1"
bgp 1 {
}
sap 1/1/c11/1:1 {
}
bgp-evpn {
evi 1
local-attachment-circuit "AC-PE-2-MTU-1" {
eth-tag 21
}
remote-attachment-circuit "AC-PE-4-MTU-6" {
eth-tag 46
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
On PE-4, the service configuration is as follows:
# on PE-4:
configure {
service {
epipe "Epipe-1" {
admin-state enable
service-id 1
customer "1"
bgp 1 {
}
spoke-sdp 460:1 {
}
bgp-evpn {
evi 1
local-attachment-circuit "AC-PE-4-MTU-6" {
eth-tag 46
}
remote-attachment-circuit "AC-PE-2-MTU-1" {
eth-tag 21
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
sdp 460 {
admin-state enable
far-end {
ip-address 192.0.2.6
}
}
Where the following commands are relevant for the EVPN-VPWS configuration:
-
bgp 1 enables the context for the BGP configuration relevant to the service. The bgp context configures the common BGP parameters for all BGP families in the service, such as route distinguisher and route target. Even if the general BGP parameters for the service are auto-derived, the bgp context must be enabled.
[ex:/configure service epipe "Epipe-1"] A:admin@PE-2# bgp 1 ? bgp adv-service-mtu - Advertised service MTU value apply-groups - Apply a configuration group at this level apply-groups-exclude - Exclude a configuration group at this level pw-template-binding + Enter the pw-template-binding list instance route-distinguisher - High-order 6 bytes that are used as string to compose VSI-ID for use in NLRI route-target + Enter the route-target context vsi-export - VSI export policies vsi-import - VSI import policies -
The following parameters can be configured in the bgp-evpn context:
[ex:/configure service epipe "Epipe-1"] A:admin@PE-2# bgp-evpn ? bgp-evpn apply-groups - Apply a configuration group at this level apply-groups-exclude - Exclude a configuration group at this level evi - EVPN ID local-attachment- + Enter the local-attachment-circuit list instance circuit mpls + Enter the mpls list instance remote-attachment- + Enter the remote-attachment-circuit list instance circuit segment-routing-v6 + Enter the segment-routing-v6 list instance vxlan + Enter the vxlan list instance-
The evi is a two-byte or three-byte identifier used for auto-deriving the service RD (only for two-byte EVI), service RT, and for the DF election in multi-homing. The auto-derivation of RD and RT for a two-byte EVI is as follows:
-
RD <system IP address>:<evi>
-
RT <autonomous system number>:<evi>
The EVI values must be unique in the system, regardless of the type of service they are assigned to (Epipe or VPLS).
Note: Three-byte EVI values are supported in SR OS Release 21.10.R1 and later. For auto-derived RT as per RFC 8365, the evi-three-byte-auto-rt command must be configured, as described in the Three-byte EVI in EVPN Services chapter. -
-
The local-attachment-circuit and remote-attachment-circuit identify the two attachment circuits connected by the EVPN-VPWS service. The configured Ethernet tag for the local AC is advertised in the Ethernet tag field of the AD per-EVI route for the Epipe, along with the corresponding RD, RT, and MPLS label. Both local and remote Ethernet tags are mandatory to bring up the Epipe service. If the received Ethernet tag for the Epipe service matches the configured remote AC Ethernet tag, it will create an EVPN-MPLS destination to the next hop.
-
The following configuration options are available for Epipes in the bgp-evpn>mpls <bgp-instance> context:
[ex:/configure service epipe "Epipe-1" bgp-evpn] A:admin@PE-2# mpls 1 ? mpls admin-state - Administrative state of BGP EVPN MPLS apply-groups - Apply a configuration group at this level apply-groups-exclude - Exclude a configuration group at this level auto-bind-tunnel + Enter the auto-bind-tunnel context control-word - Enable the CW bit in the label message default-route-tag - Default route tag dynamic-egress- - Enables dynamic egress label limit label-limit ecmp - Maximum ECMP routes information entropy-label - Enable use of entropy-labels evi-three-byte-auto- - Auto-derive the BGP EVPN route target rt force-vc-forwarding - VC forwarding action oper-group - Operational group identifier route-next-hop + Enter the route-next-hop context send-tunnel-encap + Enter the send-tunnel-encap contextThis is a subset of the options for VPLS services; see chapter EVPN for MPLS Tunnels.
-
When the local AC (SAP 1/1/c11/1:1) is up, PE-2 sends a BGP EVPN AD per-EVI route that contains Ethernet tag 21 for the local AC:
# on PE-2:
3 2022/11/30 11:33:31.729 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:1 ESI: ESI-0, tag: 21 Label: 8388512 (Raw Label: 0x7fffa0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:1
l2-attribute:MTU: 1514 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
"The auto-derived RD is 192.0.2.2:1 and the RT is 64500:1.
When the remote AC on PE-4 (spoke-SDP 460:1) is up, PE-2 receives the following BGP update from PE-4:
# on PE-2:
5 2022/11/30 11:33:50.377 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-AD Len: 25 RD: 192.0.2.4:1 ESI: ESI-0, tag: 46 Label: 8388512 (Raw Label: 0x7fffa0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:1
l2-attribute:MTU: 1514 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
"When the received RT matches and the received Ethernet tag matches the configured remote AC, the EVPN-MPLS destination (comprised of a termination endpoint (TEP) and egress label) is created on PE-2 and PE-4:
[/]
A:admin@PE-2# show service id 1 evpn-mpls
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-id
-------------------------------------------------------------------------------
192.0.2.4 524282 11/30/2022 11:33:50
ldp:65538
-------------------------------------------------------------------------------
Number of entries : 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
The MPLS label in the debug message is not the same as in the service, because the router will strip the extra four lowest bits to get the 20-bit MPLS label. The egress label for the EVPN-MPLS destination on PE-4 is 524282. The 24-bit label value in the BGP update debug is 16 (2^4) times as high: 524282*16 = 8388512. This is because the debug message is shown before the router can parse the label field and see if it corresponds to an MPLS label (20 bits) or a VXLAN VNI (24 bits).
The BGP AD per-EVI routes for Ethernet tag 46 can be shown with the following command:
[/]
A:admin@PE-2# show router bgp routes evpn auto-disc tag 46
===============================================================================
BGP Router ID:192.0.2.2 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Auto-Disc Routes
===============================================================================
Flag Route Dist. ESI NextHop
Tag Label
-------------------------------------------------------------------------------
u*>i 192.0.2.4:1 ESI-0 192.0.2.4
46 LABEL 524282
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The following command shows the BGP EVPN information for Epipe 1:
[/]
A:admin@PE-2# show service id 1 bgp-evpn
===============================================================================
BGP EVPN Table
===============================================================================
EVI : 1 Creation Origin : manual
-------------------------------------------------------------------------------
Local AC Name Eth Tag Endpoint Ingress Label
-------------------------------------------------------------------------------
AC-PE-2-MTU-1 21 0
-------------------------------------------------------------------------------
Number of local ACs : 1
-------------------------------------------------------------------------------
Remote AC Name Eth Tag Endpoint
-------------------------------------------------------------------------------
AC-PE-4-MTU-6 46
-------------------------------------------------------------------------------
Number of Remote ACs : 1
===============================================================================
===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status : Enabled Bgp Instance : 1
Force Vlan Fwding : Disabled
Force Qinq Fwding : none
Route NextHop Type : system-ipv4
Control Word : Disabled
Max Ecmp Routes : 1
Entropy Label : Disabled
Default Route Tag : none
Oper Group :
Evi 3-byte Auto-RT : Disabled
Dyn Egr Lbl Limit : Disabled
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN MPLS Auto Bind Tunnel Information
===============================================================================
Allow-Flex-Algo-Fallback : false
Resolution : any Strict Tnl Tag : false
Max Ecmp Routes : 1
Bgp Instance : 1
Filter Tunnel Types : (Not Specified)
Weighted Ecmp : false
-------------------------------------------------------------------------------
===============================================================================
EVPN for MPLS tunnels in Epipe services with multi-homing
SR OS supports EVPN multi-homing as per RFC 8214.
The EVPN multi-homing implementation is based on the concept of the Ethernet segment (ES). An ES is a logical structure that can be defined in one or more PEs and identifies the CE (or access network) multi-homed to the EVPN PEs. An ES is associated with a port, LAG, or SDP object, and is shared by all the services defined on those objects. It can also be shared between Epipe and VPLS services.
Each ES has a unique Ethernet Segment Identifier (ESI) that is 10 bytes and is manually configured.
The ES is part of the base BGP-EVPN configuration and is not applied to any EVPN-MPLS service, by default. An ES can be shared by multiple services; the association of a specific SAP or spoke-SDP to an ES is automatically made when the SAP is defined in the same LAG or port configured in the ES, or when the spoke-SDP is defined in the same SDP configured in the ES.
Regardless of the multi-homing mode, the local Ethernet tag values must match on all the PEs that are part of the same ES. The PEs in the ES will use the AD per-EVI routes from the peer PEs to validate the PEs as DF election candidates for an EVI. The DF election is only relevant for single-active multi-homing ESs. For Epipes defined in an all-active multi-homing ES, there is no DF election required, because all PEs are forwarding traffic and all traffic is treated as unicast.
Aliasing is supported when sending traffic to an ES destination. Assuming ECMP is enabled on the ingress PE (and shared queuing or ingress policing), per-flow load-balancing will be performed among all the PEs that advertised P=1. PEs advertising P=0 are not considered as next hops for an ES destination.
The following sections show the configuration of:
-
an all-active multi-homing ES with a LAG associated with it
-
a single-active multi-homing ES linked to an SDP
The figure Example topology EVPN-VPWS with multi-homing shows an all-active ES and a single-active ES. The all-active multi-homing ES "AA-ESI-23" on PE-2 and PE-3 has a LAG associated to it; the single-active multi-homing ES "SA-ESI-45" on PE-4 and PE-5 has an SDP associated to it.
EVPN for MPLS tunnels in Epipe services with all-active multi-homing
All-active multi-homing allows for per-flow load-balancing. Unlike EVPN-MPLS in VPLS services, EVPN-VPWS has no DF election in all-active multi-homing. All PEs in the ES are active and the remote PE will do per-flow load-balancing. ES "AA-ESI-23" is configured on PE-2 and PE-3 in all-active multi-homing with LAG 1 associated to it. This LAG is used as a SAP in Epipe 2 on both PE-2 and PE-3. The configuration of the ES and Epipe 2 is identical on PE-2 and PE-3, including the local AC and remote AC names and Ethernet tags:
# on PE-2, PE-3:
configure {
service {
system {
bgp {
evpn {
ethernet-segment "AA-ESI-23" {
admin-state enable
esi 01:00:00:00:00:23:00:00:00:01
multi-homing-mode all-active
df-election {
es-activation-timer 3
}
association {
lag "lag-1" {
}
}
}
}
}
}
epipe "Epipe-2" {
admin-state enable
service-id 2
customer "1"
bgp 1 {
}
sap lag-1:2 {
}
bgp-evpn {
evi 2
local-attachment-circuit "AC-AA-ESI-23-MTU-1" {
eth-tag 231
}
remote-attachment-circuit "AC-SA-ESI-45-MTU-6" {
eth-tag 456
}
mpls 1 {
admin-state enable
ecmp 2
auto-bind-tunnel {
resolution any
}
}
}
}
See chapter EVPN for MPLS Tunnels for a detailed explanation of the configuration parameters of the ES.
In EVPN-VPWS multi-homing scenarios, three route types are exchanged: AD per-EVI, AD per-ES, and ES routes. The following ES route (route-type 4) for ESI 01:00:00:00:00:23:00:00:00:01 sent by PE-2 is imported at PE-3:
# on PE-3:
3 2022/11/30 11:44:28.466 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 71
Flag: 0x90 Type: 14 Len: 34 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-ETH-SEG Len: 23 RD: 192.0.2.2:0 ESI: 01:00:00:00:00:23:00:00:00:01, IP-Len: 4 Orig-IP-Addr: 192.0.2.2
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
df-election::DF-Type:Auto/DP:0/DF-Preference:0/AC:1
target:00:00:00:00:23:00
"The target 00:00:00:00:23:00 in the extended community is derived from the ESI (bytes 2 to 7) and is only imported by the PEs that are part of the same ES; that is, PE-2 and PE-3 in this example.
At the same time, the following AD per-ES route (route-type 1) with maximum Ethernet tag (MAX-ET, all Fs) and label 0 is sent by RR PE-2 and imported by the rest of the PEs. The following two BGP updates with MAX-ET are received by PE-4:
# on PE-4:
6 2022/11/30 11:44:28.466 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:2 ESI: 01:00:00:00:00:23:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
esi-label:524280/All-Active
bgp-tunnel-encap:MPLS
"
8 2022/11/30 11:44:30.124 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-AD Len: 25 RD: 192.0.2.3:2 ESI: 01:00:00:00:00:23:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.3
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
esi-label:524281/All-Active
bgp-tunnel-encap:MPLS
"The ESI label is in the extended community, as well as the indication that the multi-homing is all-active. Epipe services do not require ESI labels because BUM traffic is not recognized as such in EVPN-VPWS services. However, because the ES can be shared by Epipe and VPLS services, the AD per-ES route still includes a non-zero ESI label.
The following AD per-EVI routes (route-type 1) with Ethernet tag 231 sent by RR PE-2 are received and imported on PE-4:
# on PE-4:
5 2022/11/30 11:44:28.466 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:2 ESI: 01:00:00:00:00:23:00:00:00:01, tag: 231 Label: 8388496 (Raw Label: 0x7fff90) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 1 B: 0
bgp-tunnel-encap:MPLS
"7 2022/11/30 11:44:30.124 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-AD Len: 25 RD: 192.0.2.3:2 ESI: 01:00:00:00:00:23:00:00:00:01, tag: 231 Label: 8388512 (Raw Label: 0x7fffa0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.3
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 1 B: 0
bgp-tunnel-encap:MPLS
"This route contains the flags for control word (C), primary (P), and backup (B). In all-active multi-homing, all nodes are primary (P=1).
PE-4 has learned AD per-EVI/ES routes for AA-ESI-23 from PE-2 and PE-3, as shown in the following output:
[/]
A:admin@PE-4# show router bgp routes evpn auto-disc esi 01:00:00:00:00:23:00:00:00:01
===============================================================================
BGP Router ID:192.0.2.4 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Auto-Disc Routes
===============================================================================
Flag Route Dist. ESI NextHop
Tag Label
-------------------------------------------------------------------------------
u*>i 192.0.2.2:2 01:00:00:00:00:23:00:00:00:01 192.0.2.2
231 LABEL 524281
u*>i 192.0.2.2:2 01:00:00:00:00:23:00:00:00:01 192.0.2.2
MAX-ET LABEL 0
u*>i 192.0.2.3:2 01:00:00:00:00:23:00:00:00:01 192.0.2.3
231 LABEL 524282
u*>i 192.0.2.3:2 01:00:00:00:00:23:00:00:00:01 192.0.2.3
MAX-ET LABEL 0
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
For Epipe 2 on PE-4, the EVPN MPLS destination is not pointing at a specific TEP, but AA-ESI-23, as shown in the following output:
[/]
A:admin@PE-4# show service id 2 evpn-mpls
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-id
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:23:00:00:00:01 11/30/2022 11:44:50
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
When ECMP > 1 on the ingress PE, multiple TEPs can correspond to a specific ESI (aliasing). In this case, ECMP=2 and PE-4 and PE-5 have two TEP addresses and egress labels for ESI 01:00:00:00:00:23:00:00:00:01, as shown for PE-4:
[/]
A:admin@PE-4# show service id 2 evpn-mpls esi esi-1 01:00:00:00:00:23:00:00:00:01
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:23:00:00:00:01 11/30/2022 11:44:50
===============================================================================
===============================================================================
BGP EVPN-MPLS Dest TEP Info
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-Id
-------------------------------------------------------------------------------
192.0.2.2 524281 11/30/2022 11:44:50
ldp:65538
192.0.2.3 524282 11/30/2022 11:44:50
ldp:65537
-------------------------------------------------------------------------------
Number of entries : 2
-------------------------------------------------------------------------------
===============================================================================
In all-active multi-homing for EVPN-VPWS, there is no DF election and all PEs in the ES are active. For AA-ESI-23, both PE-2 and PE-3 are active/primary/DF, but there are no DF candidates, because there is no DF election:
[/]
A:admin@PE-2# show service system bgp-evpn ethernet-segment name "AA-ESI-23" evi evi-1 2
===============================================================================
EVI DF and Candidate List
===============================================================================
EVI SvcId Actv Timer Rem DF DF Last Change
-------------------------------------------------------------------------------
2 2 0 yes 11/30/2022 11:44:28
===============================================================================
===============================================================================
DF Candidates Time Added Oper Pref Do Not
Value Preempt
-------------------------------------------------------------------------------
No entries found
===============================================================================
Similarly, on PE-3:
[/]
A:admin@PE-3# show service system bgp-evpn ethernet-segment name "AA-ESI-23" evi evi-1 2
===============================================================================
EVI DF and Candidate List
===============================================================================
EVI SvcId Actv Timer Rem DF DF Last Change
-------------------------------------------------------------------------------
2 2 0 yes 11/30/2022 11:44:30
===============================================================================
===============================================================================
DF Candidates Time Added Oper Pref Do Not
Value Preempt
-------------------------------------------------------------------------------
No entries found
===============================================================================
To confirm that all-active multi-homing is working correctly, the following command shows all information related to a specific ESI; in this case, AA-ESI-23 on PE-2:
[/]
A:admin@PE-2# show service system bgp-evpn ethernet-segment name "AA-ESI-23" all
===============================================================================
Service Ethernet Segment
===============================================================================
Name : AA-ESI-23
Eth Seg Type : None
Admin State : Enabled Oper State : Up
ESI : 01:00:00:00:00:23:00:00:00:01
Oper ESI : 01:00:00:00:00:23:00:00:00:01
Auto-ESI Type : None
AC DF Capability : Include
Multi-homing : allActive Oper Multi-homing : allActive
ES SHG Label : 524280
Source BMAC LSB : None
Lag : lag-1
ES Activation Timer : 3 secs
Oper Group : (Not Specified)
Svc Carving : auto Oper Svc Carving : auto
Cfg Range Type : primary
Vprn NextHop EVI Ranges : <none>
===============================================================================
===============================================================================
EVI Information
===============================================================================
EVI SvcId Actv Timer Rem DF
-------------------------------------------------------------------------------
2 2 0 yes
-------------------------------------------------------------------------------
Number of entries: 1
===============================================================================
---snip---
EVPN for MPLS tunnels in Epipe services with single-active multi-homing
Single-active multi-homing allows for per-service load-balancing. Single-active multi-homing is configured on PE-4 and PE-5 with ES "SA-ESI-45". Both PEs have an SDP to MTU-6, which is associated with the ES and to the Epipe service. The configuration of the local and remote AC names and Ethernet tags is identical on PE-4 and PE-5.
On PE-4, the service configuration is as follows:
# on PE-4:
configure {
service {
system {
bgp {
evpn {
ethernet-segment "SA-ESI-45" {
admin-state enable
esi 01:00:00:00:00:45:00:00:00:01
multi-homing-mode single-active
df-election {
es-activation-timer 3
}
association {
sdp 46 {
}
}
}
}
}
}
epipe "Epipe-2" {
admin-state enable
service-id 2
customer "1"
bgp 1 {
}
spoke-sdp 46:2 {
}
bgp-evpn {
evi 2
local-attachment-circuit "AC-SA-ESI-45-MTU-6" {
eth-tag 456
}
remote-attachment-circuit "AC-AA-ESI-23-MTU-1" {
eth-tag 231
}
mpls 1 {
admin-state enable
ecmp 2
auto-bind-tunnel {
resolution any
}
}
}
}
sdp 46 {
admin-state enable
delivery-type mpls
ldp true
far-end {
ip-address 192.0.2.6
}
}
On PE-5, the configuration is similar, but with a different SDP:
# on PE-5:
configure {
service {
system {
bgp {
evpn {
ethernet-segment "SA-ESI-45" {
admin-state enable
esi 01:00:00:00:00:45:00:00:00:01
multi-homing-mode single-active
df-election {
es-activation-timer 3
}
association {
sdp 56 {
}
}
}
}
}
}
epipe "Epipe 2" {
admin-state enable
service-id 2
customer "1"
bgp 1 {
}
spoke-sdp 56:2 {
}
bgp-evpn {
evi 2
local-attachment-circuit "AC-SA-ESI-45-MTU-6" {
eth-tag 456
}
remote-attachment-circuit "AC-AA-ESI-23-MTU-1" {
eth-tag 231
}
mpls 1 {
admin-state enable
ecmp 2
auto-bind-tunnel {
resolution any
}
}
}
}
sdp 56 {
admin-state enable
delivery-type mpls
ldp true
far-end {
ip-address 192.0.2.6
}
}
Three route types will be exchanged between the core PEs: AD per-EVI, AD per-ES, and ES routes.
PE-4 and PE-5 advertise ES routes that are only imported by them. As an example, the following is the ES route with originator PE-4 sent by RR PE-2 to PE-5. It contains a target 00:00:00:00:45:00 in the extended community that is derived from the ESI:
# on PE-2:
56 2022/11/30 11:45:03.406 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 85
Flag: 0x90 Type: 14 Len: 34 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-ETH-SEG Len: 23 RD: 192.0.2.4:0 ESI: 01:00:00:00:00:45:00:00:00:01, IP-Len: 4 Orig-IP-Addr: 192.0.2.4
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
df-election::DF-Type:Auto/DP:0/DF-Preference:0/AC:1
target:00:00:00:00:45:00
"The AD per-ES route has a maximum Ethernet tag (MAX-ET) and an ESI label in the extended community. The multi-homing mode is single-active. As in the case of all-active multi-homing, the ESI label is not used in Epipe services. The following BGP update with originator PE-4 is sent by RR PE-2 to its client PE-5:
# on PE-2:
36 2022/11/30 11:44:47.394 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-AD Len: 25 RD: 192.0.2.4:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
esi-label:524279/Single-Active
bgp-tunnel-encap:MPLS
"The AD per-EVI route contains flags for primary and backup, which will be different for routes received from PE-4 and PE-5. In this case, PE-4 is primary in the single-active multi-homing ES (P=1):
# on PE-2:
64 2022/11/30 11:45:06.415 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-AD Len: 25 RD: 192.0.2.4:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: 456 Label: 8388480 (Raw Label: 0x7fff80) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 1 B: 0
bgp-tunnel-encap:MPLS
"PE-5 is backup in the single-active multi-homing ES (B=1):
# on PE-2:
72 2022/11/30 11:45:10.872 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.5
Type: EVPN-AD Len: 25 RD: 192.0.2.5:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: 456 Label: 8388512 (Raw Label: 0x7fffa0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 0 B: 1
bgp-tunnel-encap:MPLS
"The BGP EVPN AD routes can be shown with the following command:
[/]
A:admin@PE-2# show router bgp routes evpn auto-disc esi 01:00:00:00:00:45:00:00:00:01
===============================================================================
BGP Router ID:192.0.2.2 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Auto-Disc Routes
===============================================================================
Flag Route Dist. ESI NextHop
Tag Label
-------------------------------------------------------------------------------
u*>i 192.0.2.4:2 01:00:00:00:00:45:00:00:00:01 192.0.2.4
456 LABEL 524280
u*>i 192.0.2.4:2 01:00:00:00:00:45:00:00:00:01 192.0.2.4
MAX-ET LABEL 0
u*>i 192.0.2.5:2 01:00:00:00:00:45:00:00:00:01 192.0.2.5
456 LABEL 524282
u*>i 192.0.2.5:2 01:00:00:00:00:45:00:00:00:01 192.0.2.5
MAX-ET LABEL 0
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
For each PE in the single-active ES, there are two AD routes: the routes with MAX-ET are AD per-ES routes and the routes with a configured Ethernet tag are AD per-EVI routes.
The EVPN MPLS destination for Epipe 2 on PE-2 is SA-ESI-45, as shown in the following output:
[/]
A:admin@PE-2# show service id 2 evpn-mpls
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-id
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:45:00:00:00:01 11/30/2022 11:45:06
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
The ESI is resolved to the TEP address of the primary (DF) PE-4, as follows:
[/]
A:admin@PE-2# show service id 2 evpn-mpls esi esi-1 01:00:00:00:00:45:00:00:00:01
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:45:00:00:00:01 11/30/2022 11:45:06
===============================================================================
===============================================================================
BGP EVPN-MPLS Dest TEP Info
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-Id
-------------------------------------------------------------------------------
192.0.2.4 524280 11/30/2022 11:45:06
ldp:65538
-------------------------------------------------------------------------------
Number of entries : 1
-------------------------------------------------------------------------------
===============================================================================
The DF election is key for the forwarding and backup functions in single-active multi-homing ESs. The PE elected as DF will be the primary for the ES in the Epipe and will unblock the SAP/spoke-SDP for upstream and downstream traffic. The rest of the PEs in the ES will bring their ES SAPs or spoke-SDPs operationally down.
PE-5 is a non-DF, as follows:
[/]
A:admin@PE-5# show service system bgp-evpn ethernet-segment name "SA-ESI-45" evi evi-1 2
===============================================================================
EVI DF and Candidate List
===============================================================================
EVI SvcId Actv Timer Rem DF DF Last Change
-------------------------------------------------------------------------------
2 2 0 no 11/30/2022 11:44:55
===============================================================================
===============================================================================
DF Candidates Time Added Oper Pref Do Not
Value Preempt
-------------------------------------------------------------------------------
192.0.2.4 11/30/2022 11:45:03 0 Disabl*
192.0.2.5 11/30/2022 11:45:08 0 Disabl*
-------------------------------------------------------------------------------
Number of entries: 2
===============================================================================
* indicates that the corresponding row element may have been truncated.
In single-active multi-homing, the service spoke-SDP (or SAP) is brought operationally down on the non-DF, as shown in the following output:
[/]
A:admin@PE-5# show service id 2 sdp
===============================================================================
Services: Service Destination Points
===============================================================================
SdpId Type Far End addr Adm Opr I.Lbl E.Lbl
-------------------------------------------------------------------------------
56:2 Spok 192.0.2.6 Up Down 524280 524280
-------------------------------------------------------------------------------
Number of SDPs : 1
-------------------------------------------------------------------------------
===============================================================================
The spoke-SDP 56:2 is operationally down with a StandbyForMHProtocol flag:
[/]
A:admin@PE-5# show service id 2 sdp 56:2 detail | match Flag
Flags : StandbyForMHProtocol
Two consecutive DF elections take place: the first DF election includes all PEs in the ES for that Epipe and determines which PE is the primary PE (flags P=1, B=0). The second DF election excludes this DF and determines which PE is the backup (P=0, B=1). All other PEs signal flags P=0 and B=0.
When the primary PE fails, AD per-ES/EVI withdrawal messages are sent to the remote PE, which will update its next hop to the backup. The backup PE takes over immediately without waiting for the es-activation-timer to bring up its SAP/spoke-SDP.
Ethernet segment failures
When the SDP toward the primary (DF) fails, the backup PE needs to take over. An SDP failure is emulated and log 99 on PE-4 shows that SDP 46 is operational down and PE-4 is no longer the DF:
140 2022/11/30 12:09:36.765 CET MINOR: SVCMGR #2094 Base
"Ethernet Segment:SA-ESI-45, EVI:2, Designated Forwarding state changed to:false"
139 2022/11/30 12:09:36.764 CET MINOR: SVCMGR #2326 Base
"Status of SDP Bind 46:2 in service 2 (customer 1) local PW status bits changed to psnIngressFault psnEgressFault "
138 2022/11/30 12:09:36.764 CET MINOR: SVCMGR #2303 Base
"Status of SDP 46 changed to admin=up oper=down"
Remote PEs receive route withdrawal updates (unreachable NLRI) from former DF PE-4, for example on RR PE-2:
# on PE-2:
76 2022/11/30 12:09:36.765 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 34
Flag: 0x90 Type: 15 Len: 30 Multiprotocol Unreachable NLRI:
Address Family EVPN
Type: EVPN-AD Len: 25 RD: 192.0.2.4:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
"
75 2022/11/30 12:09:36.765 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 59
Flag: 0x90 Type: 15 Len: 55 Multiprotocol Unreachable NLRI:
Address Family EVPN
Type: EVPN-AD Len: 25 RD: 192.0.2.4:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: 456 Label: 0 (Raw Label: 0x0) PathId:
Type: EVPN-ETH-SEG Len: 23 RD: 192.0.2.4:0 ESI: 01:00:00:00:00:45:00:00:00:01, IP-Len: 4 Orig-IP-Addr: 192.0.2.4
"
The backup PE-5 is promoted to primary (P=1, B=0) and sends BGP updates accordingly. The following AD per-EVI is received on PE-2:
# on PE-2:
79 2022/11/30 12:09:36.768 CET MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.5
Type: EVPN-AD Len: 25 RD: 192.0.2.5:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: 456 Label: 8388512 (Raw Label: 0x7fffa0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 1 B: 0
bgp-tunnel-encap:MPLS
"PE-5 brings up its spoke-SDP without waiting for the es-activation-timer and takes over immediately. It is now the only DF candidate, and therefore the DF, as follows:
[/]
A:admin@PE-5# show service system bgp-evpn ethernet-segment name "SA-ESI-45" evi evi-1 2
===============================================================================
EVI DF and Candidate List
===============================================================================
EVI SvcId Actv Timer Rem DF DF Last Change
-------------------------------------------------------------------------------
2 2 0 yes 11/30/2022 11:44:55
===============================================================================
===============================================================================
DF Candidates Time Added Oper Pref Do Not
Value Preempt
-------------------------------------------------------------------------------
192.0.2.5 11/30/2022 11:45:08 0 Disabl*
-------------------------------------------------------------------------------
Number of entries: 1
===============================================================================
* indicates that the corresponding row element may have been truncated.
BGP updates are exchanged and the remote PEs will resolve the ESI to the TEP address 192.0.2.5. For example, on PE-2:
[/]
A:admin@PE-2# show service id 2 evpn-mpls esi esi-1 01:00:00:00:00:45:00:00:00:01
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:45:00:00:00:01 11/30/2022 12:09:37
===============================================================================
===============================================================================
BGP EVPN-MPLS Dest TEP Info
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-Id
-------------------------------------------------------------------------------
192.0.2.5 524282 11/30/2022 12:09:37
ldp:65539
-------------------------------------------------------------------------------
Number of entries : 1
-------------------------------------------------------------------------------
===============================================================================
This process is revertive; as soon as the SDP 46 is operationally up again, a new DF election is triggered with two DF candidates and PE-4 will be elected as DF.
Troubleshooting and debugging
The following show and debug commands can be used in EVPN-VPWS:
-
show redundancy bgp-evpn-multi-homing
-
show router bgp routes evpn (and filters)
-
show service evpn-mpls [<TEP ip-address>]
-
show service id bgp-evpn
-
show service id evpn-mpls (and modifiers)
-
show service system bgp-evpn
-
show service system bgp-evpn ethernet-segment (and modifiers)
-
debug router bgp update
-
show log log-id 99
Most of these commands have been shown in the preceding sections; some commands are shown in this section.
Information about the configured boot timers (before DF election) and ES activation timer (after the system has been elected DF) can be shown as follows:
[/]
A:admin@PE-2# show redundancy bgp-evpn-multi-homing
===============================================================================
Redundancy BGP EVPN Multi-homing Information
===============================================================================
Boot-Timer : 10 secs
Boot-Timer Remaining : 0 secs
ES Activation Timer : 3 secs
===============================================================================
See chapter EVPN for MPLS Tunnels for a description of these timers.
The following command shows that the BGP route-type 4 (ES route) messages are only imported by the PEs in the same ES; for example, on PE-3:
[/]
A:admin@PE-3# show router bgp routes evpn eth-seg
===============================================================================
BGP Router ID:192.0.2.3 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Eth-Seg Routes
===============================================================================
Flag Route Dist. ESI NextHop
OrigAddr
-------------------------------------------------------------------------------
u*>i 192.0.2.2:0 01:00:00:00:00:23:00:00:00:01 192.0.2.2
192.0.2.2
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
On PE-4:
[/]
A:admin@PE-4# show router bgp routes evpn eth-seg
===============================================================================
BGP Router ID:192.0.2.4 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Eth-Seg Routes
===============================================================================
Flag Route Dist. ESI NextHop
OrigAddr
-------------------------------------------------------------------------------
u*>i 192.0.2.5:0 01:00:00:00:00:45:00:00:00:01 192.0.2.5
192.0.2.5
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The following command shows all the EVPN MPLS destinations toward TEP 192.0.2.4. Epipe 1 has an EVPN MPLS destination toward TEP 192.0.2.4 directly and Epipe 2 has an EVPN MPLS destination to SA-ESI-45, which can be resolved to TEP 192.0.2.4. This is shown in the following output:
[/]
A:admin@PE-2# show service evpn-mpls 192.0.2.4
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
Service Id Egr Label Instance
-------------------------------------------------------------------------------
1 524282 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Service Id Eth Seg Id Egr Label
-------------------------------------------------------------------------------
2 01:00:00:00:00:45:00:00:00:01 524280
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-MPLS ES BMac Dest
===============================================================================
Service Id ES BMac Egr Label
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
The following command lists all configured ESs on the system:
[/]
A:admin@PE-2# show service system bgp-evpn ethernet-segment
===============================================================================
Service Ethernet Segment
===============================================================================
Name ESI Admin Oper
-------------------------------------------------------------------------------
AA-ESI-23 01:00:00:00:00:23:00:00:00:01 Enabled Up
-------------------------------------------------------------------------------
Entries found: 1
===============================================================================
In addition to the preceding commands, the following tools dump commands may be useful:
-
tools dump service evpn usage – This command shows the number of EVPN-MPLS (and EVPN-VXLAN) destinations in the system.
-
tools dump service system bgp-evpn ethernet-segment <name> evi <..> df – This command computes the DF election for a specific ESI and EVI. For all-active, there is no DF election and all PEs forward traffic. For single-active, one PE will be active for a service while another PE will be backup. This command shows the DF (primary), even if it is not the local PE.
The usage of EVPN resources can be shown as follows:
[/]
A:admin@PE-2# tools dump service evpn usage
vxlan-srv6-evpn-mpls usage statistics at 11/30/2022 12:15:35:
MPLS-TEP : 1
VXLAN-TEP : 0
SRV6-TEP : 0
Total-TEP : 1/ 16383
Mpls Dests (TEP, Egress Label + ES + ES-BMAC) : 2
Mpls Etree Leaf Dests : 0
Vxlan Dests (TEP, Egress VNI + ES) : 0
Srv6 Dests (TEP, SID + ES) : 0
Total-Dest : 2/196607
Sdp Bind + Evpn Dests : 2/245759
ES L2/L3 PBR : 0/ 32767
Evpn Etree Remote BUM Leaf Labels : 0
On PE-2, there is one MPLS-TEP (192.0.2.4 in Epipe 1 and Epipe 2) and there are two MPLS destinations: 192.0.2.4 and ESI 01:00:00:00:00:45:00:00:00:01. PE-5 is not an MPLS-TEP for PE-2, because it is not a primary and, therefore, not forwarding any traffic.
In all-active multi-homing, the DF election is not applicable:
[/]
A:admin@PE-2# tools dump service system bgp-evpn ethernet-segment "AA-ESI-23" evi 2 df
[11/30/2022 12:15:50] All Active VPWS or IP-ALIASING - DF N/A
In single-active multi-homing, the following command shows which PE is the DF:
[/]
A:admin@PE-5# tools dump service system bgp-evpn ethernet-segment "SA-ESI-45" evi 2 df
[11/30/2022 12:16:04] Computed DF: 192.0.2.4 (Remote) (Boot Timer Expired: Yes)
[11/30/2022 12:16:04] Computed Backup: 192.0.2.5 (This Node)
The command is launched on PE-5, which is a backup. The computed DF is PE-4 and the boot timer has expired, meaning there is no DF re-election pending.
Conclusion
EVPN-VPWS is a simplified point-to-point version of RFC 7432 - BGP MPLS-Based Ethernet VPN. When used for Epipe and VPLS services, EVPN provides a unified control plane mechanism that simplifies the network deployment and operation. Single-active and all-active multi-homing can be used in Epipes; EVPN-VPWS is a differentiator of EVPN compared to traditional TLDP or BGP Epipe redundancy mechanisms. The Ethernet Segments used for multi-homing can be shared between EVPN VPLS and EVPN Epipes.