EVPN for MPLS Tunnels in Epipe Services (EVPN-VPWS)
This chapter provides information about EVPN for MPLS tunnels in Epipe services (EVPN-VPWS).
Topics in this chapter include:
Applicability
This chapter was initially written for SR OS Release 14.0.R4, but the CLI in the current edition is based on SR OS Release 22.10.R1. Ethernet Virtual Private Network - Virtual Private Wire Service (EVPN-VPWS) is supported in SR OS Release 14.0.R1 and later. EVPN-VPWS in multi-homing scenarios is supported in SR OS Release 14.0.R4 and later.
Chapter EVPN for MPLS Tunnels is prerequisite reading.
Overview
Service providers prefer an optimized, standardized, and unified control plane for VPNs. EVPN-VPWS is supported in MPLS networks that also run EVPN-MPLS in VPLS services. From a control plane perspective, EVPN-VPWS is a simplified point-to-point version of RFC 7432, BGP MPLS-Based Ethernet VPN, because there is no need to advertise MAC routes in VPWS. EVPN-VPWS is described in RFC 8214, Virtual Private Wire Service Support in Ethernet VPN.
EVPN-VPWS supports all-active multi-homing (per-flow load-balancing multi-homing) as well as single-active multi-homing (per-service load-balancing multi-homing), using the same Ethernet segments (ESs) used for EVPN-MPLS VPLS services. EVPN-VPWS uses route-type 1 and route-type 4; it does not use route-types 2, 3, or 5, because MAC/IP routes, inclusive multicast, or IP-prefix routes are not required.
Route types and NLRIs for EVPN-VPWS shows the encoding of the required extensions for the route-types 1 and 4 for EVPN-VPWS.
Two sub-types are defined for route-type 1. Route-type 4 has no sub-types. The route types used for EVPN-VPWS have the following purposes:
Route-type 1 - Auto-discovery per EVPN instance (AD per-EVI). This route type is used in all EVPN-VPWS scenarios, with or without multi-homing. For EVPN-VPWS, the Ethernet tag field is encoded with the local Attachment Circuit (AC) of the advertising PE. This value is configured using the service epipe bgp-evpn local-attachment-circuit eth-tag <value> command. The route distinguisher (RD), MPLS label, and the Ethernet segment ID (ESI) are encoded as for EVPN-MPLS. The MPLS label field is used as service label. In case of multi-homing, AD per-EVI routes containing the same ESI are used to provide aliasing and a backup path to the PEs part of the ES. The L2 MTU is encoded with the service MTU configured in the Epipe. The following flags are used for EVPN-VPWS:
Flag C is set if a control word is configured in the service.
Flag P is set if the advertising PE is primary PE.
If no multi-homing is used, there is no primary PE (P=0).
In all-active multi-homing, all PEs in the ES are primary (P=1).
In single-active multi-homing, only one PE per-EVI in the ES is primary (P=1).
Flag B is set if the advertising PE is backup PE.
The B-flag is only set in case of single-active multi-homing and only for one PE, even if more than two PEs are present in the same single-active ES. The backup PE is the winner of the second Designated Forwarder (DF) election (excluding the DF). The remaining non-DF PEs send B=0.
If there is no multi-homing, the ESI, flag P, and flag B will be zero.
Route-type 1 - AD per Ethernet segment (AD per-ES). Same encoding as for EVPN-MPLS. AD per-ES is only used in multi-homing scenarios where it is advertised per ES from the PE. It carries the ESI label (used for split-horizon, but only for VPLS services and not for Epipe services) and can affect procedures such as the DF election, as well as the aliasing on remote PEs.
Route-type 4 - ES route. Same encoding as for EVPN-MPLS. Route-type 4 is only used in multi-homing scenarios. This route advertises a local configured ES. The exchange of this route can discover remote PEs that are part of the same ES and the DF election algorithm among them.
Configuration
EVPN-VPWS example topology shows the example topology that will be used throughout this chapter.
The example topology consists of six SR OS nodes with the following initial configuration:
Network (or hybrid) ports interconnect the core PEs with configured router interfaces.
MTU-1 is a pure Ethernet aggregator. The ports toward the core PEs are access ports. Likewise, the ports on PE-2 and PE-3 toward MTU-1 are access ports.
Core PEs and MTU-6 run IS-IS on all router interfaces. Point-to-point adjacencies are established for the exchange of system IP addresses.
Link LDP is configured between all PEs, and toward/from MTU-6.
EVPN uses BGP for exchanging reachability at service level. Therefore, BGP peering sessions must be established among the core PEs for the EVPN family. Although typically a separate router is used, in this chapter, PE-2 is used as route reflector with the following BGP configuration:
# on PE-2: configure router Base autonomous-system 64500 bgp vpn-apply-import vpn-apply-export enable-peer-tracking rapid-withdrawal split-horizon rapid-update evpn group "internal" family evpn cluster 192.0.2.2 peer-as 64500 neighbor 192.0.2.3 exit neighbor 192.0.2.4 exit neighbor 192.0.2.5 exit exit exitThe BGP configuration on the other PEs is as follows:
# on PE-3, PE-4, PE-5: configure router autonomous-system 64500 bgp vpn-apply-import vpn-apply-export enable-peer-tracking rapid-withdrawal split-horizon rapid-update evpn group "internal" family evpn peer-as 64500 neighbor 192.0.2.2 exit exit exit
The following EVPN-VPWS scenarios are described in the following sections:
EVPN for MPLS tunnels in Epipe services without multi-homing
BGP-EVPN can be enabled in Epipe services with either SAPs or spoke-SDPs at the access, as shown in Example topology for EVPN-VPWS without multi-homing.
On PE-2, Epipe 1 is configured as follows:
# on PE-2:
configure
service
epipe 1 name "Epipe-1" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit AC-PE-2-MTU-1 create
eth-tag 21
exit
remote-attachment-circuit AC-PE-4-MTU-6 create
eth-tag 46
exit
evi 1
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap 1/1/c11/1:1 create
no shutdown
exit
no shutdown
On PE-4, the service configuration is as follows:
# on PE-4:
configure
service
sdp 460 create
far-end 192.0.2.6
keep-alive
shutdown
exit
no shutdown
exit
epipe 1 name "Epipe-1" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit AC-PE-4-MTU-6 create
eth-tag 46
exit
remote-attachment-circuit AC-PE-2-MTU-1 create
eth-tag 21
exit
evi 1
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
spoke-sdp 460:1 create
no shutdown
exit
no shutdown
Where the following commands are relevant for the EVPN-VPWS configuration:
-
bgp enables the context for the BGP configuration relevant to the service. The bgp context configures the common BGP parameters for all BGP families in the service, such as route distinguisher and route target. Even if the general BGP parameters for the service are auto-derived, the bgp context must be enabled.
*A:PE-2>config>service>epipe# bgp ? - bgp - no bgp [no] adv-service-mtu - Configure service-mtu to be advertised [no] pw-template-bi* + Configure pw-template bind policy [no] route-distingu* - Configure route distinguisher [no] route-target - Configure route target [no] vsi-export - VSI export route policies [no] vsi-import - VSI import route policies The following parameters can be configured in the bgp-evpn context:
*A:PE-2>config>service>epipe# bgp-evpn ? - bgp-evpn - no bgp-evpn [no] evi - EVPN Identifier [no] local-attachme* + Configure local attachment circuit information [no] mpls + Configure BGP EVPN mpls [no] remote-attachm* + Configure remote attachment circuit information [no] segment-routin* + Configure SRv6 instance [no] vxlan + Configure BGP EVPN vxlanThe evi is a two-byte or three-byte identifier used for auto-deriving the service RD (only for two-byte EVI), service RT, and for the DF election in multi-homing. The auto-derivation of RD and RT for a two-byte EVI is as follows:
RD <system IP address>:<evi>
RT <autonomous system number>:<evi>
The EVI values must be unique in the system, regardless of the type of service they are assigned to (Epipe or VPLS).Note: Three-byte EVI values are supported in SR OS Release 21.10.R1 and later. For auto-derived RT as per RFC 8365, the evi-three-byte-auto-rt command must be configured, as described in the Three-byte EVI in EVPN Services chapter.The local-attachment-circuit and remote-attachment-circuit identify the two attachment circuits connected by the EVPN-VPWS service. The configured Ethernet tag for the local AC is advertised in the Ethernet tag field of the AD per-EVI route for the Epipe, along with the corresponding RD, RT, and MPLS label. Both local and remote Ethernet tags are mandatory to bring up the Epipe service. If the received Ethernet tag for the Epipe service matches the configured remote AC Ethernet tag, it will create an EVPN-MPLS destination to the next hop.
The local Ethernet tag cannot be modified without disabling bgp-evpn mpls in the Epipe, as shown in the following output:
*A:PE-2>config>service>epipe>bgp-evpn>local-att-cir# eth-tag 221 MINOR: SVCMGR #8036 evpn-vpws ac eth-tag not allowed - cannot change while evpn mpls/vxlan/srv6 is enabledUnlike local Ethernet tags, remote Ethernet tags can be modified without disabling bgp-evpn.
The following configuration options are available for Epipes in the bgp-evpn>mpls context:
*A:PE-2>config>service>epipe>bgp-evpn# mpls ? - mpls [bgp <bgp>] - no mpls [bgp <bgp>] <bgp> : [1..1] auto-bind-tunn* + Configure BGP EVPN mpls auto-bind-tunnel [no] control-word - Enable/disable setting the CW bit in the label message [no] default-route-* - Configure default-route-tag to match against export policies [no] dynamic-egress* - Enable/disable Dynamic Egress Label Limit ecmp - Configure maximum ECMP routes information [no] entropy-label - Enable/disable use of entropy-label [no] evi-three-byte* - Enable/Disable evi-three-byte-auto-rt [no] force-qinq-vc-* - Forces qinq-vc-type forwarding in the data-path [no] force-vlan-vc-* - Forces vlan-vc-type forwarding in the data-path [no] oper-group - Configure oper-group route-next-hop - Configure route next-hop [no] send-tunnel-en* - Configure encapsulation for this service [no] shutdown - Administratively Enable/Disable BGP-EVPN mplsThis is a subset of the options for VPLS services; see chapter EVPN for MPLS Tunnels.
When the local AC (SAP 1/1/c11/1:1) is up, PE-2 sends a BGP EVPN AD per-EVI route that contains Ethernet tag 21 for the local AC:
# on PE-2:
2 2022/11/29 09:33:44.668 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:1 ESI: ESI-0, tag: 21 Label: 8388512 (Raw Label: 0x7fffa0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:1
l2-attribute:MTU: 1514 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
"The auto-derived RD for EVI 1 is 192.0.2.2:1 and the RT is 64500:1.
When the remote AC on PE-4 (spoke-SDP 460:1) is up, PE-2 receives the following EVPN-AD per-EVI route with Ethernet tag 46 from PE-4:
# on PE-2:
4 2022/11/29 09:33:54.253 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-AD Len: 25 RD: 192.0.2.4:1 ESI: ESI-0, tag: 46 Label: 8388512 (Raw Label: 0x7fffa0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:1
l2-attribute:MTU: 1514 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
"When the received RT 64500:1 matches and the received Ethernet tag 46 matches the configured remote AC on PE-2, the following EVPN-MPLS destination (comprised of a termination endpoint (TEP) 192.0.2.4 and egress label 524282) is created on PE-2. In a similar way, an EVPN-MPLS destination is created on PE-4.
*A:PE-2# show service id 1 evpn-mpls
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-id
-------------------------------------------------------------------------------
192.0.2.4 524282 11/29/2022 09:33:54
ldp:65538
-------------------------------------------------------------------------------
Number of entries : 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
The MPLS label in the debug message is not the same as in the service, because the router will strip the extra four lowest bits to get the 20-bit MPLS label. The egress label for the EVPN-MPLS destination on PE-4 is 524282. The 24-bit label value in the BGP update debug is 16 (2^4) times as high: 524282*16 = 8388512. This is because the debug message is shown before the router can parse the label field and see if it corresponds to a 20-bit MPLS label or a 24-bit VXLAN VNI.
The BGP AD per-EVI routes for Ethernet tag 46 can be shown with the following command:
*A:PE-2# show router bgp routes evpn auto-disc tag 46
===============================================================================
BGP Router ID:192.0.2.2 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Auto-Disc Routes
===============================================================================
Flag Route Dist. ESI NextHop
Tag Label
-------------------------------------------------------------------------------
u*>i 192.0.2.4:1 ESI-0 192.0.2.4
46 LABEL 524282
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The following command shows the BGP EVPN information for Epipe 1:
*A:PE-2# show service id 1 bgp-evpn
===============================================================================
BGP EVPN Table
===============================================================================
EVI : 1 Creation Origin : manual
-------------------------------------------------------------------------------
Local AC Name Eth Tag Endpoint Ingress Label
-------------------------------------------------------------------------------
AC-PE-2-MTU-1 21 0
-------------------------------------------------------------------------------
Number of local ACs : 1
-------------------------------------------------------------------------------
Remote AC Name Eth Tag Endpoint
-------------------------------------------------------------------------------
AC-PE-4-MTU-6 46
-------------------------------------------------------------------------------
Number of Remote ACs : 1
===============================================================================
===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status : Enabled Bgp Instance : 1
Force Vlan Fwding : Disabled
Force Qinq Fwding : none
Route NextHop Type : system-ipv4
Control Word : Disabled
Max Ecmp Routes : 1
Entropy Label : Disabled
Default Route Tag : none
Oper Group :
Evi 3-byte Auto-RT : Disabled
Dyn Egr Lbl Limit : Disabled
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN MPLS Auto Bind Tunnel Information
===============================================================================
Allow-Flex-Algo-Fallback : false
Resolution : any Strict Tnl Tag : false
Max Ecmp Routes : 1
Bgp Instance : 1
Filter Tunnel Types : (Not Specified)
Weighted Ecmp : false
-------------------------------------------------------------------------------
===============================================================================
EVPN for MPLS tunnels in Epipe services with multi-homing
SR OS supports EVPN multi-homing as per RFC 8214.
The EVPN multi-homing implementation is based on the concept of the Ethernet segment (ES). An ES is a logical structure that can be defined in one or more PEs and identifies the CE (or access network) multi-homed to the EVPN PEs. An ES is associated with a port, LAG, or SDP object, and is shared by all the services defined on those objects. It can also be shared between Epipe and VPLS services.
Each ES has a unique Ethernet segment Identifier (ESI) that is 10 bytes and is manually configured.
The ESI is advertised in the control plane to all the PEs in an EVPN network; therefore, it is very important to ensure that the 10-byte ESI value is unique throughout the entire network. Single-homed CEs are assumed to be connected to an ES with ESI = 0 (single-homed ESs are not explicitly configured).
The ES is part of the base BGP-EVPN configuration and is not applied to any EVPN-MPLS service, by default. An ES can be shared by multiple services; the association of a specific SAP or spoke-SDP to an ES is automatically made when the SAP is defined in the same LAG or port configured in the ES, or when the spoke-SDP is defined in the same SDP configured in the ES.
Regardless of the multi-homing mode, the local Ethernet tag values must match on all the PEs that are part of the same ES. The PEs in the ES will use the AD per-EVI routes from the peer PEs to validate the PEs as DF election candidates for an EVI. The DF election is only relevant for single-active multi-homing ESs. For Epipes defined in an all-active multi-homing ES, there is no DF election required, because all PEs are forwarding traffic and all traffic is treated as unicast.
Aliasing is supported when sending traffic to an ES destination. Assuming ECMP is enabled on the ingress PE (and shared queuing or ingress policing), per-flow load-balancing will be performed among all the PEs that advertised P=1. PEs advertising P=0 are not considered as next hops for an ES destination.
The following sections show the configuration of:
an all-active multi-homing ES with a LAG associated with it
a single-active multi-homing ES linked to an SDP
Example topology EVPN-VPWS with multi-homing shows an all-active ES and a single-active ES. The all-active multi-homing ES "AA-ESI-23" in PE-2 and PE-3 has a LAG associated to it; the single-active multi-homing ES "SA-ESI-45" in PE-4 and PE-5 has an SDP associated to it.
EVPN for MPLS tunnels in Epipe services with all-active multi-homing
All-active multi-homing allows for per-flow load-balancing. Unlike EVPN-MPLS in VPLS services, EVPN-VPWS has no DF election in all-active multi-homing. All PEs in the ES are active and the remote PE will do per-flow load-balancing. AA-ESI-23 is configured on PE-2 and PE-3 in all-active multi-homing with LAG 1 associated to it. This LAG is used as a SAP in Epipe 2 on both PE-2 and PE-3. The configuration of the ES and Epipe 2 is identical on PE-2 and PE-3, including the local AC and remote AC names and Ethernet tags:
# on PE-2, PE-3:
configure
service
system
bgp-evpn
ethernet-segment "AA-ESI-23" create
esi 01:00:00:00:00:23:00:00:00:01
es-activation-timer 3
service-carving
mode auto
exit
multi-homing all-active
lag 1
no shutdown
exit
exit
exit
epipe 2 name "Epipe 2" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit AC-AA-ESI-23-MTU-1 create
eth-tag 231
exit
remote-attachment-circuit AC-SA-ESI-45-MTU-6 create
eth-tag 456
exit
evi 2
mpls bgp 1
ecmp 2
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap lag-1:2 create
no shutdown
exit
no shutdown
See chapter EVPN for MPLS Tunnels for a detailed explanation of the configuration parameters of the ES.
In EVPN-VPWS multi-homing scenarios, three route types are exchanged: AD per-EVI, AD per-ES, and ES routes. The following ES route (route-type 4) for ESI 01:00:00:00:00:23:00:00:00:01 sent by PE-2 is imported at PE-3:
# on PE-3:
5 2022/11/29 09:42:41.537 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 71
Flag: 0x90 Type: 14 Len: 34 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-ETH-SEG Len: 23 RD: 192.0.2.2:0 ESI: 01:00:00:00:00:23:00:00:00:01, IP-Len: 4 Orig-IP-Addr: 192.0.2.2
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
df-election::DF-Type:Auto/DP:0/DF-Preference:0/AC:1
target:00:00:00:00:23:00
"The target 00:00:00:00:23:00 in the extended community is derived from the ESI (bytes 2 to 7) and is only imported by the PEs that are part of the same ES; that is, PE-2 and PE-3 in this example.
At the same time, the following AD per-ES route (route-type 1) with maximum Ethernet tag (MAX-ET, all Fs) and label 0 is sent by route reflector (RR) PE-2 and imported by the rest of the PEs. The following two BGP updates with MAX-ET are received by PE-4:
# PE-4 receives EVPN AD per-ES (MAX-ET) from PE-2:
3 2022/11/29 09:42:41.491 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:2 ESI: 01:00:00:00:00:23:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
esi-label:524281/All-Active
bgp-tunnel-encap:MPLS
"# PE-4 receives EVPN AD per-ES (MAX-ET)(originator PE-3):
6 2022/11/29 09:42:43.033 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-AD Len: 25 RD: 192.0.2.3:2 ESI: 01:00:00:00:00:23:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.3
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
esi-label:524282/All-Active
bgp-tunnel-encap:MPLS
"The ESI label is in the extended community, as well as the indication that the multi-homing is all-active. Epipe services do not require ESI labels because BUM traffic is not recognized as such in EVPN-VPWS services. However, because the ES can be shared by Epipe and VPLS services, the AD per-ES route still includes a non-zero ESI label.
The following AD per-EVI routes (route-type 1) with Ethernet tag 231 sent by RR PE-2 are received and imported on PE-4:
# PE-4 receives EVPN AD per-ES with Ethernet tag 231 (originator PE-2):
4 2022/11/29 09:42:41.494 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:2 ESI: 01:00:00:00:00:23:00:00:00:01, tag: 231 Label: 8388480 (Raw Label: 0x7fff80) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 1 B: 0
bgp-tunnel-encap:MPLS
"## PE-4 receives EVPN AD per-ES with Ethernet tag 231 (originator PE-3):
7 2022/11/29 09:42:43.047 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-AD Len: 25 RD: 192.0.2.3:2 ESI: 01:00:00:00:00:23:00:00:00:01, tag: 231 Label: 8388496 (Raw Label: 0x7fff90) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.3
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 1 B: 0
bgp-tunnel-encap:MPLS
"
This route contains the flags for control word (C), primary (P), and backup (B). In all-active multi-homing, all nodes are primary (P=1).
PE-4 has learned AD per-EVI/ES routes for AA-ESI-23 from PE-2 and PE-3, as shown in the following output:
*A:PE-4# show router bgp routes evpn auto-disc esi 01:00:00:00:00:23:00:00:00:01
===============================================================================
BGP Router ID:192.0.2.4 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Auto-Disc Routes
===============================================================================
Flag Route Dist. ESI NextHop
Tag Label
-------------------------------------------------------------------------------
u*>i 192.0.2.2:2 01:00:00:00:00:23:00:00:00:01 192.0.2.2
231 LABEL 524280
u*>i 192.0.2.2:2 01:00:00:00:00:23:00:00:00:01 192.0.2.2
MAX-ET LABEL 0
u*>i 192.0.2.3:2 01:00:00:00:00:23:00:00:00:01 192.0.2.3
231 LABEL 524281
u*>i 192.0.2.3:2 01:00:00:00:00:23:00:00:00:01 192.0.2.3
MAX-ET LABEL 0
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
For Epipe 2 on PE-4, the EVPN MPLS destination is not pointing at a specific TEP, but AA-ESI-23, as shown in the following output:
*A:PE-4# show service id 2 evpn-mpls
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-id
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:23:00:00:00:01 11/29/2022 09:43:02
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
When ECMP > 1 on the ingress PE, multiple TEPs can correspond to a specific ESI (aliasing). In this case, ECMP=2 and PE-4 and PE-5 have two TEP addresses and egress labels for ESI 01:00:00:00:00:23:00:00:00:01, as shown for PE-4:
*A:PE-4# show service id 2 evpn-mpls esi 01:00:00:00:00:23:00:00:00:01
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:23:00:00:00:01 11/29/2022 09:43:02
===============================================================================
===============================================================================
BGP EVPN-MPLS Dest TEP Info
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-Id
-------------------------------------------------------------------------------
192.0.2.2 524280 11/29/2022 09:43:02
ldp:65537
192.0.2.3 524281 11/29/2022 09:43:02
ldp:65538
-------------------------------------------------------------------------------
Number of entries : 2
-------------------------------------------------------------------------------
===============================================================================
In all-active multi-homing for EVPN-VPWS, there is no DF election and all PEs in the ES are active. For AA-ESI-23, both PE-2 and PE-3 are active/primary/DF, but there are no DF candidates, because there is no DF election:
*A:PE-2# show service system bgp-evpn ethernet-segment name "AA-ESI-23" evi 2
===============================================================================
EVI DF and Candidate List
===============================================================================
EVI SvcId Actv Timer Rem DF DF Last Change
-------------------------------------------------------------------------------
2 2 0 yes 11/29/2022 09:42:41
===============================================================================
===============================================================================
DF Candidates Time Added Oper Pref Do Not
Value Preempt
-------------------------------------------------------------------------------
No entries found
===============================================================================
Similarly, on PE-3:
*A:PE-3# show service system bgp-evpn ethernet-segment name "AA-ESI-23" evi 2
===============================================================================
EVI DF and Candidate List
===============================================================================
EVI SvcId Actv Timer Rem DF DF Last Change
-------------------------------------------------------------------------------
2 2 0 yes 11/29/2022 09:42:43
===============================================================================
===============================================================================
DF Candidates Time Added Oper Pref Do Not
Value Preempt
-------------------------------------------------------------------------------
No entries found
===============================================================================
To confirm that all-active multi-homing is working correctly, the following command shows all information related to a specific ESI; in this case, AA-ESI-23 on PE-2:
*A:PE-2# show service system bgp-evpn ethernet-segment name "AA-ESI-23" all
===============================================================================
Service Ethernet Segment
===============================================================================
Name : AA-ESI-23
Eth Seg Type : None
Admin State : Enabled Oper State : Up
ESI : 01:00:00:00:00:23:00:00:00:01
Oper ESI : 01:00:00:00:00:23:00:00:00:01
Auto-ESI Type : None
AC DF Capability : Include
Multi-homing : allActive Oper Multi-homing : allActive
ES SHG Label : 524281
Source BMAC LSB : None
Lag Id : 1
ES Activation Timer : 3 secs
Oper Group : (Not Specified)
Svc Carving : auto Oper Svc Carving : auto
Cfg Range Type : primary
Vprn NextHop EVI Ranges : <none>
===============================================================================
===============================================================================
EVI Information
===============================================================================
EVI SvcId Actv Timer Rem DF
-------------------------------------------------------------------------------
2 2 0 yes
-------------------------------------------------------------------------------
Number of entries: 1
===============================================================================
---snip---
EVPN for MPLS tunnels in Epipe services with single-active multi-homing
Single-active multi-homing allows for per-service load-balancing. Single-active multi-homing is configured on PE-4 and PE-5 with ES "SA-ESI-45". Both PEs have an SDP to MTU-6, which is associated with the ES and to the Epipe service. The configuration of the local and remote AC names and Ethernet tags is identical on PE-4 and PE-5.
On PE-4, the service configuration is as follows:
# on PE-4:
configure
service
sdp 46 mpls create
far-end 192.0.2.6
ldp
keep-alive
shutdown
exit
no shutdown
exit
system
bgp-evpn
ethernet-segment "SA-ESI-45" create
esi 01:00:00:00:00:45:00:00:00:01
es-activation-timer 3
service-carving
mode auto
exit
multi-homing single-active
sdp 46
no shutdown
exit
exit
exit
epipe 2 name "Epipe 2" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit AC-SA-ESI-45-MTU-6 create
eth-tag 456
exit
remote-attachment-circuit AC-AA-ESI-23-MTU-1 create
eth-tag 231
exit
evi 2
mpls bgp 1
ecmp 2
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
spoke-sdp 46:2 create
no shutdown
exit
no shutdown
On PE-5, the configuration is similar, but with a different SDP:
# on PE-5:
configure
service
sdp 56 mpls create
far-end 192.0.2.6
ldp
keep-alive
shutdown
exit
no shutdown
exit
system
bgp-evpn
ethernet-segment "SA-ESI-45" create
esi 01:00:00:00:00:45:00:00:00:01
es-activation-timer 3
service-carving
mode auto
exit
multi-homing single-active
sdp 56
no shutdown
exit
exit
exit
epipe 2 name "Epipe 2" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit AC-SA-ESI-45-MTU-6 create
eth-tag 456
exit
remote-attachment-circuit AC-AA-ESI-23-MTU-1 create
eth-tag 231
exit
evi 2
mpls bgp 1
ecmp 2
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
spoke-sdp 56:2 create
no shutdown
exit
no shutdown
exit
Three route types will be exchanged between the core PEs: AD per-EVI, AD per-ES, and ES routes.
PE-4 and PE-5 advertise ES routes that are only imported by them. As an example, the following is the ES route with originator PE-4 sent by RR PE-2 to PE-5. It contains a target 00:00:00:00:45:00 in the extended community that is derived from the ESI:
# on PE-2:
64 2022/11/29 09:43:18.845 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 85
Flag: 0x90 Type: 14 Len: 34 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-ETH-SEG Len: 23 RD: 192.0.2.4:0 ESI: 01:00:00:00:00:45:00:00:00:01, IP-Len: 4 Orig-IP-Addr: 192.0.2.4
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
df-election::DF-Type:Auto/DP:0/DF-Preference:0/AC:1
target:00:00:00:00:45:00
"The AD per-ES route has a maximum Ethernet tag (MAX-ET) and an ESI label in the extended community. The multi-homing mode is single-active. As in the case of all-active multi-homing, the ESI label is not used in Epipe services. The following BGP update with originator PE-5 is sent by RR PE-2 to its client PE-4:
# on PE-2:
67 2022/11/29 09:43:18.970 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.5
Type: EVPN-AD Len: 25 RD: 192.0.2.5:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.5
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
esi-label:524282/Single-Active
bgp-tunnel-encap:MPLS
"The AD per-EVI route contains flags for primary and backup, which will be different for routes received from PE-4 and PE-5. In this case, PE-4 is primary in the single-active multi-homing ES (P=1):
# on PE-2:
70 2022/11/29 09:43:21.801 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-AD Len: 25 RD: 192.0.2.4:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: 456 Label: 8388464 (Raw Label: 0x7fff70) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.2
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 1 B: 0
bgp-tunnel-encap:MPLS
"PE-5 is backup in the single-active multi-homing ES (B=1):
# on PE-2:
78 2022/11/29 09:43:25.369 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.5
Type: EVPN-AD Len: 25 RD: 192.0.2.5:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: 456 Label: 8388496 (Raw Label: 0x7fff90) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 0 B: 1
bgp-tunnel-encap:MPLS
"The BGP EVPN AD routes can be shown with the following command:
*A:PE-2# show router bgp routes evpn auto-disc esi 01:00:00:00:00:45:00:00:00:01
===============================================================================
BGP Router ID:192.0.2.2 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Auto-Disc Routes
===============================================================================
Flag Route Dist. ESI NextHop
Tag Label
-------------------------------------------------------------------------------
u*>i 192.0.2.4:2 01:00:00:00:00:45:00:00:00:01 192.0.2.4
456 LABEL 524279
u*>i 192.0.2.4:2 01:00:00:00:00:45:00:00:00:01 192.0.2.4
MAX-ET LABEL 0
u*>i 192.0.2.5:2 01:00:00:00:00:45:00:00:00:01 192.0.2.5
456 LABEL 524281
u*>i 192.0.2.5:2 01:00:00:00:00:45:00:00:00:01 192.0.2.5
MAX-ET LABEL 0
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
For each PE in the single-active ES, there are two AD routes: the routes with MAX-ET are AD per-ES routes and the routes with a configured Ethernet tag are AD per-EVI routes.
The EVPN MPLS destination for Epipe 2 on PE-2 is SA-ESI-45, as shown in the following output:
*A:PE-2# show service id 2 evpn-mpls
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-id
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:45:00:00:00:01 11/29/2022 09:43:22
-------------------------------------------------------------------------------
Number of entries: 1
-------------------------------------------------------------------------------
===============================================================================
The ESI is resolved to the TEP address of the primary (DF) PE-4, as follows:
*A:PE-2# show service id 2 evpn-mpls esi 01:00:00:00:00:45:00:00:00:01
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:45:00:00:00:01 11/29/2022 09:43:22
===============================================================================
===============================================================================
BGP EVPN-MPLS Dest TEP Info
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-Id
-------------------------------------------------------------------------------
192.0.2.4 524279 11/29/2022 09:43:22
ldp:65538
-------------------------------------------------------------------------------
Number of entries : 1
-------------------------------------------------------------------------------
===============================================================================
The DF election is key for the forwarding and backup functions in single-active multi-homing ESs. The PE elected as DF will be the primary for the ES in the Epipe and will unblock the SAP/spoke-SDP for upstream and downstream traffic. The rest of the PEs in the ES will bring their ES SAPs or spoke-SDPs operationally down.
PE-5 is a non-DF, as follows:
*A:PE-5# show service system bgp-evpn ethernet-segment name "SA-ESI-45" evi 2
===============================================================================
EVI DF and Candidate List
===============================================================================
EVI SvcId Actv Timer Rem DF DF Last Change
-------------------------------------------------------------------------------
2 2 0 no 11/29/2022 09:43:09
===============================================================================
===============================================================================
DF Candidates Time Added Oper Pref Do Not
Value Preempt
-------------------------------------------------------------------------------
192.0.2.4 11/29/2022 09:43:19 0 Disabl*
192.0.2.5 11/29/2022 09:43:22 0 Disabl*
-------------------------------------------------------------------------------
Number of entries: 2
===============================================================================
* indicates that the corresponding row element may have been truncated.
In single-active multi-homing, the service spoke-SDP (or SAP) is brought operationally down on the non-DF, as shown in the following output:
*A:PE-5# show service id 2 sdp
===============================================================================
Services: Service Destination Points
===============================================================================
SdpId Type Far End addr Adm Opr I.Lbl E.Lbl
-------------------------------------------------------------------------------
56:2 Spok 192.0.2.6 Up Down 524280 524280
-------------------------------------------------------------------------------
Number of SDPs : 1
-------------------------------------------------------------------------------
===============================================================================
The spoke-SDP 56:2 is operationally down with a StandbyForMHProtocol flag:
*A:PE-5# show service id 2 sdp 56:2 detail | match Flag
Flags : StandbyForMHProtocol
Two consecutive DF elections take place: the first DF election includes all PEs in the ES for that Epipe and determines which PE is the primary PE (flags P=1, B=0). The second DF election excludes this DF and determines which PE is the backup (P=0, B=1). All other PEs signal flags P=0 and B=0.
When the primary PE fails, AD per-ES/EVI withdrawal messages are sent to the remote PE, which will update its next hop to the backup. The backup PE takes over immediately without waiting for the es-activation-timer to bring up its SAP/spoke-SDP.
Ethernet segment failures
When the SDP toward the primary (DF) fails, the backup PE needs to take over. An SDP failure is emulated and log 99 on PE-4 shows that SDP 46 is operational down and PE-4 is no longer the DF:
140 2022/11/29 10:05:00.118 UTC MINOR: SVCMGR #2303 Base
"Status of SDP 46 changed to admin=up oper=down"
142 2022/11/29 10:05:00.119 UTC MINOR: SVCMGR #2094 Base
"Ethernet Segment:SA-ESI-45, EVI:2, Designated Forwarding state changed to:false"Remote PEs receive route withdrawal updates (unreachable NLRI) from former DF PE-4, for example on PE-2:
# on PE-2:
82 2022/11/29 10:05:00.122 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 34
Flag: 0x90 Type: 15 Len: 30 Multiprotocol Unreachable NLRI:
Address Family EVPN
Type: EVPN-AD Len: 25 RD: 192.0.2.4:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
"
81 2022/11/29 10:05:00.122 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 59
Flag: 0x90 Type: 15 Len: 55 Multiprotocol Unreachable NLRI:
Address Family EVPN
Type: EVPN-AD Len: 25 RD: 192.0.2.4:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: 456 Label: 0 (Raw Label: 0x0) PathId:
Type: EVPN-ETH-SEG Len: 23 RD: 192.0.2.4:0 ESI: 01:00:00:00:00:45:00:00:00:01, IP-Len: 4 Orig-IP-Addr: 192.0.2.4
"The backup PE-5 is promoted to primary (P=1, B=0) and sends BGP updates accordingly. The following AD per-EVI is received on PE-2:
# on PE-2:
85 2022/11/29 10:05:00.124 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 81
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.5
Type: EVPN-AD Len: 25 RD: 192.0.2.5:2 ESI: 01:00:00:00:00:45:00:00:00:01, tag: 456 Label: 8388496 (Raw Label: 0x7fff90) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:2
l2-attribute:MTU: 1514 C: 0 P: 1 B: 0
bgp-tunnel-encap:MPLS
"PE-5 brings up its spoke-SDP without waiting for the es-activation-timer and takes over immediately. It is now the only DF candidate, and therefore the DF, as follows:
*A:PE-5# show service system bgp-evpn ethernet-segment name "SA-ESI-45" evi 2
===============================================================================
EVI DF and Candidate List
===============================================================================
EVI SvcId Actv Timer Rem DF DF Last Change
-------------------------------------------------------------------------------
2 2 0 yes 11/29/2022 09:43:09
===============================================================================
===============================================================================
DF Candidates Time Added Oper Pref Do Not
Value Preempt
-------------------------------------------------------------------------------
192.0.2.5 11/29/2022 09:43:22 0 Disabl*
-------------------------------------------------------------------------------
Number of entries: 1
===============================================================================
* indicates that the corresponding row element may have been truncated.
BGP updates are exchanged and the remote PEs will resolve the ESI to the TEP address 192.0.2.5. For example, on PE-2:
*A:PE-2# show service id 2 evpn-mpls esi 01:00:00:00:00:45:00:00:00:01
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Eth SegId Last Change
-------------------------------------------------------------------------------
01:00:00:00:00:45:00:00:00:01 11/29/2022 10:05:00
===============================================================================
===============================================================================
BGP EVPN-MPLS Dest TEP Info
===============================================================================
TEP Address Egr Label Last Change
Transport:Tnl-Id
-------------------------------------------------------------------------------
192.0.2.5 524281 11/29/2022 10:05:00
ldp:65539
-------------------------------------------------------------------------------
Number of entries : 1
-------------------------------------------------------------------------------
===============================================================================
This process is revertive; as soon as the SDP 46 is operationally up again, a new DF election is triggered with two DF candidates and PE-4 will be elected as DF.
Troubleshooting and debugging
The following show and debug commands can be used in EVPN-VPWS:
show redundancy bgp-evpn-multi-homing
show router bgp routes evpn (and filters)
show service evpn-mpls [<TEP ip-address>]
show service id bgp-evpn
show service id evpn-mpls (and modifiers)
show service system bgp-evpn
show service system bgp-evpn ethernet-segment (and modifiers)
debug router bgp update
show log log-id 99
Most of these commands have been shown in the preceding sections; some commands are shown in this section.
Information about the configured boot timers (before DF election) and ES activation timer (after the system has been elected DF) can be shown as follows:
*A:PE-2# show redundancy bgp-evpn-multi-homing
===============================================================================
Redundancy BGP EVPN Multi-homing Information
===============================================================================
Boot-Timer : 10 secs
Boot-Timer Remaining : 0 secs
ES Activation Timer : 3 secs
===============================================================================
See chapter EVPN for MPLS Tunnels for a description of these timers.
The following command shows that the BGP route-type 4 (ES route) messages are only imported by the PEs in the same ES; for example, on PE-3:
*A:PE-3# show router bgp routes evpn eth-seg
===============================================================================
BGP Router ID:192.0.2.3 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Eth-Seg Routes
===============================================================================
Flag Route Dist. ESI NextHop
OrigAddr
-------------------------------------------------------------------------------
u*>i 192.0.2.2:0 01:00:00:00:00:23:00:00:00:01 192.0.2.2
192.0.2.2
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
On PE-4:
*A:PE-4# show router bgp routes evpn eth-seg
===============================================================================
BGP Router ID:192.0.2.4 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Eth-Seg Routes
===============================================================================
Flag Route Dist. ESI NextHop
OrigAddr
-------------------------------------------------------------------------------
u*>i 192.0.2.5:0 01:00:00:00:00:45:00:00:00:01 192.0.2.5
192.0.2.5
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The following command shows all the EVPN MPLS destinations toward TEP 192.0.2.4. Epipe 1 has an EVPN MPLS destination toward TEP 192.0.2.4 directly and Epipe 2 has an EVPN MPLS destination to SA-ESI-45, which can be resolved to TEP 192.0.2.4. This is shown in the following output:
*A:PE-2# show service evpn-mpls 192.0.2.4
===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
Service Id Egr Label Instance
-------------------------------------------------------------------------------
1 524282 1
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-MPLS Ethernet Segment Dest
===============================================================================
Service Id Eth Seg Id Egr Label
-------------------------------------------------------------------------------
2 01:00:00:00:00:45:00:00:00:01 524279
-------------------------------------------------------------------------------
===============================================================================
===============================================================================
BGP EVPN-MPLS ES BMac Dest
===============================================================================
Service Id ES BMac Egr Label
-------------------------------------------------------------------------------
No Matching Entries
===============================================================================
The following command lists all configured ESs on the system:
*A:PE-2# show service system bgp-evpn ethernet-segment
===============================================================================
Service Ethernet Segment
===============================================================================
Name ESI Admin Oper
-------------------------------------------------------------------------------
AA-ESI-23 01:00:00:00:00:23:00:00:00:01 Enabled Up
-------------------------------------------------------------------------------
Entries found: 1
===============================================================================
In addition to the preceding commands, the following tools dump commands may be useful:
tools dump service evpn usage - This command shows the number of EVPN-MPLS (and EVPN-VXLAN) destinations in the system.
tools dump service system bgp-evpn ethernet-segment <name> evi <..> df - This command computes the DF election for a specific ESI and EVI. For all-active, there is no DF election and all PEs forward traffic. For single-active, one PE will be active for a service while another PE will be backup. This command shows the DF (primary), even if it is not the local PE.
The usage of EVPN resources can be shown as follows:
*A:PE-2# tools dump service evpn usage
vxlan-srv6-evpn-mpls usage statistics at 11/29/2022 10:13:11:
MPLS-TEP : 1
VXLAN-TEP : 0
SRV6-TEP : 0
Total-TEP : 1/ 16383
Mpls Dests (TEP, Egress Label + ES + ES-BMAC) : 2
Mpls Etree Leaf Dests : 0
Vxlan Dests (TEP, Egress VNI + ES) : 0
Srv6 Dests (TEP, SID + ES) : 0
Total-Dest : 2/196607
Sdp Bind + Evpn Dests : 2/245759
ES L2/L3 PBR : 0/ 32767
Evpn Etree Remote BUM Leaf Labels : 0
On PE-2, there is one MPLS-TEP (192.0.2.4 in Epipe 1 and Epipe 2) and there are two MPLS destinations: 192.0.2.4 and ESI 01:00:00:00:00:45:00:00:00:01. PE-5 is not an MPLS-TEP for PE-2, because it is not a primary and, therefore, not forwarding any traffic.
In all-active multi-homing, the DF election is not applicable:
*A:PE-2# tools dump service system bgp-evpn ethernet-segment "AA-ESI-23" evi 2 df
[11/29/2022 10:13:29] All Active VPWS or IP-ALIASING - DF N/A
In single-active multi-homing, the following command shows which PE is the DF and which PE is the backup:
*A:PE-5# tools dump service system bgp-evpn ethernet-segment "SA-ESI-45" evi 2 df
[11/29/2022 10:13:49] Computed DF: 192.0.2.4 (Remote) (Boot Timer Expired: Yes)
[11/29/2022 10:13:49] Computed Backup: 192.0.2.5 (This Node)
The command is launched on PE-5, which is a backup. The computed DF is PE-4 and the boot timer has expired, meaning there is no DF re-election pending.
Conclusion
EVPN-VPWS is a simplified point-to-point version of RFC 7432, BGP MPLS-Based Ethernet VPN. When used for Epipe and VPLS services, EVPN provides a unified control plane mechanism that simplifies the network deployment and operation. Single-active and all-active multi-homing can be used in Epipes; EVPN-VPWS is a differentiator of EVPN compared to traditional TLDP or BGP Epipe redundancy mechanisms. The Ethernet segments used for multi-homing can be shared between EVPN VPLS and EVPN Epipes.