EVPN VPWS Multihoming on PW ports
This chapter provides information about EVPN-VPWS multihoming on PW ports.
Topics in this chapter include:
Applicability
The information and configuration in this chapter are based on SR OS Release 23.10.R1.
Prerequisite reading: EVPN for MPLS Tunnels in Epipe Services (EVPN-VPWS) and EVPN VPWS Services with SRv6 Transport.
Overview
- provide SAP-based capabilities to a PW which has traditionally been a network-port-based concept within SR OS. PW payload can be extracted onto PW-port-based SAPs with granular queuing capabilities (per SAP).
- look up dot1q and qinq VLAN tags underneath the PW labels and map the traffic to different services.
- terminate subscriber traffic carried within the PW on a Broadband Network Gateway (BNG): PW-port-based SAPs are instantiated under a group interface with Enhanced Subscriber Management (ESM).
- PW port bound to a specific physical input/output port (I/O port)
- PW port independent of the I/O port on which the PW is terminated: Forwarding Path Extension (FPE) based PW port
In this chapter, FPE-based PW ports are used. The benefit of FPE-based PW ports is that they can provide services when traffic within the PW is rerouted between I/O ports because of a network failure.
Both all-active and single-active EVPN multihoming modes are supported. EVPN-MPLS single-active multihoming on Epipe PW ports shows the example topology with stitching Epipe "Epipe-123" in the access network and single-active multihoming between PE-2 and PE-3. The single-active Ethernet segment (ES) has a PW port associated to it.
In the aggregation network, the stitching EVPN-VPWS "Epipe-123" backhauls the traffic. The SAP of the stitching Epipe on access node PE-1 is 1/1/c10/1:*, so it accepts packets with VID 1 for core service "Epipe-1" and packets with VID 2 for core service "Epipe-2". The stitching between Epipe "Epipe-123" and the service Epipes is done at the Designated Forwarder (DF) PE-3. PE-2 is the Non-Designated Forwarder (NDF) which brings the PW port operationally down due to the MHStandby flag (unless the oper-up-on-mhstandby option is enabled). On the NDF PE-2, the PW SAPs pw-23:1 and pw-23:2 (in the core services "Epipe-1" and "Epipe-2") are brought operationally down when the PW port is down. If the PW port is down only due to the MHStandby flag, the AD per-ES route and AD per-EVI route for the service Epipes are still advertised, so PE-1 receives EVPN-AD routes from DF PE-3 and NDF PE-2.
Internal connectivity between switching Epipe and service Epipes shows how internal VPLS "_tmnx_InternalVplsService" with ID 2147483649 is used for the internal cross-connect between the stitching Epipe and the service Epipes.
Configuration
- cards, MDAs, ports
- router interfaces
- IS-IS on the router interfaces (alternatively, OSPF can be used)
In the EVPN-MPLS scenarios, MPLS is used in the aggregation network (between PE-1, PE-2, and PE-3) and in the core network (between PE-2, PE-3, and PE-4). In this example, LDP is applied in the aggregation network and SR-ISIS in the core network.
# on PE-1, PE-2, PE-3, PE-4:
configure
router Base
autonomous-system 64500
bgp
vpn-apply-import
vpn-apply-export
enable-peer-tracking
rapid-withdrawal
split-horizon
rapid-update evpn
group "internal"
family evpn
peer-as 64500
neighbor 192.0.2.5
exit
exit
# on RR-5:
configure
router Base
autonomous-system 64500
bgp
vpn-apply-import
vpn-apply-export
enable-peer-tracking
rapid-withdrawal
split-horizon
rapid-update evpn
group "internal"
family evpn
cluster 192.0.2.5
peer-as 64500
neighbor 192.0.2.1
exit
neighbor 192.0.2.2
exit
neighbor 192.0.2.3
exit
neighbor 192.0.2.4
exit
exit
In all scenarios, FPE-based PW ports are configured on PE-2 and PE-3, as follows:
# on PE-2, PE-3:
configure
port-xc
pxc 5 create
port 1/1/c5/1
no shutdown
exit
exit
port pxc-5.a
ethernet
encap-type dot1q
exit
no shutdown
exit
port pxc-5.b
ethernet
encap-type dot1q
exit
no shutdown
exit
port 1/1/c5/1
ethernet
dot1x
tunneling
exit
mode hybrid
encap-type dot1q
exit
no shutdown
exit
fwd-path-ext
sdp-id-range from 10000 to 10127
fpe 5 create
path pxc 5
pw-port-extension
interface-a
exit
interface-b
exit
exit
exit
exit
pw-port 23 create
encap-type dot1q # default
exitThe preceding configuration is similar to the VXLAN FPE configuration, as described in the VXLAN Forwarding Path Extension chapter
EVPN-MPLS all-active multihoming on Epipe PW ports
EVPN-MPLS all-active multi-homing on Epipe PW ports shows the stitching Epipe in the aggregation network and the core Epipe services in an all-active multihoming scenario.
# on PE-1:
configure
service
epipe 123 name "Epipe-123" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-1 create
eth-tag 1
exit
remote-attachment-circuit ac-23 create
eth-tag 23
exit
evi 123
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap 1/1/c10/1:* create
description "SAP to CEs"
no shutdown
exit
no shutdown
# on PE-2, PE-3:
configure
service
epipe 123 name "stitching-Epipe-123" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-23 create
eth-tag 23
exit
remote-attachment-circuit ac-1 create
eth-tag 1
exit
evi 123
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
pw-port 23 fpe 5 create
no shutdown
exit
no shutdown
exitThe aggregation network uses EVPN-VPWS to backhaul the traffic. The network nodes PE-2 and PE-3 apply the same Ethernet tag (23) on the local attachment circuit in the stitching Epipe. Optionally, PE-2 and PE-3 can use the same route distinguisher on the stitching service. AD per-EVI routes for the stitching service are advertised with ESI 0.
# on PE-2:
configure
service
system
bgp-evpn
ethernet-segment "AA-vES-23-1" virtual create
esi 01:00:00:00:00:23:00:00:01:01
es-activation-timer 3
service-carving
mode manual
manual
preference non-revertive create
value 100
exit
exit
exit
multi-homing all-active
pw-port 23
dot1q
q-tag-range 1 to 200
exit
no shutdown
exitPE-2 and PE-3 receive tagged traffic inside the EVPN-VPWS stitching Epipe and map each tag to a different service in the core, such as ESM services, Epipe services, or VPRN services. In this example, the following services are configured:
# on PE-2, PE-3:
configure
service
epipe 1 name "service-Epipe-1" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-23 create
eth-tag 23
exit
remote-attachment-circuit ac-4 create
eth-tag 4
exit
evi 1
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap pw-23:1 create
no shutdown
exit
no shutdown
exit
epipe 2 name "service-Epipe-2" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-23 create
eth-tag 23
exit
remote-attachment-circuit ac-4 create
eth-tag 4
exit
evi 2
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap pw-23:2 create
no shutdown
exit
no shutdown
exit
These Epipe services are also configured on PE-4:
# on PE-4:
configure
service
epipe 1 name "Epipe-1" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-4 create
eth-tag 4
exit
remote-attachment-circuit ac-23 create
eth-tag 23
exit
evi 1
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap 1/1/c10/1:1 create
description "SAP to CE-41"
no shutdown
exit
no shutdown
exit
epipe 2 name "Epipe-2" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-4 create
eth-tag 4
exit
remote-attachment-circuit ac-23 create
eth-tag 23
exit
evi 2
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap 1/1/c10/1:2 create
description "SAP to CE-42"
no shutdown
exit
no shutdown
exit
- Access node PE-1 forwards traffic based on the best AD per-EVI route
advertised by PE-2 and PE-3 for the stitching Epipe. This selection can
be either BGP-based if PE-2 and PE-3 use the same route distinguisher
(RD) in the stitching service or EVPN-based if different RDs are used.
The EVPN-based selection when the RDs are different is based on the
lowest IP address of the route. In the example, the RDs are
auto-derived, such as 192.0.2.2:123, 192.0.2.3:123.Note:
BGP-based selection is also possible when the RDs are different if the command configure service system bgp-evpn ad-per-evi-routes bgp-path-selection is configured. For example, in the case of regular BGP best path selection, it is possible to modify the local preference to influence which path is selected.
- When access node PE-1 selects the route to PE-2, PE-2 receives the traffic on the local PW SAP for Epipes "Epipe-1" or "Epipe-2" and forwards it based on the EVPN-VPWS rules in the network to PE-4.
- PE-4 forwards the traffic based on the configuration of ECMP and aliasing rules for Epipes "Epipe-1" or "Epipe-2".
- PE-4 may send the traffic to PE-3 and PE-3 to the access node PE-1.
Traffic from the core to the access network may follow an asymmetric path because the multihoming procedures are run on the PW SAPs of the core services, not on the stitching Epipe service. The AD per-EVI routes advertised in the context of the stitching Epipe use ESI 0.
The following command shows that the all-active ES applies to EVI 1 and EVI 2, not to the stitching Epipe with EVI 123:
*A:PE-2# show service system bgp-evpn ethernet-segment name "AA-vES-23-1" all
===============================================================================
Service Ethernet Segment
===============================================================================
Name : AA-vES-23-1
Eth Seg Type : Virtual
Admin State : Enabled Oper State : Up
ESI : 01:00:00:00:00:23:00:00:01:01
Oper ESI : 01:00:00:00:00:23:00:00:01:01
Auto-ESI Type : None
AC DF Capability : Include
Multi-homing : allActive Oper Multi-homing : allActive
ES Split Horizon Label : 524277
ES Split Horizon Arg : 1
Source BMAC LSB : None
PW Port Id : 23
ES Activation Timer : 3 secs
Oper Group : (Not Specified)
Svc Carving : manual Oper Svc Carving : manual
Cfg Range Type : lowest-pref
-------------------------------------------------------------------------------
DF Pref Election Information
-------------------------------------------------------------------------------
Preference Preference Last Admin Change Oper Pref Do No
Mode Value Value Preempt
-------------------------------------------------------------------------------
non-revertive 100 11/28/2023 06:39:53 100 Enabled
-------------------------------------------------------------------------------
EVI Ranges: <none>
ISID Ranges: <none>
Vprn NextHop EVI Ranges : <none>
===============================================================================
===============================================================================
EVI Information
===============================================================================
EVI SvcId Actv Timer Rem DF
-------------------------------------------------------------------------------
1 1 0 yes
2 2 0 yes
-------------------------------------------------------------------------------
Number of entries: 2
===============================================================================
-------------------------------------------------------------------------------
DF Candidate list
-------------------------------------------------------------------------------
EVI DF Address
-------------------------------------------------------------------------------
No entries found
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
---snip---
Both PE-2 and PE-3 are DF in all-active EVPN-VPWS services "Epipe-1" and "Epipe-2". The following commands are launched on PE-2, but the output on PE-3 is similar.
*A:PE-2# show service id 1 ethernet-segment
===============================================================================
SAP Ethernet-Segment Information
===============================================================================
SAP Eth-Seg Status
-------------------------------------------------------------------------------
pw-23:1 AA-vES-23-1 DF
===============================================================================
No sdp entries
No vxlan instance entries
*A:PE-2# show service id 2 ethernet-segment
===============================================================================
SAP Ethernet-Segment Information
===============================================================================
SAP Eth-Seg Status
-------------------------------------------------------------------------------
pw-23:2 AA-vES-23-1 DF
===============================================================================
No sdp entries
No vxlan instance entries
*A:PE-2# show service id 123 ethernet-segment
No sap entries
No sdp entries
No vxlan instance entries
*A:PE-2# tools dump service system bgp-evpn ethernet-segment "AA-vES-23-1" evi 123 df
[11/28/2023 07:40:39] Evi not active on ethernet-segment
*A:PE-2# tools dump service system bgp-evpn ethernet-segment "AA-vES-23-1" evi 1 df
[11/28/2023 07:40:39] All Active VPWS or IP-ALIASING - DF N/A
*A:PE-2# tools dump service system bgp-evpn ethernet-segment "AA-vES-23-1" evi 2 df
[11/28/2023 07:40:39] All Active VPWS or IP-ALIASING - DF N/A
DF election is not applicable for all-active multihoming in EVPN-VPWS services. For the stitching Epipe, the EVI 123 is not active on the Ethernet segment.
The following VPLS "_tmnx_InternalVplsService" with SDP 10001:100023 on FPE-5.a ensures the internal cross-connect between the stitching Epipe and the core Epipes in PE-2.
*A:PE-2# show service id "_tmnx_InternalVplsService" base
===============================================================================
Service Basic Information
===============================================================================
Service Id : 2147483649 Vpn Id : 0
Service Type : intVpls
MACSec enabled : no
Name : _tmnx_InternalVplsService
Description : VPLS Service for internal purposes only
Customer Id : 1 Creation Origin : manual
Last Status Change: 11/28/2023 06:39:56
Last Mgmt Change : 11/28/2023 06:39:53
Admin State : Up Oper State : Up
SAP Count : 0 SDP Bind Count : 1
-------------------------------------------------------------------------------
Service Access & Destination Points
-------------------------------------------------------------------------------
Identifier Type AdmMTU OprMTU Adm Opr
-------------------------------------------------------------------------------
sdp:10001:100023 SB(fpe_5.a) Fpe 0 8910 Up Up
===============================================================================
The following command on PE-2 shows that PW port 23 uses SDP 10001 with VC ID 100023 on SDP binding port pxc-5.b:
*A:PE-2# show pw-port 23 detail
===============================================================================
PW Port Information
===============================================================================
PW Port : 23
Encap : dot1q
SDP : 10001
IfIndex : 1526726679
VC-Id : 100023
Description : PW Port
Dot1Q Ethertype : 0x8100
Service Id : 123
Down on Peer Tldp PW Status Faults: No
Oper Up on MH Standby : No
===============================================================================
===============================================================================
Service Destination Point (Sdp Id 10001 Pw-Port 23)
===============================================================================
SDP Binding port : pxc-5.b
VC-Id : 100023 Admin Status : up
Encap : dot1q Oper Status : up
VC Type : ether
Dot1Q Ethertype : 0x8100
Control Word : Not Preferred
Entropy Label : Disabled
Service MTU : default
Admin Ingress label : 524278 Admin Egress label : 524279
Oper Flags : (Not Specified)
Monitor Oper-Group : (Not Specified)
===============================================================================
EVPN-MPLS single-active multihoming on Epipe PW ports
EVPN-MPLS single-active multihoming support on Epipe PW ports is supported in SR OS Release 22.2.R1 and later.
Service configuration on PE-2 and PE-3
On PE-2 and PE-3, the stitching Epipe is configured as follows:
# on PE-2, PE-3:
configure
service
epipe 123 name "stitching-Epipe-123" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-23 create
eth-tag 23
exit
remote-attachment-circuit ac-1 create
eth-tag 1
exit
evi 123
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
pw-port 23 fpe 5 create
no shutdown
exit
no shutdown
exit
The following (non-virtual) single-active ES is configured on PE-2 and PE-3.
# on PE-2, PE-3:
configure
service
system
bgp-evpn
ethernet-segment "SA-ES-23" create
esi 01:00:00:00:00:23:00:00:00:01
es-activation-timer 3
service-carving
mode auto
exit
multi-homing single-active
pw-port 23 pw-headend
no shutdown
exit
exit
exitThe pw-headend keyword allows PW ports to be associated with ESs in single-active mode. The pw-headend keyword ensures that the stitching Epipe is running the ES and DF election procedures similar to the mh-mode network in VPLS services. The NDF on the stitching Epipe brings the PW port down with reason MHStandby. The AD per-ES routes and AD per-EVI routes are advertised with the RD and RT of the service Epipe and the configured ESI of the ES associated with the PW port. If the PW port is down only due to MHStandby, the AD per-ES routes and AD per-EVI routes are still advertised. The oper-up-on-mhstandby option allows to keep the PW port up on the NDF, which can speed up convergence in case a large number of PW SAPs is configured on the same PW port.
The DF receives tagged traffic inside EVPN-VPWS circuits and maps each tag to a different service in the core network, such as ESM services, Epipe services, or VPRN services. In this example, the following Epipe services are configured with PW SAPs:
# on PE-2, PE-3:
configure
service
epipe 1 name "service-Epipe-1" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-23 create
eth-tag 23
exit
remote-attachment-circuit ac-4 create
eth-tag 4
exit
evi 1
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap pw-23:1 create
no shutdown
exit
no shutdown
exit
epipe 2 name "service-Epipe-2" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-23 create
eth-tag 23
exit
remote-attachment-circuit ac-4 create
eth-tag 4
exit
evi 2
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap pw-23:2 create
no shutdown
exit
no shutdown
exit
The configuration on PE-1 and PE-4 is similar to the configuration in the all-active multihoming scenario.
ES and DF election procedures on stitching Epipe
The stitching Epipe associated with the PW port is running the ES and DF election procedures. The following ES command on PE-3 shows the state for the stitching Epipe EVI 123, not for the contained PW SAP services with EVI 1 and EVI 2. PE-3 is the DF.
*A:PE-3# show service system bgp-evpn ethernet-segment name "SA-ES-23" all
===============================================================================
Service Ethernet Segment
===============================================================================
Name : SA-ES-23
Eth Seg Type : None
Admin State : Enabled Oper State : Up
ESI : 01:00:00:00:00:23:00:00:00:01
Oper ESI : 01:00:00:00:00:23:00:00:00:01
Auto-ESI Type : None
AC DF Capability : Include
Multi-homing : singleActive Oper Multi-homing : singleActive
ES Split Horizon Label : None
ES Split Horizon Arg : None
Source BMAC LSB : None
PW Port Id : 23
PW Port Headend : enabled
ES Activation Timer : 3 secs
Oper Group : (Not Specified)
Svc Carving : auto Oper Svc Carving : auto
Cfg Range Type : primary
Vprn NextHop EVI Ranges : <none>
===============================================================================
===============================================================================
EVI Information
===============================================================================
EVI SvcId Actv Timer Rem DF
-------------------------------------------------------------------------------
123 123 0 yes
-------------------------------------------------------------------------------
Number of entries: 1
===============================================================================
-------------------------------------------------------------------------------
DF Candidate list
-------------------------------------------------------------------------------
EVI DF Address
-------------------------------------------------------------------------------
123 192.0.2.2
123 192.0.2.3
-------------------------------------------------------------------------------
Number of entries: 2
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
---snip---
The following command shows that PE-2 is NDF for the stitching Epipe:
*A:PE-2# show service id 123 ethernet-segment
No sap entries
No sdp entries
No vxlan instance entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
Pw-Port Eth-Seg Status
-------------------------------------------------------------------------------
23 SA-ES-23 NDF
===============================================================================
The NDF PE-2 will bring the PW port down because of the MHStandby flag, but the AD per-ES route for the stitching Epipe service is still advertised (if MHStandby is the only reason for the PW port to be down). Therefore, PE-1 receives an AD per-ES route for the stitching Epipe from PE-2 and from PE-3:
# on PE-1:
8 2023/11/23 13:35:32.942 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:123 ESI: 01:00:00:00:00:23:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.2
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:123
esi-label:0/Single-Active
bgp-tunnel-encap:MPLS
"13 2023/11/23 13:35:32.947 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-AD Len: 25 RD: 192.0.2.3:123 ESI: 01:00:00:00:00:23:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.3
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:123
esi-label:0/Single-Active
bgp-tunnel-encap:MPLS
"Likewise, PE-1 receives an AD per-EVI route from both PE-2 and PE-3. DF PE-3 sends an AD per-EVI with primary bit P: 1, as follows:
# on PE-1:
36 2023/11/23 13:35:36.002 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-AD Len: 25 RD: 192.0.2.3:123 ESI: 01:00:00:00:00:23:00:00:00:01, tag: 23 Label: 8388496 (Raw Label: 0x7fff90) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.3
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:123
l2-attribute:MTU: 1514 F: 0 C: 0 P: 1 B: 0
bgp-tunnel-encap:MPLS
"# on PE-1:
33 2023/11/23 13:35:35.910 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:123 ESI: 01:00:00:00:00:23:00:00:00:01, tag: 23 Label: 8388480 (Raw Label: 0x7fff80) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.2
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:123
l2-attribute:MTU: 1514 F: 0 C: 0 P: 0 B: 1
bgp-tunnel-encap:MPLS
"
NDF behavior
The following command on NDF PE-2 shows that the PW port is operationally down with flag StandbyForMHProtocol:
*A:PE-2# show pw-port 23 detail
===============================================================================
PW Port Information
===============================================================================
PW Port : 23
Encap : dot1q
SDP : 10001
IfIndex : 1526726679
VC-Id : 100023
Description : PW Port
Dot1Q Ethertype : 0x8100
Service Id : 123
Down on Peer Tldp PW Status Faults: No
Oper Up on MH Standby : No
===============================================================================
===============================================================================
Service Destination Point (Sdp Id 10001 Pw-Port 23)
===============================================================================
SDP Binding port : pxc-5.b
VC-Id : 100023 Admin Status : up
Encap : dot1q Oper Status : down
VC Type : ether
Dot1Q Ethertype : 0x8100
Control Word : Not Preferred
Entropy Label : Disabled
Service MTU : default
Admin Ingress label : 524278 Admin Egress label : 524279
Oper Flags : StandbyForMHProtocol
Monitor Oper-Group : (Not Specified)
===============================================================================
On NDF PE-2, the PW SAPs contained in the PW port are also brought down:
*A:PE-2# show service id "service-Epipe-1" sap
===============================================================================
SAP(Summary), Service 1
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
pw-23:1 1 1 none 1 none Up Down
-------------------------------------------------------------------------------
Number of SAPs : 1
-------------------------------------------------------------------------------
===============================================================================
*A:PE-2# show service id "service-Epipe-2" sap
===============================================================================
SAP(Summary), Service 2
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
pw-23:2 2 1 none 1 none Up Down
-------------------------------------------------------------------------------
Number of SAPs : 1
-------------------------------------------------------------------------------
===============================================================================
The following command configures the oper-up-on-mhstandby option for the stitching Epipe on NDF PE-2:
# on PE-2:
configure
service
epipe "stitching-Epipe-123"
pw-port 23 fpe 5 create
oper-up-on-mhstandby
no shutdown
exit
With the oper-up-on-mhstandby option enabled, the PW port is operationally up on NDF PE-2:
*A:PE-2# show pw-port 23 detail
===============================================================================
PW Port Information
===============================================================================
PW Port : 23
Encap : dot1q
SDP : 10001
IfIndex : 1526726679
VC-Id : 100023
Description : PW Port
Dot1Q Ethertype : 0x8100
Service Id : 123
Down on Peer Tldp PW Status Faults: No
Oper Up on MH Standby : Yes
===============================================================================
===============================================================================
Service Destination Point (Sdp Id 10001 Pw-Port 23)
===============================================================================
SDP Binding port : pxc-5.b
VC-Id : 100023 Admin Status : up
Encap : dot1q Oper Status : up
VC Type : ether
Dot1Q Ethertype : 0x8100
Control Word : Not Preferred
Entropy Label : Disabled
Service MTU : default
Admin Ingress label : 524278 Admin Egress label : 524279
Oper Flags : StandbyForMHProtocol
Monitor Oper-Group : (Not Specified)
===============================================================================
Likewise, the PW SAPs in the service Epipes are operationally up:
*A:PE-2# show service id "service-Epipe-1" sap
===============================================================================
SAP(Summary), Service 1
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
pw-23:1 1 1 none 1 none Up Up
-------------------------------------------------------------------------------
Number of SAPs : 1
-------------------------------------------------------------------------------
===============================================================================
*A:PE-2# show service id "service-Epipe-2" sap
===============================================================================
SAP(Summary), Service 2
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
pw-23:2 2 1 none 1 none Up Up
-------------------------------------------------------------------------------
Number of SAPs : 1
-------------------------------------------------------------------------------
===============================================================================
EVPN-SRv6 single-active multihoming on Epipe PW ports
EVPN-SRv6 single-active multihoming support on Epipe PW ports is supported in SR OS Release 22.5.R1 and later.
EVPN-SRv6 single-active multihoming on Epipe PW ports shows the topology with EVPN-SRv6 in the aggregation network and both EVPN-MPLS and EVPN-SRv6 in the core network.
The stitching Epipe in the aggregation network uses SRv6 transport. Epipe 1 in the core network uses SRv6 transport, while Epipe 2 in the core network uses MPLS transport.
SRv6 configuration
The SRv6 configuration is as described in the EVPN VPWS Services with SRv6 Transport chapter:
# on PE-2:
configure
card 1
mda 1
xconnect
mac 1 create
loopback 1 create
exit
loopback 2 create
exit
exit
exit
no shutdown
info
exit
exit
port-xc
pxc 1 create
port 1/1/m1/1
no shutdown
exit
pxc 2 create
port 1/1/m1/2
no shutdown
exit
exit
port pxc-1.a
ethernet
exit
no shutdown
exit
port pxc-1.b
ethernet
exit
no shutdown
exit
port pxc-2.a
ethernet
exit
no shutdown
exit
port pxc-2.b
ethernet
exit
no shutdown
exit
port 1/1/m1/1
no shutdown
exit
port 1/1/m1/2
no shutdown
exit
fwd-path-ext
fpe 1 create
path pxc 1
srv6 origination
interface-a
exit
interface-b
exit
exit
exit
fpe 2 create
path pxc 2
srv6 termination
interface-a
exit
interface-b
exit
exit
exit
exit
router Base
segment-routing
segment-routing-v6
origination-fpe 1
source-address 2001:db8::2:2
locator "loc_Epipe"
block-length 48
termination-fpe 2
prefix
ip-prefix 2001:db8:aaaa:102::/64
exit
static-function
exit
no shutdown
exit
base-routing-instance
locator "loc_Epipe"
function
end-x-auto-allocate srh-mode psp protection unprotected
end 1
srh-mode usp
exit
exit
exit
exit
exit
exit
isis 0
area-id 49.0001
traffic-engineering
traffic-engineering-options
ipv6
application-link-attributes
exit
exit
advertise-passive-only
advertise-router-capability as
ipv6-routing native
no ipv6-multicast-routing
segment-routing-v6
locator "loc_Epipe"
level-capability level-2
level 1
exit
level 2
exit
exit
no shutdown
exitThe configuration on the other PEs is similar. IPv6 addresses are configured on all interfaces. The BGP configuration uses IPv6 addresses, as follows:
# on PE-1, PE-2, PE-3, PE-4:
configure
router Base
autonomous-system 64500
bgp
vpn-apply-import
vpn-apply-export
enable-peer-tracking
rapid-withdrawal
split-horizon
rapid-update evpn
group "internal"
family evpn
peer-as 64500
neighbor 2001:db8::2:5
exit
exit
# on RR-5:
configure
router Base
autonomous-system 64500
bgp
vpn-apply-import
vpn-apply-export
enable-peer-tracking
rapid-withdrawal
split-horizon
rapid-update evpn
group "internal"
family evpn
cluster 192.0.2.5
peer-as 64500
extended-nh-encoding ipv4 vpn-ipv4
advertise-ipv6-next-hops evpn
neighbor 2001:db8::2:1
exit
neighbor 2001:db8::2:2
exit
neighbor 2001:db8::2:3
exit
neighbor 2001:db8::2:4
exit
exit
Service configuration
On PE-1, the EVPN-VPWS "Epipe-123" is configured as follows:
# on access node PE-1:
configure
service
epipe 123 name "Epipe-123" customer 1 create
segment-routing-v6 1 create
locator "loc_Epipe"
function
end-dx2
exit
exit
exit
bgp
exit
bgp-evpn
local-attachment-circuit ac-1 create
eth-tag 1
exit
remote-attachment-circuit ac-23 create
eth-tag 23
exit
evi 123
segment-routing-v6 bgp 1 srv6-instance 1 default-locator "loc_Epipe" create
no shutdown
exit
exit
sap 1/1/c10/1:* create
description "SAP to CEs"
no shutdown
exit
no shutdown
exit
On PE-2 and PE-3, the stitching Epipe and the single-active ES are configured as follows:
# on PE-2, PE-3:
configure
service
epipe 123 name "stitching-Epipe-123" customer 1 create
segment-routing-v6 1 create
locator "loc_Epipe"
function
end-dx2
exit
exit
exit
bgp
exit
bgp-evpn
local-attachment-circuit ac-23 create
eth-tag 23
exit
remote-attachment-circuit ac-1 create
eth-tag 1
exit
evi 123
segment-routing-v6 bgp 1 srv6-instance 1 default-locator "loc_Epipe" create
no shutdown
exit
exit
pw-port 23 fpe 5 create
no shutdown
exit
no shutdown
exit
system
bgp-evpn
ethernet-segment "SA-ES-23" create
esi 01:00:00:00:00:23:00:00:00:01
es-activation-timer 3
service-carving
mode auto
exit
multi-homing single-active
pw-port 23 pw-headend
no shutdown
exit
exit
exit
The core service "service-Epipe-1" uses SRv6 transport:
# on PE-2, PE-3:
configure
service
epipe 1 name "service-Epipe-1" customer 1 create
segment-routing-v6 1 create
locator "loc_Epipe"
function
end-dx2
exit
exit
exit
bgp
exit
bgp-evpn
local-attachment-circuit ac-23 create
eth-tag 23
exit
remote-attachment-circuit ac-4 create
eth-tag 4
exit
evi 1
segment-routing-v6 bgp 1 srv6-instance 1 default-locator "loc_Epipe" create
no shutdown
exit
exit
sap pw-23:1 create
no shutdown
exit
no shutdownThe core service "service-Epipe-2" uses MPLS transport:
# on PE-2, PE-3:
configure
service
epipe 2 name "service-Epipe-2" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-23 create
eth-tag 23
exit
remote-attachment-circuit ac-4 create
eth-tag 4
exit
evi 2
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap pw-23:2 create
no shutdown
exit
no shutdown
On PE-4, the corresponding Epipe services are configured as follows:
# on PE-4:
configure
service
epipe 1 name "Epipe-1" customer 1 create
segment-routing-v6 1 create
locator "loc_Epipe"
function
end-dx2
exit
exit
exit
bgp
exit
bgp-evpn
local-attachment-circuit ac-1 create
eth-tag 4
exit
remote-attachment-circuit ac-23 create
eth-tag 23
exit
evi 1
segment-routing-v6 bgp 1 srv6-instance 1 default-locator "loc_Epipe" create
no shutdown
exit
exit
sap 1/1/c10/1:1 create
description "SAP to CE-41"
no shutdown
exit
no shutdown
exit
epipe 2 name "Epipe-2" customer 1 create
bgp
exit
bgp-evpn
local-attachment-circuit ac-1 create
eth-tag 4
exit
remote-attachment-circuit ac-23 create
eth-tag 23
exit
evi 2
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap 1/1/c10/1:2 create
description "SAP to CE-42"
no shutdown
exit
no shutdown
exit
Verification
The stitching Epipe associated with the PW port is running the ES and DF election procedures. The following service configuration output shows that the ES is applied for EVI 123 of the stitching Epipe, not for EVI 1 or EVI 2.
*A:PE-3# show service system bgp-evpn ethernet-segment name "SA-ES-23" all
===============================================================================
Service Ethernet Segment
===============================================================================
Name : SA-ES-23
Eth Seg Type : None
Admin State : Enabled Oper State : Up
ESI : 01:00:00:00:00:23:00:00:00:01
Oper ESI : 01:00:00:00:00:23:00:00:00:01
Auto-ESI Type : None
AC DF Capability : Include
Multi-homing : singleActive Oper Multi-homing : singleActive
ES Split Horizon Label : None
ES Split Horizon Arg : None
Source BMAC LSB : None
PW Port Id : 23
PW Port Headend : enabled
ES Activation Timer : 3 secs
Oper Group : (Not Specified)
Svc Carving : auto Oper Svc Carving : auto
Cfg Range Type : primary
Vprn NextHop EVI Ranges : <none>
===============================================================================
===============================================================================
EVI Information
===============================================================================
EVI SvcId Actv Timer Rem DF
-------------------------------------------------------------------------------
123 123 0 yes
-------------------------------------------------------------------------------
Number of entries: 1
===============================================================================
-------------------------------------------------------------------------------
DF Candidate list
-------------------------------------------------------------------------------
EVI DF Address
-------------------------------------------------------------------------------
123 192.0.2.2
123 192.0.2.3
-------------------------------------------------------------------------------
Number of entries: 2
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
---snip---
PE-2 is NDF for the stitching Epipe:
*A:PE-2# show service id 123 ethernet-segment
No sap entries
No sdp entries
No vxlan instance entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
Pw-Port Eth-Seg Status
-------------------------------------------------------------------------------
23 SA-ES-23 NDF
===============================================================================
On NDF PE-2, the PW port is operationally down with flag StandbyForMHProtocol:
*A:PE-2# show pw-port 23 detail
===============================================================================
PW Port Information
===============================================================================
PW Port : 23
Encap : dot1q
SDP : 10001
IfIndex : 1526726679
VC-Id : 100023
Description : PW Port
Dot1Q Ethertype : 0x8100
Service Id : 123
Down on Peer Tldp PW Status Faults: No
Oper Up on MH Standby : No
===============================================================================
===============================================================================
Service Destination Point (Sdp Id 10001 Pw-Port 23)
===============================================================================
SDP Binding port : pxc-5.b
VC-Id : 100023 Admin Status : up
Encap : dot1q Oper Status : down
VC Type : ether
Dot1Q Ethertype : 0x8100
Control Word : Not Preferred
Entropy Label : Disabled
Service MTU : default
Admin Ingress label : 524278 Admin Egress label : 524279
Oper Flags : StandbyForMHProtocol
Monitor Oper-Group : (Not Specified)
===============================================================================
On the NDF, the PW SAPs are also brought down, as follows:
*A:PE-2# show service id "service-Epipe-1" sap
===============================================================================
SAP(Summary), Service 1
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
pw-23:1 1 1 none 1 none Up Down
-------------------------------------------------------------------------------
Number of SAPs : 1
-------------------------------------------------------------------------------
===============================================================================
*A:PE-2# show service id "service-Epipe-2" sap
===============================================================================
SAP(Summary), Service 2
===============================================================================
PortId SvcId Ing. Ing. Egr. Egr. Adm Opr
QoS Fltr QoS Fltr
-------------------------------------------------------------------------------
pw-23:2 2 1 none 1 none Up Down
-------------------------------------------------------------------------------
Number of SAPs : 1
-------------------------------------------------------------------------------
===============================================================================
When the PW port is operationally down only due to MHStandby, the NDF still advertises AD per-EVI and AD per-ES routes. The following shows that PE-1 receives two AD per-EVI routes and two AD per-ES routes: one from the DF PE-3 and another one from the NDF PE-2:
*A:PE-1# show router bgp routes evpn auto-disc
===============================================================================
BGP Router ID:192.0.2.1 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Auto-Disc Routes
===============================================================================
Flag Route Dist. ESI NextHop
Tag Label
-------------------------------------------------------------------------------
u*>i 192.0.2.2:123 01:00:00:00:00:23:00:00:00:01 192.0.2.2
23 524281
u*>i 192.0.2.2:123 01:00:00:00:00:23:00:00:00:01 192.0.2.2
MAX-ET 0
u*>i 192.0.2.3:123 01:00:00:00:00:23:00:00:00:01 192.0.2.3
23 524283
u*>i 192.0.2.3:123 01:00:00:00:00:23:00:00:00:01 192.0.2.3
MAX-ET 0
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
- an ESI-label extended community with the multihomed mode (single-active) and an ESI label
- an SRv6 L2 service TLV with:
- an SRv6 SID value of zero (the locator, function, and argument equal zero)
- the used endpoint behavior code point 0x18 for End.DT2M
# on PE-1:
27 2023/11/23 06:34:24.487 UTC MINOR: DEBUG #2001 Base Peer 1: 2001:db8::2:5
"Peer 1: 2001:db8::2:5: UPDATE
Peer 1: 2001:db8::2:5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 127
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:123 ESI: 01:00:00:00:00:23:00:00:00:01, tag: MAX-ET Label: 0 (Raw Label: 0x0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.2
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
target:64500:123
esi-label:3/Single-Active
Flag: 0xc0 Type: 40 Len: 37 Prefix-SID-attr:
SRv6 Services TLV (37 bytes):-
Type: SRV6 L2 Service TLV (6)
Length: 34 bytes, Reserved: 0x0
SRv6 Service Information Sub-TLV (33 bytes)
Type: 1 Len: 30 Rsvd1: 0x0
SRv6 SID: ::
SID Flags: 0x0 Endpoint Behavior: 0x18 Rsvd2: 0x0
SRv6 SID Sub-Sub-TLV
Type: 1 Len: 6
BL:0 NL:0 FL:0 AL:0 TL:0 TO:0
"
PE-1 receives the following AD per-EVI with RD 192.0.2.3:123 of the stitching Epipe from primary (P: 1) node PE-3:
# on PE-1:
31 2023/11/23 06:34:27.499 UTC MINOR: DEBUG #2001 Base Peer 1: 2001:db8::2:5
"Peer 1: 2001:db8::2:5: UPDATE
Peer 1: 2001:db8::2:5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 127
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-AD Len: 25 RD: 192.0.2.3:123 ESI: 01:00:00:00:00:23:00:00:00:01, tag: 23 Label: 8388528 (Raw Label: 0x7fffb0) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.3
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
target:64500:123
l2-attribute:MTU: 1514 F: 0 C: 0 P: 1 B: 0
Flag: 0xc0 Type: 40 Len: 37 Prefix-SID-attr:
SRv6 Services TLV (37 bytes):-
Type: SRV6 L2 Service TLV (6)
Length: 34 bytes, Reserved: 0x0
SRv6 Service Information Sub-TLV (33 bytes)
Type: 1 Len: 30 Rsvd1: 0x0
SRv6 SID: 2001:db8:aaaa:103::
SID Flags: 0x0 Endpoint Behavior: 0x15 Rsvd2: 0x0
SRv6 SID Sub-Sub-TLV
Type: 1 Len: 6
BL:48 NL:16 FL:20 AL:0 TL:20 TO:64
"
PE-1 receives the following AD per-EVI with RD 192.0.2.2:123 of the stitching Epipe from backup (B: 1) node PE-2:
# on PE-1:
33 2023/11/23 06:34:27.522 UTC MINOR: DEBUG #2001 Base Peer 1: 2001:db8::2:5
"Peer 1: 2001:db8::2:5: UPDATE
Peer 1: 2001:db8::2:5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 127
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-AD Len: 25 RD: 192.0.2.2:123 ESI: 01:00:00:00:00:23:00:00:00:01, tag: 23 Label: 8388496 (Raw Label: 0x7fff90) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.2
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
target:64500:123
l2-attribute:MTU: 1514 F: 0 C: 0 P: 0 B: 1
Flag: 0xc0 Type: 40 Len: 37 Prefix-SID-attr:
SRv6 Services TLV (37 bytes):-
Type: SRV6 L2 Service TLV (6)
Length: 34 bytes, Reserved: 0x0
SRv6 Service Information Sub-TLV (33 bytes)
Type: 1 Len: 30 Rsvd1: 0x0
SRv6 SID: 2001:db8:aaaa:102::
SID Flags: 0x0 Endpoint Behavior: 0x15 Rsvd2: 0x0
SRv6 SID Sub-Sub-TLV
Type: 1 Len: 6
BL:48 NL:16 FL:20 AL:0 TL:20 TO:64
"
- an SRv6 SID value of 2001:db8:aaaa:103:: with:
- block length (BL) 48
- node length (NL) 16
- function length (FL) 20
- argument length (AL) 0
- transposition length (TL) 20 (for EVPN and IP-VPN) - transposition of 20 bits of the function field to the ESI label field
- transposition offset (TO) 0
- the used endpoint behavior code point 0x15 for End.DX2
Conclusion
EVPN-VPWS multihoming on PW ports is supported for all-active and for single-active multihoming. The transport on the stitching (and service) Epipe services can be MPLS or SRv6.