Flow-Aware Transport (FAT) Label Signaling in L2VPN and EVPN Services
This chapter provides information about Flow-Aware Transport (FAT) label signaling in BGP Layer 2 services and BGP EVPN services.
Topics in this chapter include:
Applicability
The information and the configuration in this chapter are based on SR OS Release 24.3.R1. FAT label signaling in BGP VPLS and BGP VPWS is supported in SR OS Release 22.10.R2 and later. FAT label signaling in EVPN VPLS and EVPN VPWS is supported in SR OS Release 23.10.R1 and later.
Overview
Some operators require the use of the FAT label, also known as the hash label, as an alternative to entropy label in order to interoperate with other vendors. The hash label adds entropy to divisible flows and creates load-balancing for unicast flows in the network. The hash label uses only one additional label, whereas the entropy label adds two labels: the entropy label indicator and the entropy label itself, as described in the Entropy Label chapter in the 7450 ESS, 7750 SR, and 7950 XRS MPLS Advanced Configuration Guide for Classic CLI. The hash label is mutually exclusive with the entropy label. The hash label works end-to-end regardless of the transport tunnels. The packet sequencing is preserved because one hash label is used per flow.
- the T and R control flags as per RFC 8395 in BGP L2VPN routes
- the F control flag as per RFC 7342bis in BGP EVPN routes.
Hash label signaling in BGP VPLS and BGP VPWS services
Control flags in the layer 2 extended community shows the control flags in the Layer 2 extended community, with T for transmit capability and R for receive capability for the hash label. The C and S control flags are beyond the scope of this chapter.
RFC 8395 extends the BGP L2VPN route signaling to indicate the capability of a node to send and receive the hash label at the bottom of the stack on a per pseudowire (PW) basis. BGP VPLS and BGP VPWS services use PW templates, that can be configured with the hash-label attribute with or without the signal-capability option.
*A:PE-2>config>service>pw-template# hash-label ?
- hash-label [signal-capability]
- no hash-label
<signal-capability> : keyword
When the PW template is configured with hash-label only, the control flags are signaled as T=R=0 (Flags=none) and the hash label is sent and expected to be received in all the packets.
When the PW template is configured with hash-label signal-capability, the control flags are signaled as T=R=1. The dynamic signaling of the capability to transmit or receive hash labels is required in mixed environments where different vendors in the same service may support the hash label or not.
For example, PE-1 signals T=R=1 to PE-2 and PE-2 signals T=R=0 to PE-1. When PE-1 receives the information that PE-2 is not capable of transmitting or receiving hash labels, PE-1 sends packets without a hash label to PE-2 and PE-1 expects packets without a hash label from PE-2. If PE-2 sends packets with a hash label while PE-2 signaled T=R=0, PE-1 drops these packets.
In principle, it is possible that a node signals different values for the T and R flag, for example, T=1 and R=0 (or T=0 and R=1), but in either case, no traffic is possible toward such a node. SR OS does not support hash label asymmetry between peers. If peer PE-3 signals R=1 to PE-1, PE-1 sends packets with a hash label to PE-3 and expects packets with hash label from PE-3, regardless of the value of the received T flag. PE-1 drops packets without a hash label received from PE-3. If peer PE-4 signals R=0 to PE-1, PE-1 sends packets without a hash label to PE-4 and PE-1 expects packets without a hash label from PE-4. PE-1 drops packets with a hash label received from PE-4.
Hash label signaling in EVPN VPLS and EVPN VPWS services
Control flags in the EVPN Layer 2 attributes community shows the control flags in the EVPN Layer 2 attributes community, with F for the presence of a flow label. The C control flag is used for the presence of a control word and C=1 in one of the configuration examples in this chapter; the P and B control flags are described in the EVPN for MPLS Tunnels in Epipe Services (EVPN-VPWS) chapter.
When the F-bit is set to 1, the PE announces the capability of both sending and receiving hash labels for unicast.
The following command enables the hash label in an EVPN VPWS service:
configure
service
epipe "EVPN-VPWS-4"
bgp-evpn
mpls
hash-label
The following command enables the hash label in an EVPN VPLS service:
configure
service
vpls "EVPN-VPLS-3"
bgp-evpn
mpls
ingress-replication-bum-label
hash-label
The following error message is raised when attempting to enable the hash label in an EVPN VPLS service without ingress-replication-bum-label:
*A:PE-2>config>service>vpls>bgp-evpn>mpls# hash-label
MINOR: SVCMGR #7886 cannot modify evpn - cannot support hash-label without ingress-replication-bum-label
The incl-mcast-l2-attributes-advertisement command enables the advertisement of the EVPN Layer 2 attributes enhanced community in Inclusive Multicast Ethernet Tag (IMET) routes for the EVPN VPLS:
configure
service
vpls "EVPN-VPLS-3"
bgp-evpn
incl-mcast-l2-attributes-advertisement
For EVPN VPWS services, the Layer 2 attributes are signaled in the AD per-EVI routes. No IMET routes are sent for EVPN VPWS services, so the incl-mcast-l2-attributes-advertisement command is not supported in Epipes.
When a node receives an IMET (for EVPN VPLS) or a BGP-EVPN AD per-EVI (for EVPN-VPWS) route that contains the F bit, the node compares the received F bit with the local configuration for the hash label. In case of a mismatch, the node brings down the EVPN destination in case of EVPN VPLS or removes the EVPN destination in case of EVPN VPWS.
Besides the F flag, the Layer 2 attributes community also includes the service MTU and the control word. A node compares the received MTU with the local service MTU. In case of an MTU mismatch, the node brings down the EVPN destination in case of EVPN VPLS or removes the EVPN destination in case of EVPN VPWS, unless the ignore-mtu-mismatch command is configured. The control word flag C is set by the advertising PE when the control-word command is configured in the service. A node compares the received C flag with the local configuration. In case of a mismatch, the node brings down the EVPN destination in case of EVPN VPLS or removes the EVPN destination in case of EVPN VPWS.
Configuration
Example topology shows the example topology with four PEs and one route reflector (RR) in AS 64496.
- cards, MDAs, ports
- router interfaces
- ECMP = 2 in the base router of all PEs
- IS-IS between all nodes
- LDP between all PEs, not toward the RR
- Hash label signaling in BGP VPLS and BGP VPWS
- BGP VPLS "BGP-VPLS-1"
- BGP VPWS "BGP-VPWS-2"
- Hash label signaling in EVPN VPLS and EVPN VPWS
- EVPN VPLS "EVPN-VPLS-3
- EVPN VPWS "EVPN-VPWS-4"
- EVPN R-VPLS "BD-5" in IES "IES-55"
Hash label signaling in BGP VPLS and BGP VPWS
BGP is configured for the L2VPN address family, as follows:
# on PE-1, PE-2, PE-3, PE-4;
configure
router Base
autonomous-system 64496
bgp
rapid-withdrawal
split-horizon
rapid-update l2-vpn
group "internal"
peer-as 64496
neighbor 192.0.2.5 # RR-5
family l2-vpn
exit
exit
- PW template "PW-1" in BGP VPLS and BGP VPWS without hash label
- PW template "PW-2-hash-label-only" in BGP VPLS and BGP VPWS with hash label only
- PW template "PW-3-hash-label-sign" in BGP VPLS and BGP VPWS with hash label and hash label signaling
All of these PW templates are used on different nodes in Different hash label configuration in BGP-VPLS-1 on different nodes.
BGP VPLS and BGP VPWS without hash label
PW template "PW-1" is configured with split-horizon-group (SHG) "SHG-1" but without hash label, as follows:
# on PE-1, PE-2, PE-3, PE-4:
configure
service
pw-template 1 name "PW-1" create
split-horizon-group "SHG-1"
exit
exit
The following services on PE-1 use PW template "PW-1":
# on PE-1:
configure
service
vpls 1 name "BGP-VPLS-1" customer 1 create
bgp
route-distinguisher 192.0.2.1:1
route-target export target:64496:1 import target:64496:1
pw-template-binding 1
exit
exit
bgp-vpls
max-ve-id 10
ve-name "PE-1"
ve-id 1
exit
no shutdown
exit
stp
shutdown
exit
sap 1/1/c10/1:1 create
no shutdown
exit
no shutdown
exit
epipe 2 name "BGP-VPWS-2" customer 1 create
bgp
route-distinguisher 192.0.2.1:2
route-target export target:64496:2 import target:64496:2
pw-template-binding 1
exit
exit
bgp-vpws
ve-name "PE-1"
ve-id 1
exit
remote-ve-name "PE-4"
ve-id 4
exit
no shutdown
exit
sap 1/1/c10/1:2 create
no shutdown
exit
no shutdown
exit
The configuration is similar on the other PEs. The BGP VPLS is configured on all PEs; the BGP VPWS only on PE-1 and PE-4.
The following BGP VPLS routes are received on PE-4:
*A:PE-4# show router bgp routes l2-vpn bgp-vpls
===============================================================================
BGP Router ID:192.0.2.4 AS:64496 Local AS:64496
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP L2VPN-VPLS Routes
===============================================================================
Flag RouteType Prefix MED
RD SiteId Label
Nexthop VeId BlockSize LocalPref
As-Path BaseOffset vplsLabelBa
se
-------------------------------------------------------------------------------
u*>i VPLS - - 0
192.0.2.1:1 - -
192.0.2.1 1 8 100
No As-Path 1 524276
u*>i VPLS - - 0
192.0.2.2:1 - -
192.0.2.2 2 8 100
No As-Path 1 524276
u*>i VPLS - - 0
192.0.2.3:1 - -
192.0.2.3 3 8 100
No As-Path 1 524276
-------------------------------------------------------------------------------
Routes : 3
===============================================================================
The following BGP VPWS route is received on PE-4:
*A:PE-4# show router bgp routes l2-vpn bgp-vpws
===============================================================================
BGP Router ID:192.0.2.4 AS:64496 Local AS:64496
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP L2VPN-VPWS Routes
===============================================================================
Flag RouteType Prefix MED
RD SiteId Label
Nexthop VeId BlockSize LocalPref
As-Path BaseOffset vplsLabelBa
se
-------------------------------------------------------------------------------
u*>i VPWS - - 0
192.0.2.1:2 - -
192.0.2.1 1 1 100
No As-Path 4 524275
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
PW template 1 does not have hash label enabled. The following command shows that the hash label is disabled on all SDPs in BGP-VPLS-1:
*A:PE-1# show service id 1 sdp detail | match Hash
Hash Label : Disabled Hash Lbl Sig Cap : Disabled
Oper Hash Label : Disabled
Hash Label : Disabled Hash Lbl Sig Cap : Disabled
Oper Hash Label : Disabled
Hash Label : Disabled Hash Lbl Sig Cap : Disabled
Oper Hash Label : Disabled
The following command shows that the hash label is disabled on the SDP in BGP-VPWS-2:
*A:PE-1# show service id 2 sdp detail | match Hash
Hash Label : Disabled Hash Lbl Sig Cap : Disabled
Oper Hash Label : Disabled
BGP VPLS and BGP VPWS with hash label only
The following command configures PW template "PW-2-hash-label-only" with SHG and hash label, but without hash label signaling capability:
# on PE-1, PE-2, PE-3, PE-4:
configure
service
pw-template 2 name "PW-2-hash-label-only" create
hash-label
split-horizon-group "SHG-1"
exit
The following command reconfigures the BGP VPLS and BGP VPWS services with PW template 2 instead of PW template 1:
# on PE-1:
configure
service
vpls "BGP-VPLS-1"
bgp-vpls
shutdown
exit
bgp
route-target export target:64496:1 import target:64496:1
no pw-template-binding 1
pw-template-binding 2
exit
exit
bgp-vpls
no shutdown
exit
exit
epipe "BGP-VPWS-2"
bgp-vpws
shutdown
exit
bgp
route-target export target:64496:2 import target:64496:2
no pw-template-binding 1
pw-template-binding 2
exit
exit
bgp-vpws
no shutdown
exit
exit
The configuration on the other PEs is similar. BGP-VPWS-2 is only configured on PE-1 and PE-4.
The following command shows that hash label is enabled, but the hash label signaling capability is disabled. The operational hash label is enabled only when both ends of the SDP have the hash label enabled.
*A:PE-1# show service id 1 sdp detail | match Hash
Hash Label : Enabled Hash Lbl Sig Cap : Disabled
Oper Hash Label : Enabled
Hash Label : Enabled Hash Lbl Sig Cap : Disabled
Oper Hash Label : Enabled
Hash Label : Enabled Hash Lbl Sig Cap : Disabled
Oper Hash Label : Enabled
*A:PE-1# show service id 2 sdp detail | match Hash
Hash Label : Enabled Hash Lbl Sig Cap : Disabled
Oper Hash Label : Enabled
The hash label signaling capability is disabled, so no T-bit or R-bit is advertised. The following shows that no flags are received in the L2VPN community for BGP-VPLS-1:
*A:PE-1# show router bgp routes l2-vpn community target:64496:1 detail | match Community context all
Community : target:64496:1
l2-vpn/vrf-imp:Encap=19: Flags=none: MTU=1514: PREF=0
Community : target:64496:1
l2-vpn/vrf-imp:Encap=19: Flags=none: MTU=1514: PREF=0
Community : target:64496:1
l2-vpn/vrf-imp:Encap=19: Flags=none: MTU=1514: PREF=0
Community : target:64496:1
l2-vpn/vrf-imp:Encap=19: Flags=none: MTU=1514: PREF=0
Community : target:64496:1
l2-vpn/vrf-imp:Encap=19: Flags=none: MTU=1514: PREF=0
Community : target:64496:1
l2-vpn/vrf-imp:Encap=19: Flags=none: MTU=1514: PREF=0
Similarly, no flags are received in the L2VPN community for BGP-VPWS-2:
*A:PE-1# show router bgp routes l2-vpn community target:64496:2 detail | match Community context all
Community : target:64496:2
l2-vpn/vrf-imp:Encap=5: Flags=none: MTU=1514: PREF=0
Community : target:64496:2
l2-vpn/vrf-imp:Encap=5: Flags=none: MTU=1514: PREF=0
BGP VPLS and BGP VPWS with hash label and hash label signaling
The following command configures PW template "PW-3-hash-label-sign" with SHG, hash label, and hash label signal capability. This PW template is applied in BGP-VPLS-1 and BGP-VPWS-2.
# on PE-1, PE-4:
configure
service
pw-template 3 name "PW-3-hash-label-sign" create
hash-label signal-capability
split-horizon-group "SHG-1"
exit
exit
vpls "BGP-VPLS-1"
bgp-vpls
shutdown
exit
bgp
route-target export target:64496:1 import target:64496:1
no pw-template-binding 2
pw-template-binding 3
exit
exit
bgp-vpls
no shutdown
exit
exit
epipe "BGP-VPWS-2"
bgp-vpws
shutdown
exit
bgp
route-target export target:64496:2 import target:64496:2
no pw-template-binding 2
pw-template-binding 3
exit
exit
bgp-vpws
no shutdown
exit
exit
The configuration on PE-2 and PE-3 is similar for BGP-VPLS-1; BGP-VPWS-2 is only configured on PE-1 and PE-4.
The following shows that hash label and hash label signaling capability is supported on all SDPs in BGP-VPLS-1 on PE-1:
*A:PE-1# show service id 1 sdp detail | match Hash
Hash Label : Enabled Hash Lbl Sig Cap : Enabled
Oper Hash Label : Enabled
Hash Label : Enabled Hash Lbl Sig Cap : Enabled
Oper Hash Label : Enabled
Hash Label : Enabled Hash Lbl Sig Cap : Enabled
Oper Hash Label : EnabledLikewise, hash label and hash label capability signaling is enabled on the SDP in BGP-VPWS-2 on PE-1:
*A:PE-1# show service id 2 sdp detail | match Hash
Hash Label : Enabled Hash Lbl Sig Cap : Enabled
Oper Hash Label : Enabled
PE-1 receives the following BGP L2VPN route for BGP-VPLS-1 with T=R=1 (Flags=-T-R) from PE-4:
140 2024/09/04 14:14:12.097 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 86
Flag: 0x90 Type: 14 Len: 28 Multiprotocol Reachable NLRI:
Address Family L2VPN
NextHop len 4 NextHop 192.0.2.4
[VPLS/VPWS] preflen 17, veid: 4, vbo: 1, vbs: 8, label-base: 524253, RD 192.0.2.4:1
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x80 Type: 4 Len: 4 MED: 0
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
target:64496:1
l2-vpn/vrf-imp:Encap=19: Flags=-T-R: MTU=1514: PREF=0 # encap 19 --> BGP VPLS
"PE-1 receives the following BGP L2VPN route for BGP-VPWS-2 with T=R=1 from PE-4:
143 2024/09/04 14:14:12.098 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 90
Flag: 0x90 Type: 14 Len: 32 Multiprotocol Reachable NLRI:
Address Family L2VPN
NextHop len 4 NextHop 192.0.2.4
[VPLS/VPWS] preflen 21, veid: 4, vbo: 1, vbs: 1, label-base: 524279, RD 192.0.2.4:2, csv: 0x00000000, type 1, len 1,
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x80 Type: 4 Len: 4 MED: 0
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
target:64496:2
l2-vpn/vrf-imp:Encap=5: Flags=-T-R: MTU=1514: PREF=0 # encap 5 --> BGP VPWS
"
The following command includes the T and R flags for the different SDPs in the L2 BGP-VPLS route information:
*A:PE-4# show service id "BGP-VPLS-1" l2-route-table bgp-vpls detail
===============================================================================
Services: L2 Bgp-Vpls Route Information - Service 1
===============================================================================
VeId : 1
PW Temp Id : 3
RD : *192.0.2.1:1
Next Hop : 192.0.2.1
State (D-Bit) : up(0)
Path MTU : 1514
Hash Label Tx : 1
Hash Label Rx : 1
Control Word : 0
Seq Delivery : 0
DF Bit : clear
Status : active
Sdp Bind Id : 32752:4294967275
VeId : 2
PW Temp Id : 3
RD : *192.0.2.2:1
Next Hop : 192.0.2.2
State (D-Bit) : up(0)
Path MTU : 1514
Hash Label Tx : 1
Hash Label Rx : 1
Control Word : 0
Seq Delivery : 0
DF Bit : clear
Status : active
Sdp Bind Id : 32750:4294967273
VeId : 3
PW Temp Id : 3
RD : *192.0.2.3:1
Next Hop : 192.0.2.3
State (D-Bit) : up(0)
Path MTU : 1514
Hash Label Tx : 1
Hash Label Rx : 1
Control Word : 0
Seq Delivery : 0
DF Bit : clear
Status : active
Sdp Bind Id : 32749:4294967272
===============================================================================
The following command includes the T and R flags in the L2 BGP-VPWS route information:
*A:PE-4# show service id "BGP-VPWS-2" l2-route-table bgp-vpws detail
===============================================================================
Services: L2 Bgp-Vpws Route Information - Service 2
===============================================================================
VeId : 1
PW Temp Id : 3
RD : *192.0.2.1:2
Next Hop : 192.0.2.1
State (D-Bit) : up(0)
Path MTU : 1514
Hash Label Tx : 1
Hash Label Rx : 1
Control Word : 0
Seq Delivery : 0
Status : active
Tx Status : active
CSV : 0
Preference : 0
Sdp Bind Id : 32751:4294967274
===============================================================================
Different hash label configuration in BGP-VPLS-1 on different nodes
BGP-VPLS-1 is reconfigured on PE-2 and PE-3 with different PW templates. The following command reconfigures BGP-VPLS-1 with PW template 1 on PE-2:
# on PE-2:
configure
service
vpls "BGP-VPLS-1"
bgp-vpls
shutdown
exit
bgp
no pw-template-binding 3
pw-template-binding 1 # PW template 1 is without hash-label
exit
exit
bgp-vpls
no shutdown
exit
- PW template 3 (hash label with hash label signaling capability) on PE-1 and PE-4
- PW template 1 (no hash label) on PE-2
- PW template 2 (hash label without hash label signaling capability) on PE-3
BGP-VPLS-1 on PE-1 (hash label with hash label signaling capability) has connectivity with BGP-VPLS-1 on PE-2 (no hash label):
*A:PE-1# ping router-instance "CE-11" 172.16.1.2 rapid
PING 172.16.1.2 56 data bytes
!!!!!
---- 172.16.1.2 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.01ms, avg = 3.91ms, max = 7.33ms, stddev = 1.71ms
BGP-VPLS-1 on PE-2 is configured without hash label, so PE-2 always sends and expects to receive frames without a hash label. Even though BGP-VPLS-1 on PE-1 is configured with hash label and hash label signalling capability, when PE-1 receives T=R=0 (Flags:none) from PE-2, PE-1 adapts to send and expect to receive frames without a hash label to and from PE-2.
BGP-VPLS-1 on PE-1 (hash label with hash label signaling capability) has no connectivity with BGP-VPLS-1 on PE-3 (hash label only):
*A:PE-1# ping router-instance "CE-11" 172.16.1.3 rapid
PING 172.16.1.3 56 data bytes
.....
---- 172.16.1.3 PING Statistics ----
5 packets transmitted, 0 packets received, 100% packet loss
BGP-VPLS-1 on PE-3 is configured with hash label only, so PE-3 always sends and expects to receive frames with a hash label. PE-3 has no signaling capability, so PE-3 sends T=R=0 (Flags:none). Even though BGP-VPLS-1 on PE-1 is configured with hash label and hash label signalling capability, when PE-1 receives T=R=0 from PE-3, PE-1 adapts to send and expect to receive frames without a hash label to and from PE-3. So, there is a mismatch, causing PE-1 to drop the frames with a hash label from PE-3 and PE-3 to drop the frames without a hash label from PE-1.
From BGP-VPLS-1 on PE-3 (hash label only), there is no connectivity to PE-1 (hash label with hash label signaling capability) or to PE-2 (no hash label):
*A:PE-3# ping router-instance "CE-13" 172.16.1.1 rapid
PING 172.16.1.1 56 data bytes
.....
---- 172.16.1.1 PING Statistics ----
5 packets transmitted, 0 packets received, 100% packet loss
*A:PE-3# ping router-instance "CE-13" 172.16.1.2 rapid
PING 172.16.1.2 56 data bytes
.....
---- 172.16.1.2 PING Statistics ----
5 packets transmitted, 0 packets received, 100% packet lossBGP-VPLS-1 on PE-2 is configured without hash label, so PE-2 always sends and expects to receive frames without a hash label. PE-3 is configured with hash label only, so it always sends and expects to receive frames with a hash label. This is a mismatch and the frames are dropped.
Hash label signaling in EVPN VPLS and EVPN VPWS
BGP is configured for the EVPN address family, as follows:
# on PE-1, PE-2, PE-3, PE-4:
configure
router Base
autonomous-system 64496
bgp
rapid-withdrawal
split-horizon
rapid-update evpn
group "internal"
peer-as 64496
neighbor 192.0.2.5 # RR-5
family evpn
exit
exit
- EVPN VPLS and EVPN VPWS services without hash label where the hash label is disabled
- EVPN VPWS services with hash label and hash label signaling capability where the F flag is set in the AD per-EVI route
- EVPN VPLS services with hash label only where the hash label is enabled, but the F flag is not included in the IMET route
- EVPN VPLS services with hash label and hash label signaling capability where the F flag is set in the IMET route
- Different hash label configuration in EVPN-VPLS-3 on different nodes
EVPN VPLS and EVPN VPWS services without hash label
The following EVPN VPLS and EVPN VPWS services are configured on PE-1:
# on PE-1:
configure
service
vpls 3 name "EVPN-VPLS-3" customer 1 create
bgp
route-target export target:64496:3 import target:64496:3
exit
bgp-evpn
evi 3
mpls
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap 1/1/c10/1:3 create
no shutdown
exit
no shutdown
exit
epipe 4 name "EVPN-VPWS-4" customer 1 create
bgp
route-target export target:64496:4 import target:64496:4
exit
bgp-evpn
local-attachment-circuit PE1 create
eth-tag 1
exit
remote-attachment-circuit PE4 create
eth-tag 4
exit
evi 4
mpls
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
sap 1/1/c10/1:4 create
description "SAP to CE-41"
no shutdown
exit
no shutdown
exit
vpls 5 name "BD-5" customer 1 create
allow-ip-int-bind
exit
bgp
route-target export target:64496:5 import target:64496:5
exit
bgp-evpn
evi 5
mpls bgp 1
auto-bind-tunnel
resolution any
exit
no shutdown
exit
exit
stp
shutdown
exit
sap 1/1/c10/1:5 create
no shutdown
exit
no shutdown
exit
ies 55 name "IES-55" customer 1 create
interface "int-BD-5" create
address 172.16.51.1/24
vpls "BD-5"
exit
exit
no shutdown
exit
The configuration on PE-4 is similar. On PE-2 and PE-3, only one EVPN L2 service is configured: EVPN-VPLS-3.
PE-4 receives the following IMET routes for EVPN-VPLS-3 and BD-5:
*A:PE-4# show router bgp routes evpn incl-mcast
===============================================================================
BGP Router ID:192.0.2.4 AS:64496 Local AS:64496
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Inclusive-Mcast Routes
===============================================================================
Flag Route Dist. OrigAddr
Tag NextHop
-------------------------------------------------------------------------------
u*>i 192.0.2.1:3 192.0.2.1
0 192.0.2.1
u*>i 192.0.2.1:5 192.0.2.1
0 192.0.2.1
u*>i 192.0.2.2:3 192.0.2.2
0 192.0.2.2
u*>i 192.0.2.3:3 192.0.2.3
0 192.0.2.3
-------------------------------------------------------------------------------
Routes : 4
===============================================================================
PE-4 receives the following AD per-EVI route for EVPN-VPWS-4:
*A:PE-4# show router bgp routes evpn auto-disc
===============================================================================
BGP Router ID:192.0.2.4 AS:64496 Local AS:64496
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN Auto-Disc Routes
===============================================================================
Flag Route Dist. ESI NextHop
Tag Label
-------------------------------------------------------------------------------
u*>i 192.0.2.1:4 ESI-0 192.0.2.1
1 LABEL 524273
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The following shows that the hash label is disabled on EVPN-VPLS-3, BGP-VPWS-4, and BD-5:
*A:PE-1# show service id 3 bgp-evpn | match Hash
Hash Label : Disabled
*A:PE-1# show service id 4 bgp-evpn | match Hash
Hash Label : Disabled
*A:PE-1# show service id 5 bgp-evpn | match Hash
Hash Label : Disabled
The following command shows the EVPN destinations for EVPN-VPLS-3 on PE-1 with their operational state and operational flags:
*A:PE-1# show service id 3 evpn-mpls instance 1 detail
===============================================================================
BGP EVPN-MPLS Dest (Instance 1)
===============================================================================
TEP Address Transport:Tnl Egr Label Oper Mcast Num
State MACs
-------------------------------------------------------------------------------
192.0.2.2 ldp:65537 524275 Up bum 1
Oper Flags : None
Sup BCast Domain : No
Last Update : 10/07/2024 09:21:23
192.0.2.3 ldp:65538 524275 Up bum 1
Oper Flags : None
Sup BCast Domain : No
Last Update : 10/07/2024 09:21:29
192.0.2.4 ldp:65539 524274 Up bum 1
Oper Flags : None
Sup BCast Domain : No
Last Update : 10/07/2024 09:21:37
-------------------------------------------------------------------------------
Number of entries: 3
-------------------------------------------------------------------------------
===============================================================================
---snip---
EVPN VPWS services with hash label and hash label signaling capability
The following command enables the hash label in BGP-VPWS-4:
# on PE-1, PE-4:
configure
service
epipe "EVPN-VPWS-4"
bgp-evpn
mpls
hash-label
The following shows that the hash label is enabled in BGP-VPWS-4:
*A:PE-1# show service id 4 bgp-evpn | match Hash
Hash Label : Enabled
PE-1 receives the following AD per EVI route from originator PE-4:
100 2024/09/04 14:06:09.098 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 95
Flag: 0x90 Type: 14 Len: 36 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-AD Len: 25 RD: 192.0.2.4:4 ESI: ESI-0, tag: 4 Label: 8388368 (Raw Label: 0x7fff10) PathId:
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64496:4
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
"
The L2 attributes community includes the MTU and the control flags. The flow label F flag is set to 1. When the hash label is enabled, it is advertised in the AD per-EVI route.
EVPN VPLS services with hash label only
The hash label is configured in EVPN-VPLS-3 and BD-5, as follows:
# on PE-1:
configure
service
vpls "EVPN-VPLS-3"
bgp-evpn
mpls
ingress-replication-bum-label # required for hash label
hash-label
no shutdown
exit
exit
info
exit
vpls "BD-5"
bgp-evpn
mpls
control-word # optional
ingress-replication-bum-label # required for hash label
hash-label
no shutdown
exit
exit
The following shows that the hash label is enabled in EVPN-VPLS-3 and BD-5:
*A:PE-1# show service id 3 bgp-evpn | match Hash
Hash Label : Enabled
*A:PE-1# show service id 5 bgp-evpn | match Hash
Hash Label : Enabled
The hash label is enabled, but the flow label control flag F is not advertised. The following IMET route for EVPN-VPLS-3 shows that there is no Layer 2 attributes community in the extended community:
101 2024/09/04 14:06:09.098 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 91
Flag: 0x90 Type: 14 Len: 28 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-INCL-MCAST Len: 17 RD: 192.0.2.4:3, tag: 0, orig_addr len: 32, orig_addr: 192.0.2.4
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
target:64496:3
bgp-tunnel-encap:MPLS
Flag: 0xc0 Type: 22 Len: 9 PMSI:
Tunnel-type Ingress Replication (6)
Flags: (0x0)[Type: None BM: 0 U: 0 Leaf: not required]
MPLS Label 8388512
Tunnel-Endpoint 192.0.2.4
"
EVPN VPLS services with hash label and hash label signaling capability
The following command is configured on the PEs to include the Layer 2 attributes community in the IMET routes:
# on PE-1:
configure
service
vpls "EVPN-VPLS-3"
bgp-evpn
incl-mcast-l2-attributes-advertisement
exit
exit
vpls "BD-5"
bgp-evpn
incl-mcast-l2-attributes-advertisement
exit
exit
The Layer 2 attributes community is included in the IMET routes. The flow label control flag is set to 1 for EVPN-VPLS-3, as follows:
*A:PE-1# show router bgp routes evpn incl-mcast community target:64496:3 detail | match Community context all
Community : target:64496:3
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
Community : target:64496:3
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
Community : target:64496:3
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
Community : target:64496:3
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
Community : target:64496:3
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
Community : target:64496:3
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
The flow label control flag is set to 1 for EVPN R-VPLS BD-5, as follows:
*A:PE-1# show router bgp routes evpn incl-mcast community target:64496:5 detail | match Community context all
Community : target:64496:5
l2-attribute:MTU: 1514 F: 1 C: 1 P: 0 B: 0
bgp-tunnel-encap:MPLS
Community : target:64496:5
l2-attribute:MTU: 1514 F: 1 C: 1 P: 0 B: 0
bgp-tunnel-encap:MPLS
PE-1 receives the following IMET route with Layer 2 attributes community for EVPN-VPLS-3:
148 2024/09/04 14:14:32.434 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.5
"Peer 1: 192.0.2.5: UPDATE
Peer 1: 192.0.2.5 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 99
Flag: 0x90 Type: 14 Len: 28 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-INCL-MCAST Len: 17 RD: 192.0.2.4:3, tag: 0, orig_addr len: 32, orig_addr: 192.0.2.4
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0x80 Type: 9 Len: 4 Originator ID: 192.0.2.4
Flag: 0x80 Type: 10 Len: 4 Cluster ID:
192.0.2.5
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64496:3
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0
bgp-tunnel-encap:MPLS
Flag: 0xc0 Type: 22 Len: 9 PMSI:
Tunnel-type Ingress Replication (6)
Flags: (0x0)[Type: None BM: 0 U: 0 Leaf: not required]
MPLS Label 8388512
Tunnel-Endpoint 192.0.2.4
"
The following command shows the operational state "Up" and operational flags "None" for the EVPN destinations for EVPN-VPLS-3 on PE-1:
*A:PE-1# show service id 3 evpn-mpls instance 1 detail
===============================================================================
BGP EVPN-MPLS Dest (Instance 1)
===============================================================================
TEP Address Transport:Tnl Egr Label Oper Mcast Num
State MACs
-------------------------------------------------------------------------------
192.0.2.2 ldp:65537 524273 Up bum 0
Oper Flags : None
Sup BCast Domain : No
Last Update : 10/07/2024 09:23:53
192.0.2.3 ldp:65538 524273 Up bum 0
Oper Flags : None
Sup BCast Domain : No
Last Update : 10/07/2024 09:23:03
192.0.2.4 ldp:65539 524268 Up bum 0
Oper Flags : None
Sup BCast Domain : No
Last Update : 10/07/2024 09:23:18
-------------------------------------------------------------------------------
Number of entries: 3
-------------------------------------------------------------------------------
===============================================================================
---snip---
The operational flags will indicate which problems occur in case of a mismatch in hash label configuration on the different PEs, as shown in the following section.
Different hash label configuration in EVPN-VPLS-3 on different nodes
The hash label configuration on PE-2 is removed:
# on PE-2:
configure
service
vpls "EVPN-VPLS-3"
bgp-evpn
mpls
no hash-label
On PE-3, the hash label remains enabled in EVPN-VPLS-3, but the Layer 2 attributes community is not advertised:
# on PE-3:
configure
service
vpls "EVPN-VPLS-3"
bgp-evpn
no incl-mcast-l2-attributes-advertisement
- on PE-1 and PE-4; hash label enabled and Layer 2 attributes community advertised
- on PE-2: no hash label configured
- on PE-3: hash label enabled, but no Layer 2 attributes community advertised
The following ping commands show that it is only possible to ping between PE-1 and PE-4, but not between PE-1 and PE-2 or between PE-1 and PE-3.
*A:PE-1# ping router-instance "CE-31" 172.16.3.2 rapid
PING 172.16.3.2 56 data bytes
.....
---- 172.16.3.2 PING Statistics ----
5 packets transmitted, 0 packets received, 100% packet loss
*A:PE-1# ping router-instance "CE-31" 172.16.3.3 rapid
PING 172.16.3.3 56 data bytes
.....
---- 172.16.3.3 PING Statistics ----
5 packets transmitted, 0 packets received, 100% packet loss
*A:PE-1# ping router-instance "CE-31" 172.16.3.4 rapid
PING 172.16.3.4 56 data bytes
!!!!!
---- 172.16.3.4 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.16ms, avg = 4.92ms, max = 10.0ms, stddev = 2.56ms
Similarly, it is possible to ping between PE-4 and PE-1, but not between PE-4 and PE-2 or between PE-4 and PE-3.
It is not possible to ping between PE-2 and PE-1, between PE-2 and PE-3, or between PE-2 and PE-4:
*A:PE-2# ping router-instance "CE-32" 172.16.3.1 rapid
PING 172.16.3.1 56 data bytes
.....
---- 172.16.3.1 PING Statistics ----
5 packets transmitted, 0 packets received, 100% packet loss
*A:PE-2# ping router-instance "CE-32" 172.16.3.3 rapid
PING 172.16.3.3 56 data bytes
.....
---- 172.16.3.3 PING Statistics ----
5 packets transmitted, 0 packets received, 100% packet loss
*A:PE-2# ping router-instance "CE-32" 172.16.3.4 rapid
PING 172.16.3.4 56 data bytes
.....
---- 172.16.3.4 PING Statistics ----
5 packets transmitted, 0 packets received, 100% packet loss
PE-1 and PE-4 advertise F=1 in the IMET routes while PE-2 advertises F=0 and PE-3 does not advertise the F flag, as follows:
*A:PE-4# show router bgp routes evpn incl-mcast community target:64496:3 hunt | match Community post-lines 5
Community : target:64496:3
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0 # originator PE-1
bgp-tunnel-encap:MPLS
Cluster : 192.0.2.5
Originator Id : 192.0.2.1 Peer Router Id : 192.0.2.5
Origin : IGP
Community : target:64496:3
l2-attribute:MTU: 1514 F: 0 C: 0 P: 0 B: 0 # originator PE-2
bgp-tunnel-encap:MPLS
Cluster : 192.0.2.5
Originator Id : 192.0.2.2 Peer Router Id : 192.0.2.5
Origin : IGP
Community : target:64496:3 bgp-tunnel-encap:MPLS # originator PE-3
Cluster : 192.0.2.5
Originator Id : 192.0.2.3 Peer Router Id : 192.0.2.5
Origin : IGP
Flags : Used Valid Best
Route Source : Internal
Community : target:64496:3
l2-attribute:MTU: 1514 F: 1 C: 0 P: 0 B: 0 # originator PE-4
bgp-tunnel-encap:MPLS
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 192.0.2.5
Origin : IGP
In case of a mismatch in F flag, the receiving node brings down the EVPN destination for EVPN VPLS. On PE-4, the only EVPN destination that is operationally up is PE-1:
*A:PE-4# show service id 3 evpn-mpls
===============================================================================
BGP EVPN-MPLS Dest (Instance 1)
===============================================================================
TEP Address Transport:Tnl Egr Label Oper Mcast Num
State MACs
-------------------------------------------------------------------------------
192.0.2.1 ldp:65538 524285 Up bum 0
192.0.2.2 ldp:65537 524285 Down bum 0
192.0.2.3 ldp:65539 524285 Down bum 0
-------------------------------------------------------------------------------
Number of entries: 3
-------------------------------------------------------------------------------
===============================================================================
---snip---
The detailed information for the preceding EVPN destinations on PE-4 shows that there is a hash label mismatch with the F flag in the IMET received from PE-2 and that there is no L2 attributes community present in the IMET received from PE-3:
*A:PE-4# show service id 3 evpn-mpls detail
===============================================================================
BGP EVPN-MPLS Dest (Instance 1)
===============================================================================
TEP Address Transport:Tnl Egr Label Oper Mcast Num
State MACs
-------------------------------------------------------------------------------
192.0.2.1 ldp:65538 524285 Up bum 0
Oper Flags : None
Sup BCast Domain : No
Last Update : 09/05/2024 06:05:47
192.0.2.2 ldp:65537 524285 Down bum 0
Oper Flags : hashLblMismatch
Sup BCast Domain : No
Last Update : 09/05/2024 06:05:47
192.0.2.3 ldp:65539 524285 Down bum 0
Oper Flags : noL2comm
Sup BCast Domain : No
Last Update : 09/05/2024 06:05:47
-------------------------------------------------------------------------------
Number of entries: 3
-------------------------------------------------------------------------------
===============================================================================
---snip---
Conclusion
BGP L2VPN and BGP EVPN support hash label to enable a more efficient load balancing in the MPLS network. BGP L2VPN and BGP EVPN routes can signal the hash label capability dynamically, which is useful in networks where not all nodes support hash label.