Entropy Label
This chapter provides information about the Entropy Label (EL).
Topics in this chapter include:
Applicability
This chapter was initially written for SR OS Release 14.0.R4. The CLI in the current edition corresponds to SR OS Release 19.10.R1.
RFC 6391 hash label or flow-aware transport label is supported in SR OS Release 8.0.R1, and later. RFC 6790 Entropy Labels (ELs) are supported on RSVP and BGP tunnels in Release 14.0.R1, and later, and supported on LDP tunnels in Release 14.0.R4, and later.
Overview
Entropy is the degree of disorder or uncertainty in a system. SR OS supports both the MPLS EL and the hash label, but they are mutually exclusive. These labels allow Label Switched Routers (LSRs) to load-balance labeled packets in a granular way without the need to inspect the IPv4 or IPv6 header. The hash label is applicable to services such as Epipe VLL, VPLS, IES (spoke-SDP), and VPRN services. The main advantage of the EL compared to the hash label is that the EL can be applied to a wider range of services, such as EVPN, BGP VPWS, Fpipe VLL, Ipipe VLL, H-VPLS, and BGP RFC 3107 tunnels.
Load-balancing of flows based on hash label or entropy label shows that different flows from an ingress Label Edge Router (iLER) are load-balanced across different paths in the MPLS network toward the egress Label Edge Router (eLER).
The EL is inserted below the innermost Tunnel Label (TL) in the label stack, along with the Entropy Label Indicator (ELI), which is a special purpose MPLS label with a value of 7 to indicate that the next label in the stack is an EL. As with the hash label, the value of the EL is calculated based on a hash of the packet payload header (IP and Layer 4). The EL is inserted as deep as possible in the stack to ensure preservation for as far as possible through the network. When the EL and ELI are present, load-balancing on the transit LSRs can be configured to only take into account the EL label for Link Aggregation Group (LAG) and Equal Cost Multi-Path (ECMP). Load-balancing on the LSR can be configured to take into account the IP header also, but that is not required when the EL and ELI are present. The eLER removes the EL and ELI before forwarding the packet to its final destination.
The EL requires two additional labels in the label stack, which might result in an unsupported label stack depth in an intermediate (possibly third-party) LSR. SR OS allows control of the EL and ELI insertion and accounting of extra labels in the tunnel table. The supported label stack depth is 12 in SR OS Release 14.0, including transport, service, hash, and OAM labels.
Label stack with hash label versus label stack with EL and ELI shows a comparison between a label stack with a hash label and a label stack with an EL and ELI.
Hash labels or ELs are inserted by the iLER, based on the incoming packet header. The iLER uses a single label to represent one or more flows, based on the hash of the incoming packet header IP and Layer 4. The hash label is a single label that is inserted at the bottom of the stack, below the service label. The EL is inserted together with the ELI below the innermost tunnel label, but above the service label. The entropy and hash label value is outside of the range for MPLS label values, because the most significant bit equals 1 for entropy and hash labels and 0 for MPLS labels.
The EL is used as part of the LSR hashing algorithm for spraying packets over multi-port LAGs, ECMP, or BGP tunnels with multiple downstream interfaces. The packet ordering is preserved because one label is used per conversation. Hashing based on a label stack containing an EL per service will become more granular than when based on a standard label stack alone.
Downstream LERs signal EL capability to ILER shows how the eLER signals its EL capability to the iLER.
The Entropy Label Capability (ELC) is signaled by the eLER and indicates the ability to receive and process the ELs. This can be advertised for LDP and RSVP. However, ELC signaling is not supported for BGP tunnels, because no agreed standard exists in the IETF. RFC 6790 introduced the ELC BGP attribute that can be signaled by the eLER to indicate that it supports EL. According to RFC 6790, LSRs incapable of processing ELs must remove the ELC BGP attribute, but this requirement could not be guaranteed; therefore, the ELC BGP attribute has been deprecated in RFC 7447.
As a workaround, at the iLER, an override of the ELC for a BGP tunnel can be manually configured. After this ELC has been overridden, the BGP sender assumes that the receiver is capable of receiving and processing the ELs, regardless of the signaled ELC. The iLER inserts an EL on a tunnel for which the ELC is confirmed by the downstream peer or when the ELC is overridden by configuration.
ELC signaling can be enabled for LDP on the LERs, as follows:
# on PE-1, PE-2:
configure
router
ldp
entropy-label-capability
ELC signaling for RSVP can be enabled as follows:
# on PE-1, PE-2:
configure
router
rsvp
entropy-label-capability
As previously described, the lack of an IETF standard for BGP tunnels means that ELC is ignored by the receiving LER, and no EL is inserted. Therefore, an override is required that assumes that the far-end LER has ELC, and allows insertion of the EL. The override is enabled as follows:
# on PE-1, PE-2:
configure
router
bgp
override-tunnel-elc
When the ELC is overridden for BGP, the iLER assumes that the receiver can handle the EL.
Configuration
In this section, an EVPN-VPLS will be configured on PE-1 and PE-2 to illustrate LAG hashing based on EL. Example topology shows the topology used for this example. Load-balancing of the traffic will be performed in the P-routers that are connected by a LAG with eight links.
The initial configuration of the PE/P-routers includes the following:
Cards, MDAs, ports
LAG with eight network links between P-3 and P-4
Router interfaces
IGP (IS-IS or OSPF)
LDP enabled on all interfaces
MPLS enabled on all interfaces. RSVP enabled.
iBGP configured with P-3 as route reflector (RR)
For the configuration of EVPN-MPLS, the BGP configuration needs to include the address family EVPN, as follows. See chapter EVPN for MPLS Tunnels for more information.
# on PE-1, PE-2:
configure
router
autonomous-system 64500
bgp
rapid-update evpn
group "iBGP"
family evpn
peer-as 64500
neighbor 192.0.2.3
exit
exit
exit
EVPN service using RSVP tunnel with EL
RSVP LSP "LSP-PE-1-PE-2" from PE-1 to PE-2 via P-3 and P-4 shows LSP "LSP-PE-1-PE-2" from PE-1 to PE-2 via core routers P-3 and P-4.
The LSP "LSP-PE-1-PE-2" is configured on PE-1, as follows:
# on PE-1:
configure
router
mpls
path "path-PE-1-PE-2"
hop 10 192.168.13.2 strict
hop 20 192.168.34.2 strict
hop 30 192.168.24.1 strict
no shutdown
exit
lsp "LSP-PE-1-PE-2"
to 192.0.2.2
primary "path-PE-1-PE-2"
exit
no shutdown
exit
exit
exit
exit
The configuration for LSP "LSP-PE-2-PE-1" on PE-2 is similar.
ELC is disabled by default, and needs to be enabled for RSVP on the eLERs, as follows:
# on PE-1, PE-2:
configure
router
rsvp
entropy-label-capability
The EL can be disabled (force-disable) or enabled in the mpls context on the iLER, as follows:
configure
router
mpls
entropy-label
- entropy-label rsvp-te <rsvp-te>
- entropy-label sr-te <sr-te>
<rsvp-te> : <force-disable | enable>
<sr-te> : <force-disable | enable>
The EL can also be disabled (force-disable) or enabled per LSP, but there is a third option to inherit the EL settings from the mpls context, as follows:
configure
router
mpls
lsp "LSP-PE-1-PE-2"
entropy-label
- entropy-label {force-disable | inherit | enable}
<force-disable | i*> : force-disable|inherit|enable
By default, the EL on the LSP inherits the EL settings in the mpls context, as follows:
*A:PE-1>config>router>mpls>lsp# info detail | match entropy-label
entropy-label inherit
When LSP templates are used, EL can be configured within the lsp-template context for single-hop and mesh point-to-point LSPs, as follows:
configure
router
mpls
lsp-template "LSPtemplate1" one-hop-p2p
entropy-label
- entropy-label {force-disable | inherit | enable}
<force-disable | i*> : force-disable|inherit|enable
configure
router
mpls
lsp-template "LSPtemplate2" mesh-p2p
entropy-label
- entropy-label {force-disable | inherit | enable}
<force-disable | i*> : force-disable|inherit|enable
When the EL settings are modified, for example, from inherit to enabled, the changes only take effect after the LSP has been cleared or MPLS has been bounced, using shutdown/no shutdown. The following message is raised when the configuration is modified for an LSP in no shutdown state:
*A:PE-1>config>router>mpls>lsp# entropy-label enable
INFO: MPLS #1029 Entropy Label change is not operational - LSP must be bounced
After the LSP is bounced (shutdown/no shutdown), the following command shows that EL is enabled in MPLS:
*A:PE-1# show router mpls status | match Label
Entropy Label RSVP-TE : Enabled Entropy Label SR-TE : Enabled
The following command shows that EL is enabled in RSVP:
*A:PE-1# show router rsvp status | match Label
Implicit Null Label: Disabled Node-id in RRO : Exclude
DiffServTE AdmModel: Basic Entropy Label : Enabled
The following command shows that EL is enabled and operational in LSP "LSP-PE-1-PE-2":
*A:PE-1# show router mpls lsp "LSP-PE-1-PE-2" detail | match Label
Entropy Label : Enabled Oper Entropy Label : Enabled
The following command shows that the LSP "LSP-PE-1-PE-2" has the flag "entropy-label-capable" in the RSVP tunnel table:
*A:PE-1# show router tunnel-table protocol rsvp detail
===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination : 192.0.2.2/32
NextHop : 192.168.13.2
Tunnel Flags : exclude-for-lfa entropy-label-capable
Age : 00h14m59s
CBF Classes : (Not Specified)
Owner : rsvp Encap : MPLS
Tunnel ID : 1 Preference : 7
Tunnel Label : 524281 Tunnel Metric : 16777215
Tunnel MTU : 1560 Max Label Stack : 1
LSP ID : 62466 Bypass Label : 0
LSP Bandwidth : 0 LSP Weight : 0
-------------------------------------------------------------------------------
Number of tunnel-table entries : 1
Number of tunnel-table entries with LFA : 0
===============================================================================
EL is supported on many Layer 2 and Layer 3 services. In this example, VPLS 1 is configured on PE-1 with BGP EVPN-MPLS, as follows:
# on PE-1:
configure
service
vpls 1 name "VPLS 1" customer 1 create
bgp
exit
bgp-evpn
evi 1
mpls bgp 1
entropy-label
auto-bind-tunnel
resolution-filter
rsvp
exit
resolution filter
exit
no shutdown
exit
exit
sap 1/1/2:1 create
exit
sap 1/2/1:11 create
exit
no shutdown
Auto-bind-tunnel is resolved to the RSVP LSP "LSP-PE-1-PE-2". A similar configuration is applied on PE-2.
The following command shows that the EL is enabled for BGP-EVPN, as follows:
*A:PE-1# show service id 1 bgp-evpn
===============================================================================
BGP EVPN Table
===============================================================================
---snip---
===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status : Enabled Bgp Instance : 1
Force Vlan Fwding : Disabled
Control Word : Disabled
Max Ecmp Routes : 1
Entropy Label : Enabled
Default Route Tag : none
Split Horizon Group: (Not Specified)
Ingress Rep BUM Lbl: Disabled
Ingress Ucast Lbl : 524281 Ingress Mcast Lbl : 524281
RestProtSrcMacAct : none
Evpn Mpls Encap : Enabled Evpn MplsoUdp : Disabled
Oper Group :
===============================================================================
===============================================================================
BGP EVPN MPLS Auto Bind Tunnel Information
===============================================================================
Resolution : filter Strict Tnl Tag : false
Max Ecmp Routes : 1
Bgp Instance : 1
Filter Tunnel Types: rsvp
===============================================================================
The iLER PE-1 will add the EL and ELI to the label stack. Traffic load-balancing will be performed in P-3 where the traffic will be sprayed over all eight links of the LAG. The options for LSR load-balancing are the following:
# on LSRs P-3 and P-4:
configure
system
load-balancing
lsr-load-balancing # system-wide LSR load balancing
- lsr-load-balancing <hashing-algorithm>
- no lsr-load-balancing
<hashing-algorithm> : lbl-only | lbl-ip | ip-only | eth-encap-ip | lbl-ip-l4-teid
By default, the load-balancing settings on P-3 are as follows:
*A:P-3>config>system>load-balancing# info detail
----------------------------------------------
no l2tp-load-balancing
no l4-load-balancing
lsr-load-balancing lbl-only
no system-ip-load-balancing
no mc-enh-load-balancing
no service-id-lag-hashing
----------------------------------------------
When the EL and ELI are included in the stack, only the EL label is used for the load-balancing (lbl-only). It is not required to inspect the IP header or any other header.
For the reverse path, PE-2 is the iLER, and P-4 will do the spraying of the packets over all links of the LAG. Therefore, LSR load-balancing with lbl-only is also configured on P-4.
EVPN service using LDP tunnel with EL
ELC is disabled by default for LDP, as follows:
*A:PE-1# show router ldp status
===============================================================================
LDP Status for IPv4 LSR ID 192.0.2.1
IPv6 LSR ID ::
===============================================================================
---snip---
Admin State : Up
IPv4 Oper State : Up IPv6 Oper State : Down
---snip---
Entropy Label Capa*: False
---snip---
The command to enable ELC is as follows:
# on PE-1:
configure
router
ldp
entropy-label-capability
In the list of LDP active bindings, the egress labels that are pushed by iLER PE-1 on traffic toward an eLER capable of handling an EL get the indication "e", as follows:
*A:PE-1# show router ldp bindings active prefixes ipv4
===============================================================================
LDP Bindings (IPv4 LSR ID 192.0.2.1)
(IPv6 LSR ID ::)
===============================================================================
Label Status:
U - Label In Use, N - Label Not In Use, W - Label Withdrawn
WP - Label Withdraw Pending, BU - Alternate For Fast Re-Route
e - Label ELC
FEC Flags:
LF - Lower FEC, UF - Upper FEC, M - Community Mismatch,
BA - ASBR Backup FEC
(S) - Static (M) - Multi-homed Secondary Support
(B) - BGP Next Hop (BU) - Alternate Next-hop for Fast Re-Route
(I) - SR-ISIS Next Hop (O) - SR-OSPF Next Hop
(C) - FEC resolved with class-based-forwarding
===============================================================================
LDP IPv4 Prefix Bindings (Active)
===============================================================================
Prefix Op
IngLbl EgrLbl
EgrNextHop EgrIf/LspId
-------------------------------------------------------------------------------
192.0.2.1/32 Pop
524287 --
-- --
192.0.2.2/32 Push
-- 524285e
192.168.13.2 1/1/1
192.0.2.2/32 Swap
524282 524285
192.168.13.2 1/1/1
192.0.2.3/32 Push
-- 524287
192.168.13.2 1/1/1
192.0.2.4/32 Push
-- 524284
192.168.13.2 1/1/1
192.0.2.4/32 Swap
524283 524284
192.168.13.2 1/1/1
-------------------------------------------------------------------------------
No. of IPv4 Prefix Active Bindings: 6
===============================================================================
Because the iLER adds the EL, only labels which are pushed get the "e"-indication. Labels which are swapped or popped do not get this label.
The details of the tunnel table for LDP show that the LDP tunnel toward PE-2 has tunnel flag entropy-label-capable, as follows:
*A:PE-1# show router tunnel-table protocol ldp detail
===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination : 192.0.2.2/32
NextHop : 192.168.13.2
Tunnel Flags : entropy-label-capable
Age : 00h00m30s
CBF Classes : (Not Specified)
Owner : ldp Encap : MPLS
Tunnel ID : 65539 Preference : 9
Tunnel Label : 524285 Tunnel Metric : 30
Tunnel MTU : 1560 Max Label Stack : 1
-------------------------------------------------------------------------------
---snip---
The following EVPN-VPLS uses an LDP transport tunnel and has EL enabled:
# on PE-1:
configure
service
vpls 2 name "VPLS 2" customer 1 create
bgp
exit
bgp-evpn
evi 2
mpls bgp 1
entropy-label
auto-bind-tunnel
resolution-filter
ldp
exit
resolution filter
exit
no shutdown
exit
exit
sap 1/1/2:16 create
exit
sap 1/2/1:16 create
exit
no shutdown
The configuration on PE-1 and PE-2 is similar.
EL is enabled in this service, as follows:
*A:PE-2# show service id 2 bgp-evpn
===============================================================================
BGP EVPN Table
===============================================================================
---snip---
===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status : Enabled Bgp Instance : 1
Force Vlan Fwding : Disabled
Control Word : Disabled
Max Ecmp Routes : 1
Entropy Label : Enabled
Default Route Tag : none
Split Horizon Group: (Not Specified)
Ingress Rep BUM Lbl: Disabled
Ingress Ucast Lbl : 524279 Ingress Mcast Lbl : 524279
RestProtSrcMacAct : none
Evpn Mpls Encap : Enabled Evpn MplsoUdp : Disabled
Oper Group :
===============================================================================
===============================================================================
BGP EVPN MPLS Auto Bind Tunnel Information
===============================================================================
Resolution : filter Strict Tnl Tag : false
Max Ecmp Routes : 1
Bgp Instance : 1
Filter Tunnel Types: ldp
===============================================================================
LSR load-sharing based on EL
The LSR load-sharing based on the EL results in a granular load-sharing for great numbers of flows, for example, thousands of flows. Unfortunately, for the following simulation, only 9 flows were generated (so the load is not evenly spread and some ports may even be idle); each flow with four unique variables: source IP address, destination IP address, source UDP port, and destination UDP port. Each flow gets a unique EL and the P-nodes distribute the load over all eight links in the LAG, based on the hashing algorithm "lbl-only", as follows:
A:P-3# monitor lag 1 rate
===============================================================================
Monitor statistics for LAG ID 1
===============================================================================
Port-id Input packets Output packets
Input bytes Output bytes
Input errors [Input util %] Output errors [Output util %]
-------------------------------------------------------------------------------
---snip---
-------------------------------------------------------------------------------
At time t = 30 sec (Mode: Rate)
-------------------------------------------------------------------------------
1/1/3 758 785
1170716 1213196
0 0.09 0 0.09
1/1/4 745 777
1151461 1200469
0 0.09 0 0.09
1/1/5 748 787
1156897 1217049
0 0.09 0 0.09
1/1/6 741 765
1146204 1182559
0 0.09 0 0.09
1/1/7 752 774
1162616 1197068
0 0.09 0 0.09
1/1/8 1488 1522
2300757 2352239
0 0.18 0 0.19
1/1/9 740 771
1144195 1191966
0 0.09 0 0.09
1/1/10 758 775
1172486 1198768
0 0.09 0 0.09
-------------------------------------------------------------------------------
Totals 6730 6956
10405332 10753314
0 0.10 0 0.10
However, when thousands of flows are sent via P-3 and P-4, the flows are evenly spread in each direction.
Conclusion
The EL is a standard-compliant way to maintain packet ordering within a conversation while load-balancing across multiple ECMP paths or links in a LAG. The EL is supported for both Layer 2 and Layer 3 services and is, therefore, a more general solution than the MPLS hash label.