Entropy Label
This chapter provides information about the Entropy Label (EL).
Topics in this chapter include:
Applicability
This chapter was initially written for SR OS Release 14.0.R4. The CLI in the current edition corresponds to SR OS Release 24.3.R1.
RFC 6391 hash label or flow-aware transport label is supported in SR OS Release 8.0.R1, and later. RFC 6790 Entropy Labels (ELs) are supported on RSVP and BGP tunnels in Release 14.0.R1, and later, and supported on LDP tunnels in Release 14.0.R4, and later.
Overview
Entropy is the degree of disorder or uncertainty in a system. SR OS supports both the MPLS EL and the hash label, but they are mutually exclusive. These labels allow Label Switched Routers (LSRs) to load-balance labeled packets in a granular way without the need to inspect the IPv4 or IPv6 header. The hash label is applicable to services such as Epipe VLL, VPLS, IES (spoke-SDP), and VPRN services. The main advantage of the EL compared to the hash label is that the EL can be applied to a wider range of services, such as EVPN, BGP VPWS, Fpipe VLL, Ipipe VLL, H-VPLS, and BGP RFC 3107 tunnels.
Load-balancing of flows based on hash label or entropy label shows that different flows from an ingress Label Edge Router (iLER) are load-balanced across different paths in the MPLS network toward the egress Label Edge Router (eLER).
The EL is inserted below the innermost Tunnel Label (TL) in the label stack, along with the Entropy Label Indicator (ELI), which is a special purpose MPLS label with a value of 7 to indicate that the next label in the stack is an EL. As with the hash label, the value of the EL is calculated based on a hash of the packet payload header (IP and Layer 4). The EL is inserted as deep as possible in the stack to ensure preservation for as far as possible through the network. When the EL and ELI are present, load-balancing on the transit LSRs can be configured to only take into account the EL label for Link Aggregation Group (LAG) and Equal Cost Multi-Path (ECMP). Load-balancing on the LSR can be configured to take into account the IP header also, but that is not required when the EL and ELI are present. The eLER removes the EL and ELI before forwarding the packet to its final destination.
The EL requires two additional labels in the label stack, which may result in an unsupported label stack depth in an intermediate (possibly third-party) LSR. SR OS allows control of the EL and ELI insertion and accounting of extra labels in the tunnel table. The supported label stack depth is 12 in SR OS Release 14.0, including transport, service, hash, and OAM labels.
Label stack with hash label versus label stack with EL and ELI shows a comparison between a label stack with a hash label and a label stack with an EL and ELI.
Hash labels or ELs are inserted by the iLER, based on the incoming packet header. The iLER uses a single label to represent one or more flows, based on the hash of the incoming packet header IP and Layer 4. The hash label is a single label that is inserted at the bottom of the stack, below the service label. The EL is inserted together with the ELI below the innermost tunnel label, but above the service label. The entropy and hash label value is outside of the range for MPLS label values, because the most significant bit equals 1 for entropy and hash labels and 0 for MPLS labels.
The EL is used as part of the LSR hashing algorithm for spraying packets over multi-port LAGs, ECMP, or BGP tunnels with multiple downstream interfaces. The packet ordering is preserved because one label is used per conversation. Hashing based on a label stack containing an EL per service will become more granular than when based on a standard label stack alone.
Downstream LERs signal EL capability to ILER shows how the eLER signals its EL capability to the iLER.
The Entropy Label Capability (ELC) is signaled by the eLER and indicates the ability to receive and process the ELs. This can be advertised for LDP and RSVP. However, ELC signaling is not supported for BGP tunnels, because no agreed standard exists in the IETF. RFC 6790 introduced the ELC BGP attribute that can be signaled by the eLER to indicate that it supports EL. According to RFC 6790, LSRs incapable of processing ELs must remove the ELC BGP attribute, but this requirement could not be guaranteed; therefore, the ELC BGP attribute has been deprecated in RFC 7447.
As a workaround, at the iLER, an override of the ELC for a BGP tunnel can be manually configured. After this ELC has been overridden, the BGP sender assumes that the receiver is capable of receiving and processing the ELs, regardless of the signaled ELC. The iLER inserts an EL on a tunnel for which the ELC is confirmed by the downstream peer or when the ELC is overridden by configuration.
ELC signaling can be enabled for LDP on the LERs, as follows:
# on PE-1, PE-2:
configure
router
ldp
entropy-label-capabilityELC signaling for RSVP can be enabled as follows:
# on PE-1, PE-2:
configure
router
rsvp
entropy-label-capabilityAs previously described, the lack of an IETF standard for BGP tunnels means that ELC is ignored by the receiving LER, and no EL is inserted. Therefore, an override is required that assumes that the far-end LER has ELC, and allows insertion of the EL. The override is enabled as follows:
# on PE-1, PE-2:
configure
router
bgp
override-tunnel-elcWhen the ELC is overridden for BGP, the iLER assumes that the receiver can handle the EL.
Configuration
In this section, an EVPN-VPLS is configured on PE-1 and PE-2 to illustrate LAG hashing based on EL. Example topology shows the topology used for this example. Load-balancing of the traffic is performed in the P-routers that are connected by a LAG with eight links.
The initial configuration of the PE/P-routers includes the following:
Cards, MDAs, ports
LAG with eight network links between P-3 and P-4
Router interfaces
IGP (IS-IS or OSPF)
LDP enabled on all interfaces
MPLS enabled on all interfaces. RSVP enabled.
iBGP configured with P-3 as route reflector (RR)
For the configuration of EVPN-MPLS, the BGP configuration needs to include the address family EVPN, as follows. See chapter EVPN for MPLS Tunnels for more information.
# on PE-1, PE-2:
configure
router
autonomous-system 64500
bgp
rapid-update evpn
group "iBGP"
family evpn
peer-as 64500
neighbor 192.0.2.3
exit
exit
exitEVPN service using RSVP tunnel with EL
RSVP LSP "LSP-PE-1-PE-2" from PE-1 to PE-2 via P-3 and P-4 shows LSP "LSP-PE-1-PE-2" from PE-1 to PE-2 via core routers P-3 and P-4.
The LSP "LSP-PE-1-PE-2" is configured on PE-1, as follows:
# on PE-1:
configure
router
mpls
path "path-PE-1-PE-2"
hop 10 192.168.13.2 strict
hop 20 192.168.34.2 strict
hop 30 192.168.24.1 strict
no shutdown
exit
lsp "LSP-PE-1-PE-2"
to 192.0.2.2
primary "path-PE-1-PE-2"
exit
no shutdown
exit
exitThe configuration for LSP "LSP-PE-2-PE-1" on PE-2 is similar.
ELC is disabled by default, and needs to be enabled for RSVP on the eLERs, as follows:
# on PE-1, PE-2:
configure
router
rsvp
entropy-label-capabilityThe EL can be disabled (force-disable) or enabled in the mpls context on the iLER, as follows:
# on PE-1, PE-2:
configure
router
mpls
entropy-label ?
- entropy-label rsvp-te <rsvp-te>
- entropy-label sr-te <sr-te>
<rsvp-te> : <force-disable | enable>
<sr-te> : <force-disable | enable>The EL can also be disabled (force-disable) or enabled per LSP, but there is a third option to inherit the EL settings from the mpls context, as follows:
# on PE-1:
configure
router
mpls
lsp "LSP-PE-1-PE-2"
entropy-label ?
- entropy-label {force-disable | inherit | enable}
<force-disable | i*> : force-disable|inherit|enableBy default, the EL on the LSP inherits the EL settings in the mpls context, as follows:
# On PE-1:
configure
router
mpls
lsp "lsp-PE-1-PE-2"
info detail | match "entropy-label"
entropy-label inheritWhen LSP templates are used, EL can be configured within the lsp-template context for single-hop and mesh point-to-point LSPs, as follows:
# on PE-1:
configure
router
mpls
lsp-template "LSPtemplate1" one-hop-p2p
entropy-label ?
- entropy-label {force-disable | inherit | enable}
<force-disable | i*> : force-disable|inherit|enable# on PE-1:
configure
router
mpls
lsp-template "LSPtemplate2" mesh-p2p
entropy-label ?
- entropy-label {force-disable | inherit | enable}
<force-disable | i*> : force-disable|inherit|enableWhen the EL settings are modified, for example, from inherit to enabled, the changes only take effect after the LSP has been cleared or MPLS has been bounced, using shutdown/no shutdown. The following message is raised when the configuration is modified for an LSP in no shutdown state:
# On PE-1:
configure
router
mpls
lsp "lsp-PE-1-PE-2"
entropy-label enable
INFO: MPLS #1029 Entropy Label change is not operational - LSP must be bounced
After the LSP is bounced (shutdown/no shutdown), the following command shows that EL is enabled in MPLS:
*A:PE-1# show router mpls status | match "Label"
Entropy Label RSVP-TE : Enabled Entropy Label SR-TE : EnabledThe following command shows that EL is enabled in RSVP:
*A:PE-1# show router rsvp status | match "Label"
Implicit Null Label: Disabled Node-id in RRO : Exclude
DiffServTE AdmModel: Basic Entropy Label : EnabledThe following command shows that EL is enabled and operational in LSP "LSP-PE-1-PE-2":
*A:PE-1# show router mpls lsp "LSP-PE-1-PE-2" detail | match "Label"
Entropy Label : Enabled Oper Entropy Label : EnabledThe following command shows that the LSP "LSP-PE-1-PE-2" has the flag "entropy-label-capable" in the RSVP tunnel table:
*A:PE-1# show router tunnel-table protocol rsvp detail
===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination : 192.0.2.2/32
NextHop : 192.168.13.2
Tunnel Flags : exclude-for-lfa entropy-label-capable
Age : 00h06m42s
CBF Classes : (Not Specified)
Owner : rsvp Encap : MPLS
Tunnel ID : 1 Preference : 7
Tunnel Label : 524281 Tunnel Metric : 16777215
Tunnel MTU : 8918 Max Label Stack : 1
LSP ID : 32258 Bypass Label : 0
LSP Bandwidth : 0 LSP Weight : 0
-------------------------------------------------------------------------------
Number of tunnel-table entries : 1
Number of tunnel-table entries with LFA : 0
===============================================================================EL is supported on many Layer 2 and Layer 3 services. In this example, VPLS 1 is configured on PE-1 with BGP EVPN-MPLS, as follows:
# on PE-1:
configure
service
vpls 1 name "VPLS 1" customer 1 create
bgp
exit
bgp-evpn
evi 1
mpls bgp 1
entropy-label
auto-bind-tunnel
resolution-filter
rsvp
exit
resolution filter
exit
no shutdown
exit
exit
sap 1/1/c2/1:11 create
exit
sap 1/1/c9/1:11 create
exit
no shutdownAuto-bind-tunnel is resolved to the RSVP LSP "LSP-PE-1-PE-2". A similar configuration is applied on PE-2.
The following command shows that the EL is enabled for BGP-EVPN, as follows:
*A:PE-1# show service id 1 bgp-evpn
===============================================================================
BGP EVPN
===============================================================================
---snip---
===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status : Enabled Bgp Instance : 1
Force Vlan Fwding : Disabled
Force Qinq Fwding : none
Route NextHop Type : system-ipv4
Control Word : Disabled
Max Ecmp Routes : 1
Entropy Label : Enabled
Default Route Tag : none
Split Horizon Group: (Not Specified)
Ingress Rep BUM Lbl: Disabled
Ingress Ucast Lbl : 524281 Ingress Mcast Lbl : 524281
RestProtSrcMacAct : none
Evpn Mpls Encap : Enabled Evpn MplsoUdp : Disabled
Oper Group : (none)
MH Mode : network
Evi 3-byte Auto-RT : Disabled
Dyn Egr Lbl Limit : Disabled
Hash Label : Disabled
Local AC Ingr Lbl : <not-allocated>
BGP EVPN MPLS Auto Bind Tunnel Information
-------------------------------------------------------------------------------
Allow-Flex-Algo-FB : Disabled
Resolution : filter Strict Tnl Tag : Disabled
Max Ecmp Routes : 1
Filter Tunnel Types: rsvp
Weighted Ecmp : Disabled
===============================================================================The iLER PE-1 adds the EL and ELI to the label stack. Traffic load-balancing is performed in P-3 where the traffic is sprayed over all eight links of the LAG. The options for LSR load-balancing are the following:
# on LSRs P-3 and P-4:
configure
system
load-balancing
lsr-load-balancing ? # system-wide LSR load balancing
- lsr-load-balancing <hashing-algorithm>
- no lsr-load-balancing
<hashing-algorithm> : lbl-only|lbl-ip|ip-only|eth-encap-ip|lbl-ip-l4-teidBy default, the load-balancing settings on P-3 are as follows:
# On P-3:
configure
system
load-balancing
info detail
----------------------------------------------
no l2tp-load-balancing
no l4-load-balancing
lsr-load-balancing lbl-only
no system-ip-load-balancing
no mc-enh-load-balancing
no service-id-lag-hashing
----------------------------------------------When the EL and ELI are included in the stack, only the EL label is used for the load-balancing (lbl-only). It is not required to inspect the IP header or any other header.
For the reverse path, PE-2 is the iLER, and P-4 does the spraying of the packets over all links of the LAG. Therefore, LSR load-balancing with lbl-only is also configured on P-4.
EVPN service using LDP tunnel with EL
ELC is disabled by default for LDP, as follows:
*A:PE-1# show router ldp status
===============================================================================
LDP Status for IPv4 LSR ID 192.0.2.1
IPv6 LSR ID ::
===============================================================================
---snip---
Admin State : Up
IPv4 Oper State : Up IPv6 Oper State : Down
---snip---
Entropy Label Capa*: False
---snip---
===============================================================================
* indicates that the corresponding row element may have been truncated.The command to enable ELC is as follows:
# on PE-1:
configure
router
ldp
entropy-label-capabilityIn the list of LDP active bindings, the egress labels that are pushed by iLER PE-1 on traffic toward an eLER capable of handling an EL get the indication "e", as follows:
*A:PE-1# show router ldp bindings active prefixes ipv4
===============================================================================
LDP Bindings (IPv4 LSR ID 192.0.2.1)
(IPv6 LSR ID ::)
===============================================================================
Label Status:
U - Label In Use, N - Label Not In Use, W - Label Withdrawn
WP - Label Withdraw Pending, BU - Alternate For Fast Re-Route
e - Label ELC
FEC Flags:
LF - Lower FEC, UF - Upper FEC, M - Community Mismatch,
BA - ASBR Backup FEC
(S) - Static (M) - Multi-homed Secondary Support
(B) - BGP Next Hop (BU) - Alternate Next-hop for Fast Re-Route
(I) - SR-ISIS Next Hop (O) - SR-OSPF Next Hop
(C) - FEC resolved with class-based-forwarding
===============================================================================
LDP IPv4 Prefix Bindings (Active)
===============================================================================
Prefix Op
IngLbl EgrLbl
EgrNextHop EgrIf/LspId
-------------------------------------------------------------------------------
192.0.2.1/32 Pop
524287 --
-- --
192.0.2.2/32 Push
-- 524281e
192.168.13.2 1/1/c1/1
192.0.2.2/32 Swap
524279 524281
192.168.13.2 1/1/c1/1
192.0.2.3/32 Push
-- 524287
192.168.13.2 1/1/c1/1
192.0.2.4/32 Push
-- 524282
192.168.13.2 1/1/c1/1
192.0.2.4/32 Swap
524280 524282
192.168.13.2 1/1/c1/1
-------------------------------------------------------------------------------
No. of IPv4 Prefix Active Bindings: 6
===============================================================================Because the iLER adds the EL, only labels which are pushed get the "e"-indication. Labels which are swapped or popped do not get this label.
The detail of the tunnel table for LDP shows that the LDP tunnel toward PE-2 has tunnel flag entropy-label-capable, as follows:
*A:PE-1# show router tunnel-table protocol ldp detail
===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination : 192.0.2.2/32
NextHop : 192.168.13.2
Tunnel Flags : entropy-label-capable
Age : 00h01m32s
CBF Classes : (Not Specified)
Owner : ldp Encap : MPLS
Tunnel ID : 65556 Preference : 9
Tunnel Label : 524281 Tunnel Metric : 30
Tunnel MTU : 8918 Max Label Stack : 1
-------------------------------------------------------------------------------
---snip---
===============================================================================The following EVPN-VPLS uses an LDP transport tunnel and has EL enabled:
# on PE-1:
configure
service
vpls 2 name "VPLS 2" customer 1 create
bgp
exit
bgp-evpn
evi 2
mpls bgp 1
entropy-label
auto-bind-tunnel
resolution-filter
ldp
exit
resolution filter
exit
no shutdown
exit
exit
sap 1/1/c2/1:16 create
exit
sap 1/1/c9/1:16 create
exit
no shutdownThe configuration on PE-1 and PE-2 is similar.
EL is enabled in this service, as follows:
*A:PE-2# show service id 2 bgp-evpn
===============================================================================
BGP EVPN
===============================================================================
---snip---
===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status : Enabled Bgp Instance : 1
Force Vlan Fwding : Disabled
Force Qinq Fwding : none
Route NextHop Type : system-ipv4
Control Word : Disabled
Max Ecmp Routes : 1
Entropy Label : Enabled
Default Route Tag : none
Split Horizon Group: (Not Specified)
Ingress Rep BUM Lbl: Disabled
Ingress Ucast Lbl : 524283 Ingress Mcast Lbl : 524283
RestProtSrcMacAct : none
Evpn Mpls Encap : Enabled Evpn MplsoUdp : Disabled
Oper Group : (none)
MH Mode : network
Evi 3-byte Auto-RT : Disabled
Dyn Egr Lbl Limit : Disabled
Hash Label : Disabled
Local AC Ingr Lbl : <not-allocated>
BGP EVPN MPLS Auto Bind Tunnel Information
-------------------------------------------------------------------------------
Allow-Flex-Algo-FB : Disabled
Resolution : filter Strict Tnl Tag : Disabled
Max Ecmp Routes : 1
Filter Tunnel Types: ldp
Weighted Ecmp : Disabled
===============================================================================LSR load-sharing based on EL
The LSR load-sharing based on the EL results in a granular load-sharing for great numbers of flows, for example, thousands of flows. Each flow has four unique variables: source IP address, destination IP address, source UDP port, and destination UDP port. Each flow gets a unique EL and the P-nodes distribute the load over all eight links in the LAG, based on the hashing algorithm "lbl-only".
In a virtual test setup, such great numbers of flows are difficult to simulate. Nevertheless, the spread may be illustrated by comparing the traffic on each link of the LAG, once in the absence and once in the presence of externally launched traffic, as follows:
- clear lag 1 statistics
- wait for X seconds (example: X=125)
- show lag 1 statistics
or
# On P-3:
- clear port <port> statistics (where <port> is a port in the LAG)
- wait for X seconds (example: X=125)
- show port statistics
- monitor lag 1 rate packets
-
monitor port <port> rate
As an example:, on P-3, in the absence of externally launched traffic:
*A:P-3# show lag 1 statistics
===============================================================================
LAG Statistics
===============================================================================
Description : N/A
Port-id Input(Packets) Output(Packets)
Input(Bytes) Output(Bytes)
Input Errors Output Errors
-------------------------------------------------------------------------------
1/1/c3/1 236 241
25384 25559
0 0
1/1/c4/1 125 125
16000 16000
0 0
1/1/c5/1 158 158
20125 20125
0 0
1/1/c6/1 133 154
16636 19480
0 0
1/1/c7/1 153 125
19360 16000
0 0
1/1/c8/1 125 125
16000 16000
0 0
1/1/c9/1 125 125
16000 16000
0 0
1/1/c10/1 125 125
16000 16000
0 0
-------------------------------------------------------------------------------
Totals 1180 1178
145505 145164
0 0
===============================================================================while in the presence of externally launched traffic:
*A:P-3# show lag 1 statistics
===============================================================================
LAG Statistics
===============================================================================
Description : N/A
Port-id Input(Packets) Output(Packets)
Input(Bytes) Output(Bytes)
Input Errors Output Errors
-------------------------------------------------------------------------------
1/1/c3/1 307 320
33488 34595
0 0
1/1/c4/1 161 165
20608 21219
0 0
1/1/c5/1 204 207
25983 26511
0 0
1/1/c6/1 176 215
21895 26472
0 0
1/1/c7/1 217 174
27406 23241
0 0
1/1/c8/1 167 167
21913 21742
0 0
1/1/c9/1 168 166
22062 21545
0 0
1/1/c10/1 163 165
21062 21156
0 0
-------------------------------------------------------------------------------
Totals 1563 1579
194417 196481
0 0
===============================================================================Conclusion
The EL is a standard-compliant way to maintain packet ordering within a conversation while load-balancing across multiple ECMP paths or links in a LAG. The EL is supported for both Layer 2 and Layer 3 services and is, therefore, a more general solution than the MPLS hash label.