Entropy Label
This chapter provides information about the Entropy Label (EL).
Topics in this chapter include:
Applicability
This chapter was initially written for SR OS Release 14.0.R4. The CLI in the current edition corresponds to SR OS Release 24.3.R1.
RFC 6391 hash label or flow-aware transport label is supported in SR OS Release 8.0.R1, and later. RFC 6790 Entropy Labels (ELs) are supported on RSVP and BGP tunnels in Release 14.0.R1, and later, and supported on LDP tunnels in Release 14.0.R4, and later.
Overview
Entropy is the degree of disorder or uncertainty in a system. SR OS supports both the MPLS EL and the hash label, but they are mutually exclusive. These labels allow Label Switched Routers (LSRs) to load-balance labeled packets in a granular way without the need to inspect the IPv4 or IPv6 header. The hash label is applicable to services such as Epipe VLL, VPLS, IES (spoke-SDP), and VPRN services. The main advantage of the EL compared to the hash label is that the EL can be applied to a wider range of services, such as EVPN, BGP VPWS, Fpipe VLL, Ipipe VLL, H-VPLS, and BGP RFC 3107 tunnels.
Load-balancing of flows based on hash label or entropy label shows that different flows from an ingress Label Edge Router (iLER) are load-balanced across different paths in the MPLS network toward the egress Label Edge Router (eLER).
The EL is inserted below the innermost Tunnel Label (TL) in the label stack, along with the Entropy Label Indicator (ELI), which is a special purpose MPLS label with a value of 7 to indicate that the next label in the stack is an EL. As with the hash label, the value of the EL is calculated based on a hash of the packet payload header (IP and Layer 4). The EL is inserted as deep as possible in the stack to ensure preservation for as far as possible through the network. When the EL and ELI are present, load-balancing on the transit LSRs can be configured to only take into account the EL label for Link Aggregation Group (LAG) and Equal Cost Multi-Path (ECMP). Load-balancing on the LSR can be configured to take into account the IP header also, but that is not required when the EL and ELI are present. The eLER removes the EL and ELI before forwarding the packet to its final destination.
The EL requires two additional labels in the label stack, which may result in an unsupported label stack depth in an intermediate (possibly third-party) LSR. SR OS allows control of the EL and ELI insertion and accounting of extra labels in the tunnel table. The supported label stack depth is 12 in SR OS Release 14.0, including transport, service, hash, and OAM labels.
Label stack with hash label versus label stack with EL and ELI shows a comparison between a label stack with a hash label and a label stack with an EL and ELI.
Hash labels or ELs are inserted by the iLER, based on the incoming packet header. The iLER uses a single label to represent one or more flows, based on the hash of the incoming packet header IP and Layer 4. The hash label is a single label that is inserted at the bottom of the stack, below the service label. The EL is inserted together with the ELI below the innermost tunnel label, but above the service label. The entropy and hash label value is outside of the range for MPLS label values, because the most significant bit equals 1 for entropy and hash labels and 0 for MPLS labels.
The EL is used as part of the LSR hashing algorithm for spraying packets over multi-port LAGs, ECMP, or BGP tunnels with multiple downstream interfaces. The packet ordering is preserved because one label is used per conversation. Hashing based on a label stack containing an EL per service will become more granular than when based on a standard label stack alone.
Downstream LERs signal EL capability to ILER shows how the eLER signals its EL capability to the iLER.
The Entropy Label Capability (ELC) is signaled by the eLER and indicates the ability to receive and process the ELs. This can be advertised for LDP and RSVP. However, ELC signaling is not supported for BGP tunnels, because no agreed standard exists in the IETF. RFC 6790 introduced the ELC BGP attribute that can be signaled by the eLER to indicate that it supports EL. According to RFC 6790, LSRs incapable of processing ELs must remove the ELC BGP attribute, but this requirement could not be guaranteed; therefore, the ELC BGP attribute has been deprecated in RFC 7447.
As a workaround, at the iLER, an override of the ELC for a BGP tunnel can be manually configured. After this ELC has been overridden, the BGP sender assumes that the receiver is capable of receiving and processing the ELs, regardless of the signaled ELC. The iLER inserts an EL on a tunnel for which the ELC is confirmed by the downstream peer or when the ELC is overridden by configuration.
ELC signaling can be enabled for LDP on the LERs, as follows:
# on PE-1, PE-2:
configure {
router "Base" {
ldp {
entropy-label-capability trueELC signaling for RSVP can be enabled as follows:
# on PE-1, PE-2:
configure {
router "Base" {
rsvp {
entropy-label-capability trueAs previously described, the lack of an IETF standard for BGP tunnels means that ELC is ignored by the receiving LER, and no EL is inserted. Therefore, an override is required that assumes that the far-end LER has ELC, and allows insertion of the EL. The override is enabled as follows:
# on PE-1, PE-2:
configure {
router "Base" {
bgp {
override-tunnel-elc trueWhen the ELC is overridden for BGP, the iLER assumes that the receiver can handle the EL.
Configuration
In this section, an EVPN-VPLS is configured on PE-1 and PE-2 to illustrate LAG hashing based on EL. Example topology shows the topology used for this example. Load-balancing of the traffic is performed in the P-routers that are connected by a LAG with eight links.
The initial configuration of the PE/P-routers includes the following:
Cards, MDAs, ports
LAG with eight network links between P-3 and P-4
Router interfaces
IGP (IS-IS or OSPF)
LDP enabled on all interfaces
MPLS enabled on all interfaces. RSVP enabled.
iBGP configured with P-3 as route reflector (RR)
For the configuration of EVPN-MPLS, the BGP configuration needs to include the address family EVPN, as follows. See chapter EVPN for MPLS Tunnels for more information.
# on PE-1, PE-2:
configure {
router "Base" {
autonomous-system 64500
bgp {
ebgp-default-reject-policy {
import false
export false
}
rapid-update {
evpn true
}
group "iBGP" {
peer-as 64500
family {
evpn true
}
}
neighbor "192.0.2.3" {
group "iBGP"
}
}EVPN service using RSVP tunnel with EL
RSVP LSP "LSP-PE-1-PE-2" from PE-1 to PE-2 via P-3 and P-4 shows LSP "LSP-PE-1-PE-2" from PE-1 to PE-2 via core routers P-3 and P-4.
The LSP "LSP-PE-1-PE-2" is configured on PE-1, as follows:
# on PE-1:
configure {
router "Base" {
mpls {
path "path-PE-1-PE-2" {
admin-state enable
hop 10 {
ip-address 192.168.13.2
type strict
}
hop 20 {
ip-address 192.168.34.2
type strict
}
hop 30 {
ip-address 192.168.24.1
type strict
}
}
lsp "LSP-PE-1-PE-2" {
admin-state enable
type p2p-rsvp
to 192.0.2.2
entropy-label true
primary "path-PE-1-PE-2" { }
}
}The configuration for LSP "LSP-PE-2-PE-1" on PE-2 is similar.
ELC is disabled by default, and needs to be enabled for RSVP on the eLERs, as follows:
# on PE-1, PE-2:
configure {
router "Base" {
rsvp {
entropy-label-capability trueThe EL can be disabled or enabled in the mpls context on the iLER, as follows:
# on PE-1, PE-2:
configure {
router "Base" {
mpls {
entropy-label ?
entropy-label
rsvp-te - Apply entropy labels to RSVP-TE LSPs
sr-te - Apply entropy labels to SR-TE LSPsThe EL can also be disabled or enabled per LSP, as follows:
# on PE-1:
configure {
router "Base" {
mpls {
lsp "LSP-PE-1-PE-2" {
entropy-label ?
entropy-label <keyword>
<keyword> - (false|true)By default, the EL on the LSP inherits the EL settings in the mpls context, as follows:
# On PE-1:
configure {
router "Base" {
mpls {
lsp "LSP-PE-1-PE-2" {
info detail | match "entropy-label"
entropy-label trueWhen LSP templates are used, EL can be configured within the lsp-template context for single-hop and mesh point-to-point LSPs, as follows:
# on PE-1:
configure {
router "Base" {
mpls {
lsp-template "LSPtemplate1" {
type p2p-rsvp-one-hop
entropy-label ?
entropy-label <keyword>
<keyword> - (false|true)
Entropy label for an LSP template
Warning: Modifying this element toggles
'configure router "Base" mpls lsp-template "LSPtemplate1" admin-state' automatically for the new
value to take effect.# on PE-1:
configure {
router "Base" {
mpls {
lsp-template "LSPtemplate2" {
type p2p-rsvp-mesh
entropy-label ?
entropy-label <keyword>
<keyword> - (false|true)
Entropy label for an LSP template
Warning: Modifying this element toggles
'configure router "Base" mpls lsp-template "LSPtemplate2" admin-state' automatically for the new
value to take effect.When the EL settings are modified, the changes take effect as a result of an automatic clearing of the LSP or an automatic bouncing of the MPLS.
The following command shows that EL is enabled in MPLS:
[/]
A:admin@PE-1# show router mpls status | match "Label"
Entropy Label RSVP-TE : Enabled Entropy Label SR-TE : EnabledThe following command shows that EL is enabled in RSVP:
[/]
A:admin@PE-1# show router rsvp status | match "Label"
Implicit Null Label: Disabled Node-id in RRO : Exclude
DiffServTE AdmModel: Basic Entropy Label : EnabledThe following command shows that EL is enabled and operational in LSP "LSP-PE-1-PE-2":
[/]
A:admin@PE-1# show router mpls lsp "LSP-PE-1-PE-2" detail | match "Label"
Entropy Label : Enabled Oper Entropy Label : EnabledThe following command shows that the LSP "LSP-PE-1-PE-2" has the flag "entropy-label-capable" in the RSVP tunnel table:
[/]
A:admin@PE-1# show router tunnel-table protocol rsvp detail
===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination : 192.0.2.2/32
NextHop : 192.168.13.2
Tunnel Flags : exclude-for-lfa entropy-label-capable
Age : 00h04m22s
CBF Classes : (Not Specified)
Owner : rsvp Encap : MPLS
Tunnel ID : 1 Preference : 7
Tunnel Label : 524283 Tunnel Metric : 16777215
Tunnel MTU : 8918 Max Label Stack : 1
LSP ID : 5632 Bypass Label : 0
LSP Bandwidth : 0 LSP Weight : 0
-------------------------------------------------------------------------------
Number of tunnel-table entries : 1
Number of tunnel-table entries with LFA : 0
===============================================================================EL is supported on many Layer 2 and Layer 3 services. In this example, VPLS 1 is configured on PE-1 with BGP EVPN-MPLS, as follows:
# on PE-1:
configure {
service {
vpls "VPLS 1" {
service-id 1
customer "1"
bgp 1 { }
bgp-evpn {
evi 1
mpls 1 {
admin-state enable
entropy-label true
auto-bind-tunnel {
resolution filter
resolution-filter {
rsvp true
}
}
}
}
sap 1/1/c2/1:11 { }
sap 1/1/c9/1:11 { }
}Auto-bind-tunnel is resolved to the RSVP LSP "LSP-PE-1-PE-2". A similar configuration is applied on PE-2.
The following command shows that the EL is enabled for BGP-EVPN, as follows:
[/]
A:admin@PE-1# show service id 1 bgp-evpn
===============================================================================
BGP EVPN
===============================================================================
---snip---
===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status : Enabled Bgp Instance : 1
Force Vlan Fwding : Disabled
Force Qinq Fwding : none
Route NextHop Type : system-ipv4
Control Word : Disabled
Max Ecmp Routes : 1
Entropy Label : Enabled
Default Route Tag : none
Split Horizon Group: (Not Specified)
Ingress Rep BUM Lbl: Disabled
Ingress Ucast Lbl : 524280 Ingress Mcast Lbl : 524280
RestProtSrcMacAct : none
Evpn Mpls Encap : Enabled Evpn MplsoUdp : Disabled
Oper Group : (none)
MH Mode : network
Evi 3-byte Auto-RT : Disabled
Dyn Egr Lbl Limit : Disabled
Hash Label : Disabled
Local AC Ingr Lbl : <not-allocated>
BGP EVPN MPLS Auto Bind Tunnel Information
-------------------------------------------------------------------------------
Allow-Flex-Algo-FB : Disabled
Resolution : filter Strict Tnl Tag : Disabled
Max Ecmp Routes : 1
Filter Tunnel Types: rsvp
Weighted Ecmp : Disabled
===============================================================================The iLER PE-1 adds the EL and ELI to the label stack. Traffic load-balancing is performed in P-3 where the traffic is sprayed over all eight links of the LAG. The options for LSR load-balancing are the following:
# on LSRs P-3 and P-4:
configure {
system {
load-balancing
lsr-load-balancing ?
lsr-load-balancing <keyword>
<keyword> - (lbl-only|lbl-ip|ip-only|eth-encap-ip|lbl-ip-l4-teid)
Dynamic Default - lbl-only
Algorithm for system-wide LSR load balancingBy default, the load-balancing settings on P-3 are as follows:
# On P-3:
configure {
system {
load-balancing
info detail
----------------------------------------------
l2tp-load-balancing false
l4-load-balancing false
lsr-load-balancing lbl-only
mc-enh-load-balancing false
service-id-lag-hashing false
system-ip-load-balancing false
----------------------------------------------When the EL and ELI are included in the stack, only the EL label is used for the load-balancing (lbl-only). It is not required to inspect the IP header or any other header.
For the reverse path, PE-2 is the iLER, and P-4 does the spraying of the packets over all links of the LAG. Therefore, LSR load-balancing with lbl-only is also configured on P-4.
EVPN service using LDP tunnel with EL
ELC is disabled by default for LDP, as follows:
[/]
A:admin@PE-1# show router ldp status
===============================================================================
LDP Status for IPv4 LSR ID 192.0.2.1
IPv6 LSR ID ::
===============================================================================
---snip---
Admin State : Up
IPv4 Oper State : Up IPv6 Oper State : Down
---snip---
Entropy Label Capa*: False
---snip---
===============================================================================
* indicates that the corresponding row element may have been truncated.The command to enable ELC is as follows:
# on PE-1:
configure {
router "Base" {
ldp {
entropy-label-capability trueIn the list of LDP active bindings, the egress labels that are pushed by iLER PE-1 on traffic toward an eLER capable of handling an EL get the indication "e", as follows:
[/]
A:admin@PE-1# show router ldp bindings active prefixes ipv4
===============================================================================
LDP Bindings (IPv4 LSR ID 192.0.2.1)
(IPv6 LSR ID ::)
===============================================================================
Label Status:
U - Label In Use, N - Label Not In Use, W - Label Withdrawn
WP - Label Withdraw Pending, BU - Alternate For Fast Re-Route
e - Label ELC
FEC Flags:
LF - Lower FEC, UF - Upper FEC, M - Community Mismatch,
BA - ASBR Backup FEC
(S) - Static (M) - Multi-homed Secondary Support
(B) - BGP Next Hop (BU) - Alternate Next-hop for Fast Re-Route
(I) - SR-ISIS Next Hop (O) - SR-OSPF Next Hop
(C) - FEC resolved with class-based-forwarding
===============================================================================
LDP IPv4 Prefix Bindings (Active)
===============================================================================
Prefix Op
IngLbl EgrLbl
EgrNextHop EgrIf/LspId
-------------------------------------------------------------------------------
192.0.2.1/32 Pop
524287 --
-- --
192.0.2.2/32 Push
-- 524284e
192.168.13.2 1/1/c1/1
192.0.2.2/32 Swap
524282 524284
192.168.13.2 1/1/c1/1
192.0.2.3/32 Push
-- 524287
192.168.13.2 1/1/c1/1
192.0.2.4/32 Push
-- 524285
192.168.13.2 1/1/c1/1
192.0.2.4/32 Swap
524284 524285
192.168.13.2 1/1/c1/1
-------------------------------------------------------------------------------
No. of IPv4 Prefix Active Bindings: 6
===============================================================================Because the iLER adds the EL, only labels which are pushed get the "e"-indication. Labels which are swapped or popped do not get this label.
The detail of the tunnel table for LDP shows that the LDP tunnel toward PE-2 has tunnel flag entropy-label-capable, as follows:
[/]
A:admin@PE-1# show router tunnel-table protocol ldp detail
===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination : 192.0.2.2/32
NextHop : 192.168.13.2
Tunnel Flags : entropy-label-capable
Age : 00h00m38s
CBF Classes : (Not Specified)
Owner : ldp Encap : MPLS
Tunnel ID : 65539 Preference : 9
Tunnel Label : 524284 Tunnel Metric : 30
Tunnel MTU : 8918 Max Label Stack : 1
-------------------------------------------------------------------------------
---snip---
===============================================================================The following EVPN-VPLS uses an LDP transport tunnel and has EL enabled:
# on PE-1:
configure {
service {
vpls "VPLS 2" {
admin-state enable
service-id 2
customer "1"
bgp 1 { }
bgp-evpn {
evi 2
mpls 1 {
admin-state enable
entropy-label true
auto-bind-tunnel {
resolution filter
resolution-filter {
ldp true
}
}
}
}
sap 1/1/c2/1:16 { }
sap 1/1/c9/1:16 { }
}The configuration on PE-1 and PE-2 is similar.
EL is enabled in this service, as follows:
[/]
A:admin@PE-2# show service id 2 bgp-evpn
===============================================================================
BGP EVPN
===============================================================================
---snip---
===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status : Enabled Bgp Instance : 1
Force Vlan Fwding : Disabled
Force Qinq Fwding : none
Route NextHop Type : system-ipv4
Control Word : Disabled
Max Ecmp Routes : 1
Entropy Label : Enabled
Default Route Tag : none
Split Horizon Group: (Not Specified)
Ingress Rep BUM Lbl: Disabled
Ingress Ucast Lbl : 524279 Ingress Mcast Lbl : 524279
RestProtSrcMacAct : none
Evpn Mpls Encap : Enabled Evpn MplsoUdp : Disabled
Oper Group : (none)
MH Mode : network
Evi 3-byte Auto-RT : Disabled
Dyn Egr Lbl Limit : Disabled
Hash Label : Disabled
Local AC Ingr Lbl : <not-allocated>
BGP EVPN MPLS Auto Bind Tunnel Information
-------------------------------------------------------------------------------
Allow-Flex-Algo-FB : Disabled
Resolution : filter Strict Tnl Tag : Disabled
Max Ecmp Routes : 1
Filter Tunnel Types: ldp
Weighted Ecmp : Disabled
===============================================================================LSR load-sharing based on EL
The LSR load-sharing based on the EL results in a granular load-sharing for great numbers of flows, for example, thousands of flows. Each flow has four unique variables: source IP address, destination IP address, source UDP port, and destination UDP port. Each flow gets a unique EL and the P-nodes distribute the load over all eight links in the LAG, based on the hashing algorithm "lbl-only".
In a virtual test setup, such great numbers of flows are difficult to simulate. Nevertheless, the spread may be illustrated by comparing the traffic on each link of the LAG, once in the absence and once in the presence of externally launched traffic, as follows:
- clear lag 1 statistics
- wait for X seconds (example: X=125)
- show lag "lag-1" statistics
or
# On P-3:
- clear port <port> statistics (where <port> is a port in the LAG)
- wait for X seconds (example: X=125)
- show port statistics
- monitor lag 1 rate packets
-
monitor port <port> rate monitor port all-ethernet-rates
As an example:, on P-3, in the absence of externally launched traffic:
[/]
A:admin@P-3# show lag "lag-1" statistics
===============================================================================
LAG Statistics
===============================================================================
Description : N/A
Port-id Input(Packets) Output(Packets)
Input(Bytes) Output(Bytes)
Input Errors Output Errors
-------------------------------------------------------------------------------
1/1/c3/1 228 237
24186 25052
0 0
1/1/c4/1 124 124
15872 15872
0 0
1/1/c5/1 157 156
19997 19872
0 0
1/1/c6/1 132 153
16508 19352
0 0
1/1/c7/1 153 124
19352 15872
0 0
1/1/c8/1 124 124
15872 15872
0 0
1/1/c9/1 124 124
15872 15872
0 0
1/1/c10/1 124 124
15872 15872
0 0
-------------------------------------------------------------------------------
Totals 1166 1166
143531 143636
0 0
===============================================================================while in the presence of externally launched traffic:
[/]
A:admin@P-3# show lag "lag-1" statistics
===============================================================================
LAG Statistics
===============================================================================
Description : N/A
Port-id Input(Packets) Output(Packets)
Input(Bytes) Output(Bytes)
Input Errors Output Errors
-------------------------------------------------------------------------------
1/1/c3/1 278 287
30260 30668
0 0
1/1/c4/1 150 150
19200 19200
0 0
1/1/c5/1 189 189
24075 24075
0 0
1/1/c6/1 160 186
19995 23408
0 0
1/1/c7/1 186 150
23464 19200
0 0
1/1/c8/1 150 150
19200 19200
0 0
1/1/c9/1 150 150
19200 19200
0 0
1/1/c10/1 150 150
19200 19200
0 0
-------------------------------------------------------------------------------
Totals 1413 1412
174594 174151
0 0
===============================================================================Conclusion
The EL is a standard-compliant way to maintain packet ordering within a conversation while load-balancing across multiple ECMP paths or links in a LAG. The EL is supported for both Layer 2 and Layer 3 services and is, therefore, a more general solution than the MPLS hash label.