Entropy Label

This chapter provides information about the Entropy Label (EL).

Topics in this chapter include:

Applicability

This chapter was initially written for SR OS Release 14.0.R4. The CLI in the current edition corresponds to SR OS Release 24.3.R1.

RFC 6391 hash label or flow-aware transport label is supported in SR OS Release 8.0.R1, and later. RFC 6790 Entropy Labels (ELs) are supported on RSVP and BGP tunnels in Release 14.0.R1, and later, and supported on LDP tunnels in Release 14.0.R4, and later.

Overview

Entropy is the degree of disorder or uncertainty in a system. SR OS supports both the MPLS EL and the hash label, but they are mutually exclusive. These labels allow Label Switched Routers (LSRs) to load-balance labeled packets in a granular way without the need to inspect the IPv4 or IPv6 header. The hash label is applicable to services such as Epipe VLL, VPLS, IES (spoke-SDP), and VPRN services. The main advantage of the EL compared to the hash label is that the EL can be applied to a wider range of services, such as EVPN, BGP VPWS, Fpipe VLL, Ipipe VLL, H-VPLS, and BGP RFC 3107 tunnels.

Load-balancing of flows based on hash label or entropy label shows that different flows from an ingress Label Edge Router (iLER) are load-balanced across different paths in the MPLS network toward the egress Label Edge Router (eLER).

Figure 1. Load-balancing of flows based on hash label or entropy label

The EL is inserted below the innermost Tunnel Label (TL) in the label stack, along with the Entropy Label Indicator (ELI), which is a special purpose MPLS label with a value of 7 to indicate that the next label in the stack is an EL. As with the hash label, the value of the EL is calculated based on a hash of the packet payload header (IP and Layer 4). The EL is inserted as deep as possible in the stack to ensure preservation for as far as possible through the network. When the EL and ELI are present, load-balancing on the transit LSRs can be configured to only take into account the EL label for Link Aggregation Group (LAG) and Equal Cost Multi-Path (ECMP). Load-balancing on the LSR can be configured to take into account the IP header also, but that is not required when the EL and ELI are present. The eLER removes the EL and ELI before forwarding the packet to its final destination.

The EL requires two additional labels in the label stack, which may result in an unsupported label stack depth in an intermediate (possibly third-party) LSR. SR OS allows control of the EL and ELI insertion and accounting of extra labels in the tunnel table. The supported label stack depth is 12 in SR OS Release 14.0, including transport, service, hash, and OAM labels.

Label stack with hash label versus label stack with EL and ELI shows a comparison between a label stack with a hash label and a label stack with an EL and ELI.

Figure 2. Label stack with hash label versus label stack with EL and ELI

Hash labels or ELs are inserted by the iLER, based on the incoming packet header. The iLER uses a single label to represent one or more flows, based on the hash of the incoming packet header IP and Layer 4. The hash label is a single label that is inserted at the bottom of the stack, below the service label. The EL is inserted together with the ELI below the innermost tunnel label, but above the service label. The entropy and hash label value is outside of the range for MPLS label values, because the most significant bit equals 1 for entropy and hash labels and 0 for MPLS labels.

The EL is used as part of the LSR hashing algorithm for spraying packets over multi-port LAGs, ECMP, or BGP tunnels with multiple downstream interfaces. The packet ordering is preserved because one label is used per conversation. Hashing based on a label stack containing an EL per service will become more granular than when based on a standard label stack alone.

Downstream LERs signal EL capability to ILER shows how the eLER signals its EL capability to the iLER.

Figure 3. Downstream LERs signal EL capability to ILER

The Entropy Label Capability (ELC) is signaled by the eLER and indicates the ability to receive and process the ELs. This can be advertised for LDP and RSVP. However, ELC signaling is not supported for BGP tunnels, because no agreed standard exists in the IETF. RFC 6790 introduced the ELC BGP attribute that can be signaled by the eLER to indicate that it supports EL. According to RFC 6790, LSRs incapable of processing ELs must remove the ELC BGP attribute, but this requirement could not be guaranteed; therefore, the ELC BGP attribute has been deprecated in RFC 7447.

As a workaround, at the iLER, an override of the ELC for a BGP tunnel can be manually configured. After this ELC has been overridden, the BGP sender assumes that the receiver is capable of receiving and processing the ELs, regardless of the signaled ELC. The iLER inserts an EL on a tunnel for which the ELC is confirmed by the downstream peer or when the ELC is overridden by configuration.

ELC signaling can be enabled for LDP on the LERs, as follows:

# on PE-1, PE-2:
configure
    router
        ldp
            entropy-label-capability

ELC signaling for RSVP can be enabled as follows:

# on PE-1, PE-2:
configure
    router
        rsvp
            entropy-label-capability

As previously described, the lack of an IETF standard for BGP tunnels means that ELC is ignored by the receiving LER, and no EL is inserted. Therefore, an override is required that assumes that the far-end LER has ELC, and allows insertion of the EL. The override is enabled as follows:

# on PE-1, PE-2:
configure
    router
        bgp
            override-tunnel-elc

When the ELC is overridden for BGP, the iLER assumes that the receiver can handle the EL.

Configuration

In this section, an EVPN-VPLS is configured on PE-1 and PE-2 to illustrate LAG hashing based on EL. Example topology shows the topology used for this example. Load-balancing of the traffic is performed in the P-routers that are connected by a LAG with eight links.

Figure 4. Example topology

The initial configuration of the PE/P-routers includes the following:

  • Cards, MDAs, ports

  • LAG with eight network links between P-3 and P-4

  • Router interfaces

  • IGP (IS-IS or OSPF)

  • LDP enabled on all interfaces

  • MPLS enabled on all interfaces. RSVP enabled.

  • iBGP configured with P-3 as route reflector (RR)

For the configuration of EVPN-MPLS, the BGP configuration needs to include the address family EVPN, as follows. See chapter EVPN for MPLS Tunnels for more information.

# on PE-1, PE-2:
configure
    router
        autonomous-system 64500
        bgp
            rapid-update evpn
            group "iBGP"
                family evpn
                peer-as 64500
                neighbor 192.0.2.3
                exit
            exit
        exit

EVPN service using RSVP tunnel with EL

RSVP LSP "LSP-PE-1-PE-2" from PE-1 to PE-2 via P-3 and P-4 shows LSP "LSP-PE-1-PE-2" from PE-1 to PE-2 via core routers P-3 and P-4.

Figure 5. RSVP LSP "LSP-PE-1-PE-2" from PE-1 to PE-2 via P-3 and P-4

The LSP "LSP-PE-1-PE-2" is configured on PE-1, as follows:

# on PE-1:
configure
    router
        mpls
            path "path-PE-1-PE-2"
                hop 10 192.168.13.2 strict
                hop 20 192.168.34.2 strict
                hop 30 192.168.24.1 strict
                no shutdown
            exit
            lsp "LSP-PE-1-PE-2"
                to 192.0.2.2
                primary "path-PE-1-PE-2"
                exit
                no shutdown
            exit
        exit

The configuration for LSP "LSP-PE-2-PE-1" on PE-2 is similar.

ELC is disabled by default, and needs to be enabled for RSVP on the eLERs, as follows:

# on PE-1, PE-2:
configure
    router
        rsvp
            entropy-label-capability

The EL can be disabled (force-disable) or enabled in the mpls context on the iLER, as follows:

# on PE-1, PE-2:
configure
    router
        mpls
            entropy-label ?
  - entropy-label rsvp-te <rsvp-te>
  - entropy-label sr-te <sr-te>

 <rsvp-te>            : <force-disable | enable>
 <sr-te>              : <force-disable | enable>

The EL can also be disabled (force-disable) or enabled per LSP, but there is a third option to inherit the EL settings from the mpls context, as follows:

# on PE-1:
configure
    router
        mpls
            lsp "LSP-PE-1-PE-2"
                entropy-label ?
  - entropy-label {force-disable | inherit | enable}

 <force-disable | i*> : force-disable|inherit|enable

By default, the EL on the LSP inherits the EL settings in the mpls context, as follows:

# On PE-1:
configure
    router
        mpls
            lsp "lsp-PE-1-PE-2"
                info detail | match "entropy-label"
entropy-label inherit

When LSP templates are used, EL can be configured within the lsp-template context for single-hop and mesh point-to-point LSPs, as follows:

# on PE-1:
configure
    router
        mpls
            lsp-template "LSPtemplate1" one-hop-p2p
                entropy-label ?
  - entropy-label {force-disable | inherit | enable}

 <force-disable | i*> : force-disable|inherit|enable
# on PE-1:
configure
    router
        mpls
            lsp-template "LSPtemplate2" mesh-p2p
                entropy-label ?
  - entropy-label {force-disable | inherit | enable}

 <force-disable | i*> : force-disable|inherit|enable

When the EL settings are modified, for example, from inherit to enabled, the changes only take effect after the LSP has been cleared or MPLS has been bounced, using shutdown/no shutdown. The following message is raised when the configuration is modified for an LSP in no shutdown state:

# On PE-1:
configure
    router
        mpls
            lsp "lsp-PE-1-PE-2"
                entropy-label enable
INFO: MPLS #1029 Entropy Label change is not operational - LSP must be bounced

After the LSP is bounced (shutdown/no shutdown), the following command shows that EL is enabled in MPLS:

*A:PE-1# show router mpls status | match "Label"
Entropy Label RSVP-TE     : Enabled     Entropy Label SR-TE       : Enabled

The following command shows that EL is enabled in RSVP:

*A:PE-1# show router rsvp status | match "Label"
Implicit Null Label: Disabled           Node-id in RRO     : Exclude
DiffServTE AdmModel: Basic              Entropy Label      : Enabled

The following command shows that EL is enabled and operational in LSP "LSP-PE-1-PE-2":

*A:PE-1# show router mpls lsp "LSP-PE-1-PE-2" detail | match "Label"
Entropy Label   : Enabled                   Oper Entropy Label   : Enabled

The following command shows that the LSP "LSP-PE-1-PE-2" has the flag "entropy-label-capable" in the RSVP tunnel table:

*A:PE-1# show router tunnel-table protocol rsvp detail

===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination      : 192.0.2.2/32
NextHop          : 192.168.13.2
Tunnel Flags     : exclude-for-lfa entropy-label-capable
Age              : 00h06m42s
CBF Classes      : (Not Specified)
Owner            : rsvp                 Encap            : MPLS
Tunnel ID        : 1                    Preference       : 7
Tunnel Label     : 524281               Tunnel Metric    : 16777215
Tunnel MTU       : 8918                 Max Label Stack  : 1
LSP ID           : 32258                Bypass Label     : 0
LSP Bandwidth    : 0                    LSP Weight       : 0
-------------------------------------------------------------------------------
Number of tunnel-table entries          : 1
Number of tunnel-table entries with LFA : 0
===============================================================================

EL is supported on many Layer 2 and Layer 3 services. In this example, VPLS 1 is configured on PE-1 with BGP EVPN-MPLS, as follows:

# on PE-1:
configure
    service
        vpls 1 name "VPLS 1" customer 1 create
            bgp
            exit
            bgp-evpn
                evi 1
                mpls bgp 1
                    entropy-label
                    auto-bind-tunnel
                        resolution-filter
                            rsvp
                        exit
                        resolution filter
                    exit
                    no shutdown
                exit
            exit
            sap 1/1/c2/1:11 create
            exit
            sap 1/1/c9/1:11 create
            exit
            no shutdown

Auto-bind-tunnel is resolved to the RSVP LSP "LSP-PE-1-PE-2". A similar configuration is applied on PE-2.

The following command shows that the EL is enabled for BGP-EVPN, as follows:

*A:PE-1# show service id 1 bgp-evpn 

===============================================================================
BGP EVPN
===============================================================================
---snip---

===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status       : Enabled            Bgp Instance       : 1
Force Vlan Fwding  : Disabled           
Force Qinq Fwding  : none               
Route NextHop Type : system-ipv4        
Control Word       : Disabled           
Max Ecmp Routes    : 1                  
Entropy Label      : Enabled            
Default Route Tag  : none               
Split Horizon Group: (Not Specified)
Ingress Rep BUM Lbl: Disabled           
Ingress Ucast Lbl  : 524281             Ingress Mcast Lbl  : 524281
RestProtSrcMacAct  : none               
Evpn Mpls Encap    : Enabled            Evpn MplsoUdp      : Disabled
Oper Group         : (none)             
MH Mode            : network            
Evi 3-byte Auto-RT : Disabled           
Dyn Egr Lbl Limit  : Disabled           
Hash Label         : Disabled           
Local AC Ingr Lbl  : <not-allocated>

BGP EVPN MPLS Auto Bind Tunnel Information
-------------------------------------------------------------------------------
Allow-Flex-Algo-FB : Disabled           
Resolution         : filter             Strict Tnl Tag     : Disabled
Max Ecmp Routes    : 1                  
Filter Tunnel Types: rsvp
Weighted Ecmp      : Disabled           
===============================================================================

The iLER PE-1 adds the EL and ELI to the label stack. Traffic load-balancing is performed in P-3 where the traffic is sprayed over all eight links of the LAG. The options for LSR load-balancing are the following:

# on LSRs P-3 and P-4:
configure
    system
        load-balancing
            lsr-load-balancing ?            # system-wide LSR load balancing
  - lsr-load-balancing <hashing-algorithm>
  - no lsr-load-balancing

 <hashing-algorithm>  : lbl-only|lbl-ip|ip-only|eth-encap-ip|lbl-ip-l4-teid

By default, the load-balancing settings on P-3 are as follows:

# On P-3:
configure
    system
        load-balancing
            info detail
----------------------------------------------
            no l2tp-load-balancing
            no l4-load-balancing
            lsr-load-balancing lbl-only
            no system-ip-load-balancing
            no mc-enh-load-balancing
            no service-id-lag-hashing
----------------------------------------------

When the EL and ELI are included in the stack, only the EL label is used for the load-balancing (lbl-only). It is not required to inspect the IP header or any other header.

For the reverse path, PE-2 is the iLER, and P-4 does the spraying of the packets over all links of the LAG. Therefore, LSR load-balancing with lbl-only is also configured on P-4.

EVPN service using LDP tunnel with EL

ELC is disabled by default for LDP, as follows:

*A:PE-1# show router ldp status 

===============================================================================
LDP Status for IPv4 LSR ID 192.0.2.1
               IPv6 LSR ID ::
===============================================================================
---snip---
Admin State        : Up                   
IPv4 Oper State    : Up                   IPv6 Oper State      : Down
---snip---

Entropy Label Capa*: False                
---snip---
===============================================================================
* indicates that the corresponding row element may have been truncated.

The command to enable ELC is as follows:

# on PE-1:
configure
    router
        ldp
            entropy-label-capability

In the list of LDP active bindings, the egress labels that are pushed by iLER PE-1 on traffic toward an eLER capable of handling an EL get the indication "e", as follows:

*A:PE-1# show router ldp bindings active prefixes ipv4 

===============================================================================
LDP Bindings (IPv4 LSR ID 192.0.2.1)
             (IPv6 LSR ID ::)
===============================================================================
Label Status:
        U - Label In Use, N - Label Not In Use, W - Label Withdrawn
        WP - Label Withdraw Pending, BU - Alternate For Fast Re-Route
        e - Label ELC
FEC Flags:
        LF - Lower FEC, UF - Upper FEC, M - Community Mismatch,
        BA - ASBR Backup FEC
        (S) - Static           (M) - Multi-homed Secondary Support
        (B) - BGP Next Hop     (BU) - Alternate Next-hop for Fast Re-Route
        (I) - SR-ISIS Next Hop (O) - SR-OSPF Next Hop
        (C) - FEC resolved with class-based-forwarding
===============================================================================
LDP IPv4 Prefix Bindings (Active)
===============================================================================
Prefix                                      Op
IngLbl                                      EgrLbl
EgrNextHop                                  EgrIf/LspId
-------------------------------------------------------------------------------
192.0.2.1/32                                Pop
524287                                        --
  --                                          --
                                             
192.0.2.2/32                                Push
  --                                        524281e
192.168.13.2                                1/1/c1/1
                                             
192.0.2.2/32                                Swap
524279                                      524281
192.168.13.2                                1/1/c1/1
                                             
192.0.2.3/32                                Push
  --                                        524287
192.168.13.2                                1/1/c1/1
                                             
192.0.2.4/32                                Push
  --                                        524282
192.168.13.2                                1/1/c1/1
                                             
192.0.2.4/32                                Swap
524280                                      524282
192.168.13.2                                1/1/c1/1
                                             
-------------------------------------------------------------------------------
No. of IPv4 Prefix Active Bindings: 6
===============================================================================

Because the iLER adds the EL, only labels which are pushed get the "e"-indication. Labels which are swapped or popped do not get this label.

The detail of the tunnel table for LDP shows that the LDP tunnel toward PE-2 has tunnel flag entropy-label-capable, as follows:

*A:PE-1# show router tunnel-table protocol ldp detail 

===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination      : 192.0.2.2/32
NextHop          : 192.168.13.2
Tunnel Flags     : entropy-label-capable
Age              : 00h01m32s
CBF Classes      : (Not Specified)
Owner            : ldp                  Encap            : MPLS
Tunnel ID        : 65556                Preference       : 9
Tunnel Label     : 524281               Tunnel Metric    : 30
Tunnel MTU       : 8918                 Max Label Stack  : 1
-------------------------------------------------------------------------------
---snip---
===============================================================================

The following EVPN-VPLS uses an LDP transport tunnel and has EL enabled:

# on PE-1:
configure
    service
        vpls 2 name "VPLS 2" customer 1 create
            bgp
            exit
            bgp-evpn
                evi 2
                mpls bgp 1
                    entropy-label
                    auto-bind-tunnel
                        resolution-filter
                            ldp
                        exit
                        resolution filter
                    exit
                    no shutdown
                exit
            exit
            sap 1/1/c2/1:16 create
            exit
            sap 1/1/c9/1:16 create
            exit
            no shutdown

The configuration on PE-1 and PE-2 is similar.

EL is enabled in this service, as follows:

*A:PE-2# show service id 2 bgp-evpn 

===============================================================================
BGP EVPN
===============================================================================
---snip---
===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status       : Enabled            Bgp Instance       : 1
Force Vlan Fwding  : Disabled           
Force Qinq Fwding  : none               
Route NextHop Type : system-ipv4        
Control Word       : Disabled           
Max Ecmp Routes    : 1                  
Entropy Label      : Enabled            
Default Route Tag  : none               
Split Horizon Group: (Not Specified)
Ingress Rep BUM Lbl: Disabled           
Ingress Ucast Lbl  : 524283             Ingress Mcast Lbl  : 524283
RestProtSrcMacAct  : none               
Evpn Mpls Encap    : Enabled            Evpn MplsoUdp      : Disabled
Oper Group         : (none)             
MH Mode            : network            
Evi 3-byte Auto-RT : Disabled           
Dyn Egr Lbl Limit  : Disabled           
Hash Label         : Disabled           
Local AC Ingr Lbl  : <not-allocated>

BGP EVPN MPLS Auto Bind Tunnel Information
-------------------------------------------------------------------------------
Allow-Flex-Algo-FB : Disabled           
Resolution         : filter             Strict Tnl Tag     : Disabled
Max Ecmp Routes    : 1                  
Filter Tunnel Types: ldp
Weighted Ecmp      : Disabled           
===============================================================================

LSR load-sharing based on EL

The LSR load-sharing based on the EL results in a granular load-sharing for great numbers of flows, for example, thousands of flows. Each flow has four unique variables: source IP address, destination IP address, source UDP port, and destination UDP port. Each flow gets a unique EL and the P-nodes distribute the load over all eight links in the LAG, based on the hashing algorithm "lbl-only".

In a virtual test setup, such great numbers of flows are difficult to simulate. Nevertheless, the spread may be illustrated by comparing the traffic on each link of the LAG, once in the absence and once in the presence of externally launched traffic, as follows:

# On P-3:
  1. clear lag 1 statistics
  2. wait for X seconds (example: X=125)
  3. show lag 1 statistics

or

# On P-3:

  1. clear port <port> statistics (where <port> is a port in the LAG)
  2. wait for X seconds (example: X=125)
  3. show port statistics
In the presence of great numbers of flows, the spread may be illustrated with:
  • monitor lag 1 rate packets
  • monitor port <port> rate

As an example:, on P-3, in the absence of externally launched traffic:

*A:P-3# show lag 1 statistics 

===============================================================================
LAG Statistics
===============================================================================
Description        : N/A
Port-id                              Input(Packets)       Output(Packets)
                                     Input(Bytes)         Output(Bytes)
                                     Input Errors         Output Errors
-------------------------------------------------------------------------------
1/1/c3/1                             236                  241
                                     25384                25559
                                     0                    0
1/1/c4/1                             125                  125
                                     16000                16000
                                     0                    0
1/1/c5/1                             158                  158
                                     20125                20125
                                     0                    0
1/1/c6/1                             133                  154
                                     16636                19480
                                     0                    0
1/1/c7/1                             153                  125
                                     19360                16000
                                     0                    0
1/1/c8/1                             125                  125
                                     16000                16000
                                     0                    0
1/1/c9/1                             125                  125
                                     16000                16000
                                     0                    0
1/1/c10/1                            125                  125
                                     16000                16000
                                     0                    0
-------------------------------------------------------------------------------
Totals                               1180                 1178
                                     145505               145164
                                     0                    0
===============================================================================

while in the presence of externally launched traffic:

*A:P-3# show lag 1 statistics 

===============================================================================
LAG Statistics
===============================================================================
Description        : N/A
Port-id                              Input(Packets)       Output(Packets)
                                     Input(Bytes)         Output(Bytes)
                                     Input Errors         Output Errors
-------------------------------------------------------------------------------
1/1/c3/1                             307                  320
                                     33488                34595
                                     0                    0
1/1/c4/1                             161                  165
                                     20608                21219
                                     0                    0
1/1/c5/1                             204                  207
                                     25983                26511
                                     0                    0
1/1/c6/1                             176                  215
                                     21895                26472
                                     0                    0
1/1/c7/1                             217                  174
                                     27406                23241
                                     0                    0
1/1/c8/1                             167                  167
                                     21913                21742
                                     0                    0
1/1/c9/1                             168                  166
                                     22062                21545
                                     0                    0
1/1/c10/1                            163                  165
                                     21062                21156
                                     0                    0
-------------------------------------------------------------------------------
Totals                               1563                 1579
                                     194417               196481
                                     0                    0
===============================================================================
When thousands of flows are sent via P-3 and P-4, the flows are evenly spread in each direction.

Conclusion

The EL is a standard-compliant way to maintain packet ordering within a conversation while load-balancing across multiple ECMP paths or links in a LAG. The EL is supported for both Layer 2 and Layer 3 services and is, therefore, a more general solution than the MPLS hash label.