Congestion notification

SR OS supports the following marking-based congestion notification mechanisms:

Explicit Congestion Notification (ECN)
Low Latency, Low Loss, and Scalable Throughput (L4S)

When using these mechanisms, instead of discarding a packet when congestion is detected, the router marks packets that would otherwise be dropped, so that the endpoints can reduce their sending rate before packet loss and retransmission occur.

Both ECN and L4S use the two-bit ECN field in the IP and IPv6 headers, as described in ECN codepoints, to determine whether a packet should be remarked

For information about each mechanism, see Classic ECN and L4S.

ECN codepoints

The ECN field is carried in the two least significant bits of the Type of Service octet in the IPv4 header and of the Traffic Class octet in the IPv6 header. The following table describes the ECN field values.

Table 1. ECN codepoints
ECN field value	Codepoint	Description
00	Not-ECT	The endpoints do not support ECN. The packet is never CE-marked and remains subject to the normal WRED discard behavior of the queue.
01	ECT(1)	The packet belongs to a flow that uses L4S. The packet is not eligible for CE marking by ECN; see L4S.
10	ECT(0)	The packet belongs to a flow that uses classic ECN. The packet is eligible for CE marking when the queue is congested.
11	CE	Congestion was already experienced at an upstream node. The codepoint is preserved and the packet is forwarded.

Only packets that are presented to the forwarding path as Layer 3 IP or IPv6 packets are remarked. No remarking is performed for Layer 2 SAPs or for MPLS-encapsulated traffic.

Classic ECN

Classic ECN allows the router to signal congestion to the traffic endpoints by marking a packet instead of discarding it. When an egress queue becomes congested, the router sets the CE codepoint in the ECN field of the IP or IPv6 header and forwards the packet.

The traffic endpoint reacts to the CE-marked packet by reducing its transmission rate, as it would typically do to in response to packet loss. Because the packet is delivered instead of discarded, the endpoint avoid the retransmission and the additional round-trip delay that a WRED discard incurs. The congestion is relieved with lower end-to-end delay than is achieved using WRED alone.

ECN marking is performed on egress queues and is enabled in the slope policy that is applied to those queues; see Enabling ECN in a slope policy. ECN is supported on FP5-based hardware only.

Note: ECN is not supported in subscriber management contexts, however, the system does not block the configuration. The display of per-queue counters may be affected if ECN is enabled.

ECN marking on egress queues

SR OS supports ECN marking only on the following:

network egress queues
SAP egress queues
queues within an egress queue group

ECN is not supported on ingress queues.

Only ECT(0) packets are eligible for CE marking. A packet is marked CE when the WRED algorithm determines that the packet would have been discarded and buffer space is still available in the queue. The packet is then queued and forwarded with the CE codepoint set, instead of being discarded.

If no buffer space is available, the packet is discarded even when it is eligible for CE marking. Packets that are not eligible, such as Not-ECT packets, continue to be discarded by the WRED algorithm as normal.

Enabling ECN in a slope policy

ECN takes effect only when an ECN-enabled slope policy is assigned to egress queues, either through the egress pool of a port or directly to a queue using WRED per queue.

Use the following command to enable ECN in a slope policy:

MD-CLI
```
configure qos slope-policy ecn
```
classic CLI
```
configure qos slope-policy ecn-enable
```

ECN queue statistics

SR OS typically maintains the following queue statistics counters:

forwarded in-profile and inplus-profile
forwarded out-of-profile and exceed-profile
dropped in-profile and inplus-profile
dropped out-of-profile and exceed-profile

When a valid ECN slope policy is assigned to a queue, the preceding queue statistics counters are replaced with ECN-specific statistics counters, as described in the following table.

Table 2. ECN queue statistics
ECN statistic	Description	Queue counter used
Forwarded unmarked	Packets and octets forwarded without the CE codepoint set	Forwarded in-profile and inplus-profile
Forwarded marked	Packets and octets forwarded with the CE codepoint set	Forwarded out-of-profile and exceed-profile
ECN dropped	Packets and octets that were eligible for CE marking but were discarded because no buffer space was available	Dropped in-profile and inplus-profile

Note: When a queue operates in ECN mode, no statistics are accumulated in the "dropped out-of-profile and exceed-profile" counter, and the statistic is omitted from show outputs.

The following output shows the per-queue statistics for a SAP with an egress queue operating in ECN mode. In ECN mode, the queue heading includes the stats mode and the three ECN statistics are displayed as "For. ECN unmarked", "For. ECN marked", and "Dro. ECN", instead of the per-profile forwarded and dropped labels shown for the other queues.

show service id "10" sap "1/2/c1/3:1" stats

===============================================================================
Service Access Points(SAP)
===============================================================================
...
-------------------------------------------------------------------------------
Sap per Queue stats
-------------------------------------------------------------------------------
                        Packets                 Octets
 
Ingress Queue 1 (Unicast) (Priority)
Off. HiPrio           : 0                       0
Off. LowPrio          : 0                       0
Dro. HiPrio           : 0                       0
Dro. LowPrio          : 0                       0
For. InProf           : 0                       0
For. OutProf          : 0                       0
 
Egress Queue 1 (Stats mode: ecn)
For. ECN unmarked     : 0                       0
For. ECN marked       : 0                       0
Dro. ECN              : 0                       0
===============================================================================

ECN LAG and satellite port considerations

ECN is supported on standalone satellite ports and satellite ports configured as members of a LAG.

The user must ensure that slope policies are assigned consistently across all LAG member ports. The system does not perform a consistency check for ECN across LAG member ports that reside in different buffer pools. If ECN is enabled on any member port of a LAG, the statistics for all member ports of that LAG are presented as ECN-marked and ECN-unmarked, including the statistics for member ports where ECN is not enabled.

Configuring ECN

The following examples show a slope policy with ECN enabled, applied to a SAP egress queue using WRED per queue. The low and exceed slopes are enabled in the slope policy so that the WRED algorithm is active on the queue; ECT(0) packets that the algorithm selects for discard are CE-marked and forwarded while buffer space remains available.

For information about WRED per queue and the slope usage that each WRED queue mode supports, see WRED per queue.

The following example shows an ECN slope policy and egress queue configuration.

MD-CLI

[ex:/configure qos]
A:admin@node-2# info
    slope-policy "ecn-policy" {
        ecn true
        low-slope {
            admin-state enable
            start-avg 50
            max-avg 75
        }
        exceed-slope {
            admin-state enable
            start-avg 30
            max-avg 55
        }
    }
    sap-egress "ecn-egr" {
        policy-id 2
        queue 1 {
            wred-queue {
                policy "ecn-policy"
                mode native
                usage exceed-low
            }
        }
    }

classic CLI

A:node-2>config>qos# info
----------------------------------------------
        slope-policy "ecn-policy" create
            highplus-slope
                shutdown
            exit
            high-slope
                shutdown
            exit
            low-slope
                no shutdown
            exit
            exceed-slope
                no shutdown
            exit
            ecn-enable
        exit
#--------------------------------------------------
echo "QoS Policy Configuration"
#--------------------------------------------------
        sap-egress 2 name "ecn-egr" create
            queue 1 create
                wred-queue policy "ecn-policy" mode native slope-usage exceed-low
            exit
        exit
----------------------------------------------

L4S

L4S is intended for residential applications that are sensitive to latency and jitter rather than to throughput.

An L4S-capable application marks its packets with the ECT(1) codepoint in the ECN field, which identifies the flow as L4S-capable. The network marks those packets with the CE codepoint as soon as congestion begins to build, rather than waiting for a queue to fill. Because an L4S sender reacts to the proportion of CE-marked packets instead of to packet loss, it can be signaled early and often, which keeps queueing delay and loss low while throughput remains scalable. In SR OS, the marking of L4S capable flows is achieved with static rate management (SRM) using a two-rate, three-color marking policer.

L4S is supported on FP4- and FP5-based hardware. L4S is supported on standalone satellite ports and satellite ports configured as members of a LAG.

SR OS does not support L4S for Layer 2 or MPLS-encapsulated packets. Only packets presented to the forwarding path as IP or IPv6 packets are CE-marked.

L4S forwarding class classification

SR OS supports L4S in the following classification contexts:

SAP ingress DSCP and dot1p classification
SAP egress DSCP classification

A single forwarding class carries either L4S traffic or non-L4S traffic, but not both. L4S and non-L4S traffic must therefore be separated at classification time so that each can be mapped to its own forwarding class and policed independently.

The l4s-fc command option allows a single DSCP or dot1p codepoint to resolve to two forwarding classes. The system inspects the ECN field of each packet matching the codepoint and splits the traffic as follows:

Packets carrying ECT(1) or CE are assigned to the forwarding class specified by the l4s-fc command option.
All other packets, including Not-ECN-Capable Transport (Not-ECT) and ECT(0) packets, are assigned to the forwarding class specified by the fc command option.

When an l4s-fc is configured for DSCP or dot1p, the regular fc for resolving non-L4S traffic must also be specified. Also, both must be assigned a full forwarding class; subclasses are not supported when using L4S.

L4S policer operation

Use the following commands to configure L4S mode on a policer.

configure qos sap-ingress policer mode
configure qos sap-egress policer mode

A policer configured in L4S mode provides the congestion signal for L4S traffic and processes traffic as follows:

Traffic up to the CIR is forwarded unchanged.
Traffic above the CIR and up to the PIR is CE-marked and forwarded.
Traffic above the PIR is discarded.

Only ECT(1) packets are eligible for CE marking. A packet that already carries the CE codepoint on arrival is forwarded with CE set and is counted as marked.

An L4S-mode policer ignores the profile information derived from classification and does not assign a profile to its output. The profile configured in the classification entry may still be used by SAP ingress queues and network egress queues, but it has no effect on the policing decision of an L4S-mode policer. As a result, the profile-related policer behaviors described in Policer interaction with profile, discard eligibility, and ingress priority do not apply.

L4S is supported for all traffic types (unicast, multicast, broadcast, and BUM) at SAP ingress.

L4S-mode policers support all existing policer features.

L4S policer statistics

An L4S-mode policer supports the following counters in addition to the normal statistics counters:

ecn-marked — packets and octets forwarded with the CE codepoint set, including packets that already carried CE when they arrived at the policer. Packets and octets forwarded as ECN unmarked (all other ECN codepoints)
ecn-codepoints — packets and octets counted per ECN codepoint value as received, before any CE marking is applied by the policer

Use the following commands to configure the statistics accounting mode of an L4S policer:

configure qos sap-ingress policer stat-mode
configure qos sap-egress policer stat-mode

L4S queuing and marking considerations

SR OS does not create a dedicated low-latency queue for the L4S forwarding class and does not validate the queue configuration against the L4S policer configuration. To achieve minimum latency and jitter, Nokia recommends assigning a higher-priority, expedited queue with a shallow drop tail to policers when using L4S.

The system does not validate slope policies applied to the queues that carry L4S traffic against the L4S configuration. The user must ensure that a WRED or drop-tail configuration does not result in the router discarding L4S packets before the policer can CE-mark.

L4S does not affect egress packet marking. L4S traffic is forwarded as normal egress traffic, and the egress marking configuration for that forwarding class determines how L4S traffic is presented to the next hop.

Configuring L4S

The following example shows an L4S configuration in the upstream direction where traffic flows from SAP ingress to network egress. The DSCP=AF11 codepoint is used by the application for both its L4S and its non-L4S flows, so a single classification entry maps the codepoint to the "be" forwarding class for non-L4S traffic and to the "h1" forwarding class for L4S traffic. The "h1" traffic is policed by an L4S-mode policer that reports ecn-marked statistics, while the "be" traffic is policed by a default-mode policer with minimal statistics.

MD-CLI

[ex:/configure qos]
A:admin@node-2# info
    sap-ingress "l4s-ingress" {
        policy-id 2
        dscp af11 {
            fc "be"
            l4s-fc h1
        }
        policer 1 {
            stat-mode ecn-marked
            mode l4s
            percent-rate {
                pir 50.0
                cir 20.0
            }
        }
        policer 2 {
            stat-mode minimal
            mode default
            percent-rate {
                pir 100.0
                cir 0.0
            }
        }
        fc "be" {
            policer 2
        }
        fc "h1" {
            policer 1
        }
    }
    network-queue "l4s-net-queue" {
        fc be {
            queue 1
        }
        fc h1 {
            queue 3
        }
        queue 1 {
            rate {
                cir 0
            }
        }
        queue 3 {
            queue-type expedited
            rate {
                pir 100
                cir 20
            }
        }
    }
    network "l4s-network" {
        policy-id 2
        egress {
            fc be {
                lsp-exp-in-profile 0
                lsp-exp-out-profile 0
            }
            fc h1 {
                lsp-exp-in-profile 3
                lsp-exp-out-profile 3
            }
        }
    }

classic CLI

A:node-2>config>qos# info
----------------------------------------------
        network-queue "l4s-net-queue" create
            queue 1 create
                mbs 50
            exit
            queue 3 expedite create
                rate 100 cir 20
            exit
            queue 9 multipoint create
                mbs 50
            exit
            fc be create
                multicast-queue 9
                queue 1
            exit
            fc h1 create
                multicast-queue 9
                queue 3
            exit
        exit
#--------------------------------------------------
echo "QoS Policy Configuration"
#--------------------------------------------------
        sap-ingress 2 name "l4s-ingress" create
            queue 1 create
            exit
            queue 11 multipoint create
            exit
            policer 1 create
                stat-mode ecn-codepoints
                mode l4s
                percent-rate 50.00 cir 20.00
            exit
            policer 2 create
                percent-rate 100.00
            exit
            fc "be" create
                policer 2
            exit
            fc "h1" create
                policer 1
            exit
            dscp af11 fc "be" l4s-fc "h1"
        exit
        network 2 name "l4s-network" create
            ingress
            exit
            egress
                fc h1
                    lsp-exp-in-profile 3
                    lsp-exp-out-profile 3
                exit
            exit
        exit

----------------------------------------------