QoS and QoS policies

On the 2-port 10GigE (Ethernet) Adapter card and 2-port 10GigE (Ethernet) module, traffic can flow between the Layer 2 bridging domain and the Layer 3 IP domain (see Ingress and egress traffic on a 2-port 10GigE (Ethernet) Adapter card). In the bridging domain, ring traffic flows from one ring port to another, as well as to and from the add/drop port. From the network point of view, traffic from the ring toward the add/drop port and the v-port is considered ingress traffic (drop traffic). Similarly, traffic from the fabric toward the v-port and the add/drop port is considered egress traffic (add traffic).

The 2-port 10GigE (Ethernet) Adapter card or 2-port 10GigE (Ethernet) module functions as an add/drop card to a network side 10 Gb/s optical ring. Conceptually, the card or module should be envisioned as having two domains: a Layer 2 bridging domain where the add/drop function operates and a Layer 3 IP domain where the normal IP processing and IP nodal traffic flows are managed. Ingress and egress traffic flow remains in the context of the nodal fabric. The ring ports are considered to be east-facing and west-facing and are referenced as Port 1 and Port 2. A virtual port (or v-port) provides the interface to the IP domain within the structure of the card or module.

The 7705 SAR supports two types of buffer pools that allocate memory as follows:

• reserved pool – represents the CBS that is guaranteed for all queues. The reserved pool is limited to a maximum of 75% of the total buffer space.

• shared pool – represents the buffer space that remains after the reserved pool has been allocated. The shared pool always has at least 25% of the total buffer space.

Both buffer pools can be displayed in the CLI using the show pools command.

On the access side, CBS is configured in bytes and MBS in bytes or kilobytes using the CLI. See, for example, the config>qos>sap-ingress/egress>queue>cbs and mbs configuration commands.

On the network side, CBS and MBS values are expressed as a percentage of the total number of available buffers. If the buffer space is further segregated into pools (for example, ingress and egress, access and network, or a combination of these), the CBS and MBS values are expressed as a percentage of the applicable buffer pool. See the config>qos>network-queue>queue>cbs and mbs configuration commands.

The configured CBS and MBS values are converted to the number of buffers by dividing the CBS or MBS value by a fixed buffer size of 512 bytes or 2304 bytes, depending on the type of adapter card or platform. The number of buffers can be displayed for an adapter card using the show pools command.

Packetization buffers and queues are supported in the packet memory of each adapter card or platform. All adapter cards and platforms allocate a fixed space for each buffer. The 7705 SAR supports two buffer sizes: 512 bytes or 2304 bytes, depending on the type of adapter card or platform.

The adapter cards and platforms that support a buffer size of 2304 bytes do not support buffer chaining (see the description below) and only allow a 1-to-1 correspondence of packets to buffers.

The adapter cards and platforms that support a buffer of size of 512 bytes use a method called buffer chaining to process packets that are larger than 512 bytes. To accommodate packets that are larger than 512 bytes, these adapter cards or platforms divide the packet dynamically into a series of concatenated 512-byte buffers. An internal 64-byte header is prepended to the packet, so only 448 bytes of buffer space is available for customer traffic in the first buffer. The remaining customer traffic is split among concatenated 512-byte buffers.

The following table shows the supported buffer sizes on the 7705 SAR adapter cards and platforms. If a version number or variant is not specified, this implies all versions of the adapter card or variants of the platform. Adapter cards and platforms that support buffer chaining have 512 byte buffer size (‟Yes”); those that do not support buffer chaining have 2304 byte buffer size (‟No”).

Shaped SAPs have user-configured rate limits (PIR and CIR)—called the aggregate rate limit—and must use 16-priority scheduling mode. Unshaped SAPs use default rate limits (PIR is maximum and CIR is 0 kb/s) and can use 4-priority or 16-priority scheduling mode.

Shaped 16-priority SAPs are configured with a PIR and a CIR using the agg-rate-limit command in the config>service>service-type service-id>sap context, where service-type is epipe, ipipe, ies, vprn, or vpls (including routed VPLS). The PIR is set using the agg-rate variable and the CIR is set using the cir-rate variable.

Unshaped 4-priority SAPs are considered unshaped by definition of the default PIR and CIR values (PIR is maximum and CIR is 0 kb/s). Therefore, they do not require any configuration other than to be set to 4-priority scheduling mode.

Unshaped 16-priority SAPs are created when 16-priority scheduling mode is selected, when the default PIR is maximum and the default CIR is 0 kb/s, which are same default settings of a 4-priority SAP. The main reason for preferring unshaped SAPs using 16-priority scheduling over unshaped SAPs using 4-priority scheduling is to have a coherent scheduler behavior (one scheduling model) across all SAPs.

In order for unshaped 4-priority SAPs to compete fairly for bandwidth with 16-priority shaped and unshaped SAPs, a single, aggregate CIR for all the 4-priority SAPs can be configured. This aggregate CIR is applied to all the 4-priority SAPs as a group, not to individual SAPs. In addition, the aggregate CIR is configured differently for access ingress and access egress traffic. On the 7705 SAR-8 Shelf V2 and 7705 SAR-18, access ingress is configured in the config>qos>fabric-profile context. On the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, and 7705 SAR-Wx, access ingress is configured in the config>system>qos>access-ingress-aggregate-rate context. For all platforms, access egress configuration uses the config>port>ethernet context.

For more information about access ingress scheduling and traffic arbitration from the 16-priority and 4-priority schedulers toward the fabric, see Access ingress per-SAP aggregate shapers (access ingress H-QoS).

A typical example in which H-QoS is used is where a transport provider uses a 7705 SAR as a PE device and sells 100 Mb/s of fixed bandwidth for point-to-point Internet access, and offers premium treatment to 10% of the traffic. A customer can mark up to 10% of their critical traffic such that it is classified into high-priority queues and serviced prior to low-priority traffic.

Without H-QoS, there is no way to enforce a limit to ensure that the customer does not exceed the leased 100 Mb/s bandwidth, as illustrated in the following two scenarios:

• If a queue hosting high-priority traffic is serviced at 10 Mb/s and the low-priority queue is serviced at 90 Mb/s, then at a moment when the customer transmits less than 10 Mb/s of high-priority traffic, the customer bandwidth requirement is not met (the transport provider transports less traffic than the contracted rate).

• If the scheduling rate for the high-priority queue is set to 10 Mb/s and the rate for low-priority traffic is set to 100 Mb/s, then when the customer transmits both high- and low-priority traffic, the aggregate amount of bandwidth consumed by customer traffic exceeds the contracted rate of 100 Mb/s and the transport provider transports more traffic than the contracted rate.

The second-tier shaper—that is, the per-SAP aggregate shaper—is used to limit the traffic at a configured rate on a per-SAP basis. The per-queue rates and behavior are not affected when the aggregate shaper is enabled. That is, as long as the aggregate rate is not reached then there are no changes to the behavior. If the aggregate rate limit is reached, then the per-SAP aggregate shaper throttles the traffic at the configured aggregate rate while preserving the 16-priority scheduling priorities that are used on shaped SAPs.

Shaped VLANs have user-configured rate limits (PIR and CIR)—called the aggregate rate limit—and must use 16-priority scheduling mode. Shaped VLANs operate on a per-interface basis and are enabled after a network queue policy is assigned to the interface. If a VLAN does not have a network queue policy assigned to the interface, it is considered an unshaped VLAN.

To configure a shaped VLAN with aggregate rate limits, use the agg-rate-limit command in the config>router>if>egress context. If the VLAN shaper is not enabled, the agg-rate-limit settings do not apply. The default aggregate rate limit (PIR) is set to the port egress rate.

Unshaped VLANs use default rate limits (PIR is the maximum possible port rate and CIR is 0 kb/s) and use 16-priority scheduling mode. All unshaped VLANs are classified, queued, buffered, and scheduled into an aggregate flow that gets prepared for third-tier arbitration by a single VLAN aggregate shaper.

In order for the aggregated unshaped VLANs to compete fairly for bandwidth with the shaped VLANs, a single, aggregate CIR for all the unshaped VLANs can be configured using the unshaped-if-cir command. The aggregate CIR is applied to all the unshaped VLANs as a group, not to individual VLANs, and is configured in the config>port>ethernet> network>egress context.

The following cards and nodes support network egress per-VLAN shapers:

• 6-port Ethernet 10Gbps Adapter card

• 8-port Gigabit Ethernet Adapter card

• 10-port 1GigE/1-port 10GigE X-Adapter card (1-port 10GigE mode and 10-port 1GigE mode)

• Packet Microwave Adapter card (includes 1+1 redundancy)

• v-port on the 2-port 10GigE (Ethernet) Adapter card/module

• 7705 SAR-A

• 7705 SAR-Ax

• 7705 SAR-H

• 7705 SAR-Hc

• 7705 SAR-M

• 7705 SAR-Wx

• 7705 SAR-X

This section describes the following two scenarios:

• VLAN shapers for dual uplinks

• VLAN shapers for aggregation site

The following cards, modules, and platforms support MSS:

• 6-port Ethernet 10Gbps Adapter card

• 8-port Gigabit Ethernet Adapter card

• 10-port 1GigE/1-port 10GigE X-Adapter card

• Packet Microwave Adapter card

• 6-port SAR-M Ethernet module

• 7705 SAR-A

• 7705 SAR-Ax

• 7705 SAR-H

• 7705 SAR-Hc

• 7705 SAR-M

• 7705 SAR-Wx

• 7705 SAR-X

A SAP that uses a LAG can include two or more ports from the same adapter card or two different adapter cards.

In the access egress direction, each port can be assigned a shaper policy for access and can have shaper groups configured with different shaping rates. If a shaper group is not defined, the default shaper group is used. The port egress shaper policy, when configured on a LAG, must be configured on the primary LAG member. This shaper policy is propagated to each of the LAG port members, ensuring that each LAG port member has the same shaper policy.

The following egress MSS restrictions ensure that both active and standby LAG members have the same configuration:

• When a SAP is created using a LAG, whether the LAG has any port members and whether the egress scheduler mode is 4-priority or 16-priority, the default shaper group is automatically assigned to the SAP.

• Shaper groups cannot be changed from the default if they are assigned to SAPs that use LAGs with no port members.

• The last LAG port member cannot be deleted from a LAG that is used by any SAP that is assigned a non-default shaper group.

• The shaper policy or shaper group is not checked when the first port is added as a member of a LAG. When a second port is added as a member of a LAG, it can only be added if the shaper policy on the second port is the same as the shaper policy on the first member port of the LAG.

• The shaper group of a LAG SAP can be changed to a non-default shaper group only if the new shaper group exists in the shaper policy used by the active LAG port member.

• A shaper group cannot be deleted if it is assigned to unshaped SAPs (unshaped-sap-shaper-group command) or if it is used by at least one LAG SAP or non-LAG SAP.

• The shaper policy assigned to a port cannot be changed unless all of the SAPs on that port are assigned to the default shaper group.

In the ingress direction, there can be two different shaper policies on two different adapter cards for the two port members in a LAG. When assigning a shaper group to an ingress LAG SAP, each shaper policy assigned to the LAG port MDAs must contain that shaper group or the shaper group cannot be assigned. In addition, after a LAG activity switch occurs, the CIR/PIR configuration from the subgroup of the policy of the adapter card of the newly active member will be used.

The following ingress MSS restrictions allow the configuration of shaper groups for LAG SAPs, but the router ignores shaper groups that do not meet the restrictions:

• When a SAP is created using a LAG, whether the LAG has any port members and whether the ingress scheduler mode is 4-priority or 16-priority, the default shaper group is automatically assigned to the SAP.

• Shaper groups cannot be changed from the default if they are assigned to SAPs that use LAGs with no port members.

• The last LAG port member cannot be deleted from a LAG that is used by any SAP that is assigned a non-default shaper group.

• The shaper policy or shaper group is not checked when the first port is added as a member of a LAG. When a second port is added as a member of a LAG, all SAPs using the LAG are checked to ensure that any non-default shaper groups already configured on the SAPs are part of the shaper policy assigned to the adapter card of the second port. If the check fails, the port member is rejected.

• The shaper group of a LAG SAP can be changed to a non-default shaper group only if the new shaper group exists in the shaper policies used by all adapter cards of all LAG port members.

• A shaper group cannot be deleted if it is assigned to unshaped SAPs (unshaped-sap-shaper-group command) or if it is used by at least one LAG SAP or non-LAG SAP.

• The shaper policy assigned to an adapter card cannot be changed unless all of the SAPs on that adapter card are assigned to the default shaper group.

The egress datapath shapers on a 6-port SAR-M Ethernet module operate on the same frame size as any other shaper. These egress datapath shapers are:

• per-queue shapers

• per-SAP aggregate shapers

• per-customer aggregate (MSS) shapers

The egress port shaper on a 6-port SAR-M Ethernet module does not account for the 4-byte FCS. Packet byte offset can be used to make adjustments to match the desired operational rate or eliminate the implications of FCS. See Packet byte offset for more information.

For network egress traffic through a network port on Gen-3 hardware, the behavior of 4-priority scheduling is as follows: traffic priority is determined at the queue-level scheduler, which is based on the queue PIR and CIR and the queue type. The queue-level priority is carried through the various shaping stages and is used by the 4-priority Gen-3 VLAN scheduler at network egress. See 4-priority scheduling at network egress (Gen-3 hardware) on a network port and its accompanying description.

For hybrid ports, both access and network egress traffic use 4-priority scheduling that is similar to 4-priority scheduling on Gen-2 hardware. See 4-priority scheduling for hybrid port egress (Gen-3 hardware) and its accompanying description.

In the following figure, the shaper groups all belong within one shaper policy and only one shaper policy is assigned to an ingress adapter card. Each SAP can be associated with one shaper group. Multiple SAPs can be associated with the same shaper group. All the access ingress traffic flows to the access ingress fabric shaper, in-profile (conforming) traffic first, then out-of-profile (non-conforming) traffic. Network ingress traffic functions similarly.

The 4-priority schedulers on Gen-2 and Gen-3 hardware are very similar, except that 4-priority scheduling on Gen-3 hardware is done on a per-SAP basis.

The following figure shows 4-priority scheduling for access egress on Gen-3 hardware. QoS behavior for access egress is similar to QoS behavior for access ingress.

The 4-priority schedulers on Gen-2 and Gen-3 hardware are very similar, except that 4-priority scheduling on Gen-3 hardware is done on a per-SAP basis.

4-priority scheduling at network ingress (Gen-3 hardware): per-destination mode and 4-priority scheduling at network ingress (Gen-3 hardware): aggregate mode show network ingress scheduling for per-destination and aggregate modes, which are configured under the fabric-profile command. Traffic arriving on a network port is examined for its destination MDA and directed to the QoS block that sends traffic to the appropriate MDA. There is one set of queues for each block, and an additional set for multipoint traffic.

In the following figure, there is one per-destination shaper for each destination MDA.

In the following figure, there is a single shaper to handle all the traffic.

The following figure shows 4-priority scheduling for Gen-3 hardware at network egress. Queue-level CIR and PIR values and the queue type are determined at queuing and provide the scheduling priority for a specific flow across all shapers toward the egress port (#1 in the figure). At the per-VLAN aggregate level (#2), only a single rate—the total aggregate rate (PIR)—can be configured; CIR configuration is not supported at the per-VLAN aggregate-shaper level for network egress traffic. All VLANs are aggregated and scheduled by a 4-priority aggregate scheduler (#3). The flow is then fed to the port shaper and processed at the egress rate. In case of congestion, the port shaper provides backpressure, resulting in the buffering of traffic by individual FC queues until the congested state ends.

The following figure shows 4-priority scheduling for Gen-3 hardware where ports are in hybrid mode. The QoS behavior for both access and network egress traffic is similar except that the access egress path includes tier 3, per-customer aggregate shapers. Access and network shapers prepare and present traffic to the port shaper, which arbitrates between access and network traffic.

The 4-priority schedulers on the Gen-2 and Gen-3 hardware are very similar, except that 4-priority scheduling on Gen-3 hardware is done on a per-SAP or a per-VLAN basis (for access egress and network egress, respectively).

When a Gen-3-based port and a Gen-2-based port are attached to a LAG SAP (also referred to as mix-and-match LAG), configuring scheduler mode for the LAG SAP is required because it is used by Gen-2 ports, but it is ignored by the Gen-3 port.

For more information, see the ‟LAG Support on Third-Generation Ethernet Adapter Cards, Ports, and Platforms” section in the 7705 SAR Interface Configuration Guide.

All ports on a ring adapter card or module are possible congestion points and therefore can have network queue QoS policies applied to them.

In the bridging domain, a single ring type network QoS policy can be applied at the adapter card level and operates on the ring ports and the add/drop port. In the IP domain, IP interface type network QoS policies can be applied to router interfaces.

Network QoS policies are created using the config>qos>network command, which includes the network-policy-type keyword to specify the type of policy:

• ring (for bridging domain policies)

• ip-interface (for network ingress and network egress IP domain policies)

• default (for setting the policy to policy 1, the system default policy)

When the policy has been created, its default action and classification mapping can be configured.

Network QoS policies are applied to the ring ports and the add/drop port using the qos-policy command found under the config>card>mda context. These ports are not explicitly specified in the command.

Network queue QoS policies are applied to the ring ports and the v-port using the queue-policy command found under the config>port context. Similarly, a network queue policy is applied to the add/drop port using the add-drop-port- queue-policy command, found under the config>card>mda context. The add/drop port is not explicitly specified in this command.

The CLI commands for applying QoS policies are listed in this guide. The CLI command descriptions are in the 7705 SAR Interface Configuration Guide.

The following notes apply to configuring and applying QoS policies to a ring adapter card or module, as well as other adapter cards:

1. The ring ports and the add/drop port cannot use a network queue policy that is being used by the v-port or the network ingress port or any other port on other cards, and vice versa. This does not apply to the default network queue policy.

2. If a network-queue policy is assigned to a ring port or the add/drop port, all queues that are configured in the network-queue policy are created regardless of any FC mapping to the queues. All FC queue mapping in the network-queue policy is meaningless and is ignored.

3. If the QoS policy has a dot1p value mapped to a queue that is not configured in the network-queue policy, the traffic of this dot1p value is sent to queue 1.

4. If a dot1q-to-queue mapping is defined in a network QoS policy, and if the queue is not configured on any ring port or the add/drop port, all traffic received from a port will be sent out from queue 1 on the other two ports. For example, if traffic is received on port 1, it will be sent out on port 2 and/or the add/drop port.

5. Upon provisioning an MDA slot for a ring adapter card or module (config>card>mda> mda-type) an additional eight network ingress queues are allocated to account for the network queues needed for the add/drop port. When the ring adapter card or module is deprovisioned, the eight queues are deallocated.

6. The number of ingress network queues is shown by using the tools>dump> system-resource command.

The various traffic classification methods used on the 7705 SAR are described in the following table. A list of classification rules follows the table.

Each queue is serviced based on the user-configured CIR and PIR values. If the packets that are collected by a scheduler from a queue are flowing at a rate that is less than or equal to the CIR value, the packets are scheduled as in-profile. Packets with a flow rate that exceeds the CIR value but is less than the PIR value are scheduled as out-of-profile. 4-priority scheduling depicts this behavior by the ‟In-Prof” and ‟Out-Prof” labels. This behavior is comparable to the dual leaky bucket implementation in ATM networks. With in-profile and out-of-profile scheduling, traffic that flows at rates up to the traffic contract (that is, CIR) from all the queues is serviced prior to traffic that flows at rates exceeding the traffic contract. This mode of operation ensures that service-level agreements (SLAs) are honored and traffic that is committed to be transported is switched prior to traffic that exceeds the contract agreement.

As well as profiled scheduling described above, queue-type scheduling is supported at access ingress. Queues are divided into two categories, those that are serviced by the Expedited scheduler and those that are serviced by the Best Effort scheduler.

The Expedited scheduler has precedence over the Best Effort scheduler. Therefore, at access ingress, CoS queues that are marked with an Expedited priority are serviced first. Then the Best Effort marked queues are serviced. In a default configuration, the Expedited scheduler services the following CoS queues before the Best Effort scheduler services the rest:

• Expedited scheduler: NC, H1, EF, H2

• Best Effort scheduler: L1, AF, L2, BE

If a packet with an Expedited forwarding class arrives while a Best Effort marked queue is being serviced, the Expedited scheduler takes over and services the Expedited marked CoS queue as soon as the packet from the Best Effort scheduler is serviced.

The schedulers at access ingress in the 7705 SAR service the group of all Expedited queues exhaustively ahead of the group of all Best Effort queues. This means that all expedited queues have to be empty before any packet from a Best Effort queue is serviced.

The following basic rules apply to the queue-type scheduling of CoS queues:

1. Queues marked for Expedited scheduling are serviced in a round-robin fashion before any queues marked as Best Effort (in a default configuration, these would be queues CoS-8 through CoS-5).

2. These Expedited queues are serviced exhaustively within the round robin. For example, if in a default configuration there are two packets scheduled for service in both CoS-8 and CoS-5, one packet from CoS-8 is serviced, then one packet from CoS-5 is serviced, and then the scheduler returns back to CoS-8, until all the packets are serviced.

3. After the Expedited scheduler has serviced all the packets in the queues marked for Expedited scheduling, the Best Effort scheduler starts serving the queues marked as Best Effort. The same principle described in step 2 is followed, until all the packets in the Best Effort queues are serviced.

4. If a packet arrives at any of the queues marked for Expedited scheduling while the scheduler is servicing a packet from a Best Effort queue, the Best Effort scheduler finishes servicing the current packet and then the Expedited scheduler immediately activates to service the packet in the Expedited queue. If there are no other packets to be serviced in any of the Expedited queues, the Best Effort scheduler resumes servicing the packets in the Best Effort queues. If the queues are configured according to the tables and defaults described in this guide, CoS-4 will be scheduled prior to CoS-1 among queues marked as Best Effort.

5. After one cycle is completed across all the queues marked as Best Effort, the next pass is started until all the packets in all the queues marked as Best Effort are serviced, or a packet arrives to a queue marked as Expedited and is serviced as described in step 2.

With 4-priority scheduling, profiled scheduling and queue-type scheduling are combined and the combination is applied to all of the access ingress queues. The profile and queue-type schedulers are combined and applied to the queues to provide maximum flexibility and scalability that meet the stringent QoS requirements of modern network applications. See Profiled (rate-based) scheduling and Queue-type scheduling for information about these types of scheduling.

Packets with a flow rate that is less than or equal to the CIR value of a queue are scheduled as in-profile. Packets with a flow rate that exceeds the CIR value but is less than the PIR value of a queue are scheduled as out-of-profile.

The scheduling cycle for 4-priority scheduling of CoS queues is shown in 4-priority scheduling. The following basic steps apply:

1. In-profile traffic from Expedited queues is serviced in round-robin fashion up to the CIR value. When a queue exceeds its configured CIR value, its state is changed to out-of-profile.

2. When all of the in-profile packets from the Expedited queues are serviced, in-profile packets from Best Effort queues are serviced in a round-robin fashion until the configured CIR value is exceeded. When a queue exceeds its configured CIR value, its state is changed to out-of-profile.

3. When all of the in-profile packets from the Best Effort queues are serviced, out-of-profile packets from Expedited queues are serviced in a round-robin fashion.

4. When all of the out-of-profile packets from the Expedited queues are serviced, the out-of-profile packets from the Best Effort queues are serviced in a round-robin fashion.

   Note: If a packet arrives at any of the queues marked for Expedited scheduling while the scheduler is servicing a packet from a Best Effort queue or is servicing an out-of-profile packet, the scheduler finishes servicing the current packet and then returns to the Expedited queues immediately.

   

At access ingress, 4-priority scheduling for Gen-3 hardware is the same as 4-priority scheduling for Gen-2 hardware, except that scheduling is done on a per-SAP basis. For more information, see QoS for Gen-3 adapter cards and platforms.

For 16-priority scheduling, the rate-based schedulers (CIR and PIR) are combined with the strict priority schedulers (CoS-8 queue first to CoS-1 queue last).

For general information about 16-priority scheduling, see Network egress 16-priority scheduling. Access ingress 16-priority scheduling functions in a similar fashion to network egress 16-priority scheduling.

The 7705 SAR treats broadcast, multicast, and unknown traffic in the same way as unicast traffic. After being classified, the BMU traffic can be mapped to individual queues in order to be forwarded to the fabric. Classification of unicast and BMU traffic does not differ, which means that BMU traffic that has been classified to a BMU-designated queue can be shaped at its own rate, offering better control and fairer usage of fabric resources. For more information, see BMU support.

The 7705 SAR provides per-SAP aggregate shapers for access ingress SAPs. With this feature, both shaped and unshaped SAPs can coexist on the same adapter card. When switching traffic from shaped and unshaped SAPs to the fabric, arbitration is required.

Access ingress per-SAP arbitration to fabric shows how the 7705 SAR arbitrates traffic to the fabric between 4-priority unshaped SAPs, and 16-priority shaped and unshaped SAPs.

All SAPs support configurable CIR and PIR rates on a per-CoS queue basis (per-queue level). In addition, each 16-priority SAP has its own configurable per-SAP aggregate CIR and PIR rates that operate one level above the per-queue rates.

To allow the 4-priority unshaped SAPs to compete for fabric bandwidth with the aggregate CIR rates of the shaped SAPs, the 4-priority unshaped SAPs (as a group) have their own configurable unshaped SAP aggregate CIR rate, which is configured on the 7705 SAR-8 Shelf V2 and 7705 SAR-18 under the config>qos> fabric-profile aggregate-mode context using the unshaped-sap-cir parameter. On the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, and 7705 SAR-Wx, the CIR rate is configured in the config>system>qos>access-ingress-aggregate-rate context.

The configured CIR and PIR for the 16-priority shaped SAPs dictates committed and uncommitted fabric bandwidth for each of these SAPs. Configuring the unshaped-sap-cir parameter for the group (aggregate) of 4-priority unshaped SAPs ensures that the unshaped SAPs will compete for fabric bandwidth with the aggregate CIR rate of the shaped SAPs. Otherwise, the unshaped SAPs would only be able to send traffic into the fabric after the aggregate CIR rates of all the shaped SAPs were serviced. The 16-priority unshaped SAPs are serviced as if they are non-conforming traffic for 16-priority shaped SAPs.

The aggregate fabric shaper shown in the figure performs round-robin selection between the 16-priority SAPs (shaped and unshaped) and the 4-priority unshaped SAP aggregate until:

• the aggregate fabric shaper rate is exceeded

• the conforming (CIR) traffic for every 16-priority SAP and the 4-priority unshaped SAP aggregate is exceeded

• the non-conforming traffic for every 16-priority SAP and the 4-priority unshaped SAP aggregate is completed, provided that the aggregate PIR rate is not exceeded

Fabric shapers support both unicast and multipoint traffic. Multipoint traffic can be any combination of broadcast, multicast, and unknown (BMU) frames. From access ingress to the fabric, BMU traffic is treated as unicast traffic. A single copy of BMU traffic is handed off to the fabric, where it is replicated and sent to all potential destination adapter cards.

Fabric shapers support access ingress traffic being switched from a SAP to another SAP residing on a port that is part of a link aggregation group (LAG). Either the aggregate mode or destination mode can be used for fabric shaping.

When the aggregate mode is used, one aggregate rate sets the rate on all adapter cards. When the destination mode is used, the multipoint shaper is used to set the fabric shaping rate for traffic switched to a LAG SAP.

Network Ethernet ports support network egress per-VLAN (per-interface) shapers with eight CoS queues per VLAN, which is an extension to the eight CoS queues per port shared by all unshaped VLANs. Eight unique per-VLAN CoS queues are created for each VLAN when the VLAN shaper is enabled. These per-VLAN CoS queues are separate from the eight unshaped VLAN queues. The eight CoS queues that are shared by all the remaining unshaped VLANs are referred to as unshaped VLAN CoS queues. VLAN shapers are enabled when the queue-policy command is used to assign a network queue policy to the interface.

For details on per-VLAN network egress queuing and scheduling, see Per-VLAN network egress shapers.

The implementation of network egress scheduling on the cards and ports listed in Scheduling modes supported by adapter cards and ports at network egress under ‟4-Priority” is very similar to the scheduling mechanisms used for adapter cards that are configured for access ingress traffic. 4-priority scheduling is a combination of queue-type scheduling (Expedited vs. Best-effort scheduling) and profiled scheduling (rate-based scheduling).

Packets less than or up to the CIR are scheduled as in-profile. Packets that arrive at rates greater than the CIR, but less than the PIR, are scheduled as out-of-profile. In-profile traffic is exhaustively transmitted from the queues before out-of-profile traffic is transmitted. That is, all of the in-profile packets must be transmitted before any out-of-profile packets are transmitted. In addition, Expedited queues are always scheduled before Best Effort queues.

The default configuration of scheduling CoS queues provides a logical and consistent means to manage the traffic priorities. The default configuration is as follows:

• CoS-8 to CoS-5 Expedited in-profile

• CoS-4 to CoS-1 Best Effort in-profile

• CoS-8 to CoS-5 Expedited out-of-profile

• CoS-4 to CoS-1 Best Effort out-of-profile

The order shown below is maintained when scheduling the traffic on the adapter card’s network ports. A strict priority is applied between the four schedulers, and all four schedulers are exhaustive:

• Expedited in-profile traffic

• Best Effort in-profile traffic

• Expedited out-of-profile traffic

• Best Effort out-of-profile traffic

The adapter cards and ports that support 4-priority scheduling for network egress traffic on Gen-3 hardware are identified in Scheduling modes supported by adapter cards and ports at network egress. This type of scheduling takes into consideration the traffic’s profile type and the CoS queue priority. It also uses priority information to apply backpressure to lower-level CoS queues. See QoS for Gen-3 adapter cards and platforms for details.

The adapter cards and ports that support 16-priority scheduling for network egress traffic are listed in Scheduling modes supported by adapter cards and ports at network egress. This type of scheduling takes into consideration the traffic’s profile type and the priority of the CoS queue that the traffic is coming from.

Packets less than or up to the CIR are scheduled as in-profile. Packets that arrive at rates greater than the CIR, but less than the PIR, are scheduled as out-of-profile. Eight CoS queues in total are available for packets to go through.

In-profile traffic is exhaustively transmitted from the queues, starting with the highest-priority CoS queue. A strict priority is applied between the eight CoS queues. If a packet arrives at a queue of higher priority than the one being serviced, the scheduler services the packet at the higher-priority queue as soon as it finishes servicing the current packet.

When all the in-profile traffic is transmitted, the out-of-profile traffic is transmitted, still maintaining priority of the queues. If an in-profile packet arrives and the scheduler is servicing an out-of-profile packet, the scheduler finishes servicing the out-of-profile packet and then immediately services the in-profile packet.

The order of priority in the default configuration is as follows:

• CoS-8 in-profile traffic

• CoS-7 in-profile traffic

• CoS-6 in-profile traffic

• CoS-5 in-profile traffic

• CoS-4 in-profile traffic

• CoS-3 in-profile traffic

• CoS-2 in-profile traffic

• CoS-1 in-profile traffic

• CoS-8 out-of-profile traffic

• CoS-7 out-of-profile traffic

• CoS-6 out-of-profile traffic

• CoS-5 out-of-profile traffic

• CoS-4 out-of-profile traffic

• CoS-3 out-of-profile traffic

• CoS-2 out-of-profile traffic

• CoS-1 out-of-profile traffic

For shaped VLANs, the configured CIR and PIR limits dictate committed and uncommitted port bandwidth for each of these VLANs. To ensure that the bulk (aggregate) of unshaped VLANs can compete for port bandwidth with the aggregate of CIR rates for the shaped VLANs, the unshaped VLANs (as a group) have their own aggregate CIR rate, which is configured using the unshaped-if-cir command (under the config>port> ethernet>network>egress context). Otherwise, without their own aggregate CIR rate, the unshaped VLANs are only able to send traffic into the port after the aggregate CIR rates of all the shaped VLANs are serviced. Shaped VLANs using default aggregate rate limits (PIR = maximum and CIR = 0 kb/s) are serviced as if they are non-conforming traffic for shaped VLANs.

Referring to Network egress shaped and unshaped VLAN queuing and scheduling, at the port shaper, conforming (CIR) traffic has priority over non-conforming traffic. The arbitration between the shaped VLANs and unshaped VLANs is handled in the following priority order:

• committed traffic: the per-VLAN committed rate (CIR) for shaped VLANs as set by the agg-rate-limit command, and the aggregate committed rate for all the unshaped VLANs as set by the unshaped-if-cir command

• uncommitted traffic: the per-VLAN uncommitted rate (PIR) for shaped VLANs as set by the agg-rate-limit command, and aggregate uncommitted rate for all the unshaped VLANs as set by the unshaped-if-cir command

For MPLS tunnels, if network egress Ethernet ports are used, dot1p bit marking can be enabled in conjunction with EXP bit marking. In this case, the tunnel and pseudowire EXP bits do not have to be the same as the dot1p bits.

For GRE and IP tunnels, dot1p marking and pseudowire EXP marking can be enabled, and DSCP marking can also be enabled.

Network egress dot1p is supported for Ethernet frames, which can carry IPv4, IPv6, or MPLS packets. EXP re-marking is supported for MPLS packets.

To simplify QoS management through the network core, some operators aggregate multiple forwarding classes of traffic at the ingress LER or PE and use two or three QoS markings instead of the eight different QoS markings that a customer device may be using to dictate QoS treatment. However, to ensure the end-to-end QoS enforcement required by the customer, the aggregated markings must be mapped back to their original forwarding classes at the egress LER (eLER) or PE.

For IP traffic (including IPSec packets) riding over MPLS or GRE tunnels that will be routed to the base router, a VPRN interface, or an IES interface at the tunnel termination point (the eLER), the 7705 SAR can be configured to ignore the EXP/DSCP bits in the tunnel header when the packets arrive at the eLER. Instead, classification is based on the inner IP header, which is essentially the customer IP packet header. This configuration is done using the ler-use-dscp command.

When the command is enabled on an ingress network IP interface, the IP interface will ignore the tunnel’s QoS mapping and will derive the internal forwarding class and associated profile state based on the DSCP values of the IP header ToS field rather than on the network QoS policy defined on the IP interface. This function is useful when the mapping for the tunnel QoS marking does not completely reflect the required QoS handling for the IP packet. The command applies only on the eLER where the tunnel or service is terminated and the next header in the packet is IP.

At network ingress, broadcast, multicast, and unknown (BMU) traffic identified using DSCP and/or EXP (also known as LSP TC) is mapped to a forwarding class (FC). Because BMU traffic is considered to be multipoint traffic, the queue hosting BMU traffic must be configured with the multipoint keyword. Queues 9 through 16 support multipoint traffic (see Default network ingress QoS policy). For any adapter card hosting any number of network ports, up to 16 queues can be configured to host 8 unicast and 8 multicast queues.

Similar to unicast queues, BMU queues require configuration of:

• queue depth (committed and maximum)

• scheduled rate (committed and peak)

In addition, as is the case for unicast queues, all other queue-based congestion management techniques apply to multipoint queues.

The benefits of using multipoint queues occur when the to-fabric shapers begin scheduling traffic toward the destination line card. To-fabric shapers can be configured for aggregate or per-destination mode. For more information, see BMU support.

The adapter cards listed in Scheduling modes supported by adapter cards and ports at network ingress under ‟4-Priority” can receive network ingress traffic. One or more ports on the card are configured for PPP/MLPPP for this purpose.

The implementation of network ingress scheduling on the cards listed in the table under ‟4-Priority” is very similar to the scheduling mechanisms used for adapter cards that are configured for access ingress traffic. That is, 4-priority scheduling is used (queue-type scheduling combined with profiled scheduling).

The adapter cards provide sets of eight queues for incoming traffic: 7 sets of queues for the 7705 SAR-8 Shelf V2 and 17 sets of queues for the 7705 SAR-18. Each set of queues is specific to a destination adapter card. For the 7705 SAR-8 Shelf V2 and 7705 SAR-18 (respectively), 6 and 16 sets of queues are automatically created for each access egress adapter card, plus 1 set of queues for multicast traffic.

There is one additional set of queues for slow-path (control) traffic destined for the CSMs.

The individual queues within each set of queues provide buffer space for traffic isolation based on the CoS values being applied (from the received EXP bits).

All of the network ingress ports of the adapter card share the same sets of queues, which are created automatically.

When the packets received from the network are mapped to queues, four access ingress-like queue-type and profile (rate-based) schedulers per destination card service the queues in strict priority. The following queue-type and profiled schedulers service the queues in the order listed:

1. Expedited in-profile scheduler

2. Best Effort in-profile scheduler

3. Expedited out-of-profile scheduler

4. Best Effort out-of-profile scheduler

To complete the operation, user-configurable shapers send the traffic into the fabric. See Configurable ingress shaping to fabric (access and network) for details. Throughout this operation, each packet retains its individual CoS value.

The adapter cards and ports that support 4-priority (Gen-3) scheduling for network ingress traffic are listed in Scheduling modes supported by adapter cards and ports at network ingress. See QoS for Gen-3 adapter cards and platforms for details.

The cards and ports that support 16-priority scheduling for network ingress traffic are listed in Scheduling modes supported by adapter cards and ports at network ingress.

For a detailed description of how 16-priority scheduling functions, see Network egress 16-priority scheduling.

The 7705 SAR-8 Shelf V2 and 7705 SAR-18 adapter cards, and the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, and 7705 SAR-Wx ports provide sets of 8 queues for incoming traffic: 7 sets of queues for the 7705 SAR-8 Shelf V2, 17 sets of queues for the 7705 SAR-18, and 4 sets of queues for the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, and 7705 SAR-Wx.

Each set of queues is specific to a destination adapter card. For the 7705 SAR-8 Shelf V2, 6 sets of queues are automatically created for each access egress adapter card, plus 1 set of queues for multicast traffic. For the 7705 SAR-18, 16 sets of queues are automatically created, plus 1 set of queues for multicast traffic. For the 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, and 7705 SAR-Wx, 3 sets of queues are automatically created, plus 1 set of queues for multicast traffic. For all these platforms, there is 1 additional set of queues for slow-path (control) traffic that is destined for the CSMs.

Each queue within each set provides buffer space for traffic isolation based on the classification carried out on EXP bits of the MPLS packet header (that is, the CoS setting).

All of the network ingress ports on an adapter card on a 7705 SAR-8 Shelf V2 or 7705 SAR-18 share the same sets of queues, which are created automatically. All of the network ingress ports across the entire 7705 SAR-M, 7705 SAR-H, 7705 SAR-Hc, 7705 SAR-A, 7705 SAR-Ax, or 7705 SAR-Wx also share the same sets of queues, which are created automatically.

At access egress, the 7705 SAR handles traffic management for unicast and BMU traffic in the same way. Unicast or BMU traffic is mapped to a queue and the mapping is based on the FC classification. Individual queues are then scheduled based on the available traffic.

After the ATM pseudowire is terminated at the access egress, all the ATM cells are mapped to the default queue, which is queue 1, and queuing is performed per SAP. ATM access egress queuing and scheduling applies to the 16-port T1/E1 ASAP Adapter card, 32-port T1/E1 ASAP Adapter card, and 2-port OC3/STM1 Channelized Adapter card with atm/ima encapsulation. ATM access egress queuing and scheduling applies to the 4-port OC3/STM1 Clear Channel Adapter card and 4-port DS3/E3 Adapter card with atm encapsulation.

After the per-SAP queuing takes place, the ATM scheduler services these queues in the fashion and order defined below, based on the service categories assigned to each of these SAPs.

At access egress, CBR and rt-VBR VCs are always shaped because there is no option for the user to turn shaping off. Shaping for nrt-VBR is optional.

Strict priority scheduling in an exhaustive fashion takes place for the shaped VCs in the following order:

1. CBR (always shaped)

2. rt-VBR (always shaped)

3. nrt-VBR (when shaped, user-configurable for shaped or unshaped)

UBR traffic is not shaped. To offer maximum flexibility to the user, nrt-VBR unshaped (also known as scheduled) is implemented.

ATM traffic is serviced in priority order. CBR traffic has the highest priority and is serviced ahead of all other traffic. After all of the CBR traffic has been serviced, rt-VBR traffic is serviced. Then, nrt-VBR traffic is serviced.

After scheduling all the other traffic from the CBR and VBR service categories, UBR is serviced. If there is no other traffic, UBR can burst up to the line rate. Scheduled nrt-VBR is treated the same way as UBR. Both UBR and unshaped nrt-VBR are scheduled using the weighted round-robin scheduler.

The scheduler weight assigned to queues hosting scheduled nrt-VBR and UBR traffic is determined by the configured traffic rate. The weight used by the scheduler for UBR+ VCs is dependent on the minimum information rate (MIR) defined by the user. UBR with no MIR traffic has an MIR of 0.

Similarly, the scheduler weight is dependent on the sustained information rate (SIR) for scheduled nrt-VBR. Weight used by the scheduler is programmed automatically based on the user-configured MIR/SIR value and is not user-configurable.

For UBR+, the following tables are used to determine the weight of a UBR+ VC. These tables are also applicable to scheduled nrt-VBR weight determination. Instead of the MIR, the SIR is used to determine the scheduler weight.

The access egress ATM scheduling behavior is shown in the following table. For UBR traffic, the scheduler weight of the lowest possible value is always used, which is the value of 1. Only cell-based operations are carried out.

Ethernet access egress queuing and scheduling is very similar to the Ethernet access ingress behavior. When the Ethernet pseudowire is terminated, traffic is mapped to up to eight different forwarding classes per SAP. Mapping traffic to different forwarding classes is performed based on the EXP bit settings of the received Ethernet pseudowire by network ingress classification.

Queue-type and profile scheduling are both supported for Ethernet access egress ports. If the queues are configured according to the tables and defaults described in this guide (implying a default mode of operation), the configuration is as follows:

• CoS-8 to CoS-5 Expedited in-profile

• CoS-4 to CoS-1 Best Effort in-profile

• CoS-8 to CoS-5 Expedited out-of-profile

• CoS-4 to CoS-1 Best Effort out-of-profile

In this default configuration, for queue-type scheduling, CoS-8 to CoS-5 are serviced by the Expedited scheduler, and CoS-4 to CoS-1 are serviced by the Best Effort scheduler. This default mode of operation can be altered to better fit the operating characteristics of specific SAPs.

With profile scheduling, the Ethernet frames can be either in-profile or out-of-profile, and scheduling takes into account the state of the Ethernet frames in conjunction with the configured CIR and PIR rates.

After the queuing, an aggregate queue-type and profile scheduling takes place in the following order:

1. Expedited in-profile traffic

2. Best Effort in-profile traffic

3. Expedited out-of-profile traffic

4. Best Effort out-of-profile traffic

After the traffic is scheduled using the aggregate queue-type and profile schedulers, the per-port shapers shape the traffic at a sub-rate (that is, at the configured/shaped port rate). Per-port shapers ensure that a sub-rate is met and attainable at all times.

The arbitration of traffic from 4-priority and 16-priority schedulers toward an access egress port is achieved by configuring a committed aggregate rate limit for the aggregate of all the 4-priority unshaped SAPs. By configuring the 4-priority unshaped SAPs committed aggregate rate, the arbitration between the 16-priority shaped SAPs, 16-priority unshaped SAPs, and 4-priority unshaped SAPs is handled in the following priority order:

• committed traffic: 16-priority per-SAP agg-rate-limit committed for shaped SAPs and 4-priority aggregate committed rate for all the unshaped SAPs

• uncommitted traffic: 16-priority per-SAP agg-rate-limit uncommitted for shaped and unshaped SAPs and 4-priority aggregate uncommitted rate for all the unshaped SAPs

The following figure illustrates the traffic treatment for a single Ethernet port. It also illustrates that the shaped SAP aggregate CIR rate competes with the unshaped 4-priority aggregate CIR rate for port bandwidth. When the aggregate CIR rates are satisfied, the shaped SAP aggregate PIR rate competes with the 4-priority PIR rate (always maximum) for port bandwidth.

The egress aggregate CIR rate limit for all the unshaped 4-priority SAPs is configured using the config>port>ethernet>access>egress>unshaped-sap-cir command.

Network QoS policies define egress QoS marking and ingress QoS classification for traffic on network interfaces. The 7705 SAR automatically creates egress queues for each of the forwarding classes on network interfaces.

A network QoS policy defines ingress, egress, and ring handling of QoS on the network interface. The following functions are defined:

• ingress

  • defines label switched path Experimental bit (LSP EXP) value mappings to forwarding classes

  • defines DSCP name mappings to forwarding classes

• egress

  • defines forwarding class to LSP EXP and dot1p value markings

  • defines forwarding class to DSCP value markings

• ring

  • defines dot1p bit value mappings to queue and profile state

The required elements to be defined in a network QoS policy are:

• a unique network QoS policy ID

• egress forwarding class to LSP EXP value mappings for each forwarding class used

• a default ingress forwarding class and in-profile/out-of-profile state

• a default queue and in-profile/out-of-profile state for ring type network QoS policy

Optional ip-interface type network QoS policy elements include the LSP EXP value or DSCP name to forwarding class and profile state mappings for all EXP values or DSCP values received. Optional ring type network QoS policy elements include the dot1p bits value to queue and profile state mappings for all dot1p bit values received.

Network policy ID 1 is reserved as the default network QoS policy. The default policy cannot be deleted or changed. The default network QoS policy is applied to all network interfaces and ring ports (for ring adapter cards) that do not have another network QoS policy explicitly assigned.

The following tables list the various network QoS policy default mappings:

• Default network QoS policy egress marking
• Default network QoS policy DSCP-to-forwarding class mappings
• Default network QoS policy LSP EXP-to-forwarding class mappings
• Default network QoS policy dot1p-to-queue class mappings

The following table lists the default mapping of forwarding class to LSP EXP values and DSCP names for network egress.

For network ingress, the following table lists the default mapping of DSCP name to forwarding class and profile state for the default network QoS policy.

The following table lists the default mapping of LSP EXP values to forwarding class and profile state for the default network QoS policy.

The following table lists the default mapping of dot1p values to queue and profile state for the default network QoS policy.

Network queue policies define the queue characteristics that are used in determining the scheduling and queuing behavior for a forwarding class. Network queue policies are applied on ingress and egress network ports as well as on the ring ports and the add/drop port on the 2-port 10GigE (Ethernet) Adapter card and 2-port 10GigE (Ethernet) module.

Network queue policies are identified with a unique policy name that conforms to the standard 7705 SAR alphanumeric naming conventions. The policy name is user-configured when the policy is created.

Network queue policies can be configured to use up to 16 queues (8 unicast and 8 multicast). This means that the number of queues can vary. Not all user-created policies will require and use 16 queues; however, the system default network queue policy (named "default") does define 16 queues.

The queue characteristics that can be configured on a per-forwarding class basis are:

• committed buffer size (CBS) as a percentage of the buffer pool

• maximum buffer size (MBS) as a percentage of the buffer pool

• high-priority-only buffers as a percentage of MBS

• peak information rate (PIR) as a percentage of egress port bandwidth

• committed information rate (CIR) as a percentage of egress port bandwidth

The following table describes the default network queue policy definition.

The queue ID is used to uniquely identify the queue. The queue ID is only unique within the context of the QoS policy within which the queue is defined.

The CIR for a queue defines a limit for scheduling. Packets queued at service ingress queues are serviced by in-profile or out-of-profile schedulers based on the queue’s CIR and the rate at which the packets are flowing. For each packet in a service ingress queue, the CIR is checked with the current transmission rate of the queue. If the current rate is at or below the CIR threshold, the transmitted packet is internally marked in-profile. If the flow rate is above the threshold, the transmitted packet is internally marked out-of-profile.

All 7705 SAR queues support the concept of in-profile and out-of-profile. The network QoS policy applied at network egress determines how or if the profile state is marked in packets transmitted into the network core. This is done by enabling or disabling the appropriate priority marking of network egress packets within a particular forwarding class. If the profile state is marked in the packets that are sent toward the network core, then out-of-profile packets are preferentially dropped over in-profile packets at congestion points in the network.

When defining the CIR for a queue, the value specified is the administrative CIR for the queue. The 7705 SAR maps a user-configured value to a hardware supported rate that it uses to determine the operational CIR for the queue. The user has control over how the administrative CIR is converted to an operational CIR if a slight adjustment is required. The interpretation of the administrative CIR is discussed in Adaptation rule.

The CIR value for a service queue is assigned to ingress and egress service queues based on service ingress QoS policies and service egress QoS policies, respectively.

The CIR value for a network queue is defined within a network queue policy specifically for the forwarding class. The queue-id parameter links the CIR values to the forwarding classes. The CIR values for the forwarding class queues are defined as a percentage of the network interface bandwidth.

The PIR value defines the maximum rate at which packets are allowed to exit the queue. It does not specify the maximum rate at which packets may enter the queue; this is governed by the queue’s ability to absorb bursts and is user-configurable using its maximum burst size (MBS) value.

The PIR value is provisioned on ingress and egress service queues within service ingress QoS policies and service egress QoS policies, respectively.

The PIR values for network queues are defined within network queue policies and are specific for each forwarding class. The PIR value for each queue for the forwarding class is defined as a percentage of the network interface bandwidth.

When defining the PIR for a queue, the value specified is the administrative PIR for the queue. The 7705 SAR maps a user-configured value to a hardware supported rate that it uses to determine the operational PIR for the queue. The user has control over how the administrative PIR is converted to an operational CIR if a slight adjustment is required. The interpretation of the administrative PIR is discussed in Adaptation rule.

The schedulers on the network processor can only operate with a finite set of rates. These rates are called the operational rates. The configured rates for PIR and CIR do not necessarily correspond to the operational rates. In order to offer maximum flexibility to the user, the adaptation-rule command can be used to choose how an operational rate is selected based on the configured PIR or CIR rate.

The max parameter causes the network processor to be programmed at an operational rate that is less than the configured PIR or CIR rate by up to 1.0%. The min parameter causes the network processor to be programmed at an operational rate that is greater than the configured PIR or CIR rate by up to 1.0%. The closest parameter causes the network processor to be programmed at an operational rate that is closest to the configured PIR or CIR rate.

A 4-priority scheduler on the network processor of a third-generation (Gen-3) Ethernet adapter card or platform can be programmed at an operational CIR rate that exceeds 1.0% of the configured CIR rate. The PIR rate (that is, the maximum rate for the queue) and the SAP aggregate rates (CIR and PIR), maintain an accuracy of +/- 1.0% of the configured rates.

The average difference between the configured CIR rate and the programmed (operational) CIR rate is as follows:

• 2.0% for frame sizes that are less than 2049 bytes

• 4.0% for other frame sizes

The Gen-3 network processor PIR rate is programmed to an operational PIR rate that is within 1.0% of the configured rate, which ensures that the FC/CoS queue does not exceed its fair share of the total bandwidth.

The CBS parameter specifies the committed buffer space allocated for a specific queue.

The CBS is provisioned on ingress and egress service queues within service ingress QoS policies and service egress QoS policies, respectively. The CBS for a queue is specified in kilobytes.

The CBS values for network queues are defined within network queue policies based on the forwarding class. The CBS values for the queues for the forwarding class are defined as a percentage of buffer space for the pool.

When the reserved buffers for a queue have been used, the queue contends with other queues for additional buffer resources up to the maximum burst size. The MBS parameter specifies the maximum queue depth to which a queue can grow. This parameter ensures that a traffic flow (that is, a customer or a traffic type within a customer port) that is massively or continuously oversubscribing the PIR of a queue will not consume all the available buffer resources. For high-priority forwarding class service queues, the MBS can be small because the high-priority service packets are scheduled with priority over other service forwarding classes. In other words, very small queues would be needed for high-priority traffic because the contents of the queues should have been scheduled by the best available scheduler.

The MBS value is provisioned on ingress and egress service queues within service ingress QoS policies and service egress QoS policies, respectively. The MBS value for a queue is specified in bytes or kilobytes.

The MBS values for network queues are defined within network queue policies based on the forwarding class. The MBS values for the queues for the forwarding class are defined as a percentage of buffer space for the pool.

High-priority-only buffers are defined on a queue and allow buffers to be reserved for traffic classified as high priority. When the queue depth reaches a specified level, only high-priority traffic can be enqueued. The high-priority-only reservation for a queue is defined as a percentage of the MBS value and has a default value of 10% of the MBS value.

On service ingress, the high-priority-only reservation for a queue is defined in the service ingress QoS policy. High-priority traffic is specified in the match criteria for the policy.

On service egress, the high-priority-only reservation for a queue is defined in the service egress QoS policy. Service egress queues are specified by forwarding class. High-priority traffic for a given traffic class is traffic that has been marked as in-profile either on ingress classification or based on interpretation of the QoS markings.

The high-priority-only buffers for network queues are defined within network queue policies based on the forwarding class. High-priority-only traffic for a specific traffic class is marked as in-profile either on ingress classification or based on interpretation of the QoS markings.

The high/low priority feature allows a provider to offer a customer the ability to have some packets treated with a higher priority when buffered to the ingress queue. If the queue is configured with a high-prio-only setting (which set the high-priority MBS threshold higher than the queue’s low-priority MBS threshold), then a portion of the ingress queue’s allowed buffers are reserved for high-priority traffic. An access ingress packet must hit an ingress QoS action in order for the ingress forwarding plane to treat the packet as high priority (the default is low priority).

If the packet’s ingress queue is above the low-priority MBS, the packet will be discarded unless it has been classified as high priority. The priority of the packet is not retained after the packet is placed into the ingress queue. After the packet is scheduled out of the ingress queue, the packet will be considered in-profile or out-of-profile based on the dynamic rate of the queue relative to the queue’s CIR parameter.

If an ingress queue is not configured with a high-prio-only parameter (the parameter is set to 0%), the low-priority and high-priority MBS thresholds are the same. There is no difference in high-priority and low-priority packet handling. At access ingress, the priority of a packet has no effect on which packets are scheduled first. Only the first buffering decision is affected. At ingress and egress, the current dynamic rate of the queue relative to the queue’s CIR does affect the scheduling priority between queues going to the same destination (egress port).

From highest to lowest, the strict operating priority for queues is:

• expedited queues within the CIR (conform)

• best effort queues within the CIR (conform)

• expedited queues above the CIR (exceed)

• best effort queues above the CIR (exceed)

For access ingress, the CIR controls both dynamic scheduling priority and the marking threshold. At network ingress, the queue’s CIR affects the scheduling priority but does not provide a profile marking function (as the network ingress policy trusts the received marking of the packet based on the network QoS policy).

At egress, the profile of a packet is only important for egress queue buffering decisions and egress marking decisions, not for scheduling priority. The egress queue’s CIR determines the dynamic scheduling priority, but does not affect the packet’s ingress determined profile.

The 7705 SAR maintains extensive counters for queues within the system to allow granular or extensive debugging and planning; that is, the usage of queues and the scheduler used for servicing a queue or packet is extremely useful in network planning activities. The following separate billing and accounting counters are maintained for each queue:

• counters for packets and octets accepted into the queue

• counters for packets and octets rejected at the queue

• counters for packets and octets transmitted in-profile

• counters for packets and octets transmitted out-of-profile

The 7705 SAR allows two kinds of queue types: Expedited queues and Best Effort queues. Users can configure the queue type manually using the expedite and best-effort keywords, or automatically using the auto-expedite keyword. The queue type is specified as part of the queue command (for example, config>qos>sap-ingress>queue queue-id queue-type create).

With expedite, the queue is treated in an expedited manner, independent of the forwarding classes mapped to the queue.

With best-effort, the queue is treated in a non-expedited (best-effort) manner, independent of the forwarding classes mapped to the queue.

With auto-expedite, the queue type is automatically determined by the forwarding classes that are assigned to the queue. The queues that are set as auto-expedite are still either Expedited or Best Effort queues, but whether a queue is Expedited or Best Effort is determined by its assigned forwarding classes. In the default configuration, four of the eight forwarding classes (NC, H1, EF, and H2) result in an Expedited queue type, while the other four forwarding classes (L1, AF, L2, and BE) result in a Best Effort queue type.

Assigning one or more L1, AF, L2, and BE forwarding class to an Expedited queue results in a Best Effort queue type. See Forwarding classes for more information about default configuration values.

The expedite, best-effort, and auto-expedite queue types are mutually exclusive. Each defines the method that the system uses to service the queue from a hardware perspective.

The 7705 SAR supports two queue modes: priority mode and profile mode. Users can configure the queue mode using the priority-mode or profile-mode keywords when issuing the config>qos>sap-ingress>queue queue-id queue-mode create command. The default is priority-mode.The queue mode defines how an ingress access packet is categorized as in-profile or out-of-profile.

With priority mode, an access packet’s in-profile or out-of-profile state is based on the dynamic rate of the ingress queue before being forwarded to the fabric. When the queue rate is lower than or equal to the configured CIR, the packet is considered in-profile. When the queue rate is higher than the CIR, the packet is considered out-of-profile. The profile state is determined when the packet is scheduled out of the queue, not when the packet is buffered into the queue.

With profile mode, the in-profile or out-of-profile state for packets assigned to a particular forwarding class is explicitly configured using the config>qos>sap-ingress>fc>profile command. This configuration places a forwarding class in a color-aware profile mode. Packets assigned to this forwarding class profile are only marked based on this profile marking if the forwarding class is mapped to a queue configured for profile-mode. If the forwarding class is mapped to a queue configured for priority-mode, the forwarding class profile setting is ignored and the packet’s state is defined as in-profile or out-of-profile based on the dynamic rate of the ingress queue.

When the profile in command is executed on a forwarding class that is mapped to a queue operating in profile mode, all packets associated with the class are handled as in-profile. When the profile out command is executed on a forwarding class that is mapped to a queue operating in profile mode, all packets associated with the class are handled as out-of-profile.

When the no profile command is executed on a forwarding class that is mapped to a queue operating in profile mode, the data packets using the forwarding class are marked as in-profile or out-of-profile based on the dynamic rate of the ingress queue relative to its CIR.

Color-aware profiling adds the ability to selectively treat packets received on a SAP as in-profile (green) or out-of-profile (yellow) regardless of the queue forwarding rate. For example, a network operator can color a packet out-of-profile with the intention of preserving in-profile bandwidth for higher-priority packets.

A queue operating in profile mode can support in-profile, out-of-profile, and non-profiled packets simultaneously because multiple forwarding classes with different forwarding class profiles can be assigned to a single queue.

All non-profiled and profiled packets are forwarded through the same ingress access queue to prevent out-of-sequence forwarding. Profiled packets that are in-profile are counted against the total number of packets flowing through the queue that are marked in-profile. This reduces the amount of CIR available to non-profiled packets, causing fewer packets to be marked in-profile. Profiled packets that are out-of-profile are not counted against the total number of packets flowing through the queue that are marked in-profile. This ensures that the number of non-profiled packets marked in-profile is not affected by the profiled out-of-profile packet rate.

A SAP ingress queue operating in profile mode is classified as high-priority or low-priority based on the configuration of the forwarding class profile rather than on the high-priority or low-priority configuration specified for DSCP or dot1p. All non-profile packets flowing through the queue are considered high priority. Profiled in-profile packets are also handled as high priority, while profiled out-of-profile packets are handled as low priority.

For SAP ingress queues in profile mode, statistics are collected for color in (for a forwarding class configured for profile in), color out (for a forwarding class configured for profile out), and uncolor (for a forwarding class configured for no profile).

The 7705 SAR supports egress-rate limiting and ingress-rate limiting on Ethernet ports.

The egress rate is set at the port level in the config>port>ethernet context.

Egress-rate limiting sets a limit on the amount of traffic that can leave the port to control the total bandwidth on the interface. If the egress-rate limit is reached, the port applies backpressure on the queues, which stops the flow of traffic until the queue buffers are emptied. This feature is useful in scenarios where there is a fixed amount of bandwidth; for example, a mobile operator who has leased a fixed amount of bandwidth from the service provider.

The ingress-rate command configures a policing action to rate-limit the ingress traffic. Ingress-rate enforcement uses dedicated hardware for rate limiting; however, software configuration is required at the port level (ingress-rate limiter) to ensure that the network processor or adapter card or port never receives more traffic than they are optimized for.

The configured ingress rate ensures that the network processor does not receive traffic greater than this configured value on a per-port basis. When the ingress-rate value is reached, all subsequent frames are dropped. The ingress-rate limiter drops excess traffic without determining whether the traffic has a higher or lower priority.

For more information about egress and ingress rate limiting, see the egress-rate and ingress-rate command descriptions in the 7705 SAR Interface Configuration Guide.

CBS is configured in kilobytes through the CLI; MBS is configured in bytes or kilobytes. These configured values are converted to the corresponding number of buffers. The conversion factor is a non-user-configurable, fixed default value that is equal to the system-defined maximum frame size, ensuring that even the largest frames can be hosted in the allocated buffer pools. This type of WRED is called buffer-based WRED.

User-defined minThreshold and maxThreshold values, each defined as a percentage, are also converted to the number of buffers. The minT is converted to the system-minThreshold, and the maxT is converted to the system-maxThreshold.

The system-minT must be the absolute closest value to the minT that satisfies the formula below (2^x means 2 to the exponent x):

      system-maxThreshold – system-minThreshold = 2^x

The bridging domain queues support the following DP (discard probability) values: 0% to 10%, 25%, 50%, 75%, and 100%. User-configured values are rounded down to match these DP values.

For example, configuring a DP to be 74% means that the actual value used is 50%.

The third-generation Ethernet adapter cards and platforms use payload-based WRED rather than buffer-based WRED (see WRED MinThreshold and MaxThreshold computation). Payload-based WRED does not count the unused overhead space (empty space in the buffer) when making discard decisions, whereas buffer-based WRED counts the unused overhead space. Payload-based WRED is also referred to as byte-based WRED.

When a queue on an adapter card that uses payload-based WRED reaches its maximum fill (that is, the total byte count exceeds the configured maximum threshold), tail drop begins and operates in the same way as it does on any other adapter card or platform.

With payload-based WRED, the discard decision is based on the number of bytes in the queue instead of the number of buffers in the queue. For example, to accumulate 512 bytes of payload in a queue will take four buffers if the frame size is 128 bytes, but will take one buffer if the frame size is 512 bytes or more. Basing discards on bytes rather than buffers improves the efficient use of queues. In either case, byte- or buffer-based WRED, random discards begin at the minimum threshold (minT) point.

For example, assume a queue has MBS set to 512 kB (converts to 1000 buffers), minT (start-avg) is set to 10% (100 buffers), and maxT (max-avg) is set to 80% (800 buffers). The following table shows when discards and tail drop start when payload-based WRED is used.

For tail drop, if the high-priority-only threshold is set to 10%:

• when any frame size is greater than or equal to 512 bytes, tail drop starts after 900 buffers are in use for low-priority traffic