High scale QoS IOM

Introduction

This section describes the capabilities of the High Scale QoS (HSQ) IOM. The HSQ IOM is an FP3-based IOM that has a multicore CPU and accepts up to two MDA-e cards. The HSQ IOM supports an enhanced egress QoS traffic manager architecture to provide scalable network, service, and subscriber QoS. At ingress, the HSQ IOM supports regular FP3 QoS with a high ingress policer scaling.

The HSQ IOM (iom4-e-hs) supports six scheduling classes across multiple hierarchical levels of hardware egress shaping with very stringent egress burst control. The scheduling allows a mix of strict priority and weighted round-robin (WRR) queues. A flexible buffer pool structure allows both buffer isolation and buffer oversubscription for the queue buffer allocation.

The HSQ IOM supports 768k queues, which are grouped into 96k queue groups; each comprises eight queues (referred to as HSQ queue groups). HSQ queue groups are used for SAP and subscriber egress queues, network egress queues, and both access and network egress queue group instance queues.

The components of the HSQ IOM operation are described in subsequent sections.

HSQ shaping

The HSQ egress shaping uses the following objects:

  • HSQ queue groups

    An HSQ queue group comprises eight egress queues with two WRR groups. One HS queue group is allocated to each of the following:

    • an egress SAP

    • an egress network port

    • an egress access queue group instance

    • an egress network queue group instance

    • a subscriber egress (a single SLA profile instance in single HS SLA mode)

  • primary shapers

    A primary shaper is allocated for each secondary shaper because it is required in the hierarchy. It does not perform any QoS control except when allocated for subscriber egress configured with multiple SLA profile instances in extended HS SLA mode.

  • secondary shapers

    Secondary shapers provide an abstraction to be used for QoS control of traffic to a downstream device, such as an access node. Shaping can be performed on all traffic or on each scheduling class within the secondary shaper.

  • ports

    The traffic forwarded to each port can be shaped. In addition, traffic in each scheduling class within a port can be shaped individually or within a single WRR group.

Six scheduling classes are supported across all the preceding objects.

The egress QoS scheduling hierarchy is shown in Egress HSQ IOM scheduling hierarchy.

Figure 1. Egress HSQ IOM scheduling hierarchy

The available egress shaping encompasses:

  • per-queue or per-WRR group of queues

  • per-HSQ queue group aggregate

  • per-primary shaper aggregate

  • per-secondary shaper aggregate

  • per-secondary shaper per scheduling class

  • per-port aggregate

  • per-port per scheduling class

HSQ scheduling

HSQ scheduling allows a combination of strict priority and WRR. There are six scheduling classes which are implemented from the HSQ queue group queues through the primary shaper, secondary shaper, and port. The scheduling classes are serviced in a strict priority order (scheduling class 6 having the highest priority and scheduling class 1 having the lowest priority), with WRR groups at the HSQ queue group and port levels, and a dynamic weight at the primary and secondary shaper levels.

Packet forwarding is achieved using service lists. The objects at each level are on a service list at the same level if they are in a state ready to send packets. The objects are off the service list if they have exceeded their configured PIR together with its related burst. When a port has a scheduling opportunity, it selects the secondary shaper to be serviced next, which selects the primary shaper to be serviced next, which selects the HSQ queue group to be serviced next, which selects a queue to be serviced next, resulting in a packet from that queue being forwarded.

At the HSQ queue group level, queues can be attached to one of two WRR groups. Each group is scheduled at a single scheduling class, with packets being taken from the constituent queues based on a configured queue weighting.

Weighting is also supported between queues and WRR groups in different HSQ queue groups per-primary shaper scheduling class. This allows the capacity available at the primary shaper scheduling class to be shared in a WRR manner between the HSQ queue group queues and WRR groups attached to that scheduling class.

There is a single WRR group at the port level that allows multiple scheduling classes to be collapsed to a single class per port, with each class in the group being assigned a weight.

The dynamic weights at the primary and secondary shapers are managed by the system, based on the number of pending packets for each of the shapers, not on the number of attached objects in each. The more pending packets a shaper has, the higher the weight it is assigned. The goal of the scheduling is to ensure a balanced distribution of capacity between each of the primary shapers and each of the secondary shapers. For example, this allows a secondary shaper with 10,000 active HSQ queue groups to receive proportionately more scheduling opportunities than another secondary shaper with only 100 active HSQ queue groups.

HSQ buffer management

The HSQ supports a flexible buffer management configuration that allows both buffer isolation and buffer oversubscription for the queue buffer allocation. There are four levels to the buffer hierarchy with the total buffer allocation being divided into a system-reserved portion and a user-provisioned portion, as shown in HSQ buffer pool hierarchy.

  • root pools

  • mid pools

  • port class pools

  • queue group queues

Figure 2. HSQ buffer pool hierarchy

HSQ LAGs

LAGs are supported on HSQ ports. The LAG port-type must be set to hs to add an HSQ port to a LAG, at which point only HSQ ports can be added to that LAG. When an HSQ queue group is created on a LAG, an HSQ queue group is allocated on each LAG port.

HSQ IOM-specific configurations

The HSQ IOM functionality is configured using the following set of HSQ-specific policies:

HS attachment policy
configures how HSQ queue group queues and WRR groups within SAP egress QoS policies, network queue policies, and egress queue group templates are attached to a scheduling class managed by the port scheduler
HS pool policy
configures the buffer allocation and hierarchy for the root and mid pools on the HSQ IOM
HS port pool policy
configures the buffer allocation and hierarchy for the port standard and alternative class pools
HS scheduler policy
configures the scheduling behavior of the HSQ scheduler classes at a port

The HSQ queue groups are configured using SAP egress QoS policies, network queue policies, and egress queue group templates, in which there are HSQ-specific commands corresponding to the HSQ functionality.

The HS secondary shapers, which provide QoS control for traffic to a downstream device, are configured directly in the port context on which the QoS control is required.

Overrides are available under ports and SAPs for the HSQ-specific configuration.

See to the High Scale QoS IOM Command Reference section for details about command syntax and descriptions.

Non-applicable HSQ IOM functions

As the HSQ IOM traffic manager replaces several egress FP QoS functions, the following configurations are not applicable to an HSQ IOM:

  • QoS related

    • advanced configuration policies

    • egress access and network MDA and port pools

    • egress agg-rate queue-frame-based-accounting

    • egress H-QoS virtual scheduling, including Vports

    • FP egress queue

      • drop tails and CBS

      • WRED per queue (an HSQ equivalent is available)

      • CIR

      • highplus slope in a slope policy

      • burst limit (an HSQ equivalent is available)

      • dynamic MBS

      • network queue MBS (replaced by the hs-mbs command)

    • multi-service sites

    • PBB egress B-SAP per ISID shaping

  • subscriber related

    • Access Node Control Protocol Management (ANCP)

    • PCC rule-based subscriber services (Gx or RADIUS)

    • Web Authentication Protocol (WPP)

  • Service related

    • G.8031 protected Ethernet tunnels

  • System related

    • Ethernet satellite host ports

    • port cross-connects (PXC)

    • Sflow