Class fair hierarchical policing (CFHP)

At ingress, CFHP output traffic is automatically mapped to a unicast or multipoint queue in to reach the correct switch fabric destinations. To manage this automatic queuing function, a shared queue policy has been created or exists by default and is named policer-output-queues.

The unicast queues in the policy are automatically created on each destination switch fabric tap and ingress CFHP unicast packets automatically map to one of the queues based on forwarding class and destination tap. The multipoint queues within the policy are created on the IOM3-XP’s 16/IMM or XMA multicast paths; 16 multicast paths are supported by default with 28 on 7950 XRS systems and the 7750 SR 12-e systems, with the latter having setting ‟tools perform the system set-fabric-speed fabric-speed-b.” The multicast paths represent an available multicast switch fabric path; the number of each being controlled using the command:

configure mcast-management bandwidth-policy policy-name t2-paths secondary-path

    — number-paths number-of-paths [dual-sfm number-of-paths]

For ingress CFHP multicast packets (Broadcast, Unknown unicast, or Multicast—referred to as BUM traffic), the system maintains a conversation hash table per forwarding class and populates the table’s forwarding class hash result entry with the one of the multicast paths. Best-effort traffic uses the secondary paths, and expedited traffic uses the primary paths. When a BUM packet is output by ingress CFHP, a conversation hash is performed and used along with the packet’s forwarding class to select a hash table entry to derive the multicast path to be used. Each table entry maintains a bandwidth counter that is used to monitor the aggregate traffic per multicast path. The process can be optimized by enabling IMPM on any forwarding complex, which allows the system to redistribute this traffic across the IMPM paths on all forwarding complexes to achieve a more even capacity distribution. Be aware that enabling IMPM causes routed and VPLS (IGMP and PIM) snooped IPv4 multicast groups, and routed and PIM snooped (with sg-based forwarding) IPv6 multicast groups to be managed by IMPM.

Any discards performed in the ingress shared queues are reflected in the ingress child policer's discard counters and reported statistics, assuming a discard counter capable stat-mode is configured for the child policer.

When CFHP is being performed at egress, queuing of the CFHP output packets is accomplished through egress queue group queues. The system maintains a special egress queue group template (policer-output-queues) that is automatically applied to all Ethernet access ports that are up. The number of queues, queue types (expedite or best-effort), queue parameters, and the default forwarding class mappings to the queues are managed by the template. On each Ethernet port, the queue parameters may be overridden.

When a SAP egress QoS policy is applied to an Ethernet SAP and the policy contains a forwarding class mapping to a CFHP child policer, the default behavior for queuing the CFHP output is to use the egress Ethernet port’s policer-output-queues queue group and the forwarding class mapping within the group to choose the egress queue. Optionally, the SAP egress QoS policy may also explicitly define which egress queue to use within the default queue group or even map the policer output to a different, explicitly created queue group on the port.

Any discards performed in the egress queue group queues are reflected in the egress child policer’s discard counters and reported statistics assuming a discard counter capable stat-mode is configured for the child policer. Exceed-profile traffic from the policer is counted as out-of-profile traffic in the egress queue group queues.

When a subscriber packet is mapped to a child policer through the SAP egress QoS policy, the actual egress queue group is derived from the subscriber host identification process within the subscriber management module; otherwise, the default queue-group is used.

When egress policed packets are directed to a local SAP queue, and, when this is configured, the output of a show service id id sap sap-id sap-stats only counts these packets through the policer; they are not counted a second time through the queue to avoid double counting. Consequently, any packets sent directly (not through a policer) to a local SAP post-policer queue are not counted in the sap-stats output. The output of show service id id sap sap-id stats always counts these packets in both the related policer and queue. If it is required to count packets sent directly to the local SAP post-policer queue in the sap-stats output, the packets could be sent into a policer with the rate set to max, then into the local SAP queue.

To simplify subscriber destination string provisioning, a def-inter-dest-id command can be used in a sub-sla-mgmt node within a SAP, which allows the definition of a default destination string for all subscribers associated with the SAP. The command also accepts the use-top-q flag that automatically derives the string based on the topmost delineating Dot1Q tag from the SAP’s encapsulation. This feature applies only to the 7450 ESS and 7750 SR.

The command is also supported within the msap-policy allowing similar provisioning behavior for automatically created managed SAPs.

Similar to a scheduler context within a scheduler-policy, the policer-control-policy contains objects called arbiters that control the amount of bandwidth that may be distributed to a set of child policers. Each policer control policy always contains a root arbiter that represents the parent policer. The max-rate defined for the arbiter specifies the decrement rate for the parent policer that governs the overall aggregate rate of every child policer associated with the policy instance. The root arbiter also contains the parent policers MBS configuration parameters that the system uses to individually configure the priority thresholds for each policer instance.

Child policers may parent directly to the root arbiter or to one of the tier 1 or tier 2 explicitly created arbiters.

Each arbiter provides bandwidth to its children using eight strict levels. Children parented at level 8 are first to receive bandwidth. The arbiter continues to distribute bandwidth until either all of its children's bandwidth requirements are met or until the bandwidth it is allowed to distribute is exhausted. The root arbiter is special in that its strict priority levels directly represent the priority thresholds within the parent policer.

Other arbiters may be explicitly created in the policy for the purpose of creating an arbitrary bandwidth distribution hierarchy. The explicitly created arbiters must be defined within tier 1 or tier 2 on the policy. Tier 1 arbiters must always be parented by the root arbiter and therefore become a child of the root arbiter. Any child policers directly parented by a tier 1 policer treat the root arbiter as its grandparent. Inversely, the root arbiter considers the child policers as grandchildren. All grandchild policers inherit the priority level of their parent arbiter (the level that the tier 1 arbiter attaches to the root arbiter) within the parent policer.

An arbiter created on tier 2 may be parented by either an arbiter in tier 1 or by the root arbiter. If the tier 2 arbiter is parented by the root arbiter, it is internally treated the same as a tier 1 arbiter and its child policers have a grandchild to grandparent association with the root arbiter.

When a tier 2 arbiter is parented by a tier 1 arbiter, the child policers parented by a tier 2 arbiter are in a great-grandchild to great-grandparent association with the root arbiter. A great-grandchild policer inherits its indirectly parented tier 1 arbiter’s level association with the root arbiter and therefore the parent policer.

A child policer’s priority level on the root arbiter (directly or indirectly) defines which priority level discards thresholds are associated with packets mapped to the child policer for use in the parent policer (assuming the packet is not discarded by its child policer).

The bandwidth a tier 1 or tier 2 arbiter receives from its parent may be limited by the use of the rate command within the arbiter. When a rate limit is defined for a root arbiter, the system enforces the aggregate rate by calculating a per child policer PIR rate based on the distributed bandwidth per child. This calculated PIR is used to override the child's defined PIR and is represented as the child's operational PIR. The calculated rate is never greater than a child policer's provisioned rate.

A child policer parented to an arbiter can be enabled to forward traffic exceeding its PIR, in which case:

• Traffic exceeding the operational PIR of the child policer is reprofiled to be exceed-profile, where the operational PIR is determined by the H-pol algorithm from the configuration of the policer parent and the associated arbiters (root or intermediate, or both).

• Traffic exceeding the child policer's operational PIR and exceed-profile traffic entering the child policer does not consume capacity from the parent policer (meaning that it does not contribute to the parent policer bucket depth with respect to any of its thresholds).

• Traffic that did not exceed the child policer's operational PIR (when that child is configured with enable-exceed-pir) can exceed its parent rate (max-rate for the root arbiter) in which case the traffic is forwarded and reprofiled to be exceed-profile and its effect on the child policer is revoked (meaning that it does not contribute to any of the child policer bucket (PIR, CIR, FIR) depths with respect to any of its thresholds).

Access ingress packets have one of three initial profile states before processing by the policer:

• undefined

• profile in

• profile out

The SAP ingress QoS policy classification rules map each packet to either a forwarding class or a subclass within a forwarding class. The forwarding class or subclass may be defined as explicit profile in or profile out (the default is no profile). When a packet’s forwarding class or subclass is explicitly defined as profile in or profile out, the packet’s priority is ignored, and it is not handled by the ingress policer as profile ‛undefined’.

See Effect of profile-capped mode on CIR output to track the ingress behavior of initial profile and the effect of the CIR bucket on that initial state.

At egress, an ingress policer output of ‛in-profile’ is treated as ‛soft-in-profile’ and an ingress policer output of ‛out-of-profile’ is treated as ‛soft-out-of-profile’. Each may be changed by egress profile reclassification or by an egress policer with a CIR rate defined.

Packets that are explicitly ‛in-profile’ remain ‛in-profile’ in the ingress forwarding plane and are not affected by the ingress policer CIR bucket state when profile-capped mode is not enabled. They do not bypass the policer’s CIR leaky bucket but are extended with a greater threshold than the CBS derived ‛threshold-bc’. This allows the ‛undefined’ packets to backfill the remaining conforming CIR bandwidth after accounting for the explicit ‛in-profile’ packets. This does not prevent the sum of the explicit ‛in-profile’ from exceeding the configured CIR rate, but it does cause the ‛undefined’ packets that are marked ‛in-profile’ to diminish to zero when the combined explicit ‛in-profile’ rate and ‛undefined’ rate causes the bucket to reach ‛threshold-bc’.

The policer’s CIR bucket indicates that the explicit ‛in-profile’ packets should be marked ‛out-of-profile’ when the bucket reaches the greater threshold, but this indication is ignored by the ingress forwarding plane. All explicit ‛in-profile’ packets remain in-profile within the ingress forwarding plane. However, when the packet is received at egress, an ingress ‛in-profile’ packet is treated as ‛soft-in-profile’ and the profile may be changed either by explicit profile reclassification or by an egress policer with a CIR rate defined.

Explicit in-profile packets do not automatically use the high-priority threshold (‛threshold-be-high’) within the child policer’s PIR bucket. If preferential burst tolerance is needed for explicit in-profile packets, the packets should also be classified as priority high.

When profile-capped mode is enabled, the packet handling behavior defined in Ingress ‛undefined’ initial profile is altered in one aspect. The CIR output state of yellow at the greater threshold is actually honored and the packet is treated as out-of-profile. The packet is sent to egress in the ‛soft-out-of-profile’ state in this case.

Packets that are explicitly ‛out-of-profile’ remain ‛out-of-profile in the ingress forwarding plane. Unlike initially ‛in-profile’ packets, they do not consume the policer’s CIR bucket depth (accomplished by setting the ‛threshold-bc’ to 0) and therefore do not have an impact on the amount of ‛undefined’ marked as ‛in-profile’ by the policer.

While explicit ‛out-of-profile’ packets remain out-of-profile within the ingress forwarding plane, the egress forwarding plane treats ingress out-of-profile packets as ‛soft-out-of-profile’ and the profile may be changed either by explicit profile reclassification or by an egress policer with a CIR rate defined.

An egress profile reclassification overrides the ingress-derived profile of a packet and may set it to hard-inplus-profile, hard-in-profile, hard-out-of-profile, or hard-exceed-profile. A packet that has not been reclassified at egress retains its soft-in-profile or soft-out-of-profile status.

Egress inplus-profile and in-profile (including soft-in-profile and hard-in-profile) packets use the child policer’s high threshold-be value within the child policer’s PIR bucket while soft-out-of-profile and hard-out-of-profile packets use the child policer’s low threshold-be value. Egress hard-exceed-profile packets are not subject to any threshold control in the child's PIR bucket if enable-exceed-pir is configured; otherwise, they are discarded.

Traffic sent through an egress policer with a non-zero CIR is reprofiled by default based on the CIR threshold of the egress policer. To accommodate designs where traffic is set to be out-of-profile at ingress, and the out-of-profile state is required to be maintained by an egress policer, the parameter profile-out-preserve can be configured under the egress policer. Explicit egress reclassification to the profile takes precedence over the profile-out-preserve operation.

When an egress policer has been configured with a CIR (maximum or explicit rate other than 0) and profile-capped mode is not enabled, the policer’s CIR bucket state overrides the ingress soft-in-profile or soft-out-of-profile state much like the ingress policer handles initial profile undefined packets. If the CIR has not been defined or set to 0 on the egress policer, the egress policer output state is in-profile for soft-in-profile packets and out-of-profile for soft-out-of-profile packets.

If a packet’s profile has been reclassified at egress, the new profile classification is handled in the same way as the ingress policer handling of initial in-profile or out-of-profile packets. When a packet has been reclassified as hard-inplus-profile or hard-in-profile, it is applied to the egress policer’s CIR bucket using a threshold-bc higher than the threshold-bc derived from the policer’s CBS parameter, but the policer output profile state remains inplus-profile or in-profile even if the higher threshold is crossed. When a packet has been reclassified as hard-out-of-profile or hard-exceed-profile, it does not consume the egress policer’s CIR bucket depth and the policer output profile state remains out-of-profile or exceed-profile.

When profile-capped mode is enabled, the egress packet handling described in Egress policer CIR packet handling without profile-capped mode is modified in three ways.

First, the soft-in-profile packets received from ingress are handled in the same way as egress explicit profile in reclassification, unless the packet has been reclassified to profile out or profile exceed at egress.

Second, explicit egress inplus-profile, in-profile, and soft-in-profile packets that have not been reclassified to out-of-profile or exceed-profile at egress are allowed to be marked out-of-profile by an egress policer with a CIR not set to 0.

Third, when a policer has a CIR set to 0 (the default rate), all profile-capped packets are treated as out-of-profile independent of the initial profile state, except for exceed-profile packets that remain as exceed-profile.

An egress policer can be configured to forward traffic that enters the policer with an exceed-profile state or exceeds its operational PIR instead of dropping it. The traffic exceeding the PIR is assigned an exceed-profile state. This is supported for any configured (not dynamic) policer in a SAP egress QoS policy, which can be used for both SAPs and subscribers, and in an egress queue group template that can be used on egress network ports.

When enable-exceed-pir is configured under the policer, the exceed-profile state of a packet takes precedence over the hard-out/in/inplus reclassification, specifically, traffic that is reprofiled to exceed within a SAP egress policer has an exceed-profile state regardless of whether it was reclassified at egress to hard-out, hard-in, or hard-inplus.

The stat-mode offered-total-cir-exceed command provides forward and drop counters for exceed-profile traffic, as follows:

configure
    qos
        queue-group-templates
            egress
                queue-group <queue-group-name> create
                    policer <policer-id> create
                        enable-exceed-pir
                        stat-mode offered-total-cir-exceed
                    exit
                exit
            exit
        exit
        sap-egress <policy-id> create
            policer <policer-id> create
                enable-exceed-pir
                stat-mode offered-total-cir-exceed
            exit
        exit
    exit
exit

The dot1p, outer dot1p, DSCP, and precedence can be remarked for the exceed-profile traffic.

Packets processed by a SAP or subscriber egress child policer can have their forwarding class and profile state remapped to a different forwarding class and profile state after exiting the policer. Remapping is achieved by using a post-policer mapping policy containing mapping statements that determine the remapping of packets based on their forwarding class and profile state.

The post-policer mapping policy is configured as follows:

configure 
    qos
        post-policer-mapping <mapping-policy-name> [create]
            description <description-string>
            fc <fc-name> profile <profile> [create]
                maps-to fc <fc-name> profile <profile>

where:

• <mapping-policy-name> is the name of mapping policy, up to 32 characters.

• <description> is the description text string.

• <fc-name> can be af, be, ef, h1, h2, l1, l2, or nc.

• <profile> can be in, out, exceed, or inplus.

Up to 32 mapping statements are supported within a policy, covering the maximum combinations of eight forwarding classes and four profile states. A maximum of seven post-policer mapping policies can be configured per system.

After being configured, the post-policer mapping policy must be applied within a SAP egress QoS policy with, at most, one mapping policy per SAP egress QoS policy:

configure 
    qos
        sap-egress <policy-id> [create]
            post-policer-mapping <mapping-policy-name>

Packets entering a child policer are assigned a forwarding class and profile state. The configuration of the policer can change the profile state of the exiting packet, but not the forwarding class. If a post-policer mapping policy is applied within the SAP egress QoS policy, a packet exiting the policer with a specific forwarding class and profile state can be remapped to a different forwarding class and profile state. For example, consider the following post-policer mapping policy:

configure 
    qos
        post-policer-mapping ppm1 create
            fc ef profile exceed create
                maps-to fc be profile out
            exit
        exit

Packets exiting an egress child policer with forwarding class ‟ef” and profile state ‟exceed” have their forwarding class remapped to ‟be” and their profile state to ‟out”.

The mapping applies to all policers within the SAP egress QoS policy, including regular child policers and policers configured in an IP/IPv6 criterion action statement, except for dynamic policers. The remapping does not affect the policer statistics or the parent policer processing (root arbiter) as it occurs after each of these.

The new forwarding class is used to select the egress queue on which the post-policer traffic is placed. The new profile is used to determine the congestion control handling in that queue and its pool, specifically the drop tail or slope that is applied.

Egress packet remarking is based on the marking configured for the forwarding class and profile of the traffic after being policed but before it is remapped.

This remapping is supported on platforms using FP3- and higher-based hardware, with the exception of the 7750 SR-a4/a8, which does not support egress policers.

Remapping a subset of packets from one forwarding class to another could result in out-of-order packets being received at the destination if there is congestion or different latency characteristics on the paths of the different forwarding classes.

A policer has multiple types of input traffic and multiple possible output states for each input traffic type. These variations differ between ingress and egress.

For ingress policing, each offered packet has a priority and a profile state. The priority is used by the policer to choose either the high- or low-priority PIR threshold-be. Every offered packet is either priority high or priority low. The offered profile state defines how a packet interacts with the policers CIR bucket state. The combinations of priority and initial profile are as follows:

• offered priority low, undefined profile

• offered priority low, explicit profile in

• offered priority low, explicit profile out

• offered priority high, undefined profile

• offered priority high, explicit profile in

• offered priority high, explicit profile out

  Note: When de-1-out-profile is enabled, DEI = 0 is considered as undefined profile and DEI = 1 is considered the same as profile out.

The possible output results for the ingress policer are:

• output green (in-profile)

• output yellow (out-of-profile)

• output red (discard)

To conserve counter resources, the system supports a policer stat-mode command that is used to identify what counters are actually needed for the policer. Not every policer has a CIR defined, so the output green/yellow states do not exist. Also, not every policer has both high- and low-priority or explicit in-profile or out-of-profile offered traffic types. Essentially, the stat-mode command allows the counter resources to be allocated based on the accounting needs of the individual policers.

Setting the stat-mode does not modify the packet handling behavior of the policer. For example, if the configured stat-mode does not support in-profile and out-of-profile output accounting, the policer is not blocked from having a configured CIR rate. The CIR rate is enforced, but the amount of in-profile and out-of-profile traffic output from the policer is not counted separately (or maybe not at all based on the configured stat-mode).

A policer is created with minimal counters sufficient to provide total offered and total discarded (the total forwarded is computed as the sum of the offered and discarded counters). The stat-mode is defined within the sap-ingress or sap-egress QoS policy in the policer context. When defining the stat-mode, the counter resources needed to implement the mode must be available on all forwarding planes where the policer has been created using the QoS policy unless the policer instance has a stat-mode override defined. Use the tools dump resource-usage card fp command to see the resources used and available. If insufficient resources exist, the change in the mode fails without any change to the existing counters currently applied to the existing policers. If the QoS policy is being applied to a SAP or subscriber or multiservice site context and insufficient counter resources exist to implement the configured modes for the policers within the policy, the QoS policy is not applied. For SAPs, this means the previous QoS policy stays in effect. For subscribers, it could mean that the subscriber host event where the QoS policy is being applied fails and the subscriber host may be blocked or removed.

A stat-mode with at least minimal stats is required before the policer can be assigned to a parent arbiter using the parent command.

Successfully changing the stat-mode for a policer causes the counters associated with the policer to reset to zero. Any collected stats on the object the policer is created on also reset to zero.

The system uses the forwarding plane counters to generate accounting statistics and for calculating the operational PIR and FIR rates for a set of children when they are managed by a policer-control-policy. Only the offered counters are used in hierarchical policing rate management. When multiple offered stats are maintained for a child policer, they are summed to derive the total offered rate for each child policer.

All ingress policers have a default CIR value of 0, meaning that by default, all packets except packets classified as profile in is output by the policer as out-of-profile. This may have a negative impact on egress marking decisions (if in-profile and out-of-profile have different marking values) and on queue congestion handling (WRED or queue drop tail decisions when out-of-profile is less preferred). The following options exist to address this potential issue:

• If all packets handled by the policer must be output as in-profile by the policer, either the packet's forwarding class or subclass can be defined as profile in or the CIR on the policer can be defined as max

• If some packets must be output as in-profile while others output as out-of-profile, three options exist:

  • The CIR may be left at '0' while mapping the packets that must be output as in-profile to a forwarding class or subclass provisioned as profile in.

  • The CIR may be set to max while mapping the packets that must be output as out-of-profile to a forwarding class or subclass provisioned as profile out.

  • Ignore the CIR on the policer and solely rely on the forwarding class or subclass profile provisioning to the correct policer CIR output.

Make sure to use the correct stat mode if the policer’s CIR is explicitly not set or is set to 0. The no-cir version of the stat-mode must be used when the CIR has a non-zero value. Also, when overriding the policer’s CIR mode, make sure to override the stat-mode instance (CIR override can be performed using SNMP access).

Ingress supported stat-modes are:

• no-stats

• minimal - default

• offered-profile-no-cir

• offered-priority-no-cir

• offered-limited-profile-cir

• offered-profile-cir

• offered-priority-cir

• offered-total-cir

• offered-profile-capped-cir

• offered-limited-capped-cir

All packets received on the egress forwarding plane are profiled as either in-profile or out-of-profile. The egress forwarding plane treats the ingress-derived profile as a soft state that may be either overridden by an egress profile reclassification or by a CIR rate enforced by an egress policer.

Egress policers have a default CIR set to 0, but in the egress case a value of 0 disables policer profiling. Egress packets on a CIR-disabled egress policer retain their offered profile state (soft-in-profile, soft-out-of-profile, hard-inplus, hard-in-profile, hard-out-of-profile, or hard-exceed-profile) unless the enable-exceed-pir command is configured, in which case the exceed-profile state of a packet takes precedence over the hard-out/in/inplus reclassification.

For egress, the possible types of offered packets include:

• soft offered in-profile (from ingress)

• soft offered out-of-profile (from ingress)

• egress explicit inplus-profile (reclassified at egress)

• egress explicit in-profile (reclassified at egress)

• egress explicit out-of-profile (reclassified at egress)

• egress explicit exceed-profile (reclassified at egress)

The possible output results are:

• output inplus-profile

• output in-profile

• output out-of-profile

• output discard or exceed-profile

The stat-mode command follows the same counter resource rules as ingress.

Egress supported stat-modes are:

• no-stats

• minimal - default

• offered-profile-no-cir

• offered-profile-cir

• offered-total-cir

• offered-limited-capped-cir

• offered-profile-capped-cir

• offered-total-cir-exceed

• offered-four-profile-no-cir

• offered-total-cir-four-profile

Details of the output showing the stat-modes for ingress and egress child policers are in the Class Fair Hierarchical Policing for SAPs section of the SR OS Advanced Configuration Guide.